19c099360e
Backend (Rust/Actix-web): - Add video_preview_clips table and PreviewDao for tracking preview generation - Add ffmpeg preview clip generator: 10 equally-spaced 1s segments at 480p with CUDA NVENC auto-detection - Add PreviewClipGenerator actor with semaphore-limited concurrent processing - Add GET /video/preview and POST /video/preview/status endpoints - Extend file watcher to detect and queue previews for new videos - Use relative paths consistently for DB storage (matching EXIF convention) Frontend (React Native/Expo): - Add VideoWall grid view with 2-3 column layout of looping preview clips - Add VideoWallItem component with ActiveVideoPlayer sub-component for lifecycle management - Add useVideoWall hook for batch status polling with 5s refresh - Add navigation button in grid header (visible when videos exist) - Use TextureView surface type to fix Android z-ordering issues - Optimize memory: players only mount while visible via FlatList windowSize - Configure ExoPlayer buffer options and caching for short clips - Tap to toggle audio focus, long press to open in full viewer Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
92 lines
6.0 KiB
Markdown
92 lines
6.0 KiB
Markdown
# Research: VideoWall
|
|
|
|
## R1: FFmpeg Preview Clip Generation Strategy
|
|
|
|
**Decision**: Use ffmpeg's `select` filter with segment-based extraction, extending the existing `OverviewVideo` pattern in `src/video/ffmpeg.rs`.
|
|
|
|
**Rationale**: The codebase already has a nearly identical pattern at `src/video/ffmpeg.rs` using `select='lt(mod(t,{interval}),1)'` which selects 1-second frames at evenly spaced intervals across the video duration. The existing pattern outputs GIF; we adapt it to output MP4 at 480p with audio.
|
|
|
|
**Approach**:
|
|
1. Use `ffprobe` to get video duration (existing `get_video_duration()` pattern)
|
|
2. Calculate interval: `duration / 10` (or fewer segments for short videos)
|
|
3. Use ffmpeg with:
|
|
- Video filter: `select='lt(mod(t,{interval}),1)',setpts=N/FRAME_RATE/TB,scale=-2:480`
|
|
- Audio filter: `aselect='lt(mod(t,{interval}),1)',asetpts=N/SR/TB`
|
|
- Output: MP4 with H.264 video + AAC audio
|
|
- CRF 28 (lower quality acceptable for previews, reduces file size)
|
|
- Preset: `veryfast` (matches existing HLS transcoding pattern)
|
|
|
|
**Alternatives considered**:
|
|
- Generating separate segment files and concatenating: More complex, no benefit over select filter
|
|
- Using GIF output: Rejected per clarification — MP4 is 5-10x smaller with better quality
|
|
- Stream copy (no transcode): Not possible since we're extracting non-contiguous segments
|
|
|
|
## R2: Preview Clip Storage and Caching
|
|
|
|
**Decision**: Store preview clips on filesystem in a dedicated `PREVIEW_CLIPS_DIRECTORY` mirroring the source directory structure (same pattern as `THUMBNAILS` and `GIFS_DIRECTORY`).
|
|
|
|
**Rationale**: The project already uses this directory-mirroring pattern for thumbnails and GIF previews. It's simple, requires no database for file lookup (path is deterministic), and integrates naturally with the existing file watcher cleanup logic.
|
|
|
|
**Storage path formula**: `{PREVIEW_CLIPS_DIRECTORY}/{relative_path_from_BASE_PATH}.mp4`
|
|
- Example: Video at `BASE_PATH/2024/vacation.mov` → Preview at `PREVIEW_CLIPS_DIRECTORY/2024/vacation.mp4`
|
|
|
|
**Alternatives considered**:
|
|
- Database BLOBs: Too large, not suited for binary video files
|
|
- Content-addressed storage (hash-based): Unnecessary complexity for single-user system
|
|
- Flat directory with UUID names: Loses the intuitive mapping that thumbnails/GIFs use
|
|
|
|
## R3: Preview Generation Status Tracking
|
|
|
|
**Decision**: Track generation status in SQLite via a new `video_preview_clips` table with Diesel ORM, following the existing DAO pattern.
|
|
|
|
**Rationale**: The batch status endpoint (FR-004) needs to efficiently check which previews are ready for a list of video paths. A database table is the right tool — it supports batch queries (existing `get_exif_batch()` pattern), survives restarts, and tracks failure states. The file watcher already uses batch DB queries to detect unprocessed files.
|
|
|
|
**Status values**: `pending`, `processing`, `complete`, `failed`
|
|
|
|
**Alternatives considered**:
|
|
- Filesystem-only (check if .mp4 exists): Cannot track `processing` or `failed` states; race conditions on concurrent requests
|
|
- In-memory HashMap: Lost on restart; doesn't support batch queries efficiently across actor boundaries
|
|
|
|
## R4: Concurrent Generation Limits
|
|
|
|
**Decision**: Use `Arc<Semaphore>` with a limit of 2 concurrent ffmpeg preview generation processes, matching the existing `PlaylistGenerator` pattern.
|
|
|
|
**Rationale**: The `PlaylistGenerator` actor in `src/video/actors.rs` already uses this exact pattern to limit concurrent ffmpeg processes. Preview generation is CPU-intensive (transcoding), so limiting concurrency prevents server overload. The semaphore pattern is proven in this codebase.
|
|
|
|
**Alternatives considered**:
|
|
- Unbounded concurrency: Would overwhelm the server with many simultaneous ffmpeg processes
|
|
- Queue with single worker: Too slow for batch generation; 2 concurrent is a good balance
|
|
- Sharing the existing PlaylistGenerator semaphore: Would cause HLS generation and preview generation to compete for the same slots; better to keep them independent
|
|
|
|
## R5: Mobile App Video Playback Strategy
|
|
|
|
**Decision**: Use `expo-video` `VideoView` components inside FlatList items, with muted autoplay and viewport-based pause/resume.
|
|
|
|
**Rationale**: The app already uses `expo-video` (v3.0.15) for the single video player in `viewer/video.tsx`. The library supports multiple simultaneous players, `loop` mode, and programmatic mute/unmute. FlatList's `viewabilityConfig` callback can be used to pause/resume players based on viewport visibility.
|
|
|
|
**Key configuration per cell**:
|
|
- `player.loop = true`
|
|
- `player.muted = true` (default)
|
|
- `player.play()` when visible, `player.pause()` when offscreen
|
|
- `nativeControls={false}` (no controls needed in grid)
|
|
|
|
**Audio-on-focus**: On long-press, unmute the pressed player and mute all others. Track the "focused" player ID in hook state.
|
|
|
|
**Alternatives considered**:
|
|
- HLS streaming for previews: Overkill for <10s clips; direct MP4 download is simpler and faster
|
|
- Animated GIF display via Image component: Rejected per clarification — MP4 with expo-video is better
|
|
- WebView-based player: Poor performance, no native gesture integration
|
|
|
|
## R6: API Endpoint Design
|
|
|
|
**Decision**: Two new endpoints — one to serve preview clips, one for batch status checking.
|
|
|
|
**Rationale**:
|
|
- `GET /video/preview?path=...` serves the MP4 file directly (or triggers on-demand generation and returns 202 Accepted). Follows the pattern of `GET /image?path=...` for serving files.
|
|
- `POST /video/preview/status` accepts a JSON body with an array of video paths and returns their preview generation status. This allows the mobile app to efficiently determine which previews are ready in a single request (batch pattern from `get_exif_batch()`).
|
|
|
|
**Alternatives considered**:
|
|
- Single endpoint that blocks until generation completes: Bad UX — generation takes up to 30s
|
|
- WebSocket for real-time status: Overkill for this use case; polling with batch status is simpler
|
|
- Including preview URL in the existing `/photos` response: Would couple the photo listing endpoint to preview generation; better to keep separate
|