ImageApi/specs/001-video-wall/research.md

Research: VideoWall

R1: FFmpeg Preview Clip Generation Strategy

Decision: Use ffmpeg's select filter with segment-based extraction, extending the existing OverviewVideo pattern in src/video/ffmpeg.rs.

Rationale: The codebase already has a nearly identical pattern at src/video/ffmpeg.rs using select='lt(mod(t,{interval}),1)' which selects 1-second frames at evenly spaced intervals across the video duration. The existing pattern outputs GIF; we adapt it to output MP4 at 480p with audio.

Approach:

1. Use ffprobe to get video duration (existing get_video_duration() pattern)
2. Calculate interval: duration / 10 (or fewer segments for short videos)
3. Use ffmpeg with:
   • Video filter: select='lt(mod(t,{interval}),1)',setpts=N/FRAME_RATE/TB,scale=-2:480
   • Audio filter: aselect='lt(mod(t,{interval}),1)',asetpts=N/SR/TB
   • Output: MP4 with H.264 video + AAC audio
   • CRF 28 (lower quality acceptable for previews, reduces file size)
   • Preset: veryfast (matches existing HLS transcoding pattern)

Alternatives considered:

• Generating separate segment files and concatenating: More complex, no benefit over select filter
• Using GIF output: Rejected per clarification — MP4 is 5-10x smaller with better quality
• Stream copy (no transcode): Not possible since we're extracting non-contiguous segments

R2: Preview Clip Storage and Caching

Decision: Store preview clips on filesystem in a dedicated PREVIEW_CLIPS_DIRECTORY mirroring the source directory structure (same pattern as THUMBNAILS and GIFS_DIRECTORY).

Rationale: The project already uses this directory-mirroring pattern for thumbnails and GIF previews. It's simple, requires no database for file lookup (path is deterministic), and integrates naturally with the existing file watcher cleanup logic.

Storage path formula: {PREVIEW_CLIPS_DIRECTORY}/{relative_path_from_BASE_PATH}.mp4

• Example: Video at BASE_PATH/2024/vacation.mov → Preview at PREVIEW_CLIPS_DIRECTORY/2024/vacation.mp4

Alternatives considered:

• Database BLOBs: Too large, not suited for binary video files
• Content-addressed storage (hash-based): Unnecessary complexity for single-user system
• Flat directory with UUID names: Loses the intuitive mapping that thumbnails/GIFs use

R3: Preview Generation Status Tracking

Decision: Track generation status in SQLite via a new video_preview_clips table with Diesel ORM, following the existing DAO pattern.

Rationale: The batch status endpoint (FR-004) needs to efficiently check which previews are ready for a list of video paths. A database table is the right tool — it supports batch queries (existing get_exif_batch() pattern), survives restarts, and tracks failure states. The file watcher already uses batch DB queries to detect unprocessed files.

Status values: pending, processing, complete, failed

Alternatives considered:

• Filesystem-only (check if .mp4 exists): Cannot track processing or failed states; race conditions on concurrent requests
• In-memory HashMap: Lost on restart; doesn't support batch queries efficiently across actor boundaries

R4: Concurrent Generation Limits

Decision: Use Arc<Semaphore> with a limit of 2 concurrent ffmpeg preview generation processes, matching the existing PlaylistGenerator pattern.

Rationale: The PlaylistGenerator actor in src/video/actors.rs already uses this exact pattern to limit concurrent ffmpeg processes. Preview generation is CPU-intensive (transcoding), so limiting concurrency prevents server overload. The semaphore pattern is proven in this codebase.

Alternatives considered:

• Unbounded concurrency: Would overwhelm the server with many simultaneous ffmpeg processes
• Queue with single worker: Too slow for batch generation; 2 concurrent is a good balance
• Sharing the existing PlaylistGenerator semaphore: Would cause HLS generation and preview generation to compete for the same slots; better to keep them independent

R5: Mobile App Video Playback Strategy

Decision: Use expo-video VideoView components inside FlatList items, with muted autoplay and viewport-based pause/resume.

Rationale: The app already uses expo-video (v3.0.15) for the single video player in viewer/video.tsx. The library supports multiple simultaneous players, loop mode, and programmatic mute/unmute. FlatList's viewabilityConfig callback can be used to pause/resume players based on viewport visibility.

Key configuration per cell:

• player.loop = true
• player.muted = true (default)
• player.play() when visible, player.pause() when offscreen
• nativeControls={false} (no controls needed in grid)

Audio-on-focus: On long-press, unmute the pressed player and mute all others. Track the "focused" player ID in hook state.

Alternatives considered:

• HLS streaming for previews: Overkill for <10s clips; direct MP4 download is simpler and faster
• Animated GIF display via Image component: Rejected per clarification — MP4 with expo-video is better
• WebView-based player: Poor performance, no native gesture integration

R6: API Endpoint Design

Decision: Two new endpoints — one to serve preview clips, one for batch status checking.

Rationale:

• GET /video/preview?path=... serves the MP4 file directly (or triggers on-demand generation and returns 202 Accepted). Follows the pattern of GET /image?path=... for serving files.
• POST /video/preview/status accepts a JSON body with an array of video paths and returns their preview generation status. This allows the mobile app to efficiently determine which previews are ready in a single request (batch pattern from get_exif_batch()).

Alternatives considered:

• Single endpoint that blocks until generation completes: Bad UX — generation takes up to 30s
• WebSocket for real-time status: Overkill for this use case; polling with batch status is simpler
• Including preview URL in the existing /photos response: Would couple the photo listing endpoint to preview generation; better to keep separate