19c099360e
Backend (Rust/Actix-web): - Add video_preview_clips table and PreviewDao for tracking preview generation - Add ffmpeg preview clip generator: 10 equally-spaced 1s segments at 480p with CUDA NVENC auto-detection - Add PreviewClipGenerator actor with semaphore-limited concurrent processing - Add GET /video/preview and POST /video/preview/status endpoints - Extend file watcher to detect and queue previews for new videos - Use relative paths consistently for DB storage (matching EXIF convention) Frontend (React Native/Expo): - Add VideoWall grid view with 2-3 column layout of looping preview clips - Add VideoWallItem component with ActiveVideoPlayer sub-component for lifecycle management - Add useVideoWall hook for batch status polling with 5s refresh - Add navigation button in grid header (visible when videos exist) - Use TextureView surface type to fix Android z-ordering issues - Optimize memory: players only mount while visible via FlatList windowSize - Configure ExoPlayer buffer options and caching for short clips - Tap to toggle audio focus, long press to open in full viewer Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
12 KiB
Feature Specification: VideoWall
Feature Branch: 001-video-wall
Created: 2026-02-25
Status: Draft
Input: User description: "I would like to implement a new View 'VideoWall' in the React native mobile app, with supporting API/tasks to generate at most 10 second long GIF/Videos that are 10 equally spaced 1 second clips of the original video. This view will display a grid 2/3 columns wide of all these clips playing simultaneously. It should let the user view all videos in the current folder/search results."
Clarifications
Session 2026-02-25
- Q: What format should preview clips be generated in (GIF vs video)? → A: MP4 video clips (small files, hardware-accelerated playback, best quality-to-size ratio).
- Q: What resolution should preview clips be generated at? → A: 480p scaled down (sharp in grid cells, small files, smooth simultaneous playback).
- Q: How should audio be handled in preview clips? → A: Audio on focus — muted by default, audio plays when user long-presses on a clip. Audio track is preserved during generation.
User Scenarios & Testing (mandatory)
User Story 1 - Browse Videos as a Visual Wall (Priority: P1)
A user navigates to a folder containing videos and switches to the VideoWall view. The screen fills with a grid of video previews — short looping clips that give a visual summary of each video. All previews play simultaneously, creating an immersive "wall of motion" that lets the user quickly scan and identify videos of interest without opening each one individually.
Why this priority: This is the core experience. Without the visual grid of simultaneously playing previews, the feature has no value. This story delivers the primary browsing capability.
Independent Test: Can be fully tested by navigating to any folder with videos, switching to VideoWall view, and confirming that preview clips display in a grid and play simultaneously. Delivers immediate visual browsing value.
Acceptance Scenarios:
- Given a user is viewing a folder containing 6 videos, When they switch to VideoWall view, Then they see a grid of 6 video previews arranged in 2-3 columns, all playing simultaneously in a loop.
- Given a user is viewing a folder containing 20 videos, When they switch to VideoWall view, Then the grid is scrollable and loads previews progressively as they scroll.
- Given a user is in VideoWall view, When they tap on a video preview, Then they navigate to the full video player for that video.
- Given a user is in VideoWall view with all clips muted, When they long-press on a preview clip, Then that clip's audio unmutes and all other clips remain muted.
User Story 2 - Server Generates Preview Clips (Priority: P1)
When preview clips are requested for a video that has not yet been processed, the server generates a short preview clip. The preview is composed of 10 equally spaced 1-second segments extracted from the original video, concatenated into a single clip of at most 10 seconds. Once generated, the preview is cached so subsequent requests are served instantly.
Why this priority: The VideoWall view depends entirely on having preview clips available. Without server-side generation, there is nothing to display. This is co-priority with Story 1 as they are interdependent.
Independent Test: Can be tested by requesting a preview clip for any video via the API and confirming the response is a playable clip of at most 10 seconds composed of segments from different parts of the original video.
Acceptance Scenarios:
- Given a video exists that has no preview clip yet, When a preview is requested, Then the system generates a clip of at most 10 seconds composed of 10 equally spaced 1-second segments from the original video.
- Given a video is shorter than 10 seconds, When a preview is requested, Then the system generates a preview using fewer segments (as many 1-second clips as the video duration allows), resulting in a shorter preview.
- Given a preview clip was previously generated for a video, When it is requested again, Then the cached version is served without re-processing.
- Given a video file no longer exists, When a preview is requested, Then the system returns an appropriate error indicating the source video is missing.
User Story 3 - VideoWall from Search Results (Priority: P2)
A user performs a search or applies filters (tags, date range, camera, location) and the results include videos. They switch to VideoWall view to see preview clips of all matching videos displayed in the same grid layout, allowing visual browsing of search results.
Why this priority: Extends the core VideoWall browsing to work with filtered/search result sets. Important for discoverability but depends on Story 1 and 2 being functional first.
Independent Test: Can be tested by performing a search that returns videos, switching to VideoWall view, and confirming that only matching videos appear as previews in the grid.
Acceptance Scenarios:
- Given a user has search results containing 8 videos and 12 photos, When they switch to VideoWall view, Then only the 8 video previews are displayed in the grid.
- Given a user applies a tag filter that matches 3 videos, When they view the VideoWall, Then exactly 3 video previews are shown.
User Story 4 - Background Preview Generation (Priority: P3)
Preview clips are generated proactively in the background for videos discovered during file watching, so that when a user opens VideoWall, most previews are already available and the experience feels instant.
Why this priority: Enhances performance and perceived responsiveness. The feature works without this (on-demand generation), but background processing greatly improves the user experience for large libraries.
Independent Test: Can be tested by adding new video files to a monitored folder and confirming that preview clips are generated automatically within the next scan cycle, before any user requests them.
Acceptance Scenarios:
- Given a new video is added to the media library, When the file watcher detects it, Then a preview clip is generated in the background without user intervention.
- Given the system is generating previews in the background, When a user opens VideoWall, Then already-generated previews display immediately while pending ones show a placeholder.
Edge Cases
- What happens when a video is corrupted or cannot be processed? The system shows a placeholder/error state for that video and does not block other previews from loading.
- What happens when the user scrolls quickly through a large library? Previews outside the visible viewport should pause or not load to conserve resources, and resume when scrolled back into view.
- What happens when a video is extremely long (e.g., 4+ hours)? The same algorithm applies — 10 equally spaced 1-second clips — ensuring the preview still represents the full video.
- What happens when a video is exactly 10 seconds long? Each 1-second segment starts at second 0, 1, 2, ... 9, effectively previewing the entire video.
- What happens when storage for preview clips runs low? Preview clips should be reasonably compressed and sized to minimize storage impact.
- What happens when many previews are requested simultaneously (e.g., opening a folder with 100 videos)? The system should queue generation and serve already-cached previews immediately while others are processed.
Requirements (mandatory)
Functional Requirements
- FR-001: System MUST generate preview clips for videos as MP4 files scaled to 480p resolution, where each preview is composed of up to 10 equally spaced 1-second segments from the original video, resulting in a clip of at most 10 seconds.
- FR-002: System MUST cache generated preview clips so they are only generated once per source video.
- FR-003: System MUST provide an endpoint to retrieve a preview clip for a given video path.
- FR-004: System MUST provide an endpoint to retrieve preview availability status for a batch of video paths so the client knows which previews are ready.
- FR-005: The mobile app MUST display a VideoWall view showing video previews in a grid of 2 columns on smaller screens and 3 columns on larger screens.
- FR-006: All visible preview clips in the VideoWall MUST play simultaneously, muted, and loop continuously.
- FR-006a: When a user long-presses on a preview clip, the app MUST unmute that clip's audio. Only one clip may have audio at a time.
- FR-006b: Preview clips MUST retain their audio track during generation (not stripped) to support audio-on-focus playback.
- FR-007: The VideoWall MUST support browsing videos from both folder navigation and search/filter results.
- FR-008: Tapping a preview clip in the VideoWall MUST navigate the user to the full video.
- FR-009: For videos shorter than 10 seconds, the system MUST generate a preview using as many full 1-second segments as the video duration allows.
- FR-010: The system MUST display a placeholder for videos whose preview clips are not yet generated.
- FR-011: The system MUST handle unprocessable videos gracefully by showing an error state rather than failing the entire wall.
- FR-012: The VideoWall MUST support scrolling through large numbers of videos, loading previews progressively.
- FR-013: Preview clips outside the visible viewport SHOULD pause playback to conserve device resources.
Key Entities
- Video Preview Clip: A short looping MP4 video (at most 10 seconds) scaled to 480p resolution, derived from a source video. Composed of up to 10 equally spaced 1-second segments. Associated with exactly one source video by file path. Has a generation status (pending, processing, complete, failed).
- VideoWall View: A scrollable grid layout displaying video preview clips. Adapts column count based on screen size (2 or 3 columns). Operates on a set of videos from a folder or search result context.
Success Criteria (mandatory)
Measurable Outcomes
- SC-001: Users can visually browse all videos in a folder within 3 seconds of opening VideoWall (for folders with up to 50 videos with pre-generated previews).
- SC-002: Preview clips accurately represent the source video by sampling from evenly distributed points across the full duration.
- SC-003: All visible previews play simultaneously without noticeable stuttering on standard mobile devices.
- SC-004: Generated preview clips are each under 5 MB in size to keep storage and bandwidth manageable.
- SC-005: The VideoWall view correctly filters to show only videos (not photos) from the current folder or search results.
- SC-006: Users can identify and select a video of interest from the VideoWall and navigate to it in a single tap.
- SC-007: Preview generation for a single video completes within 30 seconds on typical hardware.
Assumptions
- The existing file watcher and thumbnail generation infrastructure will be extended to also trigger preview clip generation.
- Preview clips will be stored alongside existing thumbnails/GIFs in a designated directory on the server.
- The React Native mobile app already has folder navigation and search/filter capabilities that provide the video list context for VideoWall.
- The server already has ffmpeg available for video processing (used for existing HLS and GIF generation).
- Authentication and authorization follow the existing JWT-based pattern; no new auth requirements.
- "2/3 columns" means a responsive layout: 2 columns on phones (portrait), 3 columns on tablets or landscape orientation.
- Preview clips are generated as MP4 video files for optimal quality-to-size ratio and hardware-accelerated mobile playback.