From 19c099360e3e866a7b035909569a273fce9006c7 Mon Sep 17 00:00:00 2001 From: Cameron Date: Wed, 25 Feb 2026 19:40:17 -0500 Subject: [PATCH 1/9] Add VideoWall feature: server-side preview clip generation and mobile grid view Backend (Rust/Actix-web): - Add video_preview_clips table and PreviewDao for tracking preview generation - Add ffmpeg preview clip generator: 10 equally-spaced 1s segments at 480p with CUDA NVENC auto-detection - Add PreviewClipGenerator actor with semaphore-limited concurrent processing - Add GET /video/preview and POST /video/preview/status endpoints - Extend file watcher to detect and queue previews for new videos - Use relative paths consistently for DB storage (matching EXIF convention) Frontend (React Native/Expo): - Add VideoWall grid view with 2-3 column layout of looping preview clips - Add VideoWallItem component with ActiveVideoPlayer sub-component for lifecycle management - Add useVideoWall hook for batch status polling with 5s refresh - Add navigation button in grid header (visible when videos exist) - Use TextureView surface type to fix Android z-ordering issues - Optimize memory: players only mount while visible via FlatList windowSize - Configure ExoPlayer buffer options and caching for short clips - Tap to toggle audio focus, long press to open in full viewer Co-Authored-By: Claude Opus 4.6 --- .../down.sql | 1 + .../up.sql | 13 + .../001-video-wall/checklists/requirements.md | 36 +++ .../001-video-wall/contracts/api-endpoints.md | 91 ++++++ specs/001-video-wall/data-model.md | 62 ++++ specs/001-video-wall/plan.md | 79 +++++ specs/001-video-wall/quickstart.md | 115 +++++++ specs/001-video-wall/research.md | 91 ++++++ specs/001-video-wall/spec.md | 136 ++++++++ specs/001-video-wall/tasks.md | 234 ++++++++++++++ src/data/mod.rs | 23 ++ src/database/mod.rs | 2 + src/database/models.rs | 23 +- src/database/preview_dao.rs | 183 +++++++++++ src/database/schema.rs | 14 + src/main.rs | 302 +++++++++++++++++- src/state.rs | 35 +- src/video/actors.rs | 119 ++++++- src/video/ffmpeg.rs | 144 ++++++++- 19 files changed, 1691 insertions(+), 12 deletions(-) create mode 100644 migrations/2026-02-25-221921-0000_create_video_preview_clips/down.sql create mode 100644 migrations/2026-02-25-221921-0000_create_video_preview_clips/up.sql create mode 100644 specs/001-video-wall/checklists/requirements.md create mode 100644 specs/001-video-wall/contracts/api-endpoints.md create mode 100644 specs/001-video-wall/data-model.md create mode 100644 specs/001-video-wall/plan.md create mode 100644 specs/001-video-wall/quickstart.md create mode 100644 specs/001-video-wall/research.md create mode 100644 specs/001-video-wall/spec.md create mode 100644 specs/001-video-wall/tasks.md create mode 100644 src/database/preview_dao.rs diff --git a/migrations/2026-02-25-221921-0000_create_video_preview_clips/down.sql b/migrations/2026-02-25-221921-0000_create_video_preview_clips/down.sql new file mode 100644 index 0000000..96ad0d3 --- /dev/null +++ b/migrations/2026-02-25-221921-0000_create_video_preview_clips/down.sql @@ -0,0 +1 @@ +DROP TABLE IF EXISTS video_preview_clips; diff --git a/migrations/2026-02-25-221921-0000_create_video_preview_clips/up.sql b/migrations/2026-02-25-221921-0000_create_video_preview_clips/up.sql new file mode 100644 index 0000000..ea8a9ee --- /dev/null +++ b/migrations/2026-02-25-221921-0000_create_video_preview_clips/up.sql @@ -0,0 +1,13 @@ +CREATE TABLE video_preview_clips ( + id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + file_path TEXT NOT NULL UNIQUE, + status TEXT NOT NULL DEFAULT 'pending', + duration_seconds REAL, + file_size_bytes INTEGER, + error_message TEXT, + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL +); + +CREATE INDEX idx_preview_clips_file_path ON video_preview_clips(file_path); +CREATE INDEX idx_preview_clips_status ON video_preview_clips(status); diff --git a/specs/001-video-wall/checklists/requirements.md b/specs/001-video-wall/checklists/requirements.md new file mode 100644 index 0000000..297c253 --- /dev/null +++ b/specs/001-video-wall/checklists/requirements.md @@ -0,0 +1,36 @@ +# Specification Quality Checklist: VideoWall + +**Purpose**: Validate specification completeness and quality before proceeding to planning +**Created**: 2026-02-25 +**Feature**: [spec.md](../spec.md) + +## Content Quality + +- [x] No implementation details (languages, frameworks, APIs) +- [x] Focused on user value and business needs +- [x] Written for non-technical stakeholders +- [x] All mandatory sections completed + +## Requirement Completeness + +- [x] No [NEEDS CLARIFICATION] markers remain +- [x] Requirements are testable and unambiguous +- [x] Success criteria are measurable +- [x] Success criteria are technology-agnostic (no implementation details) +- [x] All acceptance scenarios are defined +- [x] Edge cases are identified +- [x] Scope is clearly bounded +- [x] Dependencies and assumptions identified + +## Feature Readiness + +- [x] All functional requirements have clear acceptance criteria +- [x] User scenarios cover primary flows +- [x] Feature meets measurable outcomes defined in Success Criteria +- [x] No implementation details leak into specification + +## Notes + +- All items pass validation. +- Assumptions section documents reasonable defaults for format choice, column layout interpretation, and infrastructure reuse. +- No [NEEDS CLARIFICATION] markers were needed — the user description was specific enough to make informed decisions for all requirements. diff --git a/specs/001-video-wall/contracts/api-endpoints.md b/specs/001-video-wall/contracts/api-endpoints.md new file mode 100644 index 0000000..9cfd8e1 --- /dev/null +++ b/specs/001-video-wall/contracts/api-endpoints.md @@ -0,0 +1,91 @@ +# API Contracts: VideoWall + +## GET /video/preview + +Retrieve the preview clip MP4 file for a given video. If the preview is not yet generated, triggers on-demand generation and returns 202. + +**Authentication**: Required (Bearer token) + +**Query Parameters**: + +| Parameter | Type | Required | Description | +|-----------|------|----------|-------------| +| path | string | yes | Relative path of the source video from BASE_PATH | + +**Responses**: + +| Status | Content-Type | Body | Description | +|--------|-------------|------|-------------| +| 200 | video/mp4 | MP4 file stream | Preview clip is ready and served | +| 202 | application/json | `{"status": "processing", "path": ""}` | Preview generation has been triggered; client should retry | +| 400 | application/json | `{"error": "Invalid path"}` | Path validation failed | +| 404 | application/json | `{"error": "Video not found"}` | Source video does not exist | +| 500 | application/json | `{"error": "Generation failed: "}` | Preview generation failed | + +**Behavior**: +1. Validate path with `is_valid_full_path()` +2. Check if preview clip exists on disk and status is `complete` → serve MP4 (200) +3. If status is `pending` or no record exists → trigger generation, return 202 +4. If status is `processing` → return 202 +5. If status is `failed` → return 500 with error detail + +--- + +## POST /video/preview/status + +Check the preview generation status for a batch of video paths. Used by the mobile app to determine which previews are ready before requesting them. + +**Authentication**: Required (Bearer token) + +**Request Body** (application/json): + +```json +{ + "paths": [ + "2024/vacation/beach.mov", + "2024/vacation/sunset.mp4", + "2024/birthday.avi" + ] +} +``` + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| paths | string[] | yes | Array of relative video paths from BASE_PATH | + +**Response** (200, application/json): + +```json +{ + "previews": [ + { + "path": "2024/vacation/beach.mov", + "status": "complete", + "preview_url": "/video/preview?path=2024/vacation/beach.mov" + }, + { + "path": "2024/vacation/sunset.mp4", + "status": "processing", + "preview_url": null + }, + { + "path": "2024/birthday.avi", + "status": "pending", + "preview_url": null + } + ] +} +``` + +| Field | Type | Description | +|-------|------|-------------| +| previews | object[] | Status for each requested path | +| previews[].path | string | The requested video path | +| previews[].status | string | One of: `pending`, `processing`, `complete`, `failed`, `not_found` | +| previews[].preview_url | string? | Relative URL to fetch the preview (only when status is `complete`) | + +**Behavior**: +1. Accept up to 200 paths per request +2. Batch query the `video_preview_clips` table for all paths +3. For paths not in the table, return status `not_found` (video may not exist or hasn't been scanned yet) +4. Return results in the same order as the input paths diff --git a/specs/001-video-wall/data-model.md b/specs/001-video-wall/data-model.md new file mode 100644 index 0000000..b72fcf2 --- /dev/null +++ b/specs/001-video-wall/data-model.md @@ -0,0 +1,62 @@ +# Data Model: VideoWall + +## Entities + +### VideoPreviewClip + +Tracks the generation status and metadata of preview clips derived from source videos. + +**Table**: `video_preview_clips` + +| Field | Type | Constraints | Description | +|-------|------|-------------|-------------| +| id | INTEGER | PRIMARY KEY, AUTOINCREMENT | Unique identifier | +| file_path | TEXT | NOT NULL, UNIQUE | Relative path of the source video from BASE_PATH | +| status | TEXT | NOT NULL, DEFAULT 'pending' | Generation status: `pending`, `processing`, `complete`, `failed` | +| duration_seconds | REAL | NULLABLE | Duration of the generated preview clip (≤10s) | +| file_size_bytes | INTEGER | NULLABLE | Size of the generated MP4 file | +| error_message | TEXT | NULLABLE | Error details if status is `failed` | +| created_at | TEXT | NOT NULL | ISO 8601 timestamp when record was created | +| updated_at | TEXT | NOT NULL | ISO 8601 timestamp when record was last updated | + +**Indexes**: +- `idx_preview_clips_file_path` on `file_path` (unique, used for lookups and batch queries) +- `idx_preview_clips_status` on `status` (used by file watcher to find pending/failed clips) + +### Relationships + +- **VideoPreviewClip → Source Video**: One-to-one via `file_path`. The preview clip file on disk is located at `{PREVIEW_CLIPS_DIRECTORY}/{file_path}.mp4`. +- **VideoPreviewClip → image_exif**: Implicit relationship via shared `file_path`. No foreign key needed — the EXIF table may not have an entry for every video. + +## State Transitions + +``` +[new video detected] → pending +pending → processing (when generation starts) +processing → complete (when ffmpeg succeeds) +processing → failed (when ffmpeg fails or times out) +failed → pending (on retry / re-scan) +``` + +## Validation Rules + +- `file_path` must be a valid relative path within BASE_PATH +- `status` must be one of: `pending`, `processing`, `complete`, `failed` +- `duration_seconds` must be > 0 and ≤ 10.0 when status is `complete` +- `file_size_bytes` must be > 0 when status is `complete` +- `error_message` should only be non-null when status is `failed` + +## Storage Layout (Filesystem) + +``` +{PREVIEW_CLIPS_DIRECTORY}/ +├── 2024/ +│ ├── vacation/ +│ │ ├── beach.mp4 # Preview for BASE_PATH/2024/vacation/beach.mov +│ │ └── sunset.mp4 # Preview for BASE_PATH/2024/vacation/sunset.mp4 +│ └── birthday.mp4 # Preview for BASE_PATH/2024/birthday.avi +└── 2025/ + └── trip.mp4 # Preview for BASE_PATH/2025/trip.mkv +``` + +All preview clips use `.mp4` extension regardless of source format. diff --git a/specs/001-video-wall/plan.md b/specs/001-video-wall/plan.md new file mode 100644 index 0000000..914b511 --- /dev/null +++ b/specs/001-video-wall/plan.md @@ -0,0 +1,79 @@ +# Implementation Plan: VideoWall + +**Branch**: `001-video-wall` | **Date**: 2026-02-25 | **Spec**: [spec.md](./spec.md) +**Input**: Feature specification from `/specs/001-video-wall/spec.md` + +## Summary + +Add a VideoWall feature spanning the Rust API backend and React Native mobile app. The backend generates 480p MP4 preview clips (up to 10 seconds, composed of 10 equally spaced 1-second segments) using ffmpeg, extending the existing `OverviewVideo` pattern in `src/video/ffmpeg.rs`. The mobile app adds a VideoWall view using `expo-video` and FlatList to display a responsive 2-3 column grid of simultaneously looping, muted preview clips with audio-on-long-press. Preview clips are cached on disk, served via new API endpoints, and generated proactively by the file watcher. + +## Technical Context + +**Language/Version**: Rust (stable, Cargo) for backend API; TypeScript / React Native (Expo SDK 52) for mobile app +**Primary Dependencies**: actix-web 4, Diesel 2.2 (SQLite), ffmpeg/ffprobe (CLI), expo-video 3.0, expo-router 6.0, react-native-reanimated 4.1 +**Storage**: SQLite (preview clip status tracking), filesystem (MP4 preview clips in `PREVIEW_CLIPS_DIRECTORY`) +**Testing**: `cargo test` for backend; manual testing for mobile app +**Target Platform**: Linux server (API), iOS/Android (mobile app via Expo) +**Project Type**: Mobile app + REST API (two separate repositories) +**Performance Goals**: <3s VideoWall load for 50 pre-generated previews; <30s per clip generation; <5MB per clip; smooth simultaneous playback of 6-12 clips +**Constraints**: Semaphore-limited concurrent ffmpeg processes (existing pattern); 480p resolution to keep bandwidth/CPU manageable; audio track preserved but muted by default +**Scale/Scope**: Hundreds to low thousands of videos per library; single user at a time + +## Constitution Check + +*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.* + +Constitution is an unfilled template — no project-specific gates defined. **PASS** (no violations possible). + +Post-Phase 1 re-check: Still PASS — no gates to evaluate. + +## Project Structure + +### Documentation (this feature) + +```text +specs/001-video-wall/ +├── plan.md # This file +├── research.md # Phase 0 output +├── data-model.md # Phase 1 output +├── quickstart.md # Phase 1 output +├── contracts/ # Phase 1 output +│ └── api-endpoints.md +└── tasks.md # Phase 2 output (/speckit.tasks command) +``` + +### Source Code (repository root) + +```text +# Backend (ImageApi - Rust) +src/ +├── video/ +│ ├── ffmpeg.rs # Add generate_preview_clip() using existing pattern +│ ├── actors.rs # Add PreviewClipGenerator actor (semaphore-limited) +│ └── mod.rs # Add generate_preview_clips() batch function +├── main.rs # Add GET /video/preview, POST /video/preview/status endpoints +│ # Extend file watcher to trigger preview generation +├── database/ +│ ├── schema.rs # Add video_preview_clips table +│ └── models.rs # Add VideoPreviewClip model +│ └── preview_dao.rs # New DAO for preview clip status tracking +└── data/ + └── mod.rs # Add PreviewClipRequest, PreviewStatusRequest types + +# Frontend (SynologyFileViewer - React Native) +app/(app)/grid/ +├── video-wall.tsx # New VideoWall view (FlatList grid) +└── _layout.tsx # Add video-wall route to stack + +components/ +└── VideoWallItem.tsx # Single preview clip cell (expo-video player) + +hooks/ +└── useVideoWall.ts # Preview clip fetching, status polling, audio state +``` + +**Structure Decision**: Mobile + API pattern. Backend changes extend existing `src/video/` module and `src/main.rs` handlers following established conventions. Frontend adds a new route under the existing grid stack navigator with a dedicated component and hook. + +## Complexity Tracking + +No constitution violations to justify. diff --git a/specs/001-video-wall/quickstart.md b/specs/001-video-wall/quickstart.md new file mode 100644 index 0000000..84ddfc6 --- /dev/null +++ b/specs/001-video-wall/quickstart.md @@ -0,0 +1,115 @@ +# Quickstart: VideoWall + +## Prerequisites + +- Rust toolchain (stable) with `cargo` +- `diesel_cli` installed (`cargo install diesel_cli --no-default-features --features sqlite`) +- ffmpeg and ffprobe available on PATH +- Node.js 18+ and Expo CLI for mobile app +- `.env` file configured with existing variables plus `PREVIEW_CLIPS_DIRECTORY` + +## New Environment Variable + +Add to `.env`: +```bash +PREVIEW_CLIPS_DIRECTORY=/path/to/preview-clips # Directory for generated preview MP4s +``` + +## Backend Development + +### 1. Create database migration + +```bash +cd C:\Users\ccord\RustroverProjects\ImageApi +diesel migration generate create_video_preview_clips +``` + +Edit the generated `up.sql`: +```sql +CREATE TABLE video_preview_clips ( + id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + file_path TEXT NOT NULL UNIQUE, + status TEXT NOT NULL DEFAULT 'pending', + duration_seconds REAL, + file_size_bytes INTEGER, + error_message TEXT, + created_at TEXT NOT NULL, + updated_at TEXT NOT NULL +); + +CREATE INDEX idx_preview_clips_file_path ON video_preview_clips(file_path); +CREATE INDEX idx_preview_clips_status ON video_preview_clips(status); +``` + +Edit `down.sql`: +```sql +DROP TABLE IF EXISTS video_preview_clips; +``` + +Regenerate schema: +```bash +diesel migration run +diesel print-schema > src/database/schema.rs +``` + +### 2. Build and test backend + +```bash +cargo build +cargo test +cargo run +``` + +Test preview endpoint: +```bash +# Check preview status +curl -X POST http://localhost:8080/video/preview/status \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{"paths": ["some/video.mp4"]}' + +# Request preview clip +curl http://localhost:8080/video/preview?path=some/video.mp4 \ + -H "Authorization: Bearer " \ + -o preview.mp4 +``` + +### 3. Verify preview clip generation + +Check that preview clips appear in `PREVIEW_CLIPS_DIRECTORY` with the expected directory structure mirroring `BASE_PATH`. + +## Frontend Development + +### 1. Start the mobile app + +```bash +cd C:\Users\ccord\development\SynologyFileViewer +npx expo start +``` + +### 2. Navigate to VideoWall + +From the grid view of any folder containing videos, switch to VideoWall mode. The view should display a 2-3 column grid of looping preview clips. + +## Key Files to Modify + +### Backend (ImageApi) +| File | Change | +|------|--------| +| `src/video/ffmpeg.rs` | Add `generate_preview_clip()` function | +| `src/video/actors.rs` | Add `PreviewClipGenerator` actor | +| `src/video/mod.rs` | Add `generate_preview_clips()` batch function | +| `src/main.rs` | Add endpoints, extend file watcher | +| `src/database/schema.rs` | Regenerated by Diesel | +| `src/database/models.rs` | Add `VideoPreviewClip` struct | +| `src/database/preview_dao.rs` | New DAO file | +| `src/data/mod.rs` | Add request/response types | +| `src/state.rs` | Add PreviewClipGenerator to AppState | + +### Frontend (SynologyFileViewer) +| File | Change | +|------|--------| +| `app/(app)/grid/video-wall.tsx` | New VideoWall view | +| `app/(app)/grid/_layout.tsx` | Add route | +| `components/VideoWallItem.tsx` | New preview clip cell component | +| `hooks/useVideoWall.ts` | New hook for preview state management | diff --git a/specs/001-video-wall/research.md b/specs/001-video-wall/research.md new file mode 100644 index 0000000..f09c340 --- /dev/null +++ b/specs/001-video-wall/research.md @@ -0,0 +1,91 @@ +# Research: VideoWall + +## R1: FFmpeg Preview Clip Generation Strategy + +**Decision**: Use ffmpeg's `select` filter with segment-based extraction, extending the existing `OverviewVideo` pattern in `src/video/ffmpeg.rs`. + +**Rationale**: The codebase already has a nearly identical pattern at `src/video/ffmpeg.rs` using `select='lt(mod(t,{interval}),1)'` which selects 1-second frames at evenly spaced intervals across the video duration. The existing pattern outputs GIF; we adapt it to output MP4 at 480p with audio. + +**Approach**: +1. Use `ffprobe` to get video duration (existing `get_video_duration()` pattern) +2. Calculate interval: `duration / 10` (or fewer segments for short videos) +3. Use ffmpeg with: + - Video filter: `select='lt(mod(t,{interval}),1)',setpts=N/FRAME_RATE/TB,scale=-2:480` + - Audio filter: `aselect='lt(mod(t,{interval}),1)',asetpts=N/SR/TB` + - Output: MP4 with H.264 video + AAC audio + - CRF 28 (lower quality acceptable for previews, reduces file size) + - Preset: `veryfast` (matches existing HLS transcoding pattern) + +**Alternatives considered**: +- Generating separate segment files and concatenating: More complex, no benefit over select filter +- Using GIF output: Rejected per clarification — MP4 is 5-10x smaller with better quality +- Stream copy (no transcode): Not possible since we're extracting non-contiguous segments + +## R2: Preview Clip Storage and Caching + +**Decision**: Store preview clips on filesystem in a dedicated `PREVIEW_CLIPS_DIRECTORY` mirroring the source directory structure (same pattern as `THUMBNAILS` and `GIFS_DIRECTORY`). + +**Rationale**: The project already uses this directory-mirroring pattern for thumbnails and GIF previews. It's simple, requires no database for file lookup (path is deterministic), and integrates naturally with the existing file watcher cleanup logic. + +**Storage path formula**: `{PREVIEW_CLIPS_DIRECTORY}/{relative_path_from_BASE_PATH}.mp4` +- Example: Video at `BASE_PATH/2024/vacation.mov` → Preview at `PREVIEW_CLIPS_DIRECTORY/2024/vacation.mp4` + +**Alternatives considered**: +- Database BLOBs: Too large, not suited for binary video files +- Content-addressed storage (hash-based): Unnecessary complexity for single-user system +- Flat directory with UUID names: Loses the intuitive mapping that thumbnails/GIFs use + +## R3: Preview Generation Status Tracking + +**Decision**: Track generation status in SQLite via a new `video_preview_clips` table with Diesel ORM, following the existing DAO pattern. + +**Rationale**: The batch status endpoint (FR-004) needs to efficiently check which previews are ready for a list of video paths. A database table is the right tool — it supports batch queries (existing `get_exif_batch()` pattern), survives restarts, and tracks failure states. The file watcher already uses batch DB queries to detect unprocessed files. + +**Status values**: `pending`, `processing`, `complete`, `failed` + +**Alternatives considered**: +- Filesystem-only (check if .mp4 exists): Cannot track `processing` or `failed` states; race conditions on concurrent requests +- In-memory HashMap: Lost on restart; doesn't support batch queries efficiently across actor boundaries + +## R4: Concurrent Generation Limits + +**Decision**: Use `Arc` with a limit of 2 concurrent ffmpeg preview generation processes, matching the existing `PlaylistGenerator` pattern. + +**Rationale**: The `PlaylistGenerator` actor in `src/video/actors.rs` already uses this exact pattern to limit concurrent ffmpeg processes. Preview generation is CPU-intensive (transcoding), so limiting concurrency prevents server overload. The semaphore pattern is proven in this codebase. + +**Alternatives considered**: +- Unbounded concurrency: Would overwhelm the server with many simultaneous ffmpeg processes +- Queue with single worker: Too slow for batch generation; 2 concurrent is a good balance +- Sharing the existing PlaylistGenerator semaphore: Would cause HLS generation and preview generation to compete for the same slots; better to keep them independent + +## R5: Mobile App Video Playback Strategy + +**Decision**: Use `expo-video` `VideoView` components inside FlatList items, with muted autoplay and viewport-based pause/resume. + +**Rationale**: The app already uses `expo-video` (v3.0.15) for the single video player in `viewer/video.tsx`. The library supports multiple simultaneous players, `loop` mode, and programmatic mute/unmute. FlatList's `viewabilityConfig` callback can be used to pause/resume players based on viewport visibility. + +**Key configuration per cell**: +- `player.loop = true` +- `player.muted = true` (default) +- `player.play()` when visible, `player.pause()` when offscreen +- `nativeControls={false}` (no controls needed in grid) + +**Audio-on-focus**: On long-press, unmute the pressed player and mute all others. Track the "focused" player ID in hook state. + +**Alternatives considered**: +- HLS streaming for previews: Overkill for <10s clips; direct MP4 download is simpler and faster +- Animated GIF display via Image component: Rejected per clarification — MP4 with expo-video is better +- WebView-based player: Poor performance, no native gesture integration + +## R6: API Endpoint Design + +**Decision**: Two new endpoints — one to serve preview clips, one for batch status checking. + +**Rationale**: +- `GET /video/preview?path=...` serves the MP4 file directly (or triggers on-demand generation and returns 202 Accepted). Follows the pattern of `GET /image?path=...` for serving files. +- `POST /video/preview/status` accepts a JSON body with an array of video paths and returns their preview generation status. This allows the mobile app to efficiently determine which previews are ready in a single request (batch pattern from `get_exif_batch()`). + +**Alternatives considered**: +- Single endpoint that blocks until generation completes: Bad UX — generation takes up to 30s +- WebSocket for real-time status: Overkill for this use case; polling with batch status is simpler +- Including preview URL in the existing `/photos` response: Would couple the photo listing endpoint to preview generation; better to keep separate diff --git a/specs/001-video-wall/spec.md b/specs/001-video-wall/spec.md new file mode 100644 index 0000000..81c4c80 --- /dev/null +++ b/specs/001-video-wall/spec.md @@ -0,0 +1,136 @@ +# Feature Specification: VideoWall + +**Feature Branch**: `001-video-wall` +**Created**: 2026-02-25 +**Status**: Draft +**Input**: User description: "I would like to implement a new View 'VideoWall' in the React native mobile app, with supporting API/tasks to generate at most 10 second long GIF/Videos that are 10 equally spaced 1 second clips of the original video. This view will display a grid 2/3 columns wide of all these clips playing simultaneously. It should let the user view all videos in the current folder/search results." + +## Clarifications + +### Session 2026-02-25 + +- Q: What format should preview clips be generated in (GIF vs video)? → A: MP4 video clips (small files, hardware-accelerated playback, best quality-to-size ratio). +- Q: What resolution should preview clips be generated at? → A: 480p scaled down (sharp in grid cells, small files, smooth simultaneous playback). +- Q: How should audio be handled in preview clips? → A: Audio on focus — muted by default, audio plays when user long-presses on a clip. Audio track is preserved during generation. + +## User Scenarios & Testing *(mandatory)* + +### User Story 1 - Browse Videos as a Visual Wall (Priority: P1) + +A user navigates to a folder containing videos and switches to the VideoWall view. The screen fills with a grid of video previews — short looping clips that give a visual summary of each video. All previews play simultaneously, creating an immersive "wall of motion" that lets the user quickly scan and identify videos of interest without opening each one individually. + +**Why this priority**: This is the core experience. Without the visual grid of simultaneously playing previews, the feature has no value. This story delivers the primary browsing capability. + +**Independent Test**: Can be fully tested by navigating to any folder with videos, switching to VideoWall view, and confirming that preview clips display in a grid and play simultaneously. Delivers immediate visual browsing value. + +**Acceptance Scenarios**: + +1. **Given** a user is viewing a folder containing 6 videos, **When** they switch to VideoWall view, **Then** they see a grid of 6 video previews arranged in 2-3 columns, all playing simultaneously in a loop. +2. **Given** a user is viewing a folder containing 20 videos, **When** they switch to VideoWall view, **Then** the grid is scrollable and loads previews progressively as they scroll. +3. **Given** a user is in VideoWall view, **When** they tap on a video preview, **Then** they navigate to the full video player for that video. +4. **Given** a user is in VideoWall view with all clips muted, **When** they long-press on a preview clip, **Then** that clip's audio unmutes and all other clips remain muted. + +--- + +### User Story 2 - Server Generates Preview Clips (Priority: P1) + +When preview clips are requested for a video that has not yet been processed, the server generates a short preview clip. The preview is composed of 10 equally spaced 1-second segments extracted from the original video, concatenated into a single clip of at most 10 seconds. Once generated, the preview is cached so subsequent requests are served instantly. + +**Why this priority**: The VideoWall view depends entirely on having preview clips available. Without server-side generation, there is nothing to display. This is co-priority with Story 1 as they are interdependent. + +**Independent Test**: Can be tested by requesting a preview clip for any video via the API and confirming the response is a playable clip of at most 10 seconds composed of segments from different parts of the original video. + +**Acceptance Scenarios**: + +1. **Given** a video exists that has no preview clip yet, **When** a preview is requested, **Then** the system generates a clip of at most 10 seconds composed of 10 equally spaced 1-second segments from the original video. +2. **Given** a video is shorter than 10 seconds, **When** a preview is requested, **Then** the system generates a preview using fewer segments (as many 1-second clips as the video duration allows), resulting in a shorter preview. +3. **Given** a preview clip was previously generated for a video, **When** it is requested again, **Then** the cached version is served without re-processing. +4. **Given** a video file no longer exists, **When** a preview is requested, **Then** the system returns an appropriate error indicating the source video is missing. + +--- + +### User Story 3 - VideoWall from Search Results (Priority: P2) + +A user performs a search or applies filters (tags, date range, camera, location) and the results include videos. They switch to VideoWall view to see preview clips of all matching videos displayed in the same grid layout, allowing visual browsing of search results. + +**Why this priority**: Extends the core VideoWall browsing to work with filtered/search result sets. Important for discoverability but depends on Story 1 and 2 being functional first. + +**Independent Test**: Can be tested by performing a search that returns videos, switching to VideoWall view, and confirming that only matching videos appear as previews in the grid. + +**Acceptance Scenarios**: + +1. **Given** a user has search results containing 8 videos and 12 photos, **When** they switch to VideoWall view, **Then** only the 8 video previews are displayed in the grid. +2. **Given** a user applies a tag filter that matches 3 videos, **When** they view the VideoWall, **Then** exactly 3 video previews are shown. + +--- + +### User Story 4 - Background Preview Generation (Priority: P3) + +Preview clips are generated proactively in the background for videos discovered during file watching, so that when a user opens VideoWall, most previews are already available and the experience feels instant. + +**Why this priority**: Enhances performance and perceived responsiveness. The feature works without this (on-demand generation), but background processing greatly improves the user experience for large libraries. + +**Independent Test**: Can be tested by adding new video files to a monitored folder and confirming that preview clips are generated automatically within the next scan cycle, before any user requests them. + +**Acceptance Scenarios**: + +1. **Given** a new video is added to the media library, **When** the file watcher detects it, **Then** a preview clip is generated in the background without user intervention. +2. **Given** the system is generating previews in the background, **When** a user opens VideoWall, **Then** already-generated previews display immediately while pending ones show a placeholder. + +--- + +### Edge Cases + +- What happens when a video is corrupted or cannot be processed? The system shows a placeholder/error state for that video and does not block other previews from loading. +- What happens when the user scrolls quickly through a large library? Previews outside the visible viewport should pause or not load to conserve resources, and resume when scrolled back into view. +- What happens when a video is extremely long (e.g., 4+ hours)? The same algorithm applies — 10 equally spaced 1-second clips — ensuring the preview still represents the full video. +- What happens when a video is exactly 10 seconds long? Each 1-second segment starts at second 0, 1, 2, ... 9, effectively previewing the entire video. +- What happens when storage for preview clips runs low? Preview clips should be reasonably compressed and sized to minimize storage impact. +- What happens when many previews are requested simultaneously (e.g., opening a folder with 100 videos)? The system should queue generation and serve already-cached previews immediately while others are processed. + +## Requirements *(mandatory)* + +### Functional Requirements + +- **FR-001**: System MUST generate preview clips for videos as MP4 files scaled to 480p resolution, where each preview is composed of up to 10 equally spaced 1-second segments from the original video, resulting in a clip of at most 10 seconds. +- **FR-002**: System MUST cache generated preview clips so they are only generated once per source video. +- **FR-003**: System MUST provide an endpoint to retrieve a preview clip for a given video path. +- **FR-004**: System MUST provide an endpoint to retrieve preview availability status for a batch of video paths so the client knows which previews are ready. +- **FR-005**: The mobile app MUST display a VideoWall view showing video previews in a grid of 2 columns on smaller screens and 3 columns on larger screens. +- **FR-006**: All visible preview clips in the VideoWall MUST play simultaneously, muted, and loop continuously. +- **FR-006a**: When a user long-presses on a preview clip, the app MUST unmute that clip's audio. Only one clip may have audio at a time. +- **FR-006b**: Preview clips MUST retain their audio track during generation (not stripped) to support audio-on-focus playback. +- **FR-007**: The VideoWall MUST support browsing videos from both folder navigation and search/filter results. +- **FR-008**: Tapping a preview clip in the VideoWall MUST navigate the user to the full video. +- **FR-009**: For videos shorter than 10 seconds, the system MUST generate a preview using as many full 1-second segments as the video duration allows. +- **FR-010**: The system MUST display a placeholder for videos whose preview clips are not yet generated. +- **FR-011**: The system MUST handle unprocessable videos gracefully by showing an error state rather than failing the entire wall. +- **FR-012**: The VideoWall MUST support scrolling through large numbers of videos, loading previews progressively. +- **FR-013**: Preview clips outside the visible viewport SHOULD pause playback to conserve device resources. + +### Key Entities + +- **Video Preview Clip**: A short looping MP4 video (at most 10 seconds) scaled to 480p resolution, derived from a source video. Composed of up to 10 equally spaced 1-second segments. Associated with exactly one source video by file path. Has a generation status (pending, processing, complete, failed). +- **VideoWall View**: A scrollable grid layout displaying video preview clips. Adapts column count based on screen size (2 or 3 columns). Operates on a set of videos from a folder or search result context. + +## Success Criteria *(mandatory)* + +### Measurable Outcomes + +- **SC-001**: Users can visually browse all videos in a folder within 3 seconds of opening VideoWall (for folders with up to 50 videos with pre-generated previews). +- **SC-002**: Preview clips accurately represent the source video by sampling from evenly distributed points across the full duration. +- **SC-003**: All visible previews play simultaneously without noticeable stuttering on standard mobile devices. +- **SC-004**: Generated preview clips are each under 5 MB in size to keep storage and bandwidth manageable. +- **SC-005**: The VideoWall view correctly filters to show only videos (not photos) from the current folder or search results. +- **SC-006**: Users can identify and select a video of interest from the VideoWall and navigate to it in a single tap. +- **SC-007**: Preview generation for a single video completes within 30 seconds on typical hardware. + +## Assumptions + +- The existing file watcher and thumbnail generation infrastructure will be extended to also trigger preview clip generation. +- Preview clips will be stored alongside existing thumbnails/GIFs in a designated directory on the server. +- The React Native mobile app already has folder navigation and search/filter capabilities that provide the video list context for VideoWall. +- The server already has ffmpeg available for video processing (used for existing HLS and GIF generation). +- Authentication and authorization follow the existing JWT-based pattern; no new auth requirements. +- "2/3 columns" means a responsive layout: 2 columns on phones (portrait), 3 columns on tablets or landscape orientation. +- Preview clips are generated as MP4 video files for optimal quality-to-size ratio and hardware-accelerated mobile playback. diff --git a/specs/001-video-wall/tasks.md b/specs/001-video-wall/tasks.md new file mode 100644 index 0000000..a9fc4d0 --- /dev/null +++ b/specs/001-video-wall/tasks.md @@ -0,0 +1,234 @@ +# Tasks: VideoWall + +**Input**: Design documents from `/specs/001-video-wall/` +**Prerequisites**: plan.md (required), spec.md (required), research.md, data-model.md, contracts/ + +**Tests**: Not explicitly requested — test tasks omitted. + +**Organization**: Tasks grouped by user story. US2 (server generation) comes before US1 (mobile view) because the mobile app depends on the API endpoints existing. + +## Format: `[ID] [P?] [Story] Description` + +- **[P]**: Can run in parallel (different files, no dependencies) +- **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3) +- Include exact file paths in descriptions + +## Path Conventions + +- **Backend (ImageApi)**: `src/` at `C:\Users\ccord\RustroverProjects\ImageApi` +- **Frontend (SynologyFileViewer)**: `app/`, `components/`, `hooks/` at `C:\Users\ccord\development\SynologyFileViewer` + +--- + +## Phase 1: Setup (Shared Infrastructure) + +**Purpose**: Database migration, new environment variable, shared types + +- [x] T001 Create Diesel migration for `video_preview_clips` table: run `diesel migration generate create_video_preview_clips`, write `up.sql` with table definition (id, file_path UNIQUE, status DEFAULT 'pending', duration_seconds, file_size_bytes, error_message, created_at, updated_at) and indexes (idx_preview_clips_file_path, idx_preview_clips_status), write `down.sql` with DROP TABLE. See `data-model.md` for full schema. +- [x] T002 Run migration and regenerate schema: execute `diesel migration run` then `diesel print-schema > src/database/schema.rs` to add the `video_preview_clips` table to `src/database/schema.rs` +- [x] T003 Add `PREVIEW_CLIPS_DIRECTORY` environment variable: read it in `src/main.rs` startup (alongside existing `GIFS_DIRECTORY`), create the directory if it doesn't exist, and add it to `AppState` or pass it where needed. Follow the pattern used for `GIFS_DIRECTORY` and `THUMBNAILS`. + +--- + +## Phase 2: Foundational (Blocking Prerequisites) + +**Purpose**: Diesel model, DAO, and request/response types that all user stories depend on + +**CRITICAL**: No user story work can begin until this phase is complete + +- [x] T004 [P] Add `VideoPreviewClip` Diesel model struct in `src/database/models.rs` with fields matching the `video_preview_clips` schema table (Queryable, Insertable derives). Add a `NewVideoPreviewClip` struct for inserts. +- [x] T005 [P] Add `PreviewClipRequest` and `PreviewStatusRequest`/`PreviewStatusResponse` types in `src/data/mod.rs`. `PreviewClipRequest` has `path: String`. `PreviewStatusRequest` has `paths: Vec`. `PreviewStatusResponse` has `previews: Vec` where each item has `path`, `status`, `preview_url: Option`. All with Serialize/Deserialize derives. +- [x] T006 Create `PreviewDao` trait and `SqlitePreviewDao` implementation in `src/database/preview_dao.rs`. Methods: `insert_preview(file_path, status) -> Result`, `update_status(file_path, status, duration_seconds?, file_size_bytes?, error_message?) -> Result`, `get_preview(file_path) -> Result>`, `get_previews_batch(file_paths: &[String]) -> Result>`, `get_by_status(status) -> Result>`. Follow the `ExifDao`/`SqliteExifDao` pattern with `Arc>` and OpenTelemetry tracing spans. +- [x] T007 Register `preview_dao` module in `src/database/mod.rs` and add `PreviewDao` to the database module exports. Wire `SqlitePreviewDao` into `AppState` in `src/state.rs` following the existing DAO pattern (e.g., how `ExifDao` is added). + +**Checkpoint**: Foundation ready — DAO, models, and types available for all stories + +--- + +## Phase 3: User Story 2 - Server Generates Preview Clips (Priority: P1) MVP + +**Goal**: Backend can generate 480p MP4 preview clips (10 equally spaced 1-second segments) and serve them via API endpoints with on-demand generation and batch status checking. + +**Independent Test**: Request `GET /video/preview?path=