Add memory-reel backend: on-demand narrated photo slideshow

New POST /reels + GET /reels/{id} (+ /video) build an MP4 slideshow of a
memory span (day/week/month), narrated by the LLM in a cloned voice.

Pipeline (src/reels/): a selector resolves which photos + reel metadata,
the scripter writes one narration line per photo via a single LLM call
(reusing each photo's cached insight as context — no fresh vision calls,
so reel generation stays off the GPU's vision slot), each line is
synthesized to speech, and the renderer assembles stills + narration via
ffmpeg. Jobs run in the background (mirroring the TTS speech-job
registry) since a reel takes minutes; the finished MP4 is cached on disk
keyed by the selection so a repeat request is instant.

The segment model is media-typed (Photo today) so a video-clip segment
(phase 2) and a nightly pre-render (phase 3) slot in without reworking
the pipeline. Ken Burns motion is implemented but defaulted off pending a
visual check on the GPU box.

Supporting changes:
- memories: extract gather_memory_items() so the reel selector reuses the
  exact window/exclusion/tz/sort logic behind /memories.
- ai::tts: add synthesize_serialized() so reel narration honors the same
  single-GPU permit + write lease as user TTS requests.
- video::ffmpeg: make get_duration_seconds() pub for narration timing.
- AppState: reels_path (REELS_DIRECTORY, defaults beside preview clips).

Pure logic (cache key, script parsing, ffmpeg arg/filter construction,
even sampling, segment timing) is unit-tested (26 tests). The runtime
path (ffmpeg render, TTS, LLM) needs a real run on the GPU host to verify
end-to-end — not exercisable in CI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Fable 5 committed 2026-06-12 22:31:08 -04:00
1 parent 98274c3301
commit e3f731b3b2
9 files changed
+1615 -30

No files matched your search

+18
View File
@@ -53,6 +53,10 @@ pub struct AppState {
pub video_path: String,
pub gif_path: String,
pub preview_clips_path: String,
/// Directory for cached memory-reel MP4s (+ title sidecars). Derived from
/// `REELS_DIRECTORY`, defaulting to a `reels` dir beside the preview clips.
/// Created lazily by the reel pipeline on first render.
pub reels_path: String,
pub excluded_dirs: Vec<String>,
pub ollama: OllamaClient,
/// `None` when `OPENROUTER_API_KEY` is not configured. Consulted only
@@ -141,6 +145,19 @@ impl AppState {
preview_dao,
);
// Reels cache dir: explicit env, else a `reels` sibling of the preview
// clips dir (a known-writable, test-safe location). Not created here —
// the reel pipeline does `create_dir_all` before its first write, so
// construction (incl. tests) never touches the filesystem.
let reels_path = std::env::var("REELS_DIRECTORY").unwrap_or_else(|_| {
std::path::Path::new(&preview_clips_path)
.parent()
.map(|p| p.join("reels"))
.unwrap_or_else(|| std::path::PathBuf::from("reels"))
.to_string_lossy()
.to_string()
});
let library_health = libraries::new_health_map(&libraries_vec);
let live_libraries = Arc::new(RwLock::new(libraries_vec.clone()));
Self {
@@ -155,6 +172,7 @@ impl AppState {
video_path,
gif_path,
preview_clips_path,
reels_path,
excluded_dirs,
ollama,
openrouter,