clip-search: backlog drain + /photos/search endpoint

Wires the persistence layer for CLIP semantic search. The watcher's
per-tick drain encodes any image_exif row with a known content_hash
but no clip_embedding via Apollo (cap CLIP_BACKLOG_MAX_PER_TICK,
default 32). On a query, /photos/search encodes the text via Apollo
and reranks every stored embedding in-memory.

ExifDao additions:
- list_clip_unencoded_candidates — partial-index scan for drain
- backfill_clip_embedding — touches only the two new columns
- list_clip_index — dedup'd (hash, embedding) pull for search

clip_watch::run_clip_encoding_pass is the parallel fan-out — tokio
runtime per pass with CLIP_ENCODE_CONCURRENCY (default 4). No marker
rows for permanent failures yet; per-tick cap bounds the retry cost.

/photos/search params: q, limit, threshold (default 0.20), library,
model_version. Response is intentionally minimal (path + score) so
the frontend joins against existing photo-metadata routes lazily.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Cameron Cordes
2026-05-14 14:00:41 -04:00
parent 8d9e76cf15
commit 32195ed89e
9 changed files with 875 additions and 0 deletions

View File

@@ -268,6 +268,7 @@ pub fn watch_files(
playlist_manager: Addr<VideoPlaylistManager>,
preview_generator: Addr<video::actors::PreviewClipGenerator>,
face_client: crate::ai::face_client::FaceClient,
clip_client: crate::ai::clip_client::ClipClient,
excluded_dirs: Vec<String>,
library_health: libraries::LibraryHealthMap,
) {
@@ -300,6 +301,14 @@ pub fn watch_files(
or APOLLO_API_BASE_URL to enable)"
);
}
if clip_client.is_enabled() {
info!(" CLIP semantic search: ENABLED");
} else {
info!(
" CLIP semantic search: DISABLED (set APOLLO_CLIP_API_BASE_URL \
or APOLLO_API_BASE_URL to enable)"
);
}
{
let libs = libs_lock.read().unwrap_or_else(|e| e.into_inner());
for lib in libs.iter() {
@@ -463,6 +472,21 @@ pub fn watch_files(
);
}
// CLIP embedding backlog. Independent of face detection —
// drain runs whenever CLIP is enabled, even on deploys
// that don't have the face engine wired up. Mirrors the
// face drain shape (capped per tick, no-op when disabled).
if clip_client.is_enabled() {
let context = opentelemetry::Context::new();
backfill::process_clip_backlog(
&context,
lib,
&clip_client,
&exif_dao,
&effective_excludes,
);
}
// Date-taken backfill: drain rows whose canonical date is
// either unresolved or only fs_time-sourced. Independent
// of face detection — runs even on deploys that don't