clip-search: backlog drain + /photos/search endpoint

Wires the persistence layer for CLIP semantic search. The watcher's
per-tick drain encodes any image_exif row with a known content_hash
but no clip_embedding via Apollo (cap CLIP_BACKLOG_MAX_PER_TICK,
default 32). On a query, /photos/search encodes the text via Apollo
and reranks every stored embedding in-memory.

ExifDao additions:
- list_clip_unencoded_candidates — partial-index scan for drain
- backfill_clip_embedding — touches only the two new columns
- list_clip_index — dedup'd (hash, embedding) pull for search

clip_watch::run_clip_encoding_pass is the parallel fan-out — tokio
runtime per pass with CLIP_ENCODE_CONCURRENCY (default 4). No marker
rows for permanent failures yet; per-tick cap bounds the retry cost.

/photos/search params: q, limit, threshold (default 0.20), library,
model_version. Response is intentionally minimal (path + score) so
the frontend joins against existing photo-metadata routes lazily.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Cameron Cordes
2026-05-14 14:00:41 -04:00
parent 8d9e76cf15
commit 32195ed89e
9 changed files with 875 additions and 0 deletions
+9
View File
@@ -31,6 +31,8 @@ use log::{error, info};
mod ai;
mod auth;
mod backfill;
mod clip_search;
mod clip_watch;
mod content_hash;
mod data;
mod database;
@@ -164,6 +166,7 @@ fn main() -> std::io::Result<()> {
playlist_mgr_for_watcher,
preview_gen_for_watcher,
app_state.face_client.clone(),
app_state.clip_client.clone(),
app_state.excluded_dirs.clone(),
app_state.library_health.clone(),
);
@@ -280,6 +283,12 @@ fn main() -> std::io::Result<()> {
.service(
web::resource("/photos/exif").route(web::get().to(files::list_exif_summary)),
)
.service(
// Semantic search via CLIP embeddings. See
// src/clip_search.rs for the request/response shape.
web::resource("/photos/search")
.route(web::get().to(clip_search::search_photos)),
)
.service(web::resource("/file/move").post(move_file::<RealFileSystem>))
.service(handlers::image::get_image)
.service(handlers::image::upload_image)