clip-search: backlog drain + /photos/search endpoint

Wires the persistence layer for CLIP semantic search. The watcher's
per-tick drain encodes any image_exif row with a known content_hash
but no clip_embedding via Apollo (cap CLIP_BACKLOG_MAX_PER_TICK,
default 32). On a query, /photos/search encodes the text via Apollo
and reranks every stored embedding in-memory.

ExifDao additions:
- list_clip_unencoded_candidates — partial-index scan for drain
- backfill_clip_embedding — touches only the two new columns
- list_clip_index — dedup'd (hash, embedding) pull for search

clip_watch::run_clip_encoding_pass is the parallel fan-out — tokio
runtime per pass with CLIP_ENCODE_CONCURRENCY (default 4). No marker
rows for permanent failures yet; per-tick cap bounds the retry cost.

/photos/search params: q, limit, threshold (default 0.20), library,
model_version. Response is intentionally minimal (path + score) so
the frontend joins against existing photo-metadata routes lazily.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Cameron Cordes
2026-05-14 14:00:41 -04:00
parent 8d9e76cf15
commit 32195ed89e
9 changed files with 875 additions and 0 deletions

View File

@@ -1,4 +1,5 @@
use crate::ai::apollo_client::ApolloClient;
use crate::ai::clip_client::ClipClient;
use crate::ai::face_client::FaceClient;
use crate::ai::insight_chat::{ChatLockMap, InsightChatService};
use crate::ai::openrouter::OpenRouterClient;
@@ -70,6 +71,10 @@ pub struct AppState {
/// nor `APOLLO_API_BASE_URL` is set; the file-watch hook (Phase 3) and
/// manual-face-create handler short-circuit in that case.
pub face_client: FaceClient,
/// CLIP inference client (calls Apollo's `/api/internal/clip/*`).
/// Same disabled semantics as `face_client`: unset env → no-op
/// backlog drain, /photos/search returns an empty result.
pub clip_client: ClipClient,
}
impl AppState {
@@ -105,6 +110,7 @@ impl AppState {
insight_chat: Arc<InsightChatService>,
preview_dao: Arc<Mutex<Box<dyn PreviewDao>>>,
face_client: FaceClient,
clip_client: ClipClient,
) -> Self {
assert!(
!libraries_vec.is_empty(),
@@ -143,6 +149,7 @@ impl AppState {
insight_generator,
insight_chat,
face_client,
clip_client,
}
}
@@ -198,6 +205,9 @@ impl Default for AppState {
.or_else(|| env::var("APOLLO_API_BASE_URL").ok());
let face_client = FaceClient::new(face_client_url);
// CLIP inference client. Same env var fallback as face_client.
let clip_client = ClipClient::from_env();
// Initialize DAOs
let insight_dao: Arc<Mutex<Box<dyn InsightDao>>> =
Arc::new(Mutex::new(Box::new(SqliteInsightDao::new())));
@@ -289,6 +299,7 @@ impl Default for AppState {
insight_chat,
preview_dao,
face_client,
clip_client,
)
}
}