clip-search: migration + client + probe binary

Probe-phase scaffolding for CLIP semantic search. Adds the column
that will hold per-photo embeddings, the HTTP client to Apollo's
inference service, and a throwaway probe binary so we can eyeball
search-result quality on the live library before building the
persistence layer (backlog drain, /photos/search endpoint, UI).

- migrations/2026-05-14-000000_add_clip_embedding/ — adds
  image_exif.clip_embedding (BLOB) and clip_model_version (TEXT),
  plus a partial index on (clip_embedding IS NULL AND content_hash
  IS NOT NULL) for the future backfill drain.
- src/database/models.rs — extends ImageExif struct to match.
- src/ai/clip_client.rs — encode_image / encode_text / health,
  same Permanent/Transient/Disabled taxonomy as face_client.
- src/bin/probe_clip_search.rs — --query <q> --library N --limit M
  --top K. Encodes a sample and prints top-K cosine similarities.
  No DB writes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Cameron Cordes
2026-05-14 12:54:07 -04:00
parent 26ffc15c8b
commit 8d9e76cf15
9 changed files with 711 additions and 0 deletions

View File

@@ -114,6 +114,15 @@ pub struct ImageExif {
/// Snapshot of the prior `date_taken_source` taken on first manual
/// override. NULL when no override is active.
pub original_date_taken_source: Option<String>,
/// L2-normalized CLIP image embedding (raw little-endian float32 bytes;
/// length depends on the model — 768×4 for ViT-L/14, 512×4 for ViT-B/32).
/// NULL until Apollo's CLIP service has encoded this photo via the
/// backfill drain. Used by `/photos/search` for semantic queries.
pub clip_embedding: Option<Vec<u8>>,
/// Which CLIP model produced `clip_embedding` (e.g. `"ViT-L/14"`). A
/// swap of `APOLLO_CLIP_MODEL` re-eligibilizes rows whose stored
/// version differs so the drain rebuilds them.
pub clip_model_version: Option<String>,
}
#[derive(Insertable)]

View File

@@ -138,6 +138,8 @@ diesel::table! {
date_taken_source -> Nullable<Text>,
original_date_taken -> Nullable<BigInt>,
original_date_taken_source -> Nullable<Text>,
clip_embedding -> Nullable<Binary>,
clip_model_version -> Nullable<Text>,
}
}