feature/clip-semantic-search #96

cameron · 2026-05-16T00:32:21Z

cameron commented

2026-05-16 00:32:21 +00:00

No description provided.

cameron added 5 commits 2026-05-16 00:32:22 +00:00

clip-search: migration + client + probe binary 8d9e76cf15

Probe-phase scaffolding for CLIP semantic search. Adds the column
that will hold per-photo embeddings, the HTTP client to Apollo's
inference service, and a throwaway probe binary so we can eyeball
search-result quality on the live library before building the
persistence layer (backlog drain, /photos/search endpoint, UI).

- migrations/2026-05-14-000000_add_clip_embedding/ — adds
  image_exif.clip_embedding (BLOB) and clip_model_version (TEXT),
  plus a partial index on (clip_embedding IS NULL AND content_hash
  IS NOT NULL) for the future backfill drain.
- src/database/models.rs — extends ImageExif struct to match.
- src/ai/clip_client.rs — encode_image / encode_text / health,
  same Permanent/Transient/Disabled taxonomy as face_client.
- src/bin/probe_clip_search.rs — --query <q> --library N --limit M
  --top K. Encodes a sample and prints top-K cosine similarities.
  No DB writes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

clip-search: backlog drain + /photos/search endpoint 32195ed89e

Wires the persistence layer for CLIP semantic search. The watcher's
per-tick drain encodes any image_exif row with a known content_hash
but no clip_embedding via Apollo (cap CLIP_BACKLOG_MAX_PER_TICK,
default 32). On a query, /photos/search encodes the text via Apollo
and reranks every stored embedding in-memory.

ExifDao additions:
- list_clip_unencoded_candidates — partial-index scan for drain
- backfill_clip_embedding — touches only the two new columns
- list_clip_index — dedup'd (hash, embedding) pull for search

clip_watch::run_clip_encoding_pass is the parallel fan-out — tokio
runtime per pass with CLIP_ENCODE_CONCURRENCY (default 4). No marker
rows for permanent failures yet; per-tick cap bounds the retry cost.

/photos/search params: q, limit, threshold (default 0.20), library,
model_version. Response is intentionally minimal (path + score) so
the frontend joins against existing photo-metadata routes lazily.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

clip-search: fmt + clippy clamp + test AppState arg 66267cc345

Pulls cargo fmt + clippy pass over the new files only — pre-existing
files left untouched even though fmt has drift on them. clamp(1,200)
swaps a manual min/max chain that clippy flagged. test AppState
constructor needed ClipClient::new(None) so the lib-test target
compiles.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

clip-search: document env knobs in .env.example ee2ed3005b

APOLLO_CLIP_API_BASE_URL (falls back to APOLLO_API_BASE_URL),
CLIP_BACKLOG_MAX_PER_TICK, CLIP_ENCODE_CONCURRENCY, and
CLIP_REQUEST_TIMEOUT_SEC — all of which the code already reads.
Apollo's side was documented earlier; this closes the parity gap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

clip-search: offset-based pagination on /photos/search 922f7df8d3

Adds `offset` query param (default 0) and `total_matching` + `offset`
response fields. Backend already computes the full sorted list of
above-threshold matches per query; pagination just slices it at
[offset, offset+limit) instead of always returning the top window.
Offsets past the end return an empty page cleanly so the client can
stop fetching naturally.

Re-scores on every page rather than caching the sorted list — at
personal-library scale (~14k embeddings, 768d) the dot-product loop
is sub-100ms and the lack of state means no eviction / staleness
concerns.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

cameron merged commit dd7b4befb6 into master

2026-05-16 00:32:33 +00:00

cameron deleted branch feature/clip-semantic-search

2026-05-16 00:32:34 +00:00

cameron referenced this issue from a commit

2026-05-16 00:32:34 +00:00

Merge pull request 'feature/clip-semantic-search' (#96) from feature/clip-semantic-search into master

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Apps/ImageApi#96