Wires the persistence layer for CLIP semantic search. The watcher's
per-tick drain encodes any image_exif row with a known content_hash
but no clip_embedding via Apollo (cap CLIP_BACKLOG_MAX_PER_TICK,
default 32). On a query, /photos/search encodes the text via Apollo
and reranks every stored embedding in-memory.
ExifDao additions:
- list_clip_unencoded_candidates — partial-index scan for drain
- backfill_clip_embedding — touches only the two new columns
- list_clip_index — dedup'd (hash, embedding) pull for search
clip_watch::run_clip_encoding_pass is the parallel fan-out — tokio
runtime per pass with CLIP_ENCODE_CONCURRENCY (default 4). No marker
rows for permanent failures yet; per-tick cap bounds the retry cost.
/photos/search params: q, limit, threshold (default 0.20), library,
model_version. Response is intentionally minimal (path + score) so
the frontend joins against existing photo-metadata routes lazily.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Probe-phase scaffolding for CLIP semantic search. Adds the column
that will hold per-photo embeddings, the HTTP client to Apollo's
inference service, and a throwaway probe binary so we can eyeball
search-result quality on the live library before building the
persistence layer (backlog drain, /photos/search endpoint, UI).
- migrations/2026-05-14-000000_add_clip_embedding/ — adds
image_exif.clip_embedding (BLOB) and clip_model_version (TEXT),
plus a partial index on (clip_embedding IS NULL AND content_hash
IS NOT NULL) for the future backfill drain.
- src/database/models.rs — extends ImageExif struct to match.
- src/ai/clip_client.rs — encode_image / encode_text / health,
same Permanent/Transient/Disabled taxonomy as face_client.
- src/bin/probe_clip_search.rs — --query <q> --library N --limit M
--top K. Encodes a sample and prints top-K cosine similarities.
No DB writes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The hash-keyed `/video/hls/{hash}/{file}` route fully covers HLS
playback now and both clients (Apollo, FileViewer-React) have
shipped updates that use it directly. Keeping the basename-keyed
fallback only encouraged stale URLs to keep flowing — every legacy
file was deleted by the startup migration, so the routes were
guaranteed 404 machines.
Dropped:
- `stream_video` handler (`GET /video/stream?path=…`) — the original
basename-keyed playlist serve.
- `get_video_part` handler (`GET /video/{path}`) — bare-filename
segment serve. The new layout's segments live in
`<shard>/<hash>/segment_NNN.ts` and reach the client via
`stream_hls_file`.
- `legacy_path` field on `GenerateVideoResponse` (serialised as
`playlist`). The field always pointed at a file the migration had
deleted; current clients ignore it entirely.
- Their service registrations in `main.rs`.
- The body-side `filename` extraction in `generate_video` (existed
only to construct `legacy_path`) and the now-unused `global`
opentelemetry import in `handlers/video.rs`.
All 707 tests still pass. Same hand-rolled validators (`is_valid_hash`
/ `is_allowed_hls_filename`) keep the new route's defense-in-depth
intact.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure mechanical cleanup of accumulated drift in files outside the
HLS-content-hash branch's main change set. No behavior change.
- `cargo fmt` on every previously-misformatted file
(`ai/insight_generator.rs`, `database/knowledge_dao.rs`,
`faces.rs`, `knowledge.rs`, `libraries.rs`).
- `cargo clippy --fix`:
- `needless_borrow`: `&library` → `library` in `handlers/image.rs`
(two sites in the photo-listing path).
- Manual clippy pass for warnings clippy emits but can't auto-apply:
- `field_reassign_with_default` in `database/reconcile.rs::run` —
consolidated into a struct-literal initializer.
- `needless_range_loop` in `database/knowledge_dao.rs::union_perceptual_tags`
— inner `for b in (a+1)..indices.len() { let ib = indices[b]; ... }`
becomes `for &ib in &indices[a + 1..] { ... }`.
- Doc-list indentation: continuation lines under nested bullets in
`database/mod.rs::get_memories_in_window` and
`database/knowledge_dao.rs::build_entity_graph` realigned to the
list-item content column.
Deliberately not touched (each deserves its own focused commit, with
testing, rather than getting bundled into a sweep):
- 4× `deprecated count_distinct` in `faces.rs` — diesel API migration
to `AggregateExpressionMethods::aggregate_distinct` may shift result
types; needs verification against the existing stats queries.
- `await_holding_lock` in `knowledge.rs:807` — `std::sync::Mutex` held
across `ollama.generate(...).await`. Genuine concurrency bug; fix
requires understanding the surrounding flow before just dropping
the guard.
- 2× `type_complexity` in `database/mod.rs` — cosmetic, would need a
`type` alias and corresponding callers updated.
- Dead `total_deleted` on `library_maintenance::GcStats` and
`file_scan::enumerate_indexable_files` — both are public surface
retained for future use; deletion is a separate decision.
All 707 tests still pass. Release build clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pure mechanical pass on the files this branch added/modified:
rustfmt reflow of a few long lines / chains, and the one
non-pre-existing clippy warning — replacing
`&[rel_path.clone()]` with `std::slice::from_ref(&rel_path)` in
`handlers::video::generate_video` to avoid the alloc + clone for a
single-element slice.
All 707 tests still pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The hash-keyed pipeline transcodes lazily, so a freshly mounted (or
freshly upgraded) library is "mostly pending" for the first hour
while the watcher works through the backlog. The operator wants a
live read on remaining work so they can tune `HLS_CONCURRENCY` and
know when to stop waiting.
Adds:
- `src/hls_stats.rs` — pure compute path (`stats_from_rows`) and an
Arc<Mutex<dyn ExifDao>> wrapper (`compute_and_publish`). Per
library: `total`, `with_playlist`, `pending`, `unsupported`,
`hashless_videos`. Dedup is by content_hash so duplicate-bytes-at-
N-paths counts once (same domain rule as `faces::stats`).
`hashless_videos` is a separate counter so the operator can see
the "hash backfill, then transcode" pipeline depth instead of
having NULL-hash rows just hide.
- Prometheus gauges labeled by library name:
`imageserver_hls_videos_total`, `..._with_playlist`, `..._pending`,
`..._unsupported`. Updated by the watcher at the end of every full-
scan tick *and* on every `/hls/stats` hit, so whichever surface the
operator is watching stays fresh. Registered in `main` alongside
the existing image/video gauges.
- `GET /hls/stats` — Claims-protected JSON snapshot of the same data
plus a top-level cross-library aggregate. Runs on a blocking pool
so it doesn't pin the actix worker; per-call cost is one
`list_paths_and_hashes_for_library` SQL query per library plus a
`stat()` per distinct video hash. Bounded — never invoked from
middleware, only from the explicit endpoint and the full-scan
tick. The watcher's end-of-tick `info!` summary line mirrors the
endpoint output for operators tailing the log.
- New `ExifDao::list_paths_and_hashes_for_library` method:
`SELECT rel_path, content_hash FROM image_exif WHERE library_id =
?`. Single round-trip; callers filter to video extensions
client-side because the schema doesn't carry media-type. Mock
impl in `files.rs` returns an empty vec.
Tests in `hls_stats::tests` exercise stats_from_rows directly (videos-
only filter, hash dedup, playlist vs sentinel decision, NULL-hash
hashless counting) plus a publish_gauges round-trip that reads the
gauge value back. Full suite (347 lib + 360 bin = 707) passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`POST /video/generate` is reshaped to return a JSON object instead of
a bare string. New fields:
- `playlist_url`: stable hash-keyed URL of the form
`/video/hls/<hash>/playlist.m3u8`. Use this with hls.js / native
players — relative segment refs inside the playlist resolve to
`/video/hls/<hash>/segment_NNN.ts` because the URL is path-based.
- `content_hash`: the blake3 hex digest that identifies the bytes.
Stable across libraries, archive ingests, renames; clients can
cache the URL by hash.
- `ready`: true iff the playlist file is already on disk. False means
a transcode was just queued; the client should retry the URL after
a short delay (or rely on hls.js's built-in retry).
- `playlist` (legacy): basename-keyed path string, echoed under the
old field name so clients that destructure `response.playlist` keep
working during the rollout. The startup migration deletes the
underlying file, so this URL will 404; clients should migrate to
`playlist_url`. Field is slated for removal once Apollo / File
Viewer ship the update.
The handler:
- resolves the source path across libraries (same logic as before),
- looks up `image_exif.content_hash` for that (library_id, rel_path),
- falls back to inline `content_hash::compute` when the row is mid-
backfill — pure read, no library mutation,
- sends a single-element `QueueVideosMessage` to `VideoPlaylistManager`
if the playlist isn't already on disk and there's no
`playlist.unsupported` sentinel,
- returns the URL immediately. The actor pipeline owns transcoding.
New route `GET /video/hls/{hash}/{file}`:
- strict validation: hash must be 64 ascii-hex chars; file must be
`playlist.m3u8` or `segment_NNN.ts` (digits only). Anything else
returns 400 so we never have to rely on path canonicalisation
alone to defend against traversal,
- belt-and-suspenders canonicalize() guard verifies the resolved
file lives under `$VIDEO_PATH`,
- serves with the standard `NamedFile::into_response` machinery.
Cleanup in `actors.rs`:
- `ProcessMessage` + its `StreamActor` handler had no senders after
the rewire — removed. `StreamActor` itself stays (still handles
`RefreshThumbnailsMessage` from `files.rs`).
- `create_playlist`, `playlist_file_for`,
`playlist_unsupported_sentinel` are gone — the legacy on-demand
transcode helper and the migration-only path helpers had no
remaining users (the migration uses its own classify() function).
- Imports tightened: dropped `Child`, `ExitStatus`, `trace`.
Tests cover both new validators (`is_valid_hash`,
`is_allowed_hls_filename`) including the strings that motivated the
defence-in-depth (traversal attempts, internal `.tmp`/`.unsupported`
artifacts, malformed segment names).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `video::legacy_migration::retire_legacy_hls_output`, called once
from `main` right after the diesel migrations run and before the
actor pipeline starts. Walks `$VIDEO_PATH` at depth 1, deletes every
`.m3u8` / `.m3u8.tmp` / `.m3u8.unsupported` / `.ts` file at root, and
logs a single info line with per-class counts. Skips directories
(the new layout's `<shard>/<hash>/` lives there) and unknown
extensions, so an operator's stashed README or `.tmp` from a
different tool is safe.
Why this needs its own one-shot pass rather than letting the rewritten
`cleanup_orphaned_playlists` handle it: the cleanup walk deliberately
only looks at `<shard>/<hash>/` dirs (so it can't accidentally `rm`
operator-stashed content), so without this migration the legacy files
would sit at root forever, never served, never refreshed. Operator
complaint count from the previous IMG_NNNN.MOV collision: ~10
duplicate-basename hits on one library alone; total .m3u8 count was
699 vs a much larger video count — i.e. the loser of every collision
was a permanent orphan. This pass collects all of them, then the
running watcher writes hash-keyed playlists going forward.
Idempotent — a second boot finds nothing and reports zero deletions,
so the call site can stay in `main` across releases until the module
is removed in a later cleanup commit. Tests cover the happy path
(legacy artifacts gone, hash dir untouched, unrelated files left
alone), idempotency, and the missing-directory case.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The cleanup walk previously looked for `$VIDEO_PATH/<basename>.m3u8`
and matched each file's stem against a recursive walk of every
library. With the hash-keyed layout now in place, every playlist's
file_stem is the literal string "playlist" — the old logic would
treat every hash-keyed playlist as orphaned on its next run and wipe
them all in one tick (default cleanup interval is 24h, so this is a
24-hour bomb on top of the prior commit).
New approach: orphan-ness is decided in the database, not on the
filesystem. The cleanup loop:
- Snapshots every distinct non-NULL `image_exif.content_hash` into a
HashSet (new `ExifDao::list_distinct_content_hashes` method —
`SELECT DISTINCT content_hash WHERE content_hash IS NOT NULL`).
- Walks `$VIDEO_PATH` two levels deep: top-level entries are filtered
to 2-char lowercase hex shard dirs, each shard's children to 64-char
hex hash dirs. Anything else (legacy `.m3u8` at root from the
pre-content-hash era, operator-stashed dirs, partial writes) is left
alone.
- Hash dirs whose hash isn't in the alive set are `remove_dir_all`'d.
Shard dirs that emptied as a result are reaped on the same pass via
`remove_dir` (no-op if non-empty).
- The library-stale safety gate is preserved: a stale library skips
the cycle even though the orphan decision is DB-only, because the
upstream missing-file scan that retires `image_exif` rows itself
pauses for stale libraries. Belt-and-suspenders — keeping a hash
dir for one extra 24h cycle is cheaper than wiping one whose source
was briefly unreachable. The gate now also filters disabled
libraries out of the stale set (they're intentionally absent from
the health map).
- The legacy `excluded_dirs` parameter is preserved on the function
signature but unused (the walk no longer crosses library trees);
flagged with a leading underscore. Callers in `main.rs` stay
unchanged.
`MockExifDao` in `files.rs` grows the new method (returns empty);
unit tests for the new `is_hash_shard` / `is_full_hash` validators
guard against an operator's stashed directory under VIDEO_PATH ever
matching the orphan-rm path. Both pass.
A follow-up commit handles the one-shot startup migration that
retires the legacy basename-keyed `.m3u8` / `.ts` files at
`$VIDEO_PATH` root.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switches the watcher → VideoPlaylistManager → PlaylistGenerator path
from the basename-keyed layout
(`$VIDEO_PATH/{basename}.m3u8`) to the hash-keyed layout
(`$VIDEO_PATH/{hash[..2]}/{hash}/playlist.m3u8`) introduced in the
prior commit. Source videos that share a basename across libraries
(or across subdirs of one library) no longer overwrite each other's
playlists. The legacy HTTP endpoints in `/video/generate` /
`/video/stream` still use the basename layout — those move in a
follow-up commit alongside the stable streaming URL.
actors.rs:
- `QueueVideosMessage.video_paths: Vec<PathBuf>` →
`videos: Vec<VideoToQueue>`. The queue handler dedups against the
hash-keyed playlist + sentinel and forwards `GeneratePlaylistMessage`
carrying the hash.
- `GeneratePlaylistMessage` now carries `content_hash: String`; the
legacy `playlist_path: String` field is gone.
- `PlaylistGenerator` takes a `video_dir: PathBuf` at construction,
computes the hash dir + playlist + sentinel + segment template via
`hls_paths`, `mkdir -p`s the shard/hash dir before ffmpeg runs, and
cleans up partial output on failure by walking the hash dir.
- `ScanDirectoryMessage` and its handler are retired entirely; their
startup-walk role is taken over by the watcher's first tick (see
`watcher.rs` below). Dropping it avoids threading an `ExifDao` into
`VideoPlaylistManager` just so the actor can resolve hashes.
- Legacy `playlist_file_for` / `playlist_unsupported_sentinel` are
retained behind `#[allow(dead_code)]` for the upcoming migration
pass that retires pre-content-hash output.
watcher.rs:
- `process_new_files` keeps `content_hash` in the EXIF-batch result
(formerly threw it away). Videos with `image_exif.content_hash =
NULL` — mid-backfill rows — are skipped this tick rather than
falling back to a basename-colliding playlist; they get picked up
after `backfill_unhashed_backlog` populates the hash on a
subsequent tick. Skipped count is logged at debug.
- The video staleness check now uses `hls_paths::playlist_for_hash`
instead of `$VIDEO_PATH/{basename}.m3u8`.
- `last_full_scan` initialises to `UNIX_EPOCH` so the watcher's first
tick is treated as a full scan. That covers the catch-up gap left
by removing `ScanDirectoryMessage` — every library's existing media
is checked once at watcher boot (≈60s after startup) instead of
waiting up to `WATCH_FULL_INTERVAL_SECONDS` (1h default).
main.rs: removes the `ScanDirectoryMessage` import and the per-library
`do_send` loop, with a comment pointing at the watcher's first-tick
behavior.
state.rs: `PlaylistGenerator::new` now takes the video dir.
Tests: existing `video::hls_paths` (4) and `watcher::tests` (4) pass.
The basename-keyed `/video/generate` endpoint still compiles and
serves; behavior change there is deferred to the follow-up commit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Foundation for migrating HLS playlist output from basename-keyed
(`$VIDEO_PATH/{basename}.m3u8`) to content-hash-keyed
(`$VIDEO_PATH/{hash[..2]}/{hash}/playlist.m3u8`). The basename layout
collides whenever two source videos share a filename — common with
iPhone-style sequential naming (`IMG_NNNN.MOV`) across libraries — so
the loser's playlist gets overwritten and ffmpeg keeps re-queueing the
file every scan.
This commit adds the path layout and type plumbing without touching the
actor pipeline, watcher, or HTTP handlers yet:
- `src/video/hls_paths.rs`: `playlist_for_hash`, `sentinel_for_hash`,
`segment_template_for_hash` built on top of `content_hash::hls_dir`,
with constants for the filenames inside the hash dir. Unit tests
cover the sharded layout and the playlist/sentinel/segment paths
all landing in the same directory (so HLS relative refs resolve).
- `src/content_hash::hls_dir` un-deaded — was waiting for this branch.
- `VideoToQueue` struct in `actors.rs`: pairs a source path with its
content hash so callers that lack a hash (rows mid-backfill) skip
the video rather than fabricate one.
- `playlist_file_for` / `playlist_unsupported_sentinel` retained as
migration-only helpers — they're only needed by the one-shot startup
pass that retires pre-content-hash output.
Follow-ups (separate commits on this branch): wire `hls_paths` through
the queue handler + `PlaylistGenerator`, update the watcher's
`process_new_files` to build `VideoToQueue`, switch `/video/generate`
and `/video/stream` to resolve path→hash and return stable URLs, add
the legacy-layout migration, rewrite `cleanup_orphaned_playlists` for
the new dir shape, and surface progress via Prometheus + `/hls/stats`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Splits parse_excluded_dirs_column on `,`, `\n`, AND `\r` so a textarea
submit with one entry per line works the same as comma-separated.
Mixed input (`a, b\nc`) parses cleanly too — the frontend can paste
from any source without preprocessing.
Motivated by the "forgot the comma" footgun: typing
`.thumbnails .thumbnails2` in a single-line input today stores a
never-matching component pattern. With newlines as a first-class
separator and the frontend switching to a textarea, the natural
one-per-line UX makes that mistake impossible.
The DB store form stays comma-joined (normalize_excluded_dirs_input
hasn't changed) so existing rows are unaffected and no migration is
needed. Newline support matters mostly for the inbound write path;
mirroring it on the read side keeps the parser round-trip safe in
case anything writes a newline form directly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reject the silent-footgun shapes that PathExcluder would store but
never match. The watcher would still walk past every photo as if the
exclude wasn't there, and the operator would have no signal that
their entry is dead. Caught at PATCH time with a descriptive 422.
Rules:
- Backslash anywhere → "use forward slashes" (catches \photos,
photos\2024, \\server\share — Windows-typed entries land in the
component-pattern bucket and never fire).
- Drive-letter prefix (Z:, Z:/...) → "relative to library root" —
excludes are root-relative, not absolute system paths.
- Multi-segment name with no leading slash (photos/2024) →
"did you mean /photos/2024?" — the common "I forgot the slash"
typo, today silently stored as a component pattern that never hits.
- `..` segments in a path entry → "doesn't normalise". base.join()
doesn't canonicalise, so the resulting prefix never matches.
- Bare "/" → "almost certainly a typo" for the library root.
Trailing slashes on path entries are stripped silently. Eight new
tests cover each rejection plus the trailing-slash normalisation
and the all-or-nothing failure mode of normalize_excluded_dirs_input
(one bad entry aborts the whole patch rather than silently applying
N-1 of N changes).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-ups to the PATCH endpoint:
1. GET /libraries now returns ``global_excluded_dirs`` alongside the
library list — the union-with-globals semantics is invisible from
the per-library row alone, and the admin UI needs to show what's
already being skipped before the operator adds entries that would
duplicate.
2. PATCH /libraries/{id} canonicalises the excluded_dirs string on
write via the new ``normalize_excluded_dirs_input``: trims per
entry, drops empties, dedupes preserving first-occurrence order,
comma-joins without inner whitespace. Empty / whitespace-only →
NULL. Round-trip stable so re-saving an entry produces an
identical row.
Five new tests cover the empty / whitespace, trim, dedup, round-trip,
and overlap-with-globals cases. effective_excluded_dirs continues to
keep overlapping entries between globals and per-library on purpose —
PathExcluder accepts repeats and there's no behavioural reason to
dedupe at merge time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an HTTP mutation surface for `libraries.enabled` and
`libraries.excluded_dirs`, replacing the SQL-only workflow noted in
CLAUDE.md. Apollo's Settings panel calls this from the LIBRARIES
section so the operator no longer has to ssh + sqlite3 to flip a
library off or edit its excludes.
Live-apply (no restart) via a new `live_libraries: Arc<RwLock<Vec<
Library>>>` field on AppState. The existing immutable `libraries`
Vec stays for hot-path handlers that only need stable id → root_path
lookups, avoiding a 19-call-site refactor. The watcher and
cleanup_orphaned_playlists now take the lock instead of a Vec
snapshot and re-read at the top of each tick, so `enabled` /
`excluded_dirs` changes are picked up within one
WATCH_QUICK_INTERVAL_SECONDS. The GET /libraries handler also reads
through the live view.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The drain queried `date_taken IS NULL OR date_taken_source = 'fs_time'`
ORDER BY id ASC LIMIT 500 every watcher tick. The resolver is
deterministic on file bytes + filename + fs metadata, so any row that
landed on fs_time once landed there again on every retry — the drain
spun on the same lowest-id rows in perpetuity, never advancing to
rows 501+ while still logging more_remain=true.
Side effect: 500 auto-commit UPDATEs per tick sustained the SQLite
write lock long enough that other writers on separate DAO connections
hit the 5s busy_timeout. Manifested as intermittent 500s on
PATCH /image/faces/{id} that succeeded on retry.
Narrow the partial index and query predicate to `date_taken IS NULL`.
If exiftool installs or a new filename regex lands, an operator can
re-resolve fs_time rows out-of-band rather than re-introducing the
steady-state churn.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The four 500-return paths in update_face_handler returned e.to_string()
in the body but never logged. When a face PATCH failed with a 16-byte
body and no log entry, the cause (SQLITE_BUSY from cross-DAO writer
contention exhausting the 5s busy_timeout) was invisible. Surface the
full anyhow chain via {:#} on each path so the diesel cause is in the
log even when the response body only shows the top-level context.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
main.rs drops from 1200 → 346 lines (90% smaller than the pre-branch
3542). What's left is the startup wiring it was always meant to be:
.env, migrations, AppState construction, route registration, server
bind. The four background-loop functions move into src/watcher.rs:
- watch_files (310 lines) — quick/full scan tick, per-library probe,
backfill drain dispatch, missing-file scan, back-ref refresh,
orphan GC.
- process_new_files (351 lines) — file walk → EXIF write →
face-candidate build → HLS / preview-clip queueing →
reconciliation. The "biggest untested chunk" from the earlier
audit.
- cleanup_orphaned_playlists (167 lines) — separate slower-tick
thread.
- playlist_needs_generation — small mtime-comparison helper.
Plus 4 unit tests for playlist_needs_generation (covers missing
playlist, newer playlist, newer video, video-missing-metadata
fallback).
main.rs's imports correspondingly shrink — Addr, HashSet, WalkDir,
Utc, InsertImageExif, and the bulk of video::actors all leave with
the watcher. CLAUDE.md updated to reflect the new module layout
(layered architecture box + module map for the face-detection
section).
cargo test --bin image-api: 329 passing (no regression).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
main.rs drops from 2935 → 1200 lines, freed for startup wiring +
the watcher. The 16 route handlers move into three domain-grouped
files under src/handlers/:
- handlers/favorites.rs (128 lines): favorites, put_add_favorite,
delete_favorite.
- handlers/video.rs (665 lines): generate_video, stream_video,
get_video_part, get_video_preview, get_preview_status. The 5
pre-existing get_preview_status integration tests move with the
handler (still pass against TestPreviewDao + AppState::test_state).
- handlers/image.rs (1003 lines): get_image (with the
hash/library-scoped/bare-legacy thumb lookup), upload_image,
get_file_metadata, set_image_gps, get_full_exif, set_image_date,
clear_image_date. Helpers (create_circular_thumbnail,
build_metadata_response_for_date_mutation) and request structs
(SetGpsRequest, SetDateRequest, ClearDateRequest, UploadQuery)
travel with them.
main.rs's import block shrinks from ~50 lines to ~22 as everything
HTTP-specific (NamedFile, mp::Multipart, BytesMut, Span, KeyValue,
StreamExt, …) moves with the handlers. The is_video_file wrapper
also goes — remaining callers in watch_files / cleanup use
file_types::is_video_file directly.
cargo test --bin image-api: 325 passing (no regression).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
main.rs drops from 3542 → ~2930 lines by moving:
- src/backfill.rs (new): backfill_unhashed_backlog,
backfill_missing_date_taken, backfill_missing_content_hashes,
build_face_candidates, process_face_backlog. Now unit-tested for
the first time — 5 tests covering cap behavior, library-id
filtering, missing-on-disk skip, and the video/unhashed/scanned
filters on face-candidate selection.
- src/thumbnails.rs (new): unsupported_thumbnail_sentinel,
generate_image_thumbnail, create_thumbnails, update_media_counts,
is_image, is_video, plus the IMAGE_GAUGE / VIDEO_GAUGE Prometheus
metrics. Replaces the no-op stubs that used to live in lib.rs.
4 new unit tests for the sentinel path math and the
walker-counts-images-vs-videos smoke path.
Supporting:
- SqliteExifDao::from_shared (test-only) so an SqliteExifDao and
SqliteFaceDao can share one in-memory connection — required to
test build_face_candidates against the real join.
- files.rs / video/{mod,actors}.rs import from crate::thumbnails::*
instead of the now-removed stubs in lib.rs.
cargo test --bin image-api: 325 passing (was 314).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two coupled changes to fight the speech-act-predicate problem
(facts like (Cameron, expressed, "I'm tempted to...")):
1. System prompt grows an explicit predicate-quality rule. The
agent is told to use relationship-shaped verbs (lives_in,
works_at, attended, is_friend_of, interested_in), and is
given an explicit DON'T list (expressed, said, mentioned,
stated, quoted, noted, discussed, thought, wondered). Plus a
concrete Bad / Good example contrasting the noise pattern
with the structured paraphrase the agent should be writing.
Stops the bleed for new insights.
2. Cleanup tools for the legacy noise that's already in the
table:
- get_predicate_stats(persona, limit) returns
[(predicate, count)] sorted desc — feeds the curation UI's
PREDICATES tab.
- bulk_reject_facts_by_predicate(persona, predicate, audit)
flips every ACTIVE fact under that predicate to 'rejected'
in one transaction, stamping last_modified_* so the action
is attributable + reversible per-fact through the entity
detail panel. REVIEWED facts under the same predicate are
left alone — the curator may have hand-approved an
exception ("interested_in" might be largely noise but a
reviewed entry is intentional).
New HTTP endpoints:
GET /knowledge/predicate-stats?limit=
POST /knowledge/predicates/{predicate}/bulk-reject
Persona-scoped via the existing X-Persona-Id header.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
One-shot migration that re-applies the synonym map from
`normalize_entity_type` over every existing row, so legacy
entries written before that helper landed in upsert_entity stop
needing client-side workarounds.
person ← person | people | human | individual | contact
place ← place | location | venue | site | area | landmark
event ← event | occasion | activity | celebration
thing ← thing | object | item | product
Unknown types ("friend", "family", etc.) get a lowercase+trim
sweep so at minimum case variants collapse — the curator can
merge or rename them via the curation UI from there.
`UPDATE OR IGNORE` skips rows that would violate UNIQUE(name,
entity_type) after the rewrite (e.g. an existing ("Sarah",
"person") + ("Sarah", "Person") pair). The duplicate survives
unchanged so it can be merged through the normal curation flow
rather than silently disappearing.
Idempotent: every UPDATE is conditional on `entity_type !=
canonical`, so re-running the migration is a no-op. The down
migration is intentionally inert — we don't have per-row
history of the original strings and the rewritten values stay
semantically correct.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New GET /knowledge/graph?type=&limit= returns the data the
curation UI's graph tab needs:
- nodes = entities with at least one in-scope fact (rejected /
superseded excluded). Carries fact_count for visual sizing.
Top-N by count desc; default cap 200 (clamped 1..1000).
- edges = relational facts (object_entity_id set) grouped by
(subject, object, predicate) so 3 "is_friend_of" facts
between the same pair collapse into one edge with count=3.
Two raw SQL queries: an INNER JOIN onto a persona-scoped fact-
count subquery for nodes (skips 0-fact entities entirely so the
sim doesn't waste time on disconnected islands), then a follow-
up GROUP BY over the persona-scoped fact set restricted to the
node id set via IN clauses (ids are i32 so inlining is safe).
Pairs with the Apollo-side GraphPanel that runs d3-force over
the returned payload and renders SVG with click-to-open.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
No logic changes - line reflow, brace placement, and method-chain splits
across handlers / personas / state / faces / knowledge / insights_dao /
knowledge_dao / populate_knowledge. Picked up incidentally while running
fmt for the sms-search work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Refactor search_messages_with_contact -> search_messages(query, &SmsSearchParams)
exposing date_from / date_to / offset / is_mms / has_media; drop the over-fetch
+ client-side date post-filter that could silently drop in-window hits past
position 100.
- Surface SMS-API's <mark>-wrapped snippet for MMS messages that only matched
via message_parts_fts (attachment text / filename) - pre-snippet, those
rendered as a blank body preview to the LLM.
- Expose is_mms / has_media on the search_messages tool schema; expand the
FTS5 syntax docs with worked examples for phrase / prefix / boolean / NEAR
/ grouping so the model picks the right operator.
- Unit tests for format_search_hits (body fallback, snippet preferred, MMS
attachment-only regression, empty-snippet fallback) and strip_mark_tags.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Finds near-duplicate entities the upsert-time cosine guard didn't
catch — typically legacy data from before that guard landed, or
pairs whose embeddings sit between 0.85 (default proposal floor)
and 0.92 (auto-collapse threshold). Pure read-side feature; the
actual merging still goes through the existing
/knowledge/entities/merge action.
New DAO method `find_consolidation_proposals(threshold,
max_groups)`:
- Loads every non-rejected entity with an embedding.
- Partitions by entity_type so a person can't cluster with a
place.
- Pairwise cosine, edges above threshold feed a union-find for
transitive grouping (Sara → Sarah → Sarah J. all land in one
cluster).
- Tracks min/max cosine per component so the UI can show "how
tight" each cluster is before clicking in.
- Returns groups of >= 2 sorted by size desc then max cosine
desc; trimmed to `max_groups`.
New endpoint `GET /knowledge/consolidation-proposals?threshold=
&limit=` accepts the threshold (clamped 0.5–0.99 to prevent the
"every entity in one mega-cluster" case) and returns groups with
per-entity persona fact-count breakdowns baked in — saves the UI
a separate query per cluster member.
ConsolidationGroup is exported through database/mod.rs so the
handler can use it without depending on knowledge_dao internals.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Entities are global; facts are persona-scoped. Under the active
persona an entity can read as "0 facts" while having plenty under
other personas the user owns — the curation UI had no way to
surface that gap. Adds a batched DAO method
`get_persona_breakdowns_for_entities` that returns
{entity_id → [(persona_id, count)]} in one query (group by
subject + persona, user-scoped, status != rejected), and wires it
into both /knowledge/entities list rows and
GET /knowledge/entities/{id}.
EntitySummary grows an optional `persona_breakdown` field
(skipped on serialization when None — keeps PATCH responses
unchanged). EntityDetailResponse carries the breakdown as a
non-optional Vec since the detail endpoint always populates it.
One extra query per list page (50 entities → 50 subject ids
batched in one IN clause); single-entity GET adds one round trip.
Indexed by (subject_entity_id, persona_id) implicitly via the
existing user-persona indexes on entity_facts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
System prompt now explicitly enumerates the markdown forms the
model shouldn't emit (bold, italics, headings, bullets, lists,
code fences) on top of the existing "no preamble, no quotes"
constraints. Some local models default to markdown-shaped
output for descriptions and the curation UI is plain-text,
which would render the asterisks and hashes literally.
The output cleaning step picks up a parallel sweep: strip code
fences, leading bullets / headings, wrapping quotes, and naive
inline emphasis markers (** and __). Rare enough that the
plain-replace is fine; not trying to parse markdown.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New POST /knowledge/entities/synthesize-merge { source_id,
target_id } that calls the local Ollama with both entities' names
+ descriptions and returns a synthesized merged-description draft.
Read-only on the database — the curation UI uses the response as
the editable seed in the merge picker; the actual merge still
requires a follow-up PATCH-target-description + POST /merge.
The handler drops the KnowledgeDao lock before the LLM call so
other knowledge reads aren't blocked while generation runs
(typically seconds). Failure mode is 503 with an explicit hint
that the UI should fall back to skip-synthesis — keeps the merge
action working when the model is offline.
Output is lightly cleaned (leading "Merged description:" /
surrounding quotes stripped) since small models reach for those
patterns even with explicit "no preamble" guidance. Heavier
parsing isn't worth it — the curator edits anyway.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bundles three coupled changes so agent-side mutations stay
auditable and reversible:
1. Audit columns on entity_facts —
`last_modified_by_model` / `last_modified_by_backend` /
`last_modified_at`. Stamped on every mutation path
(update_fact, supersede_fact, manual PATCH, manual supersede,
the new revert). NULL on rows never touched since creation.
Partial index on `last_modified_at WHERE NOT NULL` keeps the
"show me recent edits" feed fast without bloating from legacy
rows.
2. Per-persona gate `personas.allow_agent_corrections` (BOOLEAN,
default 0). Defense in depth at two layers:
- build_tool_definitions: when off, `update_fact` and
`supersede_fact` aren't in the catalog at all, so even a
hallucinated tool call by the model fails fast.
- tool_update_fact / tool_supersede_fact: re-checks the persona
flag at call time and returns an explicit "corrections
disabled" error if it's somehow off (e.g. flag flipped mid-
loop).
ToolGateOpts grows the flag; current_gate_opts splits into
`current_gate_opts` (no persona context, defaults closed) +
`current_gate_opts_for_persona` for chat callers that have a
persona id. Both call sites in insight_chat are updated.
3. Revert action — new DAO method `revert_supersession` +
`POST /knowledge/facts/{id}/restore`. Flips status back to
'active', clears `superseded_by`, clears `valid_until` (we
don't track whether it was hand-set vs auto-stamped, so the
safe reset is to drop it — user can re-bound after). Stamps
`last_modified_*` so the revert itself is attributable.
Manual paths (PATCH / supersede via HTTP, plus restore) stamp the
audit columns with `("manual", "manual")`. Agent paths stamp the
loop-time chat model and backend (mirroring the existing
created_by_* convention).
FactDetail in the HTTP response now carries the audit triple
alongside the existing provenance. Apollo wires the new field set
in the matching commit.
PersonaView / UpdatePersonaRequest grow `allowAgentCorrections`;
the PersonaPatch + InsertPersona + bulk_import paths thread it.
317 lib tests pass, including unchanged update_fact / supersede
DAO tests (now passing audit=None — None means "no provenance
context to attribute", legacy semantics).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two coupled changes to the agent's recall surface:
1. Default scope expanded. recall_facts_for_photo and recall_entities
used to filter to status='active' only — which silently dropped
'reviewed' (human-verified) facts. Now they surface active +
reviewed by default. Reviewed is strictly more trusted than
active and shouldn't have been hidden. Rejected and superseded
stay filtered.
2. New persona toggle `reviewed_only_facts` (BOOLEAN, default false,
migration 2026-05-10-000400). When set, the agent's recall on
that persona returns ONLY facts with status='reviewed' — strict
mode for tasks where hallucinated agent claims are particularly
costly. Wired:
- schema.rs / Persona / InsertPersona / PersonaPatch grow the
field.
- PersonaView returns it as `reviewedOnlyFacts` (camelCase wire).
- PUT /personas/{id} accepts it (mobile editor surfaces it).
- InsightGenerator now carries a PersonaDao reference so
recall_facts_for_photo can read the active persona's flag at
start; one extra read per recall, cheap.
Composes with include_all_memories: that operates on the persona
*scope* axis (single vs hive), reviewed_only_facts on the *status*
axis. They're orthogonal.
Legacy persona rows pick up the default false on migration; no
behavior change unless explicitly toggled. The 4 existing persona
construction sites (one production, two tests, one InsertPersona in
knowledge_dao tests) all default the field. populate_knowledge bin
+ state.rs constructors also wire the new persona_dao arg.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds two nullable TEXT columns to entity_facts —
`created_by_model` (LLM identifier) and `created_by_backend`
("local" / "hybrid" / "manual" / NULL) — so the curator can audit
which configurations produce good fact-keeping and which produce
noise.
photo_insights already carries model_version + backend, and
entity_facts.source_insight_id links to it, but:
- source_insight_id is set post-loop, so chat-continuation and
regenerated-insight facts lose the link.
- JOINing per read is more friction than embedding provenance on
the row itself.
- Manual facts (POST /knowledge/facts) have no insight at all and
need their own "manual" provenance marker.
Threading: execute_tool grows `model` + `backend` params, passed
from the three call sites (agentic insight loop, chat single-turn,
chat stream) using the loop-time `chat_backend.primary_model()` +
`effective_backend` already in scope. tool_store_fact stamps the
new fact accordingly; manual create_fact stamps backend="manual".
Legacy rows leave both NULL — pre-tracking data can't be back-
filled reliably from training_messages without burning compute.
Indexes are partial (WHERE NOT NULL) so legacy rows don't bloat
them, and "show me all facts from model X" stays fast.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two Phase-2 followups in one commit since they're coupled at the
write path:
* Agent populates valid_from from the source photo's date_taken
when calling store_fact. Loose semantics — date_taken is *evidence
at that date*, not strictly when the fact started being true — but
gives the curator a calendar anchor and pairs with supersession to
close intervals cleanly. valid_until stays NULL (a single photo
can't tell us when something stopped). Honours the existing
upsert_fact dedup (corroborated facts keep their first-recorded
valid_from).
* Supersession: new column entity_facts.superseded_by INTEGER
(migration 2026-05-10-000200), new status value 'superseded',
new DAO method supersede_fact, new HTTP endpoint
POST /knowledge/facts/{id}/supersede.
Marking an old fact as replaced by a new one atomically: flips
status to 'superseded', sets superseded_by, and stamps
valid_until from the new fact's valid_from (when not already
set). delete_fact clears dangling supersession pointers in the
same transaction so the column never points at a missing row —
no FK because SQLite can't ALTER ADD with REFERENCES, but the
DAO maintains the invariant.
Pairs with conflict detection from the previous slice: once the
old fact's valid_until is closed, its interval no longer overlaps
the new fact's, so they stop flagging — the supersede action
resolves the conflict.
Two tests pin the contract: supersede stamps valid_until from
new.valid_from while respecting an existing valid_until, and
deleting the supersedeR clears the dangling pointer while leaving
the old fact's 'superseded' status in place for history.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds bitemporal support to entity_facts. Existing `created_at` is
transaction time (when we recorded the fact); the new
`valid_from` / `valid_until` BIGINT columns are valid time (when the
fact is/was true in the real world). NULL on either side = unbounded
on that side, both NULL = "always-true / unknown" — matches the
default state of every legacy row, no backfill needed.
The split matters for time-bounded predicates like
is_in_relationship_with / lives_in / works_at: recording the fact
once doesn't mean the relationship is still ongoing. Same predicate
across different windows ("lives_in NYC 2018-2020", "lives_in SF
2020-present") is no longer a conflict — the interval-aware check
in get_entity only flags pairs whose windows overlap. Facts with no
valid-time data still flag against everything (worst case for legacy
rows — user adds dates to suppress).
API surface:
- POST /knowledge/facts accepts optional valid_from / valid_until.
- PATCH /knowledge/facts/{id} accepts both with tri-state semantics:
field omitted = leave alone, JSON null = clear to NULL, number =
set. Implemented via a small serde helper around Option<Option>.
- GET /knowledge/entities/{id} surfaces both fields per fact and
uses them in conflict detection.
Agent path (insight_generator) writes NULL/NULL for now — deriving
valid_from from the source photo's date_taken is slated for a
follow-up agent tool alongside Phase 2's supersession.
Test pins set + clear semantics via update_fact: setting both
bounds, leaving them alone on a subsequent patch, then clearing
valid_until back to NULL.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
GET /knowledge/entities/{id} now flags facts as `in_conflict` when
another active fact shares the same predicate but disagrees on the
object (entity id or text value). Pure read-time computation in the
handler — group facts by predicate, distinct-object count > 1 flags
all members. No schema change; same shape as `is_current` on photo
insights.
The flag is intentionally a *signal*, not a hard constraint. Some
predicates are legitimately multi-valued (friend_of, tagged_in,
appears_in) — the curator UI surfaces the amber accent and lets the
user reject the stale fact, accept both, or supersede one later
once the supersession column lands.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related additions to /knowledge/entities:
- New EntitySort enum (UpdatedDesc default, NameAsc, FactCountDesc)
surfaced via `?sort=updated|name|count`. NameAsc clusters near-
duplicate names so dupes stand out at a glance; FactCountDesc
surfaces heavily-used entities and demotes 0-fact noise to the
bottom.
- New `list_entities_with_fact_counts` DAO method that returns each
entity alongside a persona-scoped count of its non-rejected facts
(subject side). Persona scope follows X-Persona-Id via the
existing resolve_persona_filter chain — Single filters on
(user_id, persona_id), All unions across the user's personas.
Implemented as one raw SQL query with a LEFT JOIN to a fact-count
subquery and ORDER BY tied to the chosen sort, so count-sort needs
no second round trip.
The agent's existing list_entities call site is unchanged — it
doesn't need persona-scoped counts and the trait method stays cheap.
EntitySummary grows an Option<i64> fact_count (skip_serializing_if
none) so PATCH responses stay shaped as before.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DELETE /knowledge/entities/{id} was 500ing on any entity that was the
object of a relational fact. entity_facts.object_entity_id has
ON DELETE SET NULL, but the table also has
CHECK (object_entity_id IS NOT NULL OR object_value IS NOT NULL) —
purely relational facts (subject + predicate + object_entity_id, no
object_value, like "Alice is_friend_of Bob") would have both NULL
after SET NULL fired, the CHECK would abort, and the whole DELETE
would fail with a CHECK violation. The user just saw QueryError
because the DAO swallowed the diesel error string.
Wrap delete_entity in a transaction that first deletes facts where
the entity is the object AND object_value is null, then deletes the
entity. Surviving siblings (typed facts about the entity as subject)
are CASCADE'd by the FK as before. Also start surfacing the actual
diesel error in a warn log before collapsing to DbErrorKind so future
similar issues don't masquerade as the opaque QueryError.
A schema-level fix (changing object FK to ON DELETE CASCADE via a
table-rebuild migration) is the cleaner long-term resolution and is
slated for Phase 2; the DAO-side pre-delete is sufficient and less
invasive in the meantime.
Test pins the contract: a relational fact pointing at the deleted
entity is removed, an unrelated typed fact about an unrelated entity
survives, and the entity itself is deleted.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The PhotoLinkDetail in /knowledge/entities/{id} was dropping the
library_id field, leaving consumers no way to construct a
content-routed thumbnail URL. Apollo's curation screen was falling
through to library=0 (the FastAPI default) and getting 400s.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1 of the knowledge curation work. Three small server-side changes
to support an Apollo-side curation surface and reduce the agent's near-
duplicate output rate going forward:
- upsert_entity grows an embedding-cosine fallback after the exact name
match misses. New entities whose embedding sits above
ENTITY_DEDUP_COSINE_THRESHOLD (default 0.92) against any same-type
active entity collapse onto the existing row. Eliminates the Sarah /
Sara / Sarah J. trio the FTS5 prefix check was missing.
- POST /knowledge/facts symmetric with the existing PATCH/DELETE so the
curation UI can create facts directly. Persona-scoped via X-Persona-Id;
validates subject (and optional object) entity existence; reuses
KnowledgeDao::upsert_fact so corroboration semantics match the agent
path.
- One sentence in build_system_content telling the agent to call
recall_entities before store_entity when a name resembles something
already known. Cheap; complements the DAO-layer guard.
Includes upsert_entity_collapses_near_duplicate_by_embedding test
covering both the collapse-on-near-match path and the don't-collapse-on-
unrelated-embedding path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When a photo exists in more than one library and the user
regenerates its insight from library A's chat, the regenerate
streams cleanly, store_insight flips library A's old row to
is_current=false, and inserts a new is_current=true row tagged
(library A, rel_path). On the next history fetch the user sees
their old transcript — the regenerate appears to vanish.
The cause: get_insight(file_path) filters on rel_path + is_current
only, so library B's untouched is_current=true row for the same
rel_path satisfies the query and gets returned by SQLite's .first()
ahead of A's new row. Because get_insight is also what
chat_turn_stream uses to decide bootstrap vs. continuation, the
next chat turn after the shadow hit also routes against the
wrong insight, so update_training_messages corrupts library B's
transcript with library A's chat.
Fix: add get_current_insight_for_library(library_id, file_path)
filtered on (library_id, rel_path, is_current=true) and route the
chat surface (load_history, chat_turn{,_stream}, rewind_history)
through it. load_history falls back to the cross-library
get_insight when the scoped lookup misses — preserves the
"scalar data merges across libraries" intent for the case where
the active library has no insight but another does. The path-only
get_insight stays for callers that don't have library context
(populate_knowledge, the photo-grid metadata fetch).
chat_history_handler stops dropping the parsed library on the
floor and threads it through. Single-library deploys see no
behaviour change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When the LLM calls search_messages with { date, limit } and no
query, it's making the predictable mistake of conflating the two
"messages"-shaped tools. The previous behaviour returned an error
that pointed it at get_sms_messages — correct, but burning a turn
on the misroute. Long photo-chat threads where the user asks
"what was happening that weekend?" hit this on small models
roughly half the time.
Now the date-string-without-query case transparently dispatches
to get_sms_messages with the same args (date / limit / days_radius
/ contact name all pass through unchanged) and prepends a short
"(Note: routed to get_sms_messages — prefer it directly next time)"
to the result. The model sees real data on its first try while
still learning the right tool for next time. Cases that don't have
a get_sms_messages equivalent (numeric contact_id, or start_ts /
end_ts windows) keep the original error so the model knows to
either supply a query or restructure its call.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two persona-infrastructure correctness fixes that go together because
the second one (FK with CASCADE) requires the first (preventing the
persona row from being mutated out from under its facts).
1. update_persona handler refuses name/systemPrompt edits to built-ins
(409). includeAllMemories stays editable — that's a per-user
preference, not the persona's identity. Mirrors the existing
delete_persona guard. The DAO is intentionally permissive so the
guard sits at the HTTP layer; persona_dao test pins that contract.
2. Migration 2026-05-10 adds user_id to entity_facts and a composite
FK (user_id, persona_id) -> personas(user_id, persona_id) ON DELETE
CASCADE. This closes two issues at once:
- Persona orphans: deleting a custom persona used to leave its
facts dangling forever, readable only via PersonaFilter::All.
CASCADE now wipes them with the persona row.
- Multi-user fact leakage: PersonaFilter::Single("default") used
to surface every user's default-scoped facts. PersonaFilter is
now { user_id, persona_id } and all read paths
(get_facts_for_entity, list_facts, get_recent_activity) filter
on user_id first. upsert_fact's dedup key extends to user_id so
identical claims under shared persona names from different
users no longer corroborate-bump each other's confidence.
- user_id threads from Claims.sub.parse::<i32>().unwrap_or(1) at
the chat / insight handlers through ChatTurnRequest, the
streaming agentic loop, execute_tool, and into the leaf tools
(tool_store_fact, tool_recall_facts_for_photo). The ".unwrap_or(1)"
accommodates Apollo's service token whose sub is non-numeric on
legacy mints.
- Backfill picks the smallest user_id matching each legacy fact's
persona_id so the FK holds for already-stored rows.
Five new knowledge_dao tests with FK-on connection: persona scoping
isolation, All-variant union per-user, dedup not crossing users,
CASCADE delete, FK rejection of unknown personas. Plus
dao_update_does_not_block_built_ins documenting where the
HTTP-layer guard lives.
Apollo coordinates separately — the matching changes there add the
/api/personas proxy and start sending persona_id on photo-chat turns.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pairs with the Apollo FACES-tab change. The new
POST /api/persons/{id}/similar-unassigned route on Apollo needs to
fetch one person's embeddings cheaply to compute the centroid;
adding a person_id query param to /faces/embeddings keeps that to a
single round-trip instead of paging the whole detected set
client-side. When both person_id and unassigned=true are supplied,
person_id wins (the explicit filter is the more specific intent).
Tag-bootstrap removal: bootstrap_candidates_handler,
bootstrap_persons_handler, /persons/bootstrap and
/tags/people-bootstrap-candidates route registrations, and the
heuristic helpers (is_plausible_name_token, looks_like_person) plus
their tests. Only Apollo called these; the migration is complete.
The persons.created_from_tag column stays - it's informational on
existing rows and removing it would be a destructive migration for
no benefit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`get_duration_seconds` now returns `Option<f64>` and falls back from
`format=duration` to `stream=duration`. Empty stdout no longer
parse-panics with "cannot parse float from empty string", which was
poisoning the preview-clip row with status=failed and re-queueing every
full scan (notably for GoPro LRV files). `generate_preview_clip` handles
the unknown-duration case by transcoding the whole file (capped at 10s).
`generate_video_thumbnail` seeks to ~50% of the probed duration instead
of a hardcoded `-ss 3`, with a first-frame fallback when the probe
returns nothing. Fixes the loop where short Snapchat clips (<3s) got
"missing thumbnail" logged on every scan because ffmpeg exited 0
without writing a frame, and never wrote the .unsupported sentinel
either.
Adds unit tests for `parse_ffprobe_duration` covering the empty-output,
N/A, multi-line, non-positive, and non-finite cases.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The date-override path used to look up `image_exif` strictly by
`(library_id, rel_path)` with only the forward-slash form, while
`/image/metadata`'s `get_exif` falls back across libraries and tries
both slash forms. A photo whose row sat under a different library_id
than its filesystem-resolved one — or whose rel_path was stored with
backslashes — rendered fine in the modal but 404'd on save.
`set_manual_date_taken` / `clear_manual_date_taken` now share a
`locate_image_exif_row` helper that mirrors `get_exif`'s union
semantics (scoped lookup first, library-agnostic fallback by rel_path
in both slash forms), then update by primary key so the write hits
exactly the row read. Inner anyhow errors are logged with
`(library_id, rel_path)` so the next failure mode is debuggable.
Handler-side: `resolve_library_param` errors no longer silently fall
back to the primary library (which would have masked the original bug
with a different "row not found"); a malformed library param now
returns 400. New `DbErrorKind::NotFound` lets the handler distinguish
genuine misses (404) from real DB failures (500).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move personas off the mobile client into ImageApi as first-class
records, and scope entity_facts by persona so each one builds its own
voice over a shared entity graph. The new include_all_memories flag
lets a persona opt back into the full hive-mind pool for human
browsing of /knowledge/*; agentic generation always stays in-voice.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The recent blanket "snapchat-" prefix denylist (43f8f83) rejected ALL
Snapchat-prefixed filenames from timestamp parsing, which fixed the
sequential-ID false positives but also broke real unix-second
filenames from Snapchat's early era. `Snapchat-1383929602.jpg`
(2013-11-08 16:53:22 UTC) now falls through to fs_time — and on files
with broken filesystem metadata, fs_time pins to 1970.
Replace the blanket prefix denial with a tighter discriminator:
- exactly 10 captured digits AND timestamp >= 2011-09-23 (Snapchat
launch) → real unix epoch, accept
- any other length under this prefix → sequential ID, reject
This keeps the existing rejections intact:
Snapchat-1021849065.mp4 → 10 digits, 2002 < launch → reject
Snapchat-1751031586660373917.jpg → 19 digits truncates to 16 → reject
And restores the regression case:
Snapchat-1383929602.jpg → 10 digits, 2013 ≥ launch → accept
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The bootstrap system message gave the model a file path and (in
hybrid mode) a visual description, but no temporal anchor. Models
defaulted to today's date when calling get_sms_messages — Nov 2014
photos were getting "2024-03-11" passed as `date`, missing every
historical message and leading the model to confidently misreport
context.
This commit folds two more EXIF-sourced facts into the
--- PHOTO CONTEXT --- block:
Date taken: <YYYY-MM-DD or "unknown">
GPS: <lat, lon to 4dp> (omitted when no GPS)
Resolution waterfall for date_taken matches the documented canonical
date pipeline at the EXIF / filename steps, but intentionally stops
short of the fs-time fallback `generate_agentic_insight_for_photo`
uses — for chat we'd rather show "unknown" than mislead the model
with an inode mtime. GPS is taken straight from EXIF when both
lat/lon are populated; absent GPS suppresses the line entirely so
the model doesn't hallucinate coordinates.
InsightGenerator gains a `fetch_exif(file_path)` accessor (crate-
visible) so the chat service doesn't need its own ExifDao plumbing.
build_bootstrap_system_message picks up two new params (date,
gps); existing tests updated and 5 new tests cover:
- date present / absent / waterfall (EXIF wins, filename fallback,
None when neither source has it)
- GPS present / absent
- ordering (path → date → visual)
Total insight_chat unit tests: 33 (up from 27).
After refresh, the rendered transcript was showing two unwanted
artifacts in the initial user bubble:
Photo file path: pics/DSC_5171.jpg
please tell me about this photo and what was going on around it
Please write your final answer now without calling any more tools.
Two distinct bugs:
1. Bootstrap was prepending `Photo file path: <path>` (and, in
hybrid mode, the visual description block) into the user-turn
content. The model needed it to call file_path-keyed tools, but
the user could see it in their own bubble on replay.
2. The no-tools fallback ("Please write your final answer now…")
was a synthetic user message we never stripped from history,
so it persisted into training_messages, rendered as a second
user bubble, AND wiped the prior tool-call accumulator inside
load_history (user-turn handler clears pending_tools), which
is why the tool invocations disappeared from the assistant
bubble after refresh.
Fixes:
- New `build_bootstrap_system_message` helper composes the persona
with a `--- PHOTO CONTEXT ---` block (path + optional visual
description). Lives in the system message, not the user turn.
The user's bubble shows only what they typed.
- Streaming agentic loop's no-tools fallback now records its
insertion index and removes the synthetic user prompt from
`messages` after the model responds. Final assistant content
stays — it reads coherently on replay without the synthetic
prompt above it. Applies to both bootstrap and continuation.
3 new tests cover the system-message composer (path-only, with
visual block, persona-trim). Total insight_chat unit tests: 27.
Bug: bootstrap user_content was just the user's typed message (plus
the hybrid visual description). Tools that take a file_path arg —
recall_facts_for_photo, get_file_tags, get_faces_in_photo — had no
way to learn the canonical path. Small models would invent
placeholders like "input_file_0.png" or call the tool with a name
guessed from a hidden multimodal input handle, neither of which
matched any real photo.
Fix: prepend a single-line "Photo file path: <normalized>\n\n" block
to user_content. Same shape generate_agentic_insight_for_photo
already uses for non-chat callers — kept the bootstrap minimal
(no date / GPS / tags pre-stuffing; the agentic loop can fetch
those via tools when needed).
Hybrid still injects the visual description block between the path
block and the user message; local mode just gets path + user text.
resolve_bootstrap_system_prompt and resolve_bootstrap_backend run on
every bootstrap turn — they pick the persisted system prompt and the
chosen backend label. They were inline conditionals before; pulling
them out makes the rules testable without spinning up the full
streaming stack.
9 new tests cover:
- system prompt fallback to BOOTSTRAP_DEFAULT_SYSTEM_PROMPT for None,
empty string, whitespace-only
- supplied non-empty prompts pass through verbatim, with interior
newlines / spacing preserved (Apollo personas use multi-line tool
listings)
- backend defaults to "local" for None / empty
- "local" / "hybrid" accepted case-insensitively with edge-trim
- unknown labels return a descriptive error
Total insight_chat tests: 24 (up from 15). No behaviour change.
Tap-Discuss-on-no-insight previously failed silently: ImageApi's
/insights/chat/stream required an existing agentic insight, errored
when missing, and emitted the failure as `event: error` — which the
frontend SSE consumer ignored (it listens for `error_message`).
This commit closes both gaps with a server-side state machine:
- /insights/chat/stream now branches on insight presence. Missing
insight (or `regenerate: true` in the body) → bootstrap path:
builds [System(req.system_prompt), User(req.user_message + image)],
runs the agentic loop, generates a title, persists a new row via
store_insight (which auto-flips priors). Existing insight →
continuation path (unchanged behaviour).
- New `regenerate: bool` request field forces bootstrap even when an
insight exists. Takes precedence over `amend`.
- `done` SSE payload field-name alignment with Apollo's frontend
convention: prompt_eval_count → prompt_tokens, eval_count →
eval_tokens, num_ctx echo added.
- `amended_insight_id` semantics broaden — now populated whenever the
turn produced a new row (bootstrap, regenerate, or amend). Existing
amend clients keep working unchanged; new clients get the new row's
id for free.
- `event: error` → `event: error_message` so frontend errors stop
silently dropping.
Refactor: extracted run_streaming_agentic_loop, build_chat_clients,
and generate_title as shared helpers between bootstrap and
continuation. Continuation path's outer logic moves to
run_continuation_streaming with no behaviour change.
Mobile-ready: any client (Apollo backend, mobile, future) sends one
request to /insights/chat/stream and gets the right path. Apollo's
proxy stays a dumb pipe.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop redundant `use anyhow::Context` inside has_any_faces (already
imported at the module level).
- Drop dead `.unwrap_or("?")` on bound faces — the vec is filtered to
is_some() so the fallback can never fire.
- Reorder the face_dao constructor param + initializer to match the
struct declaration (between tag_dao and knowledge_dao). Update both
state.rs call sites and populate_knowledge.rs to match.
- Hold face_dao lock once across the library-resolver loop instead of
reacquiring per iteration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The LLM had no path to see face_detections data — get_file_tags
returns user-applied tags, but a face that's been detected and bound
to a person via the embedding-cluster auto-bind path doesn't always
have a matching tag. The new tool joins face_detections with persons
by content_hash and returns bound names + bboxes, plus unidentified
faces (so smaller models can count people in the photo without
inferring from a visual description).
Gated on face_detections being non-empty via the same has_any_*
pattern as daily_summaries.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The bare 'ffmpeg -ss 3 -i in -vframes 1 -f image2 out' command failed on
sources whose decoded pix_fmt isn't yuvj420p (e.g. older Samsung phone
videos in yuv420p). With no -vf filter chain, the decoded frame goes
straight to the mjpeg encoder, which rejects it with 'Non full-range
YUV is non-standard' and exits non-zero.
generate_image_thumbnail_ffmpeg already handles the same class of
source for HEIC/RAW by adding -vf scale=200:-1 -c:v mjpeg — the filter
chain lets ffmpeg auto-insert the pix_fmt converter the encoder needs.
Adopt the same args here. Side benefit: video thumbnails are now 200px
wide on disk, matching image thumbnails (previously full-resolution).
Pre-existing .unsupported sentinels for videos that hit this failure
will need to be deleted manually to retry — they're under
$THUMBNAILS/<lib_id>/.../*.unsupported.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
No behavior change — purely whitespace/line-break cleanup that had
accumulated since the last format run.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
generate_video_thumbnail used .output().expect(...), which only catches
spawn failure — non-zero ffmpeg exits were silently discarded. With no
thumbnail and no .unsupported sentinel left behind, the watcher
re-detected the file as missing every quick-scan tick and re-logged
"New file detected (missing thumbnail)" forever.
Mirror the image branch: return io::Result, check status.success(),
and write the sentinel from create_thumbnails on failure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add system_prompt to the /insights/chat body schema with a one-paragraph
note on the append-vs-amend semantics so future readers find the
contract alongside the rest of the chat-continuation docs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The post-PR-4 delegation kept it as a convenience for callers that
don't filter by contact, but nothing actually uses it. Delete to clear
the dead_code warning. search_messages_with_contact remains as the
single entry point.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- search_messages now delegates to search_messages_with_contact(.., None)
so the two methods share a single HTTP path. Drops the dead-code
warning and the ~30-line duplication.
- DailySummaryDao gains has_any_summaries (LIMIT 1 existence probe)
used by current_gate_opts; the SELECT COUNT(*) get_total_summary_count
added in the prior commit is removed (it had no other caller).
- current_gate_opts doc comment corrected to describe what the probes
actually do.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tools whose backing tables are empty (calendar, location_history,
daily_summaries) drop out of the catalog so the LLM doesn't waste
iteration budget calling them only to receive "no results found".
Vision and apollo gates already existed; this generalizes the pattern.
search_messages gains start_ts/end_ts/contact_id filters (date filter
is a client-side post-filter; SMS-API only accepts contact_id natively
on the search endpoint).
Descriptions follow a consistent convention: one sentence (what +
when), param semantics, examples for tools with non-obvious param
choices. No more all-caps headers, no more identity-prescriptive
language inside descriptions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Bind effective_radius once in fetch_messages_for_contact so the log
output and window math share a single source of truth for the clamp.
- Clamp tool-supplied days_radius to [1, 30] at the tool boundary so a
runaway LLM value can't produce a thousand-day window.
- Split the negative-input test into a real negative-input case
alongside the zero-input case.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The agentic tool definition advertised a days_radius parameter but
sms_client::fetch_messages_for_contact was hardcoded to ±4 days,
silently ignoring whatever value the LLM chose. Plumb the parameter
through; default 4 retained at the tool level for back-compat.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Trim the override input once via Option::map(str::trim).filter(...).
- Use matches!() in restore_system_prompt_override's Prepended arm so
it reads consistently with the Replaced arm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Append mode: applied ephemerally — original system message restored
before persistence so re-opens see the baked persona. Amend mode:
override stays in place and becomes the new insight row's system
message. Pattern mirrors annotate_system_with_budget.
Adds system_prompt field on both ChatTurnHttpRequest and ChatTurnRequest;
plumbs through chat_turn and chat_turn_stream identically.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Use Option::map instead of manual match-on-Option (drops clippy::manual_map).
- Drop redundant `max_iterations = max_iterations` from the format! call.
- Use captured identifiers consistently in the user_content format!.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The framework no longer asserts "you are a personal photo memory
assistant" alongside a user-supplied custom_system_prompt — the
persona is the authoritative identity. The procedural block (tool-use
guidance, iteration budget) stays identity-free.
The user message also stops asking for "a detailed insight with a
title and summary" since the title is regenerated post-hoc anyway and
the wording was constraining voice for no data-model benefit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lays out the cycle: split generation system prompt into identity vs
procedural blocks so personas drive voice/shape, add per-turn
system_prompt override on chat (ephemeral in append mode, persisted
on amend), gate optional tools on data presence, and fix the
days_radius bug in get_sms_messages.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
For QuickTime/MP4 files whose `moov` atom sits at the end of the
file (non-faststart — common for Snapchat exports and any MP4
muxed without `-movflags +faststart`), `-fast2` causes exiftool
to skip the trailer and return no `CreateDate` /
`MediaCreateDate`, dropping the resolver to the `fs_time`
fallback for files that actually have a real capture date.
Reported cases:
Snapchat-477624257.mp4
fs_time: 2026-05-04 (today, file was just modified)
real: QuickTime CreateDate 2018-09-02
action_compound_cc92e65b709d1deb895b4c2a9484fc6a.mp4
fs_time: 2026-05-04
real: MediaCreateDate 2018-03-01
The waterfall pre-filters to files kamadak-exif couldn't read, so
the JPEG fast-path is already covered without `-fast2`. Paying
full-scan cost on the residual is the right trade. The per-tick
drain re-resolves `source = 'fs_time'` rows, so existing rows
recover automatically on the next watcher tick after deploy — no
SQL migration needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Snapchat assigns sequential IDs that happen to overlap real epoch
values, so the 10-16 digit timestamp regex matched and produced
2002-era dates for files actually saved in 2016/2021. The digits
themselves are indistinguishable from a unix timestamp, so we
dispatch on the source-app prefix instead. Case-insensitive,
extensible for future apps that exhibit the same pattern.
Reported cases:
Snapchat-1021849065.mp4 → 2002-05-19 (actual 2021)
Snapchat-1751031586660373917.jpg → 2002-09-09 (actual 2016)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Filenames like `000227580005.jpg` (film-scan ID) and
`IMG_21323906751390.jpeg` were matched by the 10-16 digit timestamp
regex and resolved to 1970 / 2037, then written into
`image_exif.date_taken` with `source = 'filename'`. EXIF-less
photos showed up under those bogus dates everywhere date_taken is
read.
Two new guards in `extract_date_from_filename`:
- leading zero → reject (real epoch values don't have one at any
sane resolution).
- resolved year outside [1995, now+1y] → reject.
Both let the date_resolver waterfall fall through to fs_time,
which is a much better proxy for content age than a fake epoch
date. Regression tests cover the two reported filenames.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The DAO swallowed every diesel::update failure as a flat
`anyhow!("Update error")`, then trace_db_call further reduced it to
`DbError { kind: UpdateError }`. Operators saw "update failed for lib
2 Snapchat/foo.mp4: DbError { kind: UpdateError }" with no clue why
(constraint violation? type mismatch? row vanished mid-flight? DB
locked?).
Two changes:
- Preserve the diesel error in the anyhow chain along with the input
params (lib, rel_path, date_taken, source) so the cause is visible.
- Log the chain at warn-level inside the DAO before the trace wrapper
collapses it to DbErrorKind::UpdateError, so the warning at the
call site finally has something diagnosable next to it.
- Treat zero-row updates as a debug-level "row likely retired by the
missing-file scan" rather than a hard failure — that case is benign
and shouldn't poison the drain's error tally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`extract_date_from_filename` was calling `Local::from_local_datetime`
on the parsed YYYY-MM-DD-HH-MM-SS components, then `.timestamp()` was
shifting the result by the SERVER's TZ offset to produce real UTC
seconds. That made filename-sourced timestamps disagree with EXIF-
sourced timestamps by hours: kamadak-exif's `DateTimeOriginal` is a
naive string parsed AS-IF-UTC (the project's load-bearing
"naive local reinterpreted as UTC" convention), and Apollo's photo
matcher re-anchors that naive value through the BROWSER's TZ when
matching to the track. Anything stamped in server-local instead got
double-shifted on its way through the matcher and through any
`formatNaive*` display path on the client.
Visible symptom in the Apollo DETAILS modal: a photo's CURRENT date
read correctly (1:25 AM via exif) while FROM FILENAME read 4 hours
ahead (5:25 AM in EDT) for the same `IMG_20160710_012515.jpg`.
Switch to `Utc::from_utc_datetime` so `.timestamp()` returns the
wall-clock-as-UTC unix seconds — same convention as the EXIF path.
The /memories endpoint, the canonical-date waterfall (which feeds
`image_exif.date_taken` for filename-only files), and Apollo's
DETAILS modal `filename_date` field all now line up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The curated `image_exif` columns are a small slice of what exiftool
can read (camera/lens/GPS/capture/dates). Apollo's DETAILS modal wants
to surface everything — white balance, metering, MakerNotes, IPTC,
ICC profile, Composite tags, the lot — for an operator inspecting a
photo's provenance.
`read_full_exif_via_exiftool(path)` shells out to `exiftool -j -G -n`:
JSON output, group-prefixed keys (`EXIF:Make`, `MakerNotes:LensInfo`),
numeric values (callers can reformat). Spawned via web::block to keep
it off the actix worker — RAW with rich MakerNotes can take a few
seconds.
The endpoint is on-demand only; the indexer / file watcher does NOT
call it. Falls back to 503 with a clear message when exiftool isn't
on PATH so Apollo can render an "install exiftool" hint. Multi-library
union resolution mirrors set_image_gps / get_file_metadata.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add `POST /image/exif/date` and `POST /image/exif/date/clear` so an
operator can correct a row whose canonical-date waterfall landed on the
wrong value (camera clock reset, fs_time fallback for a copied-from-
backup file, etc). New `original_date_taken` / `original_date_taken_source`
columns snapshot the prior value on first override so revert is lossless.
The waterfall source set is now `'exif' | 'exiftool' | 'filename' | 'fs_time' | 'manual'`.
The existing `idx_image_exif_date_backfill` partial index already filters
to `date_taken IS NULL OR date_taken_source = 'fs_time'`, so manual rows
are naturally excluded from the per-tick drain — no index change needed.
`ExifMetadata` now exposes `date_taken_source` + originals so a UI can
render "manually set; was X via filename".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The HTTP/protobuf exporter never sent any traffic in prod (tcpdump
on port 4318 showed nothing) despite the receiver path being correct
and the bridge wiring being intact (logs reached journalctl via the
stdout exporter). Likely the BatchLogProcessor + reqwest-client combo
isn't getting the right runtime context, but debugging that on a live
deployment isn't worth holding up the rest of the speedups.
Restoring grpc-tonic transport so prod observability comes back. The
remaining build-time wins on this branch (mold linker, system sqlite3,
profile.dev tweaks, lockfile-only dep refresh) deliver most of the
original savings without touching telemetry. Operator: revert
OTLP_OTLS_ENDPOINT in prod from port 4318 back to 4317.
HTTP transport remains a viable follow-up — needs to be debugged
against a local SigNoz instance with internal SDK error visibility
enabled, on its own branch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop libsqlite3-sys 'bundled' on Linux/macOS so the SQLite C source
isn't recompiled every clean build; Windows keeps 'bundled' via a
cfg(windows) target override.
- Switch opentelemetry-otlp from grpc-tonic to http-proto + reqwest-client.
Removes the tonic + h2 + hyper-h2 stack from the build graph; reqwest
was already a dependency. Updates otel.rs to call .with_http().
- Add [profile.dev] debug = "line-tables-only" to shrink linker work
while keeping panics/backtraces useful.
- Add .cargo/config.toml selecting mold via gcc on x86_64-linux-gnu.
Requires `apt install mold`. Other platforms use the default linker.
- cargo update: lockfile-only refresh of all minor/patch bumps within
existing version constraints.
Cold debug build: ~1m 37s; touch-one-file rebuild: ~5s on Linux.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pre-resolver rows already had a populated `date_taken` from the old
kamadak-exif-only ingest path. The column-add migration left their
`date_taken_source` as NULL, and the drain's eligibility predicate
(`date_taken IS NULL OR date_taken_source = 'fs_time'`) skips them —
so they remain unlabelled forever and never benefit from the
resolver's exiftool fallback even if they're videos that should
upgrade.
Label them all `'exif'` in a one-shot UPDATE. Safe because every
write path that populated `date_taken` before the resolver landed was
a kamadak-exif read. Idempotent (the WHERE matches nothing on a
second run). Down.sql is a no-op — the labels stay correct under any
schema state, and the column-add migration is the right place to
revert if needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the EXIF-loop + WalkDir-fallback pipeline that powered
`/memories` with a single per-library SQL query
(`get_memories_in_window`) that uses `strftime('%m-%d' | '%W' | '%m',
date_taken, 'unixepoch', tz_offset)` for calendar matching in the
client's timezone, plus a `years_back` lower bound and a
no-future-dates upper bound. Returns only the matching rows; the
handler applies per-library `PathExcluder` post-query and sorts.
Drops:
- `collect_exif_memories` — replaced by the single SQL query.
- `collect_filesystem_memories` — the canonical-date pipeline now
populates `date_taken` for every row at ingest, so the WalkDir
fallback that scanned 14k+ files each request is no longer needed.
- `get_memory_date_with_priority` and friends — request-time waterfall
superseded by `date_resolver` running at ingest. The associated
three priority-tests are dropped; their replacement lives in
`date_resolver::tests`.
On a ~14k-file library this drops `/memories` from 10–15 s
(dominated by `fs::metadata` per row) to single-digit ms.
Bumps `DEFAULT_YEARS_BACK` from 15 → 20 to surface deeper archives
on matching anniversaries.
Note vs. ISO weeks: the original Rust used `chrono::iso_week().week()`
for week-span matching. SQLite's `%W` is Monday-anchored but uses week
0 for days before the first Monday, so it can disagree with ISO at
year boundaries by ±1. Acceptable for nostalgia browsing.
Adds 3 new DAO tests covering month-span filter, library scoping, and
the unknown-span-token guard. Also adds a CLAUDE.md section describing
the canonical-date pipeline end-to-end and the new
`DATE_BACKFILL_MAX_PER_TICK` env var.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds two ExifDao methods (`get_rows_needing_date_backfill` /
`backfill_date_taken`) and a `backfill_missing_date_taken` watcher pass
that runs on every tick alongside `backfill_unhashed_backlog`.
The drain queries the partial index for rows where `date_taken IS NULL`
or `date_taken_source = 'fs_time'`, batches up to
`DATE_BACKFILL_MAX_PER_TICK` paths (default 500), and feeds them through
`date_resolver::resolve_dates_batch` — a single exiftool subprocess
covers the whole tick. Rows that newly resolve to `exiftool` /
`filename` / `fs_time` get persisted via `backfill_date_taken` (touches
only `date_taken` + `date_taken_source` so EXIF / hash / perceptual
columns survive).
`filename`-sourced rows are intentionally not re-resolved — the regex
is authoritative when it matches and re-running exiftool wouldn't
change the answer. Files that have disappeared from disk are skipped
so a ghost row doesn't loop through the drain forever; the
missing-file scan in `library_maintenance` retires those separately.
Comes with two DAO unit tests (eligibility filter + column-isolation).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires `date_resolver::resolve_date_taken` into the three call sites
that build `InsertImageExif`:
- `process_new_files` (file watcher) — every newly-registered file gets
the resolver's verdict so videos and EXIF-stripped images land with a
real date instead of NULL.
- Upload handler — same waterfall on the post-multipart-write path.
- GPS-write handler — re-runs the waterfall after exiftool writes GPS
and re-reads the EXIF, in case a previously fs_time-sourced row now
has a real EXIF date to upgrade to.
This is a behavior change vs. the pre-rewrite `/memories` request-time
priority: EXIF now beats filename when both are present. A photo
named `Screenshot_2014-06-01.png` whose EXIF `DateTime` is 2021 now
appears under 2021. The reverse case (no EXIF, parseable filename) is
unchanged and continues to surface the filename date with
`date_taken_source = 'filename'`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New module that consolidates the four-step ingest waterfall:
kamadak-exif (already in process via the caller's prior result) →
exiftool fallback → filename regex → earliest_fs_time. Each step is
tagged with a `DateSource` so the caller can persist provenance.
The exiftool fallback is what makes videos and MakerNote-hosted dates
land at all — kamadak-exif can't read QuickTime/MP4 or Nikon-style
sub-IFDs. Single-file mode shells out per call; batch mode pipes paths
on stdin via `-@ -` and fans the result through one subprocess so the
upcoming per-tick drain doesn't pay startup cost per row. The
`exiftool` PATH check is cached in a `OnceLock` to keep the drain
short-circuited on deploys without exiftool installed.
`SubSecDateTimeOriginal` and `ContentCreateDate` are pulled alongside
the standard tags to capture iPhone's sub-second precision and Apple's
preferred capture-time tag respectively. `FileModifyDate` is
deliberately *not* in the tag list — it's a filesystem-derived value
the resolver already covers via the `fs_time` step, and pulling it
through exiftool would mask "no real EXIF date" with a misleading
`source = exiftool` row.
Module is registered in both `lib.rs` and `main.rs` (sibling-module
pattern the rest of the bin uses); no callers wired in yet — that
lands in the next commit. Comes with 9 unit tests covering JSON
parsing edge cases, source-priority short-circuiting, and the
fs_time-when-no-exif path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New nullable TEXT column tracks which step of the canonical-date
waterfall (kamadak-exif → exiftool → filename → fs_time) populated
`date_taken`. Lets a later per-tick drain re-resolve weak sources
(`fs_time`) once stronger ones become available, and gives the UI/debug
surface a way to answer "why does this photo show up under this date?".
Adds the column at all `InsertImageExif` construction sites with `None`
placeholders (the resolver wiring lands in a follow-up commit), and
extends the `update_exif` SET tuple so the column survives the GPS-write
re-read path. Partial index `idx_image_exif_date_backfill` is created
for the upcoming drain query.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bucket exact-dup rows by (library_id, dirname) pair on each side, then
filter by coverage = shared / min(folder_a_total, folder_b_total) and
an absolute floor on shared count. Surfaces "this folder is mostly
contained in that folder" matches that the per-file EXACT view buries
under one row each — e.g. an old phone-backup tree shadowing the
organized library, or a topic-grouped folder duplicating a date-grouped
one within the same library.
New endpoint: GET /duplicates/folder-pairs?library=&include_resolved=
&min_coverage=&min_shared=. Cached 5 min keyed on (library, include_resolved);
the user-tunable thresholds filter the cached unfiltered pair list so
slider drags don't re-bucket. Shares the resolve / unresolve flow with
the existing tabs — the frontend fans out N parallel /resolve calls,
one per shared content_hash.
Folder names carry no signal (BMW lives under Night Photos, not BMW_backup),
so bucketing is purely on (library_id, dirname) co-occurrence in
exact-dup groups. Within-folder dups (same hash twice in the same
folder) are skipped — those belong to the EXACT tab.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pass -P to exiftool so write_gps doesn't bump the file's modification
time. For phone photos with no embedded EXIF datetime, the filesystem
mtime is often the only timestamp we have — losing it on every GPS
backfill would be data loss.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Soft-marked rows used to disappear from /photos globally, including
from a library-scoped view that didn't contain the survivor at all.
A user browsing lib A who'd promoted a file from lib B as the
survivor would silently lose visibility on their own copy in lib A,
even though lib B's file isn't reachable from lib A's view.
Library-scoped queries now keep a demoted row visible when its
survivor lives in a library outside the current scope. Implemented
as a NOT EXISTS subquery against the same image_exif table aliased
as `survivor`. The unscoped (all-libraries) view is unchanged — every
survivor is reachable, so demoted rows stay hidden as before.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three changes against "still too loose at lowest sensitivity":
- Popcount entropy band tightened from [8, 56] to [16, 48]. The wider
band let too much low-frequency content through (skies, scans,
faded film) where pHash collapses to near-uniform values that
Hamming-trivially across hundreds of unrelated images.
- dHash check now uses an asymmetric stricter threshold
(dhash_threshold = max(2, threshold/2)). pHash is the candidate-
discovery signal; dHash is validation. Splitting the budget means
a real near-dup survives both while incidental pHash collisions
on uniform content get vetoed. Missing dHash on either side now
rejects the edge (was: trust pHash alone).
- Single-link union-find can chain weakly-similar images via
transitive edges. Added a medoid-validation pass: per cluster,
pick the member with smallest summed distance to others, then
drop any whose distance to it exceeds threshold. Two new tests
pin both invariants.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The perceptual cluster was producing one giant first group that
contained hundreds of unrelated images. Two causes:
- Solid-colour images (skies, black frames, monochrome scans) all
hash to near-zero pHashes that Hamming-distance-zero to each other.
- Single-link clustering on pHash alone is too permissive — a chain
of weakly-similar images all collapses into one cluster.
Fixed by skipping hashes outside the popcount [8, 56] band (uniform
content) and requiring dHash agreement within threshold before
unioning a candidate edge from the BK-tree. Two new tests pin both
invariants.
Backfill bin separately fix: decode-failed rows kept phash_64=NULL
and got re-pulled by every batch, infinite-looping on a queue of
unbreakable formats. Persist a 0/0 sentinel on decode failure so
the row leaves the candidate set; the all-zero hash is excluded
from clustering by the same entropy filter so it doesn't pollute
results.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds pHash + dHash columns alongside the existing blake3 content_hash so
near-duplicates (re-encoded, resized, format-converted copies) become
queryable. /duplicates/{exact,perceptual} return groups; /duplicates/
{resolve,unresolve} flip a duplicate_of_hash soft-mark on losing rows
and union perceptual-only tag sets onto the survivor. The default
/photos listing filters duplicate_of_hash IS NULL so demoted siblings
stop cluttering the grid; include_duplicates=true opts back in for
Apollo's review modal. Upload now hashes bytes pre-write and returns
409 with the canonical sibling when a file's bytes already exist.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
list_embeddings cloned the full FaceDetectionRow inside the filter_map
just to pair it with the base64-encoded embedding. The 2 KB BLOB was
already on the row — at 20k unassigned faces that's 40 MB of pointless
heap traffic per Apollo cluster-suggest run. Move the bytes out via
Option::take() so the row drops the BLOB instead of duplicating it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The two entry shapes for libraries.excluded_dirs / EXCLUDED_DIRS
are not symmetric:
- /sub/path → multi-segment, library-root-anchored, recursive
- name → single component anywhere in the tree
Without this pinned, a reasonable read of the column doc would be
"any path-like string works" — but a multi-segment string without a
leading slash silently never matches (the no-slash form scans path
components for exact string equality, and components are
slash-free).
No code change; just documentation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a nullable comma-separated TEXT column to the libraries table.
Effective excludes for a walk = (env-var globals) ∪
(library.excluded_dirs). Empty / NULL = no library-specific
extras; the global env var still applies.
Migration (2026-05-01-110000_libraries_excluded_dirs)
ALTER TABLE libraries ADD COLUMN excluded_dirs TEXT. NULL on every
existing row — no behavior change on upgrade.
Library struct + helpers (libraries.rs)
- Library gains excluded_dirs: Vec<String>, parsed from the column
by parse_excluded_dirs_column (drops empties / whitespace,
matches the env-var parser).
- Library::effective_excluded_dirs(globals) returns the union.
- From<LibraryRow> hydrates the field on AppState construction so
/libraries surfaces it.
Watcher / walkers / memories
Every per-library walker now consults the effective set:
- process_new_files (file-watch ingest, RAW/EXIF/face)
- process_face_backlog (filter_excluded inherits)
- create_thumbnails (startup + new-file branch)
- update_media_counts (Prometheus gauge)
- cleanup_orphaned_playlists (per-library source-existence check)
- memories endpoint (PathExcluder)
Effective set is computed once per per-library iteration in the
watcher tick and threaded through; called functions retain their
flat &[String] signature (no per-library awareness needed inside
the walker primitives).
Use case: mount a parent directory while a sibling library covers
a child subtree, and exclude the child subtree from the parent so
the libraries don't double-walk / double-write image_exif. With
hash-keyed derived data (Branches B/C), the duplication-avoidance
is the only cost prevented — face / tag / insight sharing was
already correct via content_hash.
Tests: 228 pass (226 from previous + 2 new in libraries::tests:
parse_excluded_dirs_column edge cases,
effective_excluded_dirs_unions_global_and_per_library).
CLAUDE.md gains a "Per-library excludes" subsection of the
multi-library data model.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A small follow-up to Branches A/B/C. Adds a nullable-default-1
boolean column to the `libraries` table that controls whether the
watcher considers the library at all. Useful for staging a new
mount before committing to ingest, and as a maintenance kill
switch when a library needs to be quiet without being unmounted.
Migration (2026-05-01-100000_libraries_enabled_flag)
ALTER TABLE libraries ADD COLUMN enabled BOOLEAN NOT NULL DEFAULT 1.
Existing rows stay enabled — no behavior change on upgrade.
Watcher gate (main.rs)
At the top of the per-library loop, if !lib.enabled { continue; }
— runs BEFORE the availability probe. Disabled libraries don't
enter the health map, don't get probed, don't get ingest, don't
get any maintenance pass. The initial sweep before the loop's
first sleep also skips disabled libraries.
Orphan-GC consensus (library_maintenance.rs)
all_libraries_online filters disabled libraries out of the
consensus check — they're treated as out-of-scope, not as
blockers. Otherwise flipping enabled=false would permanently
halt orphan GC for the rest of the system, which is the opposite
of the intended kill-switch semantics.
Cross-library duplicates: safe by construction. Hash-keyed derived
data (face_detections, tagged_photo with hash, photo_insights with
hash) is anchored by ANY image_exif row carrying the hash. Disabling
a library does NOT delete its image_exif rows, so a hash referenced
by a disabled library's row stays anchored — derived data survives.
collect_orphan_hashes deliberately doesn't filter image_exif by
library.enabled for exactly this reason.
No HTTP endpoint. Library mutation is rare-enough infra work that a
SQL toggle is fine, and a public mutation endpoint without a role /
permission story would be poorly-prioritized exposure for a
single-user tool. Documented in CLAUDE.md.
Tests: 226 pass (225 from Branch C + 1 new
all_libraries_online_treats_disabled_as_out_of_scope, which proves
that even an explicit Stale entry on a disabled library doesn't
block the consensus).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
"marked 2 new" parses as "2 new files" on first read — but the
unit is content_hashes, and the action is observing them as
orphaned (becoming-deleted, not appearing). Reword:
"{} new orphan hash(es) marked, {} revived"
instead of "marked {} new, revived {}". Also pluralize the deleted
counts ("row(s)") and append the pending-set size to the success
log so a tick that both deletes and re-marks doesn't lose the
trailing-state context.
No behavior change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
run_orphan_gc returned early on the !all_online branch before the
final debug/info log line, so the GC was effectively invisible
whenever any library was Stale — exactly the dry-run scenario where
operators most want to confirm the safety gate is firing. Add the
same conditional log inside the early-return branch (plus a
"deferred — at least one library Stale" hint in the info-level
variant when there's something newly marked).
No behavior change beyond observability.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
