Commit Graph
852 Commits
Author SHA1 Message Date
cameron ab16bc7da6 Merge pull request 'feat(dates): pre-cutover audit utility for the filename date class (#117)' (#123) from feature/filename-audit into master
Reviewed-on: #123
2026-09-26 14:32:12 +00:00
Cameron Cordes 89fc24ceab feat(dates): print the numeric offset in the audit report
The spec-mirroring review noted the report showed the offset SOURCE
but not the offset value — the value is what the operator spot-checks.
Each proposed line now ends with offset={+N}m (or - for utc_direct
rows). Tests pin the rendered values.
2026-09-25 18:14:02 -04:00
Cameron Cordes 3aa6ce314c feat(dates): pre-cutover audit utility for the filename date class (#117)
One-shot, operator-run, strictly read-only utility that lists every
filename-class file with its stored date/source and the date/source/
offset-source the re-based resolver would assign it — the human
spot-check that gates the production backfill drain (#118).

- src/filename_audit.rs: StoredDate/AuditRow/SkippedLibrary; walk +
  PathExcluder (depth-0 guarded) + match_filename_date -> one
  resolve_dates_batch pass with EMPTY priors (mirrors the drain
  exactly); disabled and probe-stale libraries reported as skipped,
  never partially processed; output stable-sorted by (library, path).
- src/bin/filename_audit.rs: operator binary (dotenv, DATABASE_URL,
  EXCLUDED_DIRS, libraries::load_all, format_report).
- src/memories.rs: match_filename_date canonical matcher returning
  FilenameMatch { pattern, date }; extract_date_from_filename is now
  a thin wrapper (byte-for-byte behavior-preserving, verified by
  review).

4 audit tests (inversion + utc_direct + exclusions, offline/disabled
skip, stable sort, report determinism incl. no debug quotes) + 2
matcher tests. 645 lib + 718 integration pass; clippy/fmt clean.
Standards review applied: {:?}->{}, depth-0 walk guard, dead
per-file exclude check removed, unused Default derive removed,
pattern column widened, doc comment reworded.
2026-09-25 18:03:38 -04:00
cameron 3ac011abd7 Merge pull request 'feat(dates): filename date class — per-pattern conventions + filename-over-EXIF priority (#116)' (#122) from feature/filename-date-class into master
Reviewed-on: #122
2026-09-25 21:16:53 +00:00
Cameron Cordes a97bf53603 feat(dates): filename date class — per-pattern conventions + filename-over-EXIF priority (#116)
The filename date class is now first-class (ADR-0001):

- memories::extract_date_from_filename returns FilenameDate:
  Wall(NaiveDateTime) for the screenshot/dash/compact patterns,
  Utc(DateTime<Utc>) for the unix-timestamp patterns. No patterns
  added or removed; every existing pattern carries its convention.
- Waterfall inversion in both resolve_date_taken and
  resolve_dates_batch: an in-class name is the original date of
  record and resolves BEFORE EXIF / container dates are consulted
  (shared filename_class_resolved step; in-class files skip
  exiftool entirely). Wall patterns run the offset cascade over the
  kamadak prior's evidence (empty evidence -> server-zone
  assumption, so a class file always resolves); timestamp patterns
  are true instants (tz_source utc_direct, no offset).
- Legacy date_taken stays byte-identical: wall-as-UTC for wall
  patterns, epoch for timestamp patterns, via
  FilenameDate::legacy_datetime / legacy_timestamp at the seven
  consumer call sites.
- Out-of-class files keep the global EXIF-wins waterfall unchanged.
- CLAUDE.md waterfall section rewritten (step 0 = filename class;
  the EXIF-beats-filename note becomes out-of-class-only).

Tests: five new waterfall-seam tests (wall no-evidence ->
assumed_server; wall outranks EXIF with offset-tag evidence;
wall GPS evidence; timestamp utc_direct; in-class video name
resolves without exiftool) plus enum-shape assertions in every
extractor test. Code-reviewed on both axes (standards: no
violations, 3 of 4 P2 notes applied; spec: all acceptance
criteria verified, no scope creep).

716 integration + 639 lib tests pass; fmt + clippy clean.
2026-09-25 16:54:46 -04:00
Cameron Cordes 6f73a31e08 docs(tracker): correct the Gitea native-dependency API call
The wayfinding doc pointed at a .../dependencies/blocked_by subpath
that 404s, and the working call needs the blocker's owner+repo in the
JSON body (omitting them 404s with 'repository does not exist').
Verified against Gitea 1.27: POST/DELETE .../issues/<child>/dependencies
with {"index", "owner", "repo"}.
2026-09-25 16:17:17 -04:00
cameron 467abd41d5 Merge pull request 'feat(dates): true-instant date contract — container dates, fs branch, manual edit (#114, #115)' (#121) from feature/true-utc-dates into master
Reviewed-on: #121
2026-09-25 19:55:14 +00:00
Cameron Cordes 7d7d7c713c refactor(dates): condense the two date-contract migrations into one
Neither migration had shipped, so the #115 snapshot columns join the
#114 new-contract columns in 2026-09-24-000000_add_true_utc_dates.
Pure ADD COLUMN either way — population is runtime (drains), so nothing
needs re-migrating. Column order is unchanged, so schema.rs stands.
2026-09-25 15:39:23 -04:00
Cameron Cordes d2263be6ac feat(dates): container video dates, filesystem branch, manual edit (#115)
The exiftool branch of the date waterfall now pulls raw strings with
group prefixes (one `exiftool -j -q -G1` subprocess per batch, no
`-d %s`) and classifies each candidate:

- zoned strings (own offset / Z) are true instants;
- naive strings from container groups (QuickTime/Track*/Apple) are
  UTC-rendered creation_time fields -> true instants (utc_direct).
  Running them through the camera wall-clock cascade was the
  "videos off by 4-5 hours" bug: it reinterpreted a UTC instant in
  the server zone;
- naive strings from photo-EXIF groups are camera wall clocks and run
  the offset cascade (OffsetTime* pairing + GPS evidence pulled in
  the same exiftool pass).

The legacy date_taken column stays byte-identical to the old
server-local interpretation (reproduced in Rust, DST-aware); the
new-contract date_utc / camera_offset_minutes / date_taken_tz_source
columns carry the true instant.

Manual date edit (POST /image/exif/date) reinterprets the
naive-as-UTC epoch the client sent as a wall clock in the user's
display zone (camera_tz::manual_date_resolution, tz_source =
'manual'), snapshots the pre-override new-contract triple next to the
existing legacy snapshot pair (3 new columns, migration
2026-09-25-000000), and revert restores both pairs.

ADR-0001 records the container-date finding; CLAUDE.md date-pipeline
section updated.
2026-09-25 15:26:26 -04:00
Cameron Cordes 6f020f4201 docs(utils): make the retry_with_backoff_if doctest actually compile
The example referenced an unimported DbError whose type didn't even
match the closure's anyhow::Error, and the classifier body returned
(). Rewritten as a self-contained io::Error example with an assertion.
2026-09-24 22:04:56 -04:00
Cameron Cordes 2f2107d4fa feat(dates): foundation for the true-instant date contract (#114)
Populate the new-contract date columns at ingest while keeping the
legacy date_taken column byte-identical (ADR-0001):

- camera_tz module: offset cascade (offset tag -> GPS zone -> server
  zone) as a pure, testable function; bundled offline TZF geometry for
  GPS -> IANA resolution; DST-correct offsets for the capture date;
  display-zone helpers (DEFAULT_DISPLAY_TZ, display_zone_for,
  display_zone_offset_minutes) for the read-side cutover.
- EXIF branch: kamadak-exif now collects OffsetTime* tags paired with
  the winning date tag; resolve_date_taken takes an ExifPrior and
  emits (date_utc, camera_offset_minutes, date_taken_tz_source) per
  branch: exif -> cascade, exiftool/fs_time -> utc_direct, filename
  -> unclassified (pending #116).
- Schema: migration adds image_exif.date_utc /
  camera_offset_minutes / date_taken_tz_source and
  user_ai_prefs.display_tz (all nullable; down.sql drops them).
- DAO + ingest: backfill_date_taken persists the triple alongside the
  legacy pair; watcher, upload, and set_image_gps call sites feed the
  cascade.
- Response surface: ExifMetadata and ExifSummary carry the triple so
  /image/metadata and /photos/exif expose it (null for unconverted
  rows); reel requests mirror display_tz into prefs.

cargo build + clippy clean; 625 lib + 702 bin tests pass (the one
failing doctest in utils.rs is pre-existing at the base commit).
2026-09-24 21:58:46 -04:00
Cameron Cordes ee773a34ef docs(dates): record the true-instant date contract
- CONTEXT.md: date/time vocabulary (true instant, wall clock, camera
  offset, display zone, filename date class, ...)
- ADR-0001: dates are true instants; wall clocks are a display concern
- un-ignore docs/adr/ so decisions land in git
2026-09-24 18:23:46 -04:00
cameron bf64d046d8 Merge pull request 'chore(agents): install mattpocock skills + per-repo agent config' (#112) from feature/skills-install into master
Reviewed-on: #112
2026-09-24 17:31:50 +00:00
Cameron Cordes 5b5314631b chore(agents): install mattpocock skills + per-repo agent config
- 38 skills from mattpocock/skills into .agents/skills/ (symlinked into
  .claude/skills/ for Claude Code)
- skills-lock.json for reproducible installs
- docs/agents/: issue-tracker (Gitea via tea CLI), triage-labels
  (default five-role vocabulary), domain docs (single-context)
- CLAUDE.md: add '## Agent skills' section pointing at docs/agents/
- .gitignore: un-ignore docs/agents/ under the local /docs/* rule
2026-09-24 13:24:29 -04:00
cameron 0aeeb2d1f4 Merge pull request 'Feature/agentic loop fixes' (#111) from feature/agentic-loop-fixes into master
Reviewed-on: #111
2026-09-24 15:59:58 +00:00
Cameron CordesandClaude Opus 5.5 23371ad0bc fix(watcher): refresh planner stats on full scans to stop write-lock starvation
sqlite_stat1 was last gathered by a migration while image_exif had 402
rows and an all-NULL content_hash, so the planner treated the hash index
as non-selective and scanned it for every row in the back-ref refresh's
correlated subqueries. The face_detections pass took ~20s per watcher
tick while holding the write lock, so chat/persona-chat persistence blew
past busy_timeout and returned 500s. With fresh stats the three
refreshes drop from ~22s to ~0.1s.

- library_maintenance: add refresh_planner_stats (plain ANALYZE;
  PRAGMA optimize doesn't catch this on SQLite 3.45), run on full-scan
  ticks ahead of reconcile / back-ref refresh.
- utils: retry backoff used 1_u32.pow(..), a flat ~100ms; make it the
  documented exponential 100/200/400ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 09:33:37 -04:00
Cameron CordesandClaude Opus 5.5 33fe7ce77c feat(openrouter): cap max_tokens per request (OPENROUTER_MAX_TOKENS)
OpenRouter reserves credit against the requested max_tokens, falling
back to the model's full output window when omitted, so expensive
models failed with credit-limit errors even for short replies. Default
the cap to 16384; OPENROUTER_MAX_TOKENS overrides it and 0 disables it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-22 21:32:56 -04:00
Cameron Cordes 4e3ee1973c feat(agentic-loop): retry transient knowledge-write locks + re-open tools on loop exhaustion
Transcript analysis (359 insights / ~3.3k tool results) showed 31
"database is locked" errors from knowledge-memory tools and 50/359
runs (14%) exhausting the 6-iteration budget with no final answer.
The exhaustion nudge re-invoked chat with an EMPTY tool list, so a
model that still needed information could not call anything.

- utils: add retry_with_backoff_if — same exponential backoff + jitter
  as retry_with_backoff, but retries only while the predicate classifies
  the error as transient; deterministic failures short-circuit.
- insight_generator: wrap the four knowledge write tools
  (store_fact/upsert_fact, store_entity/upsert_entity, update_fact,
  supersede_fact) plus the store_fact photo-link upsert in
  retry_with_backoff_if gated on a SQLITE_BUSY detector ("database is
  locked"/"database is busy" massage in DbError.source). WAL +
  busy_timeout absorb most contention; a parallel insight run or the
  face-watch write storm still surfaces it, and a short backoff retry
  clears it instead of surfacing a hard error to the model.
- insight_generator: exhaustion nudge now re-offers the full tool set
  instead of vec![] and explicitly says the model may call more tools;
  if it does take one, results are collected and a bounded tool-less
  turn forces the actual answer so we never store an empty insight.
  Same treatment for the read-only tool loop.
- models/knowledge_dao: derive Clone on InsertEntity, InsertEntityFact,
  InsertEntityPhotoLink and FactPatch so retried FnMut closures can
  re-issue the same write value on each attempt.
2026-09-21 19:24:28 -04:00
cameron 3a93e5cae6 Merge pull request 'Feature/open chat with persona' (#110) from feature/open-chat-with-persona into master
Reviewed-on: #110
2026-09-08 21:55:37 +00:00
Cameron CordesandClaude Opus 5 bbaa1f62a4 fix: delete a persona's conversations along with the persona
There is no FK between personas and persona_chat_conversations, and the
chat list skips conversations whose persona is gone — so deleting a persona
left its transcripts in the database, invisible and unreachable, forever.

Both deletes now run in one transaction so the two tables cannot get out of
step.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 18:39:06 -04:00
Cameron CordesandClaude Opus 5 65cceaa67c feat: multiple persona conversations with branching and generated titles
Persona chat stored a flat Vec<ChatMessage> keyed on (user_id, persona_id),
which meant one rolling transcript per persona and no way to revisit a turn.
This moves it onto the same ChatHistoryStore tree the file chat uses and
gives conversations their own identity.

Storage
- messages_json now holds a serialized ChatHistoryStore. Reads accept the
  old flat array and upgrade it in place, so existing transcripts survive
  without a data migration. The upgrade drops the v1 seed greeting, which
  the flat renderer hid but the tree renderer would surface as a bubble the
  user has never seen.
- New migration re-keys persona_chat_conversations on an opaque
  conversation_id and adds title + created_at, so one persona can hold any
  number of separate threads. Every DAO read and write is scoped by user_id
  as well: a conversation id is a bearer token for someone's transcript and
  must never grant access on its own.
- The per-conversation lock and in-flight turn slot key on conversation_id,
  so two threads with the same persona can run turns concurrently.

Endpoints
- POST/DELETE /persona_chat/conversations — start and remove a thread,
  replacing /persona_chat/reset.
- GET /persona_chat/conversations — the chat list, with snippet and counts
  derived from each tree's active branch.
- POST /persona_chat/rewind, POST /persona_chat/switch-branch,
  GET /persona_chat/branches — rewind and fork, mirroring the file chat.
  Index 0 is rewindable here (it is the user's own first question, not a
  synthetic prompt) and re-anchors on the seed node.
- history/turn/rewind/switch-branch/branches all key on conversation_id;
  history gained branch_id and now returns real fork_info, active_leaf_id
  and viewing_branch_id instead of placeholders.

The turn body no longer carries a persona at all — it is read from the
stored conversation, so a stale client cannot swap a thread's voice midway.

Titles
After the first turn persists, the conversation is named from its opening
exchange on the same backend the turn ran on. Small models wrap titles in
quotes, prefix them with "Title:" and append explanations, so sanitize_title
strips all of that and truncates on a character boundary. Any failure falls
back to the user's opening question. Generation runs after persistence: a
failed title must not cost the turn.

Fixes found along the way
- insight_chat: both file-chat turn paths captured path.len() before
  apply_context_budget drained messages out of the middle, then sliced
  messages[path_len..] for the new tree nodes. Once truncation fired that
  dropped the user turn from the tree or panicked on an out-of-range start
  index. Now read after the budget pass as history_len.
- Persona chat had no context budget at all and hardcoded truncated: false
  in the done frame, so a rolling transcript grew unbounded.
- A cancelled turn persisted a half-finished transcript and pushed a second
  terminal frame; it now returns early like the file chat.
- The seeded system prompt was frozen at conversation creation, so editing a
  persona never reached a thread already in flight. Re-resolved per turn.
- turn_count was overwritten each write with the per-turn message delta;
  it is now the cumulative assistant-turn count on the active branch.
- is_initial is always false: the file chat reserves it for its synthetic
  "describe this photo" prompt, and marking a persona chat's first question
  with it made the opening reply impossible to regenerate.

600 lib tests pass, clippy --all-targets clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-25 18:23:10 -04:00
Cameron Cordes 883e2a0e1b fix: send persona_chat wire shapes in snake_case like file chat
The mobile client dispatches POST /persona_chat/turn with snake_case
keys (persona_id, user_message, num_ctx, ...) per the file-chat
convention, but the persona request/response structs carried camelCase
serde renames (personaId, userMessage, ...), so every dispatch 400'd
with 'missing field personaId'. Drop the renames from
PersonaChatTurnRequest, PersonaChatHistoryView, RenderedPersonaMessage,
and PersonaChatResetRequest — the persona wire shapes now match the
rest of the API (history query, 202 turn_id, SSE skip_before) — and
pin the contract with serialization round-trip tests.
2026-08-24 21:34:46 -04:00
Cameron Cordes 3b9d985025 feat: add /persona_chat endpoints for open chat with persona
- PersonaChatSession dispatches turns through the shared agent loop,
  persisting transcripts keyed on (user_id, persona_id) in the new
  persona_chat table (migration included).
- GET /persona_chat/history returns a rendered transcript (tool
  invocations folded, is_initial flag) matching the file-chat shape.
- POST /persona_chat/turn returns 202 with a turn_id; SSE replay and
  cancel reuse turn_replay_impl/cancel_turn_impl, extracted from the
  actix-attributed handlers so both route families share the logic.
- POST /persona_chat/reset clears the persona transcript.
- Concurrent dispatches for the same (user, persona) are rejected with
  409 via an in-flight gate (InFlightPersonaTurns) whose RAII guard
  drops with the spawned turn task, freeing the slot on completion,
  error, or abort.
2026-08-24 21:25:48 -04:00
cameron 649863a700 Merge pull request 'Feature/insight chat branching' (#109) from feature/insight-chat-branching into master
Reviewed-on: #109
2026-08-22 02:06:24 +00:00
Cameron Cordes 4720d5653a feat: apply EXIF orientation and colorspace correction to ffmpeg thumbnails
HEIC/HEIF sources use Display P3 color primaries. Without
colorspace=bt709 the mjpeg encoder treated P3 values as sRGB,
producing warm/oversaturated output. Also bake EXIF Orientation
tag into pixels so saved JPEGs are canonically oriented.

- Extract orientation from exif-reader and pass through all
  ffmpeg thumbnail paths (small, large, xlarge previews)
- Add shared build_image_thumb_filter() for the 200px path
- Add rotation + colorspace=bt709 to large/xlarge ffmpeg paths
2026-08-10 21:15:01 -04:00
Cameron Cordes 5e6bd69405 style: run cargo fmt 2026-08-08 13:26:21 -04:00
Cameron Cordes ef54f0d646 test: add nested fork regression tests
- branch_options_handles_nested_forks: verifies branch picker returns
  correct branches for both root fork and sub-fork
- render_tree_path_nested_forks_only_emits_at_divergence: confirms fork_info
  doesn't propagate downstream (only at actual divergence point)
2026-08-08 10:58:13 -04:00
Cameron Cordes a1a2d22b5b fix: only emit fork chip at actual divergence point
Changed last_fork.clone() to last_fork.take() in render_tree_path
for both assistant and user branches. This prevents fork_info from
propagating to every downstream message, ensuring the chip appears
only at the actual divergence point in the conversation tree.
2026-08-08 10:51:39 -04:00
Cameron CordesandClaude Opus 4.8 9ce7cbf3df Branch previews show the first differing message, not the shared child
Rewind & Regenerate resends the identical question, so both branches'
first child matched and every picker option read the same. Snippets now
walk the branches' visible messages (tool scaffolding skipped) in
lockstep and preview the first position where the contents diverge; a
branch that runs out beforehand falls back to its own last message.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-07 20:07:31 -04:00
Cameron CordesandClaude Opus 4.8 ee8a11d421 Scope branch listings to a divergence point via node_id
ForkInfo now carries the divergence node's id, and the branches endpoint
accepts node_id + viewing_branch_id to return only the position-ranked
sibling branches at that fork (snippets taken from after the divergence,
skipping tool scaffolding) instead of every leaf in the tree. Options in
the same subtree as the viewing leaf anchor to that leaf so clients can
reliably identify "the branch I'm on". Tree-wide listing is unchanged
when node_id is absent.

- llm_client: descendant_leaves + branch_options_at helpers; optional
  position rank on BranchLeafInfo.
- insight_chat: ForkInfo.node_id set during render_tree_path; get_branches
  takes the scoping params.
- handlers: dedicated ChatBranchesQuery; unknown node_id maps to 400.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-07 19:39:57 -04:00
Cameron CordesandClaude Opus 4.8 1512fc5bdc Add conversation branching to insight chat (rewind preserves forks)
Store chat history as a tree in training_messages instead of a flat
array. Rewind now sets active_leaf_id to the target node rather than
truncating, so discarded paths survive as alternate branches. Fork
indicators ("X/Y") mark divergence points and let the client load or
switch to alternate paths.

- llm_client: StoredChatNode/ChatHistoryStore with tree traversal
  helpers (path_to_leaf, children_of, fork_at_node, leaves*, etc.) and
  from_flat_array for backward-compatible reads of the old flat format.
- insight_chat: load_history/chat_turn/rewind_history operate on the
  tree; new switch_branch and get_branches. render_tree_path walks every
  node (including non-rendered tool-dispatch nodes) when computing fork
  info, so a fork whose diverging child is a tool call still surfaces on
  the following rendered message.
- handlers: GET /insights/chat/branches, POST /insights/chat/switch-branch,
  and a branch_id param on the history endpoint.

Backward-compatible: flat arrays are converted to a tree lazily on read.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-07 19:19:20 -04:00
cameron 3732faef71 Merge pull request 'Retry insight persistence with exponential backoff to avoid SQLite lock contention' (#108) from feature/db-locked-fix into master
Reviewed-on: #108
2026-08-07 16:19:41 +00:00
Cameron Cordes 38aaddfd2a Retry insight persistence with exponential backoff to avoid SQLite lock contention
Add retry_with_backoff utility (100ms base, ±25% jitter, 3 attempts)
and replace all 11 direct DAO calls in insight_chat.rs and
insight_generator.rs. The Mutex guard drops between retries so the
maintenance connection can release its SQLite write lock.
2026-08-05 18:22:36 -04:00
cameron 4029625dc0 Merge pull request 'Feature/unified nl search' (#106) from feature/unified-nl-search into master
Reviewed-on: #106
2026-06-29 18:56:58 +00:00
Cameron CordesandClaude Opus 4.8 475072810e AI: add enable_thinking reasoning toggle plumbed to llama.cpp
New optional SamplingOverride forwarded to llama-server as
chat_template_kwargs.enable_thinking (gates Qwen3-style reasoning
blocks). None leaves the template default; other backends ignore it.
Wired through the agentic-insight and chat-turn request bodies/handlers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:39:16 -04:00
Cameron CordesandClaude Opus 4.8 7684220f52 Unified search: use ANY-mode tag matching, not ALL
ALL-mode over-constrains NL queries — the model maps several query words to
tags and few photos carry every one, zeroing the candidate set. Switch to
ANY (a photo matches if it has any named tag); the semantic CLIP ranking
provides precision within that pool. Exclude tags still filter out.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:39:16 -04:00
Cameron CordesandClaude Opus 4.8 b594acc4bd Unified search: rank within filtered set instead of pre-thresholding CLIP
When structured filters are present they're the constraint and CLIP only ranks
within the candidate set, so drop the global similarity threshold for that
case. Previously the 0.2 whole-library threshold ran BEFORE intersecting with
the filters, discarding filter-matching photos that scored just under it (e.g.
a 2022 beach photo at 0.18) — producing after_struct_filter=0 even when matches
existed. Plain semantic (no filters) keeps the user's threshold.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:39:16 -04:00
Cameron CordesandClaude Opus 4.8 e7f6dd56da clip_client: log encode_text failures (URL + status/body or network error)
The CLIP encode failure reason was only ever returned in the HTTP response
body, never logged server-side, making 502s from /photos/search opaque. Log
the underlying cause — network error to the URL, or the Apollo HTTP status +
response body — so CLIP-service problems are diagnosable from the ImageApi log.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:39:16 -04:00
Cameron CordesandClaude Opus 4.8 b18ab6ed99 Unified search: UNIFIED_SEARCH_MODEL env override for the translation step
Pin the NL->structured translation to a small, fast model that can stay
co-resident with CLIP (and the chat model) so it never evicts them on a tight
VRAM budget. Precedence: UNIFIED_SEARCH_MODEL env > client-selected model >
configured default. Logs the effective model (backend.model()) so model A/B
tests are visible. Documented in .env.example.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:39:16 -04:00
Cameron CordesandClaude Opus 4.8 2cde88d734 Unified search: stage-by-stage logging to debug empty results
Log the translated query (semantic/tags/place/date/media + has_struct), the
tag-filter file count, candidate-row + allowed-hash counts, and the CLIP
considered/hits/after-filter counts. Pinpoints which stage drops results to
zero (over-extracted filter, tag path mismatch, Any/All over-constraint, or
CLIP threshold). info-level for now while debugging.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:39:16 -04:00
Cameron CordesandClaude Opus 4.8 8dce536f38 Unified search: accept client model override (avoid model swapping)
Add an optional `model` query param to /photos/search/unified, passed into
resolve_backend's overrides. The client sends the user's currently-selected
local model so the translation step reuses an already-loaded model instead of
forcing a llama-swap eviction + cold start. Falls back to the configured
default when absent. Still local only (no hybrid).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:39:16 -04:00
Cameron CordesandClaude Opus 4.8 fd00dc936a Unified NL search Phase 2: /photos/search/unified endpoint
Composes the two existing engines (Path A orchestration):
- Translate NL -> StructuredQuery via local LLM, respecting LLM_BACKEND
  (resolve_backend(Local) -> ollama or llama-swap; no hybrid).
- Forward-geocode the place name into a gps circle.
- Structured filters (tags/EXIF/geo/date/media) build a candidate set of EXIF
  rows; CLIP ranks within it, joined by content_hash. Degenerate cases match
  existing behavior: semantic-only -> plain CLIP; filters-only -> date-sorted.
- Echoes the interpreted query (incl. resolved place) for editable client chips.

Refactor: extracted reusable cores from clip_search (score_photos, resolve_hits,
parse_library_scope, score_error_response) shared by both endpoints. Removed the
Phase 1 allow-until-wired attributes now that nl_query + geo are consumed.

fmt + clippy clean; 23 backend tests pass (7 geo, 12 nl_query, 4 unified).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:39:16 -04:00
Cameron CordesandClaude Opus 4.8 09ccb347c7 Unified NL search Phase 1: NL→structured-query translator + forward geocoding
Foundation for the /photos/search/unified endpoint (Phase 2). Two new,
fully unit-tested pieces, not yet wired into a route (allow-until-wired,
mirroring llm_client.rs):

- ai/nl_query.rs: translate a free-text query into a StructuredQuery via one
  grounded LLM call. Two-stage — the model emits names/ISO dates, then a pure
  resolve step maps tag names against the real vocab and converts dates to
  unix seconds. Hallucinated (non-vocab) tags are surfaced in unmatched_tags
  rather than silently used as hard filters — the anti-noise guard. 12 tests.

- geo::forward_geocode + bbox_to_circle: resolve a place name to a circle via
  Nominatim /search, collapsing the bounding box to centroid + circumscribing
  radius so "Portland" and "Italy" both map onto the existing gps circle
  filter with no schema change. Radius is the max centroid-to-corner distance
  (corners aren't equidistant on a sphere). 4 tests.

fmt + clippy clean; 19 new tests pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 12:39:16 -04:00
cameron e9d04afd4b Merge pull request 'Feature/memory reels agentic job' (#107) from feature/memory-reels-agentic-job into master
Reviewed-on: #107
2026-06-29 16:38:47 +00:00
Cameron CordesandClaude Opus 4.8 7e21213181 Reels: bound disk/ledger growth (pre-gen prune + on-demand cache sweep)
Nothing reaped reels before, so the on-disk cache and ledger grew
unbounded — each night's daily reel is a new ~4MB file + ledger row that's
stale within ~26h.

- Pre-gen self-prune: after recording a reel, prune_superseded keeps the
  newest PREGEN_KEEP_PER_SCOPE (2) rows per (span, library) and unlinks the
  superseded reels' mp4+sidecar. Caps the ledger/disk at ~spans×libraries×2.
- On-disk sweeper (spawn_reel_cache_sweeper): every 24h, removes reel mp4s
  with no ledger row and no live job older than REEL_CACHE_MAX_AGE_DAYS (7) —
  bounding the on-demand cache, which has no ledger row and otherwise grows
  forever — plus crashed-render cruft (.mp4.tmp/.concat.txt/orphan sidecars).
  Runs regardless of REEL_PREGEN_ENABLED; disable with REEL_CACHE_SWEEP_ENABLED=0.
- New DAO methods prune_superseded + all_cache_keys (with tests); env knobs
  documented in .env.example.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 23:27:32 -04:00
Cameron CordesandClaude Opus 4.8 664b3694f8 Reels pre-gen: always render the agentic reel, don't adopt on-demand mp4
Past the key-aware dedup, any mp4 already at the cache key was not
pre-generated by us (no matching ledger row) — typically an on-demand
fast-scripted reel sharing the key after the max_segments alignment.
Adopting it recorded a ledger row pointing at the fast reel, silently
defeating agentic pre-gen. Drop the adopt-existing-mp4 shortcut and
always produce_reel (atomic overwrite). Worst case is one redundant
re-render if a prior run crashed between render and ledger write.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 23:16:14 -04:00
Cameron CordesandClaude Opus 4.8 b52b1eb323 Reels pre-gen: make dedup cache-key-aware so key changes regenerate
exists_fresh only matched (span, library, render_version, age), so a
cache-key change that doesn't bump RENDER_VERSION (e.g. the max_segments
alignment, or any future selection-logic tweak) left last night's ledger
row looking 'fresh' — the nightly run would skip and the orphaned reel
would persist. Dedup now compares the stored cache_key to the freshly
computed key (and confirms the mp4 exists), so a changed key forces a
regen within the freshness window. exists_fresh stays as the HTTP
endpoint's fast gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 23:14:39 -04:00
Cameron CordesandClaude Opus 4.8 19fc1bbdf8 Reels pre-gen: use DEFAULT_MAX_SEGMENTS so cache keys match on-demand
pregen_one hardcoded max_segments: 24 while create_reel_handler defaults
to DEFAULT_MAX_SEGMENTS (40). Since the cache key encodes the raw
max_segments, the pre-generated reel's key never matched the client's
on-demand request, so POST /reels cache-hit an older max=40 reel and the
agentic pre-gen file was left orphaned. Align to DEFAULT_MAX_SEGMENTS (as
the plan specified) so the on-demand cache-hit path serves the pre-gen
reel. Content is unchanged — the actual beat count is duration-budgeted
either way; only the key descriptor differed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 23:12:54 -04:00
Cameron CordesandClaude Opus 4.8 ca007a618d Reels pre-gen: record true media count + real upsert for user_ai_prefs
- pregen_one recorded media_count as planned.len() (beat count); record
  the actual media item total (media.len(), photos + clips) in both the
  cache-hit and freshly-rendered ledger paths. Drops the redundant
  photo_count binding.
- Replace upsert_prefs's insert-then-catch-error-then-update dance with a
  single atomic INSERT ... ON CONFLICT(id) DO UPDATE. Explicit id=1 makes
  the conflict target deterministic; explicit column .set((...)) keeps
  None -> NULL overwrite semantics so the row mirrors the latest request
  exactly, and genuine insert errors surface instead of being swallowed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 15:19:41 -04:00
Cameron CordesandClaude Opus 4.8 e4d8d374fb Reels pre-gen: fix runtime breakers from review (1-5)
1. Drop the unregistered prefs_dao/reel_dao web::Data extractors from
   create_reel_handler / precomputed_reel_handler and read the DAOs off
   AppState instead (consistent with the scheduler). Missing app_data
   would have 500'd every POST /reels and /reels/precomputed at runtime.
2. Restore the dropped 'return' in the cache-hit branch — without it a
   cache hit fell through, overwrote the Done job with Queued, and
   re-ran the whole TTS+render pipeline on every request.
3. Make secs_until_next_run_hour minute/second-accurate so a batch that
   finishes inside the run hour sleeps ~24h instead of busy-looping
   (wake, re-run, sleep 0) for the rest of the hour. Tests updated.
4. Prune photo/user-bound tools (get_file_tags, get_faces_in_photo,
   recall_facts_for_photo, recall_facts_for_entity) from the agentic
   reel scripter's allow-list — they no-op/error with the empty
   file/user context and only burn iterations.
5. Align AGENTIC_SYSTEM_PROMPT's advertised tool list with the actual
   (pruned) allow-list.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-13 15:14:36 -04:00