ai: restructure agentic user message — facts up top + forcing gate

Small models (~8B) were producing generic responses regardless of persona, and bailing out of the agentic loop on iteration 1. Two underlying causes: 1. Photo facts (date, location, contact, tags, visual) were buried between "Please analyze this photo" preamble and "Use the available tools" outro. Small models skim and miss them, which is why outputs weren't anchoring to the actual photo. 2. The user message ended with "write a detailed insight" — small models took the path of least resistance and just wrote, ignoring the soft "aim to use 5 tools" floor in the system prompt. Restructured the user message: - Leads with a "## This photo" bulleted block so the metadata is visible top-down. File path, date+source, contact, location+GPS, tags, and (in hybrid) the visual description are all bullets the model can't skim past. - Replaces the prose body with a numbered "## What to do" recipe: (1) recall_facts_for_photo + recall_entities, (2) ≥3 of the time-window tools, (3) write only after tool results, referencing specific facts. "Generic narration is not acceptable" is explicit. - Ends with a hard forcing line: "YOUR FIRST RESPONSE MUST BE A TOOL CALL. Do not output any final answer text until you have called at least 5 tools." Replaces the soft "aim to" floor with a directive small models actually follow. Tradeoff: big models also follow the recipe literally and may call 5 tools when 3 would do. Optimizing for the small-model floor first; soften once that's working. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
backfill_date_taken: surface the actual diesel error in warnings
2026-05-07 10:59:39 -04:00 · 2026-05-07 10:41:09 -04:00 · 2026-05-07 10:36:05 -04:00 · 2026-05-06 22:37:32 -04:00
122 changed files with 6263 additions and 34959 deletions
@@ -53,60 +53,11 @@ AGENTIC_CHAT_MAX_ITERATIONS=6
 # OPENROUTER_HTTP_REFERER=https://your-site.example
 # OPENROUTER_APP_TITLE=ImageApi
 # ── AI Insights — local backend switch ──────────────────────────────────
 # Picks which local LLM stack the server uses for chat, vision describe,
 # and embeddings. `ollama` (default) uses the OLLAMA_* settings above;
 # `llamacpp` uses the LLAMA_SWAP_* settings below. The switch is global
 # and applies to both `backend=local` and `backend=hybrid` (hybrid keeps
 # chat on OpenRouter but still uses this stack for the describe pass).
 # Don't flip mid-deploy without re-embedding existing index rows —
 # mixed vector spaces break similarity search.
 # LLM_BACKEND=ollama
 # ── AI Insights — llama.cpp / llama-swap (optional) ─────────────────────
 # Set LLAMA_SWAP_URL plus LLM_BACKEND=llamacpp to swap the local stack
 # off Ollama. Talks OpenAI-compatible /v1 to a llama-swap proxy fronting
 # per-slot llama-server instances. Chat models receive images directly
 # via content-parts (vision-capable models assumed); a separate vision
 # slot is used only by the describe_photo tool and describe-image utility.
 # LLAMA_SWAP_URL=http://localhost:9292/v1
 # LLAMA_SWAP_PRIMARY_MODEL=chat
 # Optional dedicated vision slot for describe_image. Defaults to
 # PRIMARY_MODEL so describe_photo works without extra config.
 # LLAMA_SWAP_VISION_MODEL=vision
 # LLAMA_SWAP_EMBEDDING_MODEL=embed
 # Comma-separated allowlist surfaced by /insights/models when
 # LLM_BACKEND=llamacpp. All report has_vision=true.
 # LLAMA_SWAP_ALLOWED_MODELS=chat,vision,embed
 # LLAMA_SWAP_REQUEST_TIMEOUT_SECONDS=180
 # ── Unified search translation model (optional) ─────────────────────────
 # /photos/search/unified runs one small LLM call to translate a natural-
 # language query into structured filters + a semantic term, then CLIP-ranks.
 # That step needs an LLM AND CLIP available at once. On a tight VRAM budget a
 # large chat model can't co-reside with CLIP, so pin a small, fast model here
 # (it can stay loaded alongside CLIP and the chat model). Precedence:
 # UNIFIED_SEARCH_MODEL > the client's selected model > the configured default.
 # Use the configured backend (LLM_BACKEND); local only — no hybrid.
 # UNIFIED_SEARCH_MODEL=qwen3-0.6b
 # ── Text-to-speech (optional, requires LLAMA_SWAP_URL) ───────────────────
 # TTS routes through the same llama-swap proxy (a Chatterbox model id), so it
 # only needs LLAMA_SWAP_URL — it does NOT require LLM_BACKEND=llamacpp.
 # Powers POST /tts/speech and the /tts/voices* endpoints (read-aloud insights
 # + voice cloning in the mobile app).
 # LLAMA_SWAP_TTS_MODEL=chatterbox        # TTS model id in config.yaml
 # LLAMA_SWAP_TTS_VOICE=m                 # default voice when a request omits one
 # LLAMA_SWAP_TTS_REF_SECONDS=30          # max voice-clone reference clip length (s)
 # LLAMA_SWAP_TTS_REQUEST_TIMEOUT_SECONDS=600   # synth timeout (long chunked text)
 # ── AI Insights — sibling services (optional) ───────────────────────────
-# Apollo (places, face inference, CLIP encoders). Single-Apollo deploys
+# Apollo (places + face inference). Single Apollo deploys typically set
-# typically set only APOLLO_API_BASE_URL and let the face + CLIP
+# only APOLLO_API_BASE_URL and let the face client fall back to it.
 # clients fall back to it.
 # APOLLO_API_BASE_URL=http://apollo.lan:8000
 # APOLLO_FACE_API_BASE_URL=http://apollo.lan:8000
 # APOLLO_CLIP_API_BASE_URL=http://apollo.lan:8000
 # SMS_API_URL=http://localhost:8000
 # SMS_API_TOKEN=
@@ -129,51 +80,6 @@ FACE_DETECT_TIMEOUT_SEC=60
 FACE_BACKLOG_MAX_PER_TICK=64
 FACE_HASH_BACKFILL_MAX_PER_TICK=2000
 # ── CLIP semantic photo search ──────────────────────────────────────────
 # ImageApi calls Apollo's /api/internal/clip/{encode_image,encode_text}
 # to populate per-photo embeddings during the watcher's backlog drain
 # and to encode user queries at /photos/search time. Disabled when
 # neither APOLLO_CLIP_API_BASE_URL nor APOLLO_API_BASE_URL is set.
 #
 # Per-watcher-tick cap on the encode drain. Default 32 ≈ ~1 photo/sec
 # on CPU, ~30 photos/sec on a single-GPU host (Apollo's threadpool
 # is 1 on CUDA, so concurrency is bounded server-side regardless of
 # our setting). Bump on a fresh deploy to clear the backlog faster.
 CLIP_BACKLOG_MAX_PER_TICK=32
 # Client-side parallel encode calls per drain pass. Apollo's GPU pool
 # serializes server-side; this just overlaps file-IO with inference.
 CLIP_ENCODE_CONCURRENCY=4
 # Per-encode HTTP timeout. CPU-only Apollo deploys may need higher.
 CLIP_REQUEST_TIMEOUT_SEC=60
 # ── RAG / search ────────────────────────────────────────────────────────
 # Set to `1` to enable cross-encoder reranking on /search results.
 SEARCH_RAG_RERANK=0
 # ── Nightly reel pre-generation (Phase 3+) ──────────────────────────────
 # Set to `1` to enable the scheduler. Disabled by default.
 # REEL_PREGEN_ENABLED=1
 # Hour (0-23) when the nightly batch fires. Default 3 AM.
 # REEL_PREGEN_HOUR=3
 # Day of week for weekly reels (0=Sun, 1=Mon, …). Default Monday.
 # REEL_PREGEN_WEEK_DOW=1
 # Timezone offset in minutes from UTC (e.g., -480 = PST). Defaults to
 # the server's local timezone.
 # REEL_PREGEN_TZ_OFFSET_MINUTES=
 # Fixed timezone offset — overrides auto-detect to avoid DST shifts.
 # When set, both the DB fallback and env fallback use this value.
 # REEL_PREGEN_TZ_FIXED_MINUTES=-480
 # Voice ID for narration (e.g., "grandma"). Falls back to the value
 # stored in the user_ai_prefs DB row when set.
 # REEL_PREGEN_VOICE=
 # Library filter: a library id (e.g. "1") or "all" for every library.
 # REEL_PREGEN_LIBRARY=all
 # Max agentic tool iterations for pre-gen scripter. Default 8.
 # REEL_PREGEN_MAX_TOOL_ITERS=8
 #
 # On-disk reel cache sweep (runs every 24h, independent of pre-gen). Removes
 # reel MP4s with no ledger row + no live job that are older than the max age —
 # i.e. the on-demand cache, which otherwise grows forever. Set to 0 to disable.
 # REEL_CACHE_SWEEP_ENABLED=1
 # Age (days) before an unreferenced reel MP4 is swept. Default 7.
 # REEL_CACHE_MAX_AGE_DAYS=7
@@ -1,9 +0,0 @@
 # Normalize line endings in the repo to LF. Windows checkouts can still
 # present working-copy files as CRLF; this just keeps the committed history
 # stable so contributors on any OS don't see whitespace-only diffs every
 # time someone touches a file.
 * text=auto eol=lf
 # Migrations and SQL must be LF — SQLite parsers don't care, but diffing
 # is much cleaner with stable endings.
 *.sql text eol=lf
@@ -5,8 +5,6 @@ database/target
 *.db-shm
 *.db-wal
 .env
 # Server-local TTS pronunciation overrides (tts_pronunciations.example.json is the template)
 /tts_pronunciations.json
 /tmp
 /docs
 /specs
@@ -76,10 +76,7 @@ cargo run --bin cleanup_files -- --base-path /path/to/media --database-url ./dat
 ### Core Components
 **Layered Architecture:**
- **Startup wiring** (`main.rs`): only ~350 lines — env load, migrations, AppState, route registration, server bind. Background jobs are kicked off here but defined elsewhere.
+- **HTTP Layer** (`main.rs`): Route handlers for images, videos, metadata, tags, favorites, memories
 - **HTTP Layer** (`handlers/{image,video,favorites}.rs`, `files.rs`, `tags.rs`, `faces.rs`, `memories.rs`, `ai/handlers.rs`): the route handlers, grouped by domain.
 - **Background loops** (`watcher.rs`): the file-watcher tick (`watch_files`, `process_new_files`) and the orphaned-playlist cleanup (`cleanup_orphaned_playlists`). Per-tick drains are factored into `backfill.rs` (`backfill_unhashed_backlog`, `backfill_missing_date_taken`, `backfill_missing_content_hashes`, `process_face_backlog`, `build_face_candidates`).
 - **Thumbnails** (`thumbnails.rs`): generation pipeline + the `IMAGE_GAUGE` / `VIDEO_GAUGE` Prometheus metrics.
 - **Auth Layer** (`auth.rs`): JWT token validation, Claims extraction via FromRequest trait
 - **Service Layer** (`files.rs`, `exif.rs`, `memories.rs`): Business logic for file operations and EXIF extraction
 - **DAO Layer** (`database/mod.rs`): Trait-based data access (ExifDao, UserDao, FavoriteDao, TagDao)
@@ -395,24 +392,17 @@ under 2021, not 2014 — on the theory that EXIF is more reliable than
 import-named filenames. The reverse case (no EXIF, filename has a
 date) is unchanged.
-The `backfill_missing_date_taken` drain (`src/backfill.rs`) runs every
+The `backfill_missing_date_taken` drain (`src/main.rs`) runs every
-watcher tick alongside `backfill_unhashed_backlog` (also `src/backfill.rs`). It loads up to
+watcher tick alongside `backfill_unhashed_backlog`. It loads up to
 `DATE_BACKFILL_MAX_PER_TICK` rows (default 500) where
-`date_taken IS NULL` (backed by the `idx_image_exif_date_backfill`
+`date_taken IS NULL OR date_taken_source = 'fs_time'` (backed by the
-partial index), runs the waterfall batch via `resolve_dates_batch`,
+`idx_image_exif_date_backfill` partial index), runs the waterfall
-and writes results via the `backfill_date_taken` DAO method (touches
+batch via `resolve_dates_batch`, and writes results via the
-only `date_taken` + `date_taken_source` so EXIF / hash / perceptual
+`backfill_date_taken` DAO method (touches only `date_taken` +
-columns are preserved). Resolved rows — including the ones the
+`date_taken_source` so EXIF / hash / perceptual columns are
-waterfall could only resolve via `fs_time` — are not re-eligible:
+preserved). `filename`-sourced rows are intentionally not re-resolved
-the resolver is deterministic on file bytes + filename + fs metadata,
+— the regex is authoritative when it matches, and re-running exiftool
-so re-running on the same inputs lands on the same source every time.
+won't change the answer.
 An earlier version included `date_taken_source = 'fs_time'` in the
 eligibility predicate, but with `ORDER BY id ASC LIMIT 500` it spun on
 the same lowest-id rows in perpetuity and held the SQLite write lock
 long enough to starve face-PATCH writers (5s busy_timeout → 500). If
 a stronger tool comes online (exiftool install, new filename regex),
 re-resolve out-of-band rather than re-introducing the steady-state
 eligibility.
 `/memories` is a single SQL query against this column
 (`get_memories_in_window` in `src/database/mod.rs`), using
@@ -473,16 +463,10 @@ GET /memories?path=...&recursive=true
 POST /insights/generate              (non-agentic single-shot)
 POST /insights/generate/agentic      (tool-calling loop; body: { file_path, backend?, model?, ... })
 GET  /insights?path=...&library=...
-GET  /insights/models                (local-backend models + capabilities; Ollama OR llama-swap based on LLM_BACKEND)
+GET  /insights/models                (local Ollama models + capabilities)
 GET  /insights/openrouter/models     (curated OpenRouter allowlist)
 POST /insights/rate                  (thumbs up/down for training data)
 // Text-to-Speech (Chatterbox via llama-swap; needs LLAMA_SWAP_URL)
 POST /tts/speech                     (read-aloud: { text, voice?, ... } -> { audio_base64, format })
 GET  /tts/voices                     (Chatterbox voice library)
 POST /tts/voices/upload              (clone a voice from an uploaded clip; multipart)
 POST /tts/voices/from-library        (clone a voice from a library audio/video file)
 // Insight Chat Continuation
 POST /insights/chat                  (single-turn reply, non-streaming)
 POST /insights/chat/stream           (SSE: text / tool_call / tool_result / truncated / done)
@@ -520,9 +504,9 @@ ImageApi owns the face data; Apollo (sibling repo) hosts the insightface inferen
 **Why content_hash and not (library_id, rel_path):** ties face data to the bytes, not the path. A backup mount that copies files from the primary library naturally inherits the existing detections without re-running inference. This is the reference implementation of the multi-library data model — see "Multi-library data model" above.
-**File-watch hook** (`src/watcher.rs::process_new_files`): for each photo with a populated `content_hash`, check `FaceDao::already_scanned(hash)`; if not, send bytes (or embedded JPEG preview for RAW via `exif::extract_embedded_jpeg_preview`) to Apollo's `/api/internal/faces/detect`. K=`FACE_DETECT_CONCURRENCY` (default 8) parallel calls per scan tick; Apollo serializes them via its single-worker GPU pool. `face_watch.rs` is the Tokio orchestration layer.
+**File-watch hook** (`src/main.rs::process_new_files`): for each photo with a populated `content_hash`, check `FaceDao::already_scanned(hash)`; if not, send bytes (or embedded JPEG preview for RAW via `exif::extract_embedded_jpeg_preview`) to Apollo's `/api/internal/faces/detect`. K=`FACE_DETECT_CONCURRENCY` (default 8) parallel calls per scan tick; Apollo serializes them via its single-worker GPU pool. `face_watch.rs` is the Tokio orchestration layer.
-**Per-tick backlog drain** (`src/backfill.rs`): two passes that run on every watcher tick regardless of quick-vs-full scan:
+**Per-tick backlog drain** (also `src/main.rs`): two passes that run on every watcher tick regardless of quick-vs-full scan:
 - `backfill_unhashed_backlog` — populates `image_exif.content_hash` for photos that arrived before the hash field was retroactive. Capped by `FACE_HASH_BACKFILL_MAX_PER_TICK` (default 2000); errors don't burn the cap.
 - `process_face_backlog` — runs detection on photos that have a hash but no `face_detections` row. Capped by `FACE_BACKLOG_MAX_PER_TICK` (default 64). Selected via a SQL anti-join (`FaceDao::list_unscanned_candidates`); videos and EXCLUDED_DIRS paths filtered out client-side via `face_watch::filter_excluded` so they never reach Apollo.
@@ -537,8 +521,6 @@ ImageApi owns the face data; Apollo (sibling repo) hosts the insightface inferen
 Module map:
 - `src/faces.rs` — `FaceDao` trait + `SqliteFaceDao` impl, route handlers for `/faces/*`, `/image/faces/*`, `/persons/*`. Mirror of `tags.rs` layout.
 - `src/face_watch.rs` — Tokio orchestration for the file-watch detect pass; `filter_excluded` (PathExcluder + image-extension filter), `read_image_bytes_for_detect` (RAW preview fallback).
 - `src/backfill.rs` — per-tick drains (unhashed-hash, date_taken, face-backlog, etc.) called from `watcher::watch_files` and `watcher::process_new_files`.
 - `src/watcher.rs` — the watcher loop itself and `process_new_files` (file walk → EXIF write → face-candidate build).
 - `src/ai/face_client.rs` — HTTP client for Apollo's inference. Configured by `APOLLO_FACE_API_BASE_URL`, falls back to `APOLLO_API_BASE_URL`. Both unset → feature disabled, file-watch hook is a no-op.
 - `migrations/2026-04-29-000000_add_faces/` — schema.
@@ -637,55 +619,8 @@ OPENROUTER_EMBEDDING_MODEL=openai/text-embedding-3-small  # Optional, embeddings
 OPENROUTER_HTTP_REFERER=https://your-site.example    # Optional attribution header
 OPENROUTER_APP_TITLE=ImageApi                  # Optional attribution header
 # Local LLM backend switch. `ollama` (default) keeps the OLLAMA_* settings
 # above; `llamacpp` swaps the entire local stack (chat + vision describe +
 # embeddings) over to llama-swap. The switch is global and applies to
 # `backend=local` requests and to `backend=hybrid`'s describe pass (hybrid
 # chat still goes to OpenRouter). Don't flip mid-deploy without
 # re-embedding — mixed vector spaces break similarity search.
 LLM_BACKEND=ollama
 # Embedding model contract. Corpus and queries must be embedded by the same
 # model with matching prefixes — after changing the embed model or any of
 # these, run `cargo run --bin reembed_embeddings` (all tables) or search is
 # garbage. Prefix values may contain a literal \n (expanded to a newline).
 EMBEDDING_DIM=768           # 768 = nomic-embed-text v1.5; 1024 = Qwen3-Embedding-0.6B
 EMBED_QUERY_PREFIX=         # nomic: "search_query: " | Qwen3: "Instruct: <task>\nQuery: "
 EMBED_DOCUMENT_PREFIX=      # nomic: "search_document: " | Qwen3: leave empty
 # llama.cpp / llama-swap (used when LLM_BACKEND=llamacpp). OpenAI-compatible
 # proxy hosting one or more llama-server processes. Chat models receive
 # images directly via content-parts (all models assumed vision-capable).
 LLAMA_SWAP_URL=http://localhost:9292/v1         # Required when LLM_BACKEND=llamacpp
 LLAMA_SWAP_PRIMARY_MODEL=chat                   # Chat slot id (matches config.yaml)
 LLAMA_SWAP_VISION_MODEL=                        # Dedicated vision slot for describe_image / describe_photo
                                                # tool. Defaults to PRIMARY_MODEL when unset.
 LLAMA_SWAP_EMBEDDING_MODEL=embed                # Embedding slot id
 LLAMA_SWAP_ALLOWED_MODELS=chat,coder            # Curated allowlist surfaced by GET /insights/models
                                                # when LLM_BACKEND=llamacpp. All report has_vision=true.
                                                # Empty = picker shows only the configured primary model.
 LLAMA_SWAP_REQUEST_TIMEOUT_SECONDS=180          # Per-request timeout; bump for slow CPU offload
 # Text-to-speech (Chatterbox served behind llama-swap). Only needs
 # LLAMA_SWAP_URL — independent of LLM_BACKEND. Powers /tts/speech (read-aloud)
 # and /tts/voices* (voice cloning). Reference audio is ffmpeg-normalized to WAV
 # server-side, so any source format works.
 LLAMA_SWAP_TTS_MODEL=chatterbox                # TTS model id in config.yaml (default: chatterbox)
 LLAMA_SWAP_TTS_VOICE=m                         # Default voice when /tts/speech omits one (optional)
 LLAMA_SWAP_TTS_REF_SECONDS=30                  # Max voice-clone reference clip length, seconds
                                               # (Chatterbox is zero-shot; ~10-20s clean ref is ideal)
 LLAMA_SWAP_TTS_REQUEST_TIMEOUT_SECONDS=600     # Per-request synth timeout (long chunked insights take
                                               # minutes); overrides the shared client timeout for /tts/speech
 TTS_PRONUNCIATIONS_PATH=tts_pronunciations.json # JSON map of pronunciation overrides applied before synth
                                               # (see tts_pronunciations.example.json); hot-reloaded on change
 # Insight Chat Continuation
 AGENTIC_CHAT_MAX_ITERATIONS=6                  # Cap on tool-calling iterations per chat turn (default 6)
 AGENTIC_CHAT_DEFAULT_NUM_CTX=32768             # Assumed context window for the history-truncation budget
                                               # when a chat request omits num_ctx (default 32768). Size to
                                               # the smallest context among the chat models actually served;
                                               # too small silently guts replayed history every turn (and
                                               # destroys llama.cpp KV-cache prefix reuse).
 ```
 **AI Insights Fallback Behavior:**
@@ -703,50 +638,10 @@ The `OllamaClient` provides methods to query available models:
 This allows runtime verification of model availability before generating insights.
 **Local backend switch (`LLM_BACKEND`):**
 One env var decides which "local" stack the server runs against — `ollama`
 (default) or `llamacpp`. It's global on purpose: chat, vision, and
 embeddings all route through the same backend, so the embedding-vector
 column in SQLite stays in one vector space. Don't flip mid-deploy without
 re-embedding the affected rows — similarity search will collapse.
 - `LLM_BACKEND=ollama`: chat, vision, and embeddings use Ollama. Vision
  capability is probed per-model via `/api/show`.
 - `LLM_BACKEND=llamacpp`: chat models receive images directly via OpenAI
  content-parts (all models assumed vision-capable). Embeddings hit the
  `embed` slot. A dedicated `LLAMA_SWAP_VISION_MODEL` slot (defaults to
  the chat model) handles `describe_image` for the `describe_photo` tool.
  Requires `LLAMA_SWAP_URL`.
 The per-request `backend=hybrid` override is orthogonal: it always sends
 chat to OpenRouter (text-only, images are pre-described and inlined), but
 the describe + embed passes still route through whichever `LLM_BACKEND`
 is configured.
 **Backend dispatch (`ResolvedBackend`):**
 `InsightGenerator::resolve_backend(kind, overrides)` is the single entry
 point that builds clients for a request. Returns a `ResolvedBackend` with
 two roles: `.chat()` (the agentic/chat client) and `.local()` (local-only
 utility calls: rerank, describe_image, embeddings). `BackendKind` is an
 enum (`Local` | `Hybrid`) replacing the stringly-typed `"local"` /
 `"hybrid"` labels. `SamplingOverrides` groups model/ctx/temp/top_p/top_k/
 min_p per-request overrides. All downstream code (`execute_tool`,
 `run_streaming_agentic_loop`, etc.) takes `&ResolvedBackend` rather than
 individual client references.
 `GET /insights/models` returns the local-backend models with capabilities
 in the same envelope shape regardless of `LLM_BACKEND`: Ollama servers
 when `ollama`, llama-swap slots (from `LLAMA_SWAP_ALLOWED_MODELS`) when
 `llamacpp`. No `/insights/llamacpp/models` — the picker reads a single
 endpoint.
 **Hybrid Backend (OpenRouter):**
 - Per-request opt-in via `backend=hybrid` on `POST /insights/generate/agentic`.
- Vision describe happens before the agentic loop; the description is inlined
+- Local Ollama still describes the image (vision); the description is inlined
-  into the chat prompt and the agentic loop runs on OpenRouter. Vision
+  into the chat prompt and the agentic loop runs on OpenRouter.
  routes through whichever `LLM_BACKEND` is configured.
 - `request.model` (if provided) overrides `OPENROUTER_DEFAULT_MODEL` for that
  call. The mobile picker reads from `OPENROUTER_ALLOWED_MODELS`.
 - No live capability precheck — the operator-curated allowlist is trusted.
@@ -754,15 +649,6 @@ endpoint.
 - `GET /insights/openrouter/models` returns `{ models, default_model, configured }`
  for client picker UIs.
 **Cross-replay matrix (chat continuation):**
 - `local → local` allowed (whether served by Ollama or llama-swap; that's
  a deploy-time decision, not a request-time one).
 - `hybrid → hybrid` allowed.
 - `hybrid → local` allowed (the inlined description replays as text).
 - `local → hybrid` rejected — the stored transcript has raw images in the
  first user message and OpenRouter providers don't accept that shape
  consistently. Regenerate the insight in hybrid mode instead.
 **Insight Chat Continuation:**
 After an agentic insight is generated, the full `Vec<ChatMessage>` transcript is
@@ -772,12 +658,7 @@ clients whether chat is available for a given insight.
 - `POST /insights/chat` runs one turn of the agentic loop against the replayed
  history. Body: `{ file_path, library?, user_message, model?, backend?, num_ctx?,
-  temperature?, top_p?, top_k?, min_p?, max_iterations?, system_prompt?, amend? }`.
+  temperature?, top_p?, top_k?, min_p?, max_iterations?, amend? }`.
  `system_prompt` is a per-turn override: in append mode (default) it's applied
  ephemerally — the original system message is restored before persistence so
  the stored transcript keeps its baked persona. In amend mode the override
  stays in place and becomes the new insight row's system message. Mirrors the
  internal `annotate_system_with_budget` swap-and-restore pattern.
 - `POST /insights/chat/stream` is the SSE variant — same request body, response
  is `text/event-stream` with events: `iteration_start`, `text` (delta), `tool_call`,
  `tool_result`, `truncated`, `done`, plus a server-emitted `error_message` on
@@ -809,17 +690,14 @@ Per-`(library_id, file_path)` async mutex (`AppState.insight_chat.chat_locks`)
 serialises concurrent turns on the same insight so the JSON blob doesn't race.
 Context management is a soft bound: if the serialized history exceeds
-`num_ctx - 2048` tokens (cheap 4-byte/token heuristic; `num_ctx` defaults
+`num_ctx - 2048` tokens (cheap 4-byte/token heuristic), the oldest
-to `AGENTIC_CHAT_DEFAULT_NUM_CTX`, 32768, when the request omits it), the
+assistant-tool_call + tool_result pairs are dropped until under budget. The
 oldest assistant-tool_call + tool_result pairs are dropped until under budget. The
 initial user message (with any images) and system prompt are always preserved.
 The `truncated` event / flag is surfaced to the client when a drop occurred.
 Configurable env:
 - `AGENTIC_CHAT_MAX_ITERATIONS` — cap on tool-calling iterations per turn
  (default 6). Per-request `max_iterations` is clamped to this cap.
 - `AGENTIC_CHAT_DEFAULT_NUM_CTX` — assumed context window for the truncation
  budget when the request omits `num_ctx` (default 32768).
 **Apollo Places integration (optional):**
@@ -2051,7 +2051,7 @@ dependencies = [
 [[package]]
 name = "image-api"
-version = "1.5.0"
+version = "1.1.0"
 dependencies = [
 "actix",
 "actix-cors",
@@ -2104,7 +2104,6 @@ dependencies = [
 "tokio",
 "tokio-util",
 "urlencoding",
 "uuid",
 "walkdir",
 "zerocopy",
 ]
@@ -4392,9 +4391,7 @@ version = "1.23.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "ddd74a9687298c6858e9b88ec8935ec45d22e8fd5e6394fa1bd4e99a87789c76"
 dependencies = [
 "getrandom 0.4.2",
 "js-sys",
 "serde_core",
 "wasm-bindgen",
 ]
@@ -1,6 +1,6 @@
 [package]
 name = "image-api"
-version = "1.5.0"
+version = "1.1.0"
 authors = ["Cameron Cordes <cameronc.dev@gmail.com>"]
 edition = "2024"
@@ -66,7 +66,6 @@ image_hasher = "3.0"
 bk-tree = "0.5"
 async-trait = "0.1"
 indicatif = "0.17"
 uuid = { version = "1.10", features = ["v4", "serde"] }
 # Windows lacks system sqlite3, so re-enable the bundled C build there.
 # Linux/macOS use the system library (faster builds, smaller binary).
@@ -147,56 +147,6 @@ so you can rewrite the saved summary from within chat.
 - `AGENTIC_CHAT_MAX_ITERATIONS` - Cap on tool-calling iterations per chat turn [default: `6`]
  - Per-request `max_iterations` (when sent by the client) is clamped to this cap
 #### Text-to-Speech (Optional)
 Reads insights aloud and manages cloned voices via a Chatterbox model served
 behind the same llama-swap proxy. Only requires `LLAMA_SWAP_URL` (the TTS client
 is built whenever that's set — independent of `LLM_BACKEND`). Endpoints:
 - `POST /tts/speech` — body `{ text, voice?, format?, exaggeration?, cfg_weight?,
  temperature? }`; returns `{ audio_base64, format }`. Input is cleaned
  server-side (markdown + emoji stripped, then pronunciation overrides applied —
  see below) and the generation knobs are clamped
  to Chatterbox's ranges. Synthesis is serialized (one at a time — the upstream
  has no GPU lock of its own); a concurrent request gets a fast `429`.
 - `POST /tts/speech/jobs` — durable variant for long syntheses: same body as
  `/tts/speech`, returns `202 { job_id, status }` immediately. Jobs queue on the
  GPU permit instead of fast-failing `429`.
 - `GET /tts/speech/jobs/{id}` — poll a job: `{ job_id, status, format,
  audio_base64?, error? }` with status `queued|running|done|error|cancelled`.
  Results are kept in memory ~10 min after completion, then the job 404s.
 - `DELETE /tts/speech/jobs/{id}` — cancel a queued/running job.
 - `GET /tts/voices` — list the voice library. Served from an in-memory cache
  (so the listing doesn't make llama-swap spin up the TTS model and evict the
  resident LLM); pass `?refresh=1` to force an upstream re-query. The cache is
  invalidated by voice create/delete.
 - `POST /tts/voices/upload` — multipart `voice_name` + `voice_file`; clone a
  voice from an uploaded clip (≤25 MB).
 - `POST /tts/voices/from-library` — body `{ voice_name, path, library? }`; clone
  from a library file (audio forwarded as-is; video has its audio extracted via
  ffmpeg).
 - `DELETE /tts/voices/{name}` — remove a cloned voice from the library.
 Created voice names are tagged with the ref-clip cap in effect (e.g.
 `grandma-30s`) so the library shows which reference length produced each clone.
 Words the model mispronounces (place names, initialisms) can be rewritten
 before synthesis via a JSON map — copy `tts_pronunciations.example.json` to
 `tts_pronunciations.json` and edit; changes apply without a restart. Full
 matching rules are documented in `src/ai/pronunciation.rs`.
 Env:
 - `TTS_PRONUNCIATIONS_PATH` - pronunciation-override JSON file
  [default: `tts_pronunciations.json` in the working directory]
 - `LLAMA_SWAP_TTS_MODEL` - TTS model id in llama-swap's `config.yaml` [default: `chatterbox`]
 - `LLAMA_SWAP_TTS_VOICE` - default voice used when a `/tts/speech` request omits `voice` (optional)
 - `LLAMA_SWAP_TTS_REF_SECONDS` - max voice-clone reference clip length in seconds
  [default: `30`]. Reference audio is ffmpeg-normalized to mono 24 kHz WAV (so any
  source format works); Chatterbox is zero-shot, so a clean ~10–20s sample is the
  sweet spot — more rarely helps.
 - `LLAMA_SWAP_TTS_REQUEST_TIMEOUT_SECONDS` - per-request synthesis timeout in
  seconds [default: `600`]. Long insights are chunked + synthesized server-side
  and can take minutes; this is separate from (and overrides, for `/tts/speech`)
  the shared `LLAMA_SWAP_REQUEST_TIMEOUT_SECONDS`.
 #### Fallback Behavior
 - Primary server is tried first with 5-second connection timeout
 - On failure, automatically falls back to secondary server (if configured)
@@ -1,43 +0,0 @@
 -- Drop the persona-scoping column on entity_facts via the table-rebuild
 -- dance for SQLite-version portability (matches the pattern in
 -- 2026-04-20-000000_add_backend_to_insights/down.sql).
 DROP INDEX IF EXISTS idx_entity_facts_persona;
 CREATE TABLE entity_facts_backup AS
    SELECT id, subject_entity_id, predicate, object_entity_id, object_value,
           source_photo, source_insight_id, confidence, status, created_at
    FROM entity_facts;
 DROP TABLE entity_facts;
 CREATE TABLE entity_facts (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    subject_entity_id INTEGER NOT NULL,
    predicate TEXT NOT NULL,
    object_entity_id INTEGER,
    object_value TEXT,
    source_photo TEXT,
    source_insight_id INTEGER,
    confidence REAL NOT NULL DEFAULT 0.6,
    status TEXT NOT NULL DEFAULT 'active',
    created_at BIGINT NOT NULL,
    CONSTRAINT fk_ef_subject FOREIGN KEY (subject_entity_id) REFERENCES entities(id) ON DELETE CASCADE,
    CONSTRAINT fk_ef_object  FOREIGN KEY (object_entity_id)  REFERENCES entities(id) ON DELETE SET NULL,
    CONSTRAINT fk_ef_insight FOREIGN KEY (source_insight_id) REFERENCES photo_insights(id) ON DELETE SET NULL,
    CHECK (object_entity_id IS NOT NULL OR object_value IS NOT NULL)
 );
 INSERT INTO entity_facts
    SELECT id, subject_entity_id, predicate, object_entity_id, object_value,
           source_photo, source_insight_id, confidence, status, created_at
    FROM entity_facts_backup;
 DROP TABLE entity_facts_backup;
 CREATE INDEX idx_entity_facts_subject ON entity_facts(subject_entity_id);
 CREATE INDEX idx_entity_facts_predicate ON entity_facts(predicate);
 CREATE INDEX idx_entity_facts_status ON entity_facts(status);
 CREATE INDEX idx_entity_facts_source_photo ON entity_facts(source_photo);
 DROP INDEX IF EXISTS idx_personas_user;
 DROP TABLE IF EXISTS personas;
@@ -1,64 +0,0 @@
 -- Personas live server-side now (mobile previously stored them in
 -- AsyncStorage only). Each user gets the three built-ins seeded; custom
 -- personas land here too via POST /personas or POST /personas/migrate.
 --
 -- `entity_facts` gains a persona_id so each persona accumulates its own
 -- voice over a shared entity graph (entities themselves stay unscoped).
 -- Existing rows backfill to 'default' via the column DEFAULT — that
 -- becomes the historical baseline. The `include_all_memories` flag on
 -- personas lets any persona opt back into reading the full pool.
 CREATE TABLE personas (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    user_id INTEGER NOT NULL,
    persona_id TEXT NOT NULL,
    name TEXT NOT NULL,
    system_prompt TEXT NOT NULL,
    is_built_in BOOLEAN NOT NULL DEFAULT FALSE,
    include_all_memories BOOLEAN NOT NULL DEFAULT FALSE,
    created_at BIGINT NOT NULL,
    updated_at BIGINT NOT NULL,
    UNIQUE(user_id, persona_id),
    CONSTRAINT fk_personas_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
 );
 CREATE INDEX idx_personas_user ON personas(user_id);
 -- Seed built-ins for every existing user. System prompts copied verbatim
 -- from FileViewer-React/hooks/usePersonas.tsx so server and client agree
 -- on the canonical voice for each built-in.
 INSERT INTO personas (user_id, persona_id, name, system_prompt, is_built_in, created_at, updated_at)
 SELECT
    u.id,
    'default',
    'Default Assistant',
    'You are my long-term memory assistant. Use only the information provided. Do not invent details. Respond in 3–6 sentences in third person, leading with the most concrete moment from the photo and the surrounding context. Plain prose, no headings.',
    TRUE,
    strftime('%s', 'now') * 1000,
    strftime('%s', 'now') * 1000
 FROM users u
 UNION ALL
 SELECT
    u.id,
    'journal',
    'Personal Journal',
    'You are a personal journal writer. Write in first person, present tense, with warmth and reflection — focusing on emotions and meaningful moments. Use only the information provided; do not invent details. Aim for 4–8 sentences in a single flowing paragraph, no headings.',
    TRUE,
    strftime('%s', 'now') * 1000,
    strftime('%s', 'now') * 1000
 FROM users u
 UNION ALL
 SELECT
    u.id,
    'factual',
    'Factual Reporter',
    'You are a factual memory recorder. Be precise, objective, and concise. Lead with the date and place, then list what / when / who in 2–4 short sentences. Use only the information provided; if a detail is unknown, say so rather than guessing.',
    TRUE,
    strftime('%s', 'now') * 1000,
    strftime('%s', 'now') * 1000
 FROM users u;
 -- Persona scoping on facts only. Entities and entity_photo_links stay
 -- shared (real-world referents and shared photo ↔ entity associations).
 ALTER TABLE entity_facts ADD COLUMN persona_id TEXT NOT NULL DEFAULT 'default';
 CREATE INDEX idx_entity_facts_persona ON entity_facts(persona_id);
@@ -1,47 +0,0 @@
 -- Reverse 2026-05-10-000000_entity_facts_persona_fk: drop the
 -- composite FK and the user_id column via the same rebuild pattern.
 DROP INDEX IF EXISTS idx_entity_facts_user_persona;
 DROP INDEX IF EXISTS idx_entity_facts_persona;
 DROP INDEX IF EXISTS idx_entity_facts_source_photo;
 DROP INDEX IF EXISTS idx_entity_facts_status;
 DROP INDEX IF EXISTS idx_entity_facts_predicate;
 DROP INDEX IF EXISTS idx_entity_facts_subject;
 ALTER TABLE entity_facts RENAME TO entity_facts_old;
 CREATE TABLE entity_facts (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    subject_entity_id INTEGER NOT NULL,
    predicate TEXT NOT NULL,
    object_entity_id INTEGER,
    object_value TEXT,
    source_photo TEXT,
    source_insight_id INTEGER,
    confidence REAL NOT NULL DEFAULT 0.6,
    status TEXT NOT NULL DEFAULT 'active',
    created_at BIGINT NOT NULL,
    persona_id TEXT NOT NULL DEFAULT 'default',
    CONSTRAINT fk_ef_subject FOREIGN KEY (subject_entity_id) REFERENCES entities(id) ON DELETE CASCADE,
    CONSTRAINT fk_ef_object  FOREIGN KEY (object_entity_id)  REFERENCES entities(id) ON DELETE SET NULL,
    CONSTRAINT fk_ef_insight FOREIGN KEY (source_insight_id) REFERENCES photo_insights(id) ON DELETE SET NULL,
    CHECK (object_entity_id IS NOT NULL OR object_value IS NOT NULL)
 );
 INSERT INTO entity_facts
    (id, subject_entity_id, predicate, object_entity_id, object_value,
     source_photo, source_insight_id, confidence, status, created_at,
     persona_id)
 SELECT
    id, subject_entity_id, predicate, object_entity_id, object_value,
    source_photo, source_insight_id, confidence, status, created_at,
    persona_id
 FROM entity_facts_old;
 DROP TABLE entity_facts_old;
 CREATE INDEX idx_entity_facts_subject ON entity_facts(subject_entity_id);
 CREATE INDEX idx_entity_facts_predicate ON entity_facts(predicate);
 CREATE INDEX idx_entity_facts_status ON entity_facts(status);
 CREATE INDEX idx_entity_facts_source_photo ON entity_facts(source_photo);
 CREATE INDEX idx_entity_facts_persona ON entity_facts(persona_id);
@@ -1,82 +0,0 @@
 -- Add a real foreign key from entity_facts to personas. Until now,
 -- entity_facts.persona_id was a free-form string with no integrity
 -- guarantee — deleting a persona orphaned its facts, which then sat
 -- forever in the readable-only-via-PersonaFilter::All hive-mind view.
 --
 -- personas is keyed (user_id, persona_id) so the FK has to be
 -- composite. That requires entity_facts to carry user_id too, which
 -- has the side benefit of fixing multi-user fact leakage on the read
 -- path (without it, two users with the same 'default' persona would
 -- see each other's default-scoped facts).
 --
 -- SQLite can't ALTER TABLE to add an FK; the table-rebuild dance is
 -- the only way. Pattern matches 2026-05-09's down.sql and the older
 -- 2026-04-20-000000 migration.
 DROP INDEX IF EXISTS idx_entity_facts_subject;
 DROP INDEX IF EXISTS idx_entity_facts_predicate;
 DROP INDEX IF EXISTS idx_entity_facts_status;
 DROP INDEX IF EXISTS idx_entity_facts_source_photo;
 DROP INDEX IF EXISTS idx_entity_facts_persona;
 ALTER TABLE entity_facts RENAME TO entity_facts_old;
 CREATE TABLE entity_facts (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    subject_entity_id INTEGER NOT NULL,
    predicate TEXT NOT NULL,
    object_entity_id INTEGER,
    object_value TEXT,
    source_photo TEXT,
    source_insight_id INTEGER,
    confidence REAL NOT NULL DEFAULT 0.6,
    status TEXT NOT NULL DEFAULT 'active',
    created_at BIGINT NOT NULL,
    persona_id TEXT NOT NULL DEFAULT 'default',
    user_id INTEGER NOT NULL DEFAULT 1,
    CONSTRAINT fk_ef_subject FOREIGN KEY (subject_entity_id) REFERENCES entities(id) ON DELETE CASCADE,
    CONSTRAINT fk_ef_object  FOREIGN KEY (object_entity_id)  REFERENCES entities(id) ON DELETE SET NULL,
    CONSTRAINT fk_ef_insight FOREIGN KEY (source_insight_id) REFERENCES photo_insights(id) ON DELETE SET NULL,
    CONSTRAINT fk_ef_persona FOREIGN KEY (user_id, persona_id) REFERENCES personas(user_id, persona_id) ON DELETE CASCADE,
    CHECK (object_entity_id IS NOT NULL OR object_value IS NOT NULL)
 );
 -- Backfill: assign each legacy fact to the user that owns the matching
 -- persona. Built-ins are seeded per-user with the same persona_id
 -- string for everyone, so MIN(user_id) deterministically picks the
 -- earliest registered user (typically user 1, the operator). Custom
 -- persona_ids exist for at most one user, so MIN is also unique.
 -- Falls back to user_id=1 when no matching persona row exists; in that
 -- case the FK below would still fail, but legacy rows shouldn't be in
 -- that state because 2026-05-09 ADD COLUMN defaulted persona_id to
 -- 'default', which is seeded for every user.
 INSERT INTO entity_facts
    (id, subject_entity_id, predicate, object_entity_id, object_value,
     source_photo, source_insight_id, confidence, status, created_at,
     persona_id, user_id)
 SELECT
    old.id,
    old.subject_entity_id,
    old.predicate,
    old.object_entity_id,
    old.object_value,
    old.source_photo,
    old.source_insight_id,
    old.confidence,
    old.status,
    old.created_at,
    old.persona_id,
    COALESCE(
        (SELECT MIN(p.user_id) FROM personas p WHERE p.persona_id = old.persona_id),
        1
    )
 FROM entity_facts_old old;
 DROP TABLE entity_facts_old;
 CREATE INDEX idx_entity_facts_subject ON entity_facts(subject_entity_id);
 CREATE INDEX idx_entity_facts_predicate ON entity_facts(predicate);
 CREATE INDEX idx_entity_facts_status ON entity_facts(status);
 CREATE INDEX idx_entity_facts_source_photo ON entity_facts(source_photo);
 CREATE INDEX idx_entity_facts_persona ON entity_facts(persona_id);
 CREATE INDEX idx_entity_facts_user_persona ON entity_facts(user_id, persona_id);
@@ -1,5 +0,0 @@
 -- SQLite can drop columns since 3.35 (March 2021); embedded
 -- libsqlite3-sys is well past that. Drop in reverse insert order so
 -- a partial down still leaves the schema valid.
 ALTER TABLE entity_facts DROP COLUMN valid_until;
 ALTER TABLE entity_facts DROP COLUMN valid_from;
@@ -1,25 +0,0 @@
 -- Add valid-time columns to entity_facts.
 --
 -- entity_facts already has created_at — *transaction time*, the
 -- moment WE recorded the fact. That's not the same as the real-world
 -- period the fact was true. "Cameron is_in_relationship_with X" was
 -- only true during a window; recording it in 2026 doesn't make it
 -- true today. Without the distinction, every former relationship,
 -- former job, former address reads as currently-true.
 --
 -- Adding two BIGINT NULL columns: valid_from / valid_until (unix
 -- seconds). NULL means "unbounded on that side" — `valid_from IS
 -- NULL` reads as "always-true-back-to-the-beginning",
 -- `valid_until IS NULL` as "still-true-now-or-unknown". Both NULL =
 -- temporal validity unknown (current state of all legacy rows).
 --
 -- Conflict detection refines accordingly: same-predicate facts with
 -- different objects stop flagging when their intervals are disjoint
 -- ("lives_in NYC 2018-2020" and "lives_in SF 2020-present" are both
 -- valid, just at different times).
 ALTER TABLE entity_facts ADD COLUMN valid_from BIGINT;
 ALTER TABLE entity_facts ADD COLUMN valid_until BIGINT;
 -- Optional partial index for time-bounded scans. Skipped for now —
 -- conflict detection runs per-entity (small N) and doesn't need it.
@@ -1,2 +0,0 @@
 DROP INDEX IF EXISTS idx_entity_facts_superseded_by;
 ALTER TABLE entity_facts DROP COLUMN superseded_by;
@@ -1,31 +0,0 @@
 -- Add a supersession pointer to entity_facts.
 --
 -- Status alone is a one-way trapdoor: 'rejected' loses the link
 -- between the rejected fact and the one that replaced it. For
 -- evolving facts (Cameron's relationship, employer, address) the
 -- curator wants to *replace* a stale fact with a new one and keep
 -- the history readable: "from 2018 until 2022 this was true, then
 -- it became this other thing".
 --
 -- A nullable INTEGER column pointing at another entity_facts.id —
 -- no FK constraint because SQLite can't ALTER ADD COLUMN with REFs;
 -- the DAO's delete_fact clears dangling pointers in the same
 -- transaction as the parent delete to keep the column honest.
 --
 -- A status of 'superseded' on the old fact (alongside the existing
 -- active / reviewed / rejected) signals "replaced by a newer
 -- claim". Read paths already filter 'rejected' out of the active
 -- view; the curation UI will treat 'superseded' the same way for
 -- conflict detection so they don't keep flagging.
 --
 -- Pairs with the valid-time columns from 2026-05-10-000100: the
 -- supersede action auto-stamps the old fact's `valid_until` from
 -- the new fact's `valid_from`, closing the interval cleanly.
 ALTER TABLE entity_facts ADD COLUMN superseded_by INTEGER;
 -- Helpful index for "show me what superseded this fact" walks
 -- (rare today; cheap to add now while the table is small).
 CREATE INDEX idx_entity_facts_superseded_by
    ON entity_facts(superseded_by)
    WHERE superseded_by IS NOT NULL;
@@ -1,4 +0,0 @@
 DROP INDEX IF EXISTS idx_entity_facts_created_by_backend;
 DROP INDEX IF EXISTS idx_entity_facts_created_by_model;
 ALTER TABLE entity_facts DROP COLUMN created_by_backend;
 ALTER TABLE entity_facts DROP COLUMN created_by_model;
@@ -1,30 +0,0 @@
 -- Track which model + backend generated each fact so the curator
 -- can audit which configurations produce trustworthy knowledge.
 --
 -- photo_insights already carries `model_version` + `backend`, and
 -- entity_facts.source_insight_id links to it — but:
 --   1. source_insight_id is only set after an insight is stored
 --      (post-loop), so chat-continuation facts and facts whose insight
 --      was regenerated lose the link.
 --   2. JOINing for every read is more friction than just embedding the
 --      provenance on the fact row itself.
 --   3. Manual facts (POST /knowledge/facts) have no insight at all and
 --      need to record "manual" as their provenance.
 --
 -- Two nullable TEXT columns are enough for the audit use case: model
 -- (e.g. "qwen2.5:7b", "anthropic/claude-sonnet-4") and backend
 -- ("local", "hybrid", "manual"). Pre-existing rows leave both NULL —
 -- legacy facts predate this tracking and can't be back-filled
 -- reliably from training_messages without burning compute.
 ALTER TABLE entity_facts ADD COLUMN created_by_model TEXT;
 ALTER TABLE entity_facts ADD COLUMN created_by_backend TEXT;
 -- Indexes are cheap and useful for "show me all facts from model X"
 -- audit queries — partial so the legacy NULL rows don't bloat them.
 CREATE INDEX idx_entity_facts_created_by_model
    ON entity_facts(created_by_model)
    WHERE created_by_model IS NOT NULL;
 CREATE INDEX idx_entity_facts_created_by_backend
    ON entity_facts(created_by_backend)
    WHERE created_by_backend IS NOT NULL;
@@ -1 +0,0 @@
 ALTER TABLE personas DROP COLUMN reviewed_only_facts;
@@ -1,16 +0,0 @@
 -- Per-persona toggle: when true, agent reads only see facts whose
 -- status is exactly 'reviewed' (human-verified). When false (the
 -- default), agent reads see 'active' OR 'reviewed' — everything not
 -- rejected or superseded.
 --
 -- The mobile app surfaces this as "Strict mode" on the persona
 -- editor: useful when you want a persona's chat to be grounded
 -- exclusively on the curated subset, e.g. for tasks where
 -- hallucinated agent claims are particularly costly.
 --
 -- Note: this is separate from `include_all_memories` (which unions
 -- across personas for hive-mind reads). Reviewed-only operates on
 -- the status axis; include_all_memories operates on the persona-
 -- scope axis. They compose freely.
 ALTER TABLE personas ADD COLUMN reviewed_only_facts BOOLEAN NOT NULL DEFAULT 0;
@@ -1,5 +0,0 @@
 ALTER TABLE personas DROP COLUMN allow_agent_corrections;
 DROP INDEX IF EXISTS idx_entity_facts_last_modified_at;
 ALTER TABLE entity_facts DROP COLUMN last_modified_at;
 ALTER TABLE entity_facts DROP COLUMN last_modified_by_backend;
 ALTER TABLE entity_facts DROP COLUMN last_modified_by_model;
@@ -1,30 +0,0 @@
 -- Three coupled changes for agent self-correction safety:
 --
 -- 1. `entity_facts.last_modified_by_*` + `last_modified_at` track who
 --    most recently mutated each fact. `created_by_*` from migration
 --    2026-05-10-000300 records who first wrote the row; this records
 --    who last *changed* it. Separate columns so the create vs update
 --    audit is independently grep-able ("show me every fact gpt-5
 --    altered last week" stays a single index scan).
 --
 -- 2. `personas.allow_agent_corrections` is the gate for the new
 --    agent-side `update_fact` / `supersede_fact` tools. Default OFF —
 --    a fresh persona's agent can create but can't alter or replace.
 --    Operator opts in per-persona after the model has earned trust,
 --    typically via the strict-mode flow (curate, then ratchet up
 --    agent autonomy as confidence rises). Parallel in shape to
 --    `reviewed_only_facts` from 2026-05-10-000400; they compose.
 --
 -- 3. Index on `last_modified_at` (partial, NOT NULL) for the
 --    audit-feed reads in the curation UI ("show recent agent edits
 --    sorted newest first").
 ALTER TABLE entity_facts ADD COLUMN last_modified_by_model TEXT;
 ALTER TABLE entity_facts ADD COLUMN last_modified_by_backend TEXT;
 ALTER TABLE entity_facts ADD COLUMN last_modified_at BIGINT;
 CREATE INDEX idx_entity_facts_last_modified_at
    ON entity_facts(last_modified_at)
    WHERE last_modified_at IS NOT NULL;
 ALTER TABLE personas ADD COLUMN allow_agent_corrections BOOLEAN NOT NULL DEFAULT 0;
@@ -1,6 +0,0 @@
 -- Irreversible: we collapsed multiple raw entity_type strings to
 -- canonical forms and don't have a per-row record of the original.
 -- The down migration is intentionally a no-op (the rewritten values
 -- are still semantically correct), and the up migration is safe to
 -- re-run because every UPDATE is conditional on `!= canonical`.
 SELECT 1;
@@ -1,43 +0,0 @@
 -- Canonicalize `entities.entity_type` so legacy rows from before
 -- `normalize_entity_type` landed in upsert_entity stop polluting
 -- client-side filters. Mirrors the synonym map in
 -- `src/database/knowledge_dao.rs::normalize_entity_type`:
 --   person  ← person | people | human | individual | contact
 --   place   ← place  | location | venue | site | area | landmark
 --   event   ← event  | occasion | activity | celebration
 --   thing   ← thing  | object | item | product
 -- Types outside the synonym set (e.g. "friend", "family") are not
 -- recognized as canonical and get a lowercase+trim pass instead, so
 -- at minimum case variants collapse.
 --
 -- `UPDATE OR IGNORE` skips rows that would violate UNIQUE(name,
 -- entity_type) after the rewrite. Two rows like ("Sarah", "person")
 -- + ("Sarah", "Person") would otherwise collide — the duplicate
 -- survives unchanged so the curator can merge it via the curation
 -- UI rather than have the migration silently delete data.
 UPDATE OR IGNORE entities
 SET entity_type = 'person'
 WHERE LOWER(TRIM(entity_type)) IN ('person', 'people', 'human', 'individual', 'contact')
  AND entity_type != 'person';
 UPDATE OR IGNORE entities
 SET entity_type = 'place'
 WHERE LOWER(TRIM(entity_type)) IN ('place', 'location', 'venue', 'site', 'area', 'landmark')
  AND entity_type != 'place';
 UPDATE OR IGNORE entities
 SET entity_type = 'event'
 WHERE LOWER(TRIM(entity_type)) IN ('event', 'occasion', 'activity', 'celebration')
  AND entity_type != 'event';
 UPDATE OR IGNORE entities
 SET entity_type = 'thing'
 WHERE LOWER(TRIM(entity_type)) IN ('thing', 'object', 'item', 'product')
  AND entity_type != 'thing';
 -- Anything left ("Friend" vs "friend") gets a lowercase+trim sweep
 -- so at least case variants of the same custom type collapse.
 UPDATE OR IGNORE entities
 SET entity_type = LOWER(TRIM(entity_type))
 WHERE entity_type != LOWER(TRIM(entity_type));
@@ -1,5 +0,0 @@
 DROP INDEX IF EXISTS idx_image_exif_date_backfill;
 CREATE INDEX idx_image_exif_date_backfill
    ON image_exif (library_id, id)
    WHERE date_taken IS NULL OR date_taken_source = 'fs_time';
@@ -1,18 +0,0 @@
 -- Narrow the date-backfill partial index to NULL-only rows.
 --
 -- The original index (2026-05-06-000000_add_date_taken_source) also matched
 -- `date_taken_source = 'fs_time'` so the drain could "re-resolve weak
 -- entries when better tools become available." In practice the resolver
 -- is deterministic on file bytes + filename + fs metadata: a row that
 -- landed on fs_time once will land on fs_time again on every subsequent
 -- tick. With `ORDER BY id ASC LIMIT 500`, the drain spun on the same
 -- lowest-id fs_time rows in perpetuity, never advancing, while hammering
 -- the SQLite write lock once per row and starving other writers (face
 -- PATCHes were hitting busy_timeout and returning 500). Drop fs_time
 -- from the eligibility set; if exiftool / a new filename pattern ever
 -- comes online, a one-shot operator command can re-resolve.
 DROP INDEX IF EXISTS idx_image_exif_date_backfill;
 CREATE INDEX idx_image_exif_date_backfill
    ON image_exif (library_id, id)
    WHERE date_taken IS NULL;
@@ -1,3 +0,0 @@
 DROP INDEX IF EXISTS idx_image_exif_clip_backfill;
 ALTER TABLE image_exif DROP COLUMN clip_model_version;
 ALTER TABLE image_exif DROP COLUMN clip_embedding;
@@ -1,27 +0,0 @@
 -- CLIP semantic photo search: store a per-photo image embedding so
 -- text queries can rerank against the live library via cosine
 -- similarity. Apollo encodes the bytes via its CLIP service; ImageApi
 -- writes the resulting blob here.
 --
 -- `clip_embedding` is the raw little-endian float32 buffer of an
 -- L2-normalized vector (dim depends on the model — 768 bytes×4 for
 -- ViT-L/14, 512 bytes×4 for ViT-B/32). Apollo always returns the
 -- normalized form so the search-time dot product reduces to a plain
 -- cosine similarity.
 --
 -- `clip_model_version` echoes the upstream `APOLLO_CLIP_MODEL` (e.g.
 -- "ViT-L/14"). A model swap shouldn't silently mix geometries — the
 -- backfill drain will re-eligibilize rows whose stored model_version
 -- differs from the live engine's, and the search route refuses to
 -- mix rows from two model_versions in the same response.
 ALTER TABLE image_exif ADD COLUMN clip_embedding BLOB;
 ALTER TABLE image_exif ADD COLUMN clip_model_version TEXT;
 -- Partial index for the backfill drain. Mirrors the shape of
 -- `idx_image_exif_date_backfill`: candidate rows are those with a
 -- known content_hash (so we don't race the unhashed backlog) but no
 -- embedding yet. SELECT cost stays O(missing rows) instead of full
 -- table scan once the column is mostly populated.
 CREATE INDEX IF NOT EXISTS idx_image_exif_clip_backfill
    ON image_exif (id)
    WHERE clip_embedding IS NULL AND content_hash IS NOT NULL;
@@ -1,3 +0,0 @@
 DROP INDEX IF EXISTS idx_insight_gen_jobs_status_cleanup;
 DROP INDEX IF EXISTS idx_insight_gen_jobs_file;
 DROP TABLE IF EXISTS insight_generation_jobs;
@@ -1,23 +0,0 @@
 -- Track async insight generation jobs so the client can poll for
 -- completion after the server returns 202 Accepted. Each generation
 -- creates a new row; the application layer cancels prior running
 -- jobs before inserting.
 CREATE TABLE insight_generation_jobs (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    library_id INTEGER NOT NULL DEFAULT 1,
    file_path TEXT NOT NULL,
    generation_type TEXT NOT NULL,
    status TEXT NOT NULL DEFAULT 'running',
    started_at INTEGER NOT NULL,
    completed_at INTEGER,
    result_insight_id INTEGER,
    error_message TEXT
 );
 -- For the status endpoint: fast lookup by (library_id, file_path)
 CREATE INDEX idx_insight_gen_jobs_file
    ON insight_generation_jobs(library_id, file_path);
 -- For startup cleanup (future): prune old completed/failed jobs
 CREATE INDEX idx_insight_gen_jobs_status_cleanup
    ON insight_generation_jobs(status, started_at);
@@ -1,28 +0,0 @@
 -- Restore UNIQUE constraint
 CREATE TABLE insight_generation_jobs_new (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    library_id INTEGER NOT NULL DEFAULT 1,
    file_path TEXT NOT NULL,
    generation_type TEXT NOT NULL,
    status TEXT NOT NULL DEFAULT 'running',
    started_at INTEGER NOT NULL,
    completed_at INTEGER,
    result_insight_id INTEGER,
    error_message TEXT,
    UNIQUE(library_id, file_path, generation_type)
 );
 INSERT INTO insight_generation_jobs_new
    SELECT id, library_id, file_path, generation_type, status, started_at, completed_at, result_insight_id, error_message
    FROM insight_generation_jobs;
 DROP TABLE insight_generation_jobs;
 ALTER TABLE insight_generation_jobs_new RENAME TO insight_generation_jobs;
 CREATE INDEX idx_insight_gen_jobs_file
    ON insight_generation_jobs(library_id, file_path);
 CREATE INDEX idx_insight_gen_jobs_status_cleanup
    ON insight_generation_jobs(status, started_at);
@@ -1,30 +0,0 @@
 -- Remove UNIQUE(library_id, file_path, generation_type) constraint to allow
 -- multiple job rows per file. This enables proper cancel/regenerate semantics:
 -- a new job is always inserted on regenerate, and the old job is cancelled
 -- independently. The application layer prevents concurrent running jobs.
 CREATE TABLE insight_generation_jobs_new (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    library_id INTEGER NOT NULL DEFAULT 1,
    file_path TEXT NOT NULL,
    generation_type TEXT NOT NULL,
    status TEXT NOT NULL DEFAULT 'running',
    started_at INTEGER NOT NULL,
    completed_at INTEGER,
    result_insight_id INTEGER,
    error_message TEXT
 );
 INSERT INTO insight_generation_jobs_new
    SELECT id, library_id, file_path, generation_type, status, started_at, completed_at, result_insight_id, error_message
    FROM insight_generation_jobs;
 DROP TABLE insight_generation_jobs;
 ALTER TABLE insight_generation_jobs_new RENAME TO insight_generation_jobs;
 CREATE INDEX idx_insight_gen_jobs_file
    ON insight_generation_jobs(library_id, file_path);
 CREATE INDEX idx_insight_gen_jobs_status_cleanup
    ON insight_generation_jobs(status, started_at);
@@ -1,11 +0,0 @@
 -- SQLite doesn't support DROP COLUMN before 3.35.0; recreate the table
 -- without the new columns. This is only needed for rollback.
 CREATE TABLE photo_insights_old AS
    SELECT id, library_id, rel_path, title, summary, generated_at,
           model_version, is_current, training_messages, approved,
           backend, fewshot_source_ids, content_hash
    FROM photo_insights;
 DROP TABLE photo_insights;
 ALTER TABLE photo_insights_old RENAME TO photo_insights;
@@ -1,8 +0,0 @@
 -- Persist generation parameters on each insight row for auditing.
 ALTER TABLE photo_insights ADD COLUMN num_ctx INTEGER;
 ALTER TABLE photo_insights ADD COLUMN temperature REAL;
 ALTER TABLE photo_insights ADD COLUMN top_p REAL;
 ALTER TABLE photo_insights ADD COLUMN top_k INTEGER;
 ALTER TABLE photo_insights ADD COLUMN min_p REAL;
 ALTER TABLE photo_insights ADD COLUMN system_prompt TEXT;
 ALTER TABLE photo_insights ADD COLUMN persona_id TEXT;
@@ -1,13 +0,0 @@
 -- SQLite doesn't support DROP COLUMN before 3.35.0; recreate the table
 -- without the token-count columns. This is only needed for rollback.
 CREATE TABLE photo_insights_old AS
    SELECT id, library_id, rel_path, title, summary, generated_at,
           model_version, is_current, training_messages, approved,
           backend, fewshot_source_ids, content_hash,
           num_ctx, temperature, top_p, top_k, min_p,
           system_prompt, persona_id
    FROM photo_insights;
 DROP TABLE photo_insights;
 ALTER TABLE photo_insights_old RENAME TO photo_insights;
@@ -1,6 +0,0 @@
 -- Persist token usage on each insight row. Split from
 -- 2026-05-27-000002_add_insight_generation_params because that
 -- migration was already applied on some environments before these
 -- columns were added.
 ALTER TABLE photo_insights ADD COLUMN prompt_eval_count INTEGER;
 ALTER TABLE photo_insights ADD COLUMN eval_count INTEGER;
@@ -1,2 +0,0 @@
 DROP INDEX IF EXISTS idx_precomputed_reels_span_library;
 DROP TABLE IF EXISTS precomputed_reels;
@@ -1,14 +0,0 @@
 CREATE TABLE precomputed_reels (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    span TEXT NOT NULL,
    library_key TEXT NOT NULL,
    cache_key TEXT NOT NULL,
    output_path TEXT NOT NULL,
    title TEXT NOT NULL,
    media_count INT NOT NULL,
    render_version INT NOT NULL DEFAULT 1,
    tz_offset_minutes INT NOT NULL,
    voice TEXT,
    generated_at BIGINT NOT NULL
 );
 CREATE INDEX idx_precomputed_reels_span_library ON precomputed_reels(span, library_key, generated_at DESC);
@@ -1 +0,0 @@
 DROP TABLE IF EXISTS user_ai_prefs;
@@ -1,7 +0,0 @@
 CREATE TABLE user_ai_prefs (
    id INTEGER PRIMARY KEY CHECK(id=1),
    voice TEXT,
    tz_offset_minutes INTEGER,
    library TEXT,
    updated_at BIGINT NOT NULL
 );
@@ -1 +0,0 @@
 DROP TABLE IF EXISTS persona_chat_conversations;
@@ -1,22 +0,0 @@
 -- Open chat with persona — file-anchored insight chat is keyed by
 -- (library_id, file_path). The open chat is keyed by (user_id, persona_id)
 -- with a single rolling transcript per pair. No tree branching in v1
 -- (matches the locked-in "single rolling conversation per persona" scope);
 -- a flat JSON blob is enough and avoids forcing a tree shape onto a
 -- surface that's intentionally linear.
 --
 -- `messages_json` is the same `Vec<ChatMessage>` shape the file-anchored
 -- chat persists, so the SSE replay / UI rendering on the mobile client
 -- can reuse the same parser without a second schema.
 CREATE TABLE persona_chat_conversations (
    user_id INTEGER NOT NULL,
    persona_id TEXT NOT NULL,
    messages_json TEXT NOT NULL DEFAULT '[]',
    turn_count INTEGER NOT NULL DEFAULT 0,
    updated_at BIGINT NOT NULL,
    PRIMARY KEY (user_id, persona_id)
 );
 CREATE INDEX idx_persona_chat_updated
    ON persona_chat_conversations (user_id, updated_at DESC);
@@ -1,32 +0,0 @@
 -- Collapse back to one conversation per (user, persona). Where a persona has
 -- several, the most recently updated one wins and the rest are dropped —
 -- the v1 schema has nowhere to put them.
 CREATE TABLE persona_chat_conversations_old (
    user_id INTEGER NOT NULL,
    persona_id TEXT NOT NULL,
    messages_json TEXT NOT NULL DEFAULT '[]',
    turn_count INTEGER NOT NULL DEFAULT 0,
    updated_at BIGINT NOT NULL,
    PRIMARY KEY (user_id, persona_id)
 );
 INSERT INTO persona_chat_conversations_old (
    user_id, persona_id, messages_json, turn_count, updated_at
 )
 SELECT user_id, persona_id, messages_json, turn_count, updated_at
 FROM persona_chat_conversations c
 WHERE c.updated_at = (
    SELECT MAX(c2.updated_at)
    FROM persona_chat_conversations c2
    WHERE c2.user_id = c.user_id AND c2.persona_id = c.persona_id
 )
 GROUP BY user_id, persona_id;
 DROP INDEX IF EXISTS idx_persona_chat_updated;
 DROP INDEX IF EXISTS idx_persona_chat_persona;
 DROP TABLE persona_chat_conversations;
 ALTER TABLE persona_chat_conversations_old RENAME TO persona_chat_conversations;
 CREATE INDEX idx_persona_chat_updated
    ON persona_chat_conversations (user_id, updated_at DESC);
@@ -1,50 +0,0 @@
 -- Multiple conversations per persona.
 --
 -- v1 keyed a transcript on (user_id, persona_id), so a persona had exactly
 -- one rolling conversation and there was no way to start a fresh topic
 -- without discarding the old one. The key is now an opaque `conversation_id`,
 -- with (user_id, persona_id) demoted to an index.
 --
 -- `title` is a short generated summary of the opening exchange, used as the
 -- conversation's name in the list. Empty until the first turn completes; the
 -- client falls back to the persona name while it is blank.
 --
 -- SQLite cannot redefine a primary key in place, so this is the standard
 -- create-copy-drop-rename dance. Existing transcripts carry over with a
 -- generated id and an empty title.
 CREATE TABLE persona_chat_conversations_new (
    conversation_id TEXT NOT NULL PRIMARY KEY,
    user_id INTEGER NOT NULL,
    persona_id TEXT NOT NULL,
    title TEXT NOT NULL DEFAULT '',
    messages_json TEXT NOT NULL DEFAULT '[]',
    turn_count INTEGER NOT NULL DEFAULT 0,
    created_at BIGINT NOT NULL,
    updated_at BIGINT NOT NULL
 );
 INSERT INTO persona_chat_conversations_new (
    conversation_id, user_id, persona_id, title,
    messages_json, turn_count, created_at, updated_at
 )
 SELECT
    lower(hex(randomblob(16))),
    user_id,
    persona_id,
    '',
    messages_json,
    turn_count,
    updated_at,
    updated_at
 FROM persona_chat_conversations;
 DROP INDEX IF EXISTS idx_persona_chat_updated;
 DROP TABLE persona_chat_conversations;
 ALTER TABLE persona_chat_conversations_new RENAME TO persona_chat_conversations;
 CREATE INDEX idx_persona_chat_updated
    ON persona_chat_conversations (user_id, updated_at DESC);
 CREATE INDEX idx_persona_chat_persona
    ON persona_chat_conversations (user_id, persona_id);
@@ -1,392 +0,0 @@
 # Insight Chat improvements — design
 **Date:** 2026-05-07
 **Branch:** `feature/insight-chat-improvements` (in both `ImageApi/` and `FileViewer-React/`)
 **Scope:** ImageApi photo-anchored insight + chat surface, plus the
 FileViewer-React client. Apollo's free/visit chat is **not** in this cycle.
 ## Problem
 Three concrete gaps in today's insight + chat surface:
 1. **Tool drift.** ImageApi exposes 13 tools to the LLM. Some are gated on
   `apollo_enabled` / `has_vision`, but several optional ones
   (`search_rag`, `get_calendar_events`, `get_location_history`) are
   registered unconditionally even when their backing tables are empty.
   Descriptions vary in quality and a couple have outright bugs.
 2. **Inconsistent / incomplete tool descriptions.** Tools like
   `search_messages` describe their selection rules but omit useful
   examples; `store_fact` doesn't show the `object_entity_id` vs
   `object_value` choice; `get_sms_messages` accepts a `days_radius`
   parameter that the backing client silently ignores. The LLM is being
   instructed against a slightly wrong reality.
 3. **System prompt fights the persona.** Today's generation prompt
   prepends the user's `custom_system_prompt` and then immediately asserts
   `"You are a personal photo memory assistant..."`. The user message
   demands `"a detailed insight with a title and summary"`. Both
   contradict whatever voice / shape / POV the persona just established.
   On chat continuation the persona is baked into the stored transcript at
   generation time and can't be changed without regenerating.
 ## Goals
 - Tool catalog is **representative** — every tool registered for a turn is
  backed by data the user actually has.
 - Tool descriptions are **concise but complete**, with examples for any
  tool whose param choice has multiple modes or non-obvious interactions.
 - Persona / system prompt is **authoritative** for voice, length, and
  shape — both at generation and during chat continuation.
 - Per-turn system prompt overrides on chat work without surprising
  side-effects on the stored transcript outside `amend` mode.
 ## Non-goals
 - Apollo backend / frontend changes. Separate cycle.
 - Refactoring the `generate_photo_title` post-hoc title flow. Already
  takes `custom_system_prompt`.
 - Tool consolidation (e.g. merging `search_messages` + `get_sms_messages`).
  Considered and deferred — keeps blast radius small.
 - Removing knowledge-memory tools (`recall_*` / `store_*`). Audit
  confirmed they have a live read path via `knowledge.rs` HTTP routes.
 - Persisting persona changes to the stored transcript outside `amend`
  mode. Deliberate — re-opens use the persona currently active in the
  client, not a sticky historical setting.
 ---
 ## Design
 ### A. System prompt — generation
 Today (`insight_generator.rs:3305–3326`):
 ```
 [custom_system_prompt if any] +
 "You are a personal photo memory assistant helping to reconstruct..." +
 {owner_id_note} +
 {fewshot_block} +
 "IMPORTANT INSTRUCTIONS:
 1. You MUST call multiple tools...
 2. When calling get_sms_messages and search_rag...
 3. Use recall_facts_for_photo...
 ...
 8. You have a hard budget of {max_iterations} iterations..."
 ```
 The first concatenation is the bug: `custom` claims one identity, the
 next line asserts another.
 **New structure** — two named blocks, in order:
 ```
 [Identity / voice / format block]    ← persona-controlled (or neutral default)
 [Procedural block]                   ← always identity-free
 ```
 **Identity block:**
 - When `custom_system_prompt` is supplied: use that string verbatim, no
  pre/append.
 - When not: a neutral default that doesn't fight a future persona.
  Working text: `"You are reconstructing a memory from a photo. Use the
  gathered context to write a thoughtful summary; you decide voice,
  length, and shape."`
 **Procedural block** — identity-free, always emitted:
 ```
 Tool-use guidance:
 - You have a budget of {max_iterations} tool-calling iterations.
 - Call tools to gather context BEFORE writing your final answer; don't
  answer after one or two calls.
 - When calling get_sms_messages or search_rag, make at least one call
  WITHOUT a contact filter — surrounding events matter even when a
  contact is known.
 - Use recall_facts_for_photo + recall_entities to load any prior
  knowledge about subjects in the photo.
 - When you identify people / places / events / things, use store_entity
  + store_fact to grow the persistent memory.
 - A tool returning no results is informative; continue with the others.
 {owner_id_note if applicable}
 {fewshot_block if applicable}
 ```
 Differences from today's "IMPORTANT INSTRUCTIONS" block: removed the
 "you are a personal photo memory assistant" framing and the explicit
 "at least 5 tool calls" floor (replaced with the softer "don't answer
 after one or two"). Few-shot stays — it's pattern-of-tool-use, not
 identity.
 ### B. User message — generation
 Today (line 3357):
 ```
 {visual_block}Please analyze this photo and gather any relevant context
 from the surrounding weeks.
 Photo file path: {file_path}
 Date taken: {date}
 {contact_info}
 {gps_info}
 {tags_info}
 Use the available tools to gather more context about this moment
 (messages, calendar events, location history, etc.), then write a
 detailed insight with a title and summary.
 ```
 Problems: the trailing line bakes in output shape ("title and
 summary"), and the title from the resulting response is **discarded
 anyway** — `generate_photo_title` (line 3494) regenerates the title
 post-hoc from the summary. So the prompt is constraining voice for no
 data-model benefit.
 **New payload** — context-only, no output prescription:
 ```
 {visual_block}Photo file path: {file_path}
 Date taken: {date}
 {contact_info}
 {gps_info}
 {tags_info}
 Gather context with the available tools, then respond.
 ```
 The persona owns shape. If a user wants "title-then-paragraph" output,
 their persona prompt says so.
 ### C. System prompt — chat continuation
 Add `system_prompt: Option<String>` to `ChatTurnRequest` (and to its
 HTTP wrapper `ChatTurnHttpRequest`). It carries through both the
 non-streaming `chat_turn` and the streaming `chat_turn_stream`.
 **Append mode (default, `amend=false`)** — ephemeral
 swap-and-restore, mirroring the existing `annotate_system_with_budget`
 pattern:
 1. Load stored transcript.
 2. If `system_prompt` is `Some(s)`:
   - If first message is a `system` role: stash original content,
     replace with `s`.
   - Else: prepend a synthetic ephemeral system message with `s` (note
     it's synthetic so the restore step pops it rather than rewriting).
 3. Run `annotate_system_with_budget` on top (existing per-turn budget
   note appends to whatever's there now).
 4. Run the agentic loop.
 5. **Before persistence**, restore the original system content (or pop
   the synthetic one). Run `restore_system_content` for the budget
   annotation as today.
 6. Save.
 Result: the model sees the override; the stored transcript is
 unchanged outside the model's actual reply.
 **Amend mode (`amend=true`)**:
 - If `system_prompt` is supplied: the override stays in place during
  the serialization for the new insight row. The new row's
  `training_messages` system message is the override. `is_current=false`
  flips on prior rows as today.
 - If not supplied: behaves as today (stored transcript's system message
  carries forward unchanged).
 ### D. FileViewer-React — client wiring
 `hooks/useInsightChat.tsx`:
 - `SendTurnOptions` gains `systemPromptOverride?: string | null`.
 - Inside `sendTurn`, before issuing the streaming POST:
  1. Read the active persona's `systemPrompt` from AsyncStorage
     (already loaded for generation flows — reuse the same accessor).
  2. If a one-shot `systemPromptOverride` is set, append as a suffix
     (`${persona}\n\n${override}`) so persona voice survives + override
     tweaks the turn.
  3. Include the resulting string as `system_prompt` on the request body.
 - No history-load change. The history endpoint still returns the stored
  transcript.
 `components/InsightChatModal.tsx`:
 - Add a small "Style note" composer affordance — a one-shot text input
  that, when filled, becomes the `systemPromptOverride` for the next
  send. Cleared after send.
 - The existing persona chip continues to open `PersonaManagerModal`.
 `hooks/usePersonas.tsx` and the bundled defaults:
 - Built-in `assistant` and `journal` prompts get audited and rewritten
  to **explicitly state voice / shape / length** — since the framework
  no longer guarantees a default shape, the persona must.
 ### E. Tool catalog — gating
 Widen `build_tool_definitions` from `(has_vision: bool, apollo_enabled:
 bool)` to a single `ToolGateOpts` struct:
 ```rust
 pub struct ToolGateOpts {
    pub has_vision: bool,
    pub apollo_enabled: bool,
    pub daily_summaries_present: bool,
    pub calendar_present: bool,
    pub location_history_present: bool,
 }
 ```
 The chat / generation services compute the three new fields lazily per
 turn via `SELECT 1 FROM <table> LIMIT 1` (cheap; cached for the turn's
 duration). Lazy because operators import data after launch and we don't
 want to require a restart for the LLM to discover its new capabilities.
 Per-tool gating:
 | Tool | Existing gate | New gate |
 |---|---|---|
 | `describe_photo` | `has_vision` | unchanged |
 | `get_personal_place_at` | `apollo_enabled` | unchanged |
 | `get_calendar_events` | none | `calendar_present` |
 | `get_location_history` | none | `location_history_present` |
 | `search_rag` | none | `daily_summaries_present` |
 All other tools always-on. (`get_sms_messages` and `search_messages`
 fail informatively if SMS-API is unreachable; not worth a startup probe
 since intermittent failures are the same shape.)
 ### F. Tool descriptions — convention
 Every description follows:
 1. One sentence: **what** + **when to call**.
 2. Param semantics worth knowing (units, ranges, mode behavior,
   precedence).
 3. **Example invocation** for tools with multiple modes, optional bands,
   or non-obvious parameter interactions.
 4. Cross-references when relevant: `prefer X when both apply`.
 Banned: all-caps section headers inside descriptions
 (`"CONTENT search"`, `"TIME-BASED fetch"`); persona-prescriptive language
 (`"you are a..."`); behavioral references to other tools by description
 rather than name.
 Tools getting examples: `search_messages`, `search_rag`, `store_fact`,
 `get_sms_messages`. Trivial tools (`get_current_datetime`,
 `reverse_geocode`, `get_file_tags`) skip the example.
 Sample (`search_messages`):
 > Search SMS/MMS message bodies. Modes: `fts5` (keyword + phrase + prefix
 > + AND/OR/NOT + NEAR proximity), `semantic` (embedding similarity,
 > requires generated embeddings), `hybrid` (RRF merge, recommended;
 > degrades to `fts5` when embeddings absent). Optional `start_ts` /
 > `end_ts` (real-UTC unix seconds) and `contact_id` filters. For pure
 > date / contact browsing without keywords, prefer `get_sms_messages`.
 >
 > Examples:
 > - `{query: "trader joe's"}` — phrase across all time.
 > - `{query: "dinner", contact_id: 42, start_ts: 1700000000, end_ts: 1700604800}`
 >   — keyword within a contact and a week.
 > - `{query: "NEAR(meeting work, 5)"}` — proximity search.
 ### G. SMS tool fixes
 #### `get_sms_messages` — honor `days_radius`
 Today: `sms_client::fetch_messages_for_contact(contact, center_ts)`
 hardcodes `Duration::days(4)` (lines 31–37). The tool accepts
 `days_radius` and silently ignores it.
 **Fix:** widen the signature to
 `fetch_messages_for_contact(contact, center_ts, days_radius)`. Tool
 plumbs through. Default 4 retained for back-compat.
 #### `search_messages` — add date and contact_id filters
 Today: ImageApi's `search_messages` only forwards `query`, `mode`,
 `limit` to SMS-API.
 **Fix:** add `start_ts`, `end_ts`, `contact_id` parameters.
 - `contact_id` forwards directly to SMS-API
  (`/api/messages/search/?contact_id=`).
 - `start_ts` / `end_ts` are not natively accepted by SMS-API's search
  endpoint. Apply client-side post-filter on the response (Apollo's
  pattern: `chat_tools.py:670–680`). Bump the SMS-API `limit` to a
  larger fetch pool when a date filter is supplied so in-window matches
  aren't lost to out-of-window FTS rank.
 ---
 ## Implementation sequencing
 Each step is independently mergeable.
 ### ImageApi PRs
 1. **Split system-prompt assembly + neutralize user message.** Two
   named blocks; user message context-only. Default identity string
   added. Tests: golden snapshots of the resulting `system_content`
   with and without `custom_system_prompt`.
 2. **`system_prompt` field on chat request + swap/restore + amend
   persistence.** Mirrors `annotate_system_with_budget` pattern. Tests:
   round-trip system content unchanged in append mode; persisted in
   amend mode.
 3. **`fetch_messages_for_contact` honors `days_radius`.** Tool wires
   the param through. Tests: window math at the client level.
 4. **`ToolGateOpts` + per-tool description rewrites.** Description
   text changes are the bulk of the diff but no behavior change beyond
   gating.
 ### FileViewer-React PR
 5. **Chat hook sends `system_prompt`; modal gets style-note input;
   built-in personas updated to specify shape.** The
   `useInsightChat.sendTurn` call site picks up the persona and
   includes it on every chat turn body. Style-note input is a one-shot
   suffix.
 ## Testing & verification
 **Automated:**
 - Unit (Rust): swap-and-restore round-trip preserves stored transcript.
 - Unit (Rust): amend mode persists override into new insight row.
 - Unit (Rust): `fetch_messages_for_contact(days_radius=N)` produces a
  window of `2N` days centered on `center_ts`.
 - Unit (Rust): `build_tool_definitions(opts)` excludes gated tools when
  the corresponding flag is false.
 **Manual:**
 - Run a chat turn against an existing insight without `system_prompt` →
  output unchanged from baseline.
 - Same insight, with override → output reflects new voice.
 - Re-open chat → original baked persona still authoritative (override
  was ephemeral).
 - Regenerate an insight with the journal persona → model's voice
  matches journal style; no "memory assistant" framing leaks through.
 - Toggle data presence (delete a row from `calendar_events`) → tool
  drops from the catalog on the next turn.
 ## Risks
 - **Default identity wording matters.** A too-neutral default ("Use the
  gathered context to write a summary") might produce flatter output
  than today's "personal photo memory assistant" framing for users
  who never set a persona. Mitigation: tune the default with a small
  set of test photos before merging.
 - **Persona-suffix style notes can contradict persona voice.** A user
  who picks `journal` (first person, warm) and adds the style note
  "respond in bullet points" will get a tonal collision. Acceptable —
  the user expressed a per-turn intent and we honor it. Document the
  composition rule in the persona-manager UI.
 - **Lazy data-presence probes add a per-turn `SELECT 1`.** Negligible
  on SQLite (sub-millisecond) but adds up across many turns. Cache the
  result for the turn's duration; don't re-probe per-tool.
 ## Open questions
 None blocking. Items deferred to a possible follow-up cycle:
 - Apollo parity for the same per-turn override pattern (already
  present; just needs RN client wiring on the photo path which is
  already proxy).
 - Tool consolidation (`search_messages` + `get_sms_messages` →
  single `search_messages` with optional date filter, Apollo-style).
  Considered and deferred — separate spec.
@@ -1,146 +0,0 @@
 use anyhow::{Result, anyhow};
 use crate::ai::llm_client::LlmClient;
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub enum BackendKind {
    Local,
    Hybrid,
 }
 impl BackendKind {
    pub fn parse(s: &str) -> Result<Self> {
        match s.trim().to_lowercase().as_str() {
            "local" | "" => Ok(Self::Local),
            "hybrid" => Ok(Self::Hybrid),
            other => Err(anyhow!(
                "unknown backend '{}'; expected 'local' or 'hybrid'",
                other
            )),
        }
    }
    pub fn as_str(&self) -> &'static str {
        match self {
            Self::Local => "local",
            Self::Hybrid => "hybrid",
        }
    }
 }
 impl std::fmt::Display for BackendKind {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.write_str(self.as_str())
    }
 }
 pub struct SamplingOverrides {
    pub model: Option<String>,
    pub num_ctx: Option<i32>,
    pub temperature: Option<f32>,
    pub top_p: Option<f32>,
    pub top_k: Option<i32>,
    pub min_p: Option<f32>,
    /// Reasoning toggle. Only the llama.cpp backend honors it (forwarded as
    /// `chat_template_kwargs.enable_thinking`); other backends ignore it.
    /// `None` leaves the model/template default in place.
    pub enable_thinking: Option<bool>,
 }
 impl SamplingOverrides {
    pub fn has_sampling(&self) -> bool {
        self.temperature.is_some()
            || self.top_p.is_some()
            || self.top_k.is_some()
            || self.min_p.is_some()
    }
 }
 pub struct ResolvedBackend {
    chat: Box<dyn LlmClient>,
    local: Box<dyn LlmClient>,
    pub kind: BackendKind,
    /// `true` when the chat model receives images directly (Ollama with
    /// vision, or llamacpp). `false` for hybrid where we describe-then-inline.
    pub images_inline: bool,
 }
 impl ResolvedBackend {
    pub fn new(
        chat: Box<dyn LlmClient>,
        local: Box<dyn LlmClient>,
        kind: BackendKind,
        images_inline: bool,
    ) -> Self {
        Self {
            chat,
            local,
            kind,
            images_inline,
        }
    }
    pub fn chat(&self) -> &dyn LlmClient {
        self.chat.as_ref()
    }
    pub fn local(&self) -> &dyn LlmClient {
        self.local.as_ref()
    }
    pub fn model(&self) -> &str {
        self.chat.primary_model()
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    #[test]
    fn parse_backend_kind() {
        assert_eq!(BackendKind::parse("local").unwrap(), BackendKind::Local);
        assert_eq!(BackendKind::parse("hybrid").unwrap(), BackendKind::Hybrid);
        assert_eq!(BackendKind::parse("  Local ").unwrap(), BackendKind::Local);
        assert_eq!(BackendKind::parse("HYBRID").unwrap(), BackendKind::Hybrid);
        assert_eq!(BackendKind::parse("").unwrap(), BackendKind::Local);
        assert!(BackendKind::parse("vllm").is_err());
    }
    #[test]
    fn backend_kind_as_str_roundtrips() {
        assert_eq!(
            BackendKind::parse(BackendKind::Local.as_str()).unwrap(),
            BackendKind::Local
        );
        assert_eq!(
            BackendKind::parse(BackendKind::Hybrid.as_str()).unwrap(),
            BackendKind::Hybrid
        );
    }
    #[test]
    fn sampling_overrides_has_sampling() {
        let empty = SamplingOverrides {
            model: None,
            num_ctx: None,
            temperature: None,
            top_p: None,
            top_k: None,
            min_p: None,
            enable_thinking: None,
        };
        assert!(!empty.has_sampling());
        let with_temp = SamplingOverrides {
            model: None,
            num_ctx: Some(4096),
            temperature: Some(0.7),
            top_p: None,
            top_k: None,
            min_p: None,
            enable_thinking: None,
        };
        assert!(with_temp.has_sampling());
    }
 }
@@ -1,395 +0,0 @@
 //! Thin async HTTP client for Apollo's `/api/internal/clip/*` endpoints.
 //!
 //! Apollo hosts the OpenAI CLIP inference service (ViT-L/14 by default,
 //! configurable via `APOLLO_CLIP_MODEL`). This client is the ImageApi side
 //! of the contract: shove image bytes through `/encode_image` to populate
 //! `image_exif.clip_embedding` during backfill, and call `/encode_text` to
 //! encode a user's natural-language query at search time. The actual
 //! cosine-similarity rerank runs locally in ImageApi.
 //!
 //! Mirrors `face_client.rs` / `tag_client.rs` shape: optional base URL
 //! (None = disabled — feature off, drain and search no-op), reqwest
 //! client with a generous timeout because GPU inference under a backlog
 //! can queue server-side (Apollo's threadpool is bounded to 1 worker on
 //! CUDA).
 //!
 //! Configured via `APOLLO_CLIP_API_BASE_URL`, falling back to
 //! `APOLLO_API_BASE_URL` when the dedicated var is unset (single-Apollo
 //! deploys are the common case).
 //!
 //! Wire format:
 //! - `/encode_image`: multipart/form-data with `file=<bytes>` and
 //!   `meta=<json>` (content_hash / library_id / rel_path for logging).
 //! - `/encode_text`: JSON `{"text": "<query>"}`.
 //!
 //! Both return `{model_version, embedding_dim, duration_ms, embedding}`
 //! where `embedding` is base64 of `dim×4` little-endian float32 bytes,
 //! L2-normalized so the rerank reduces to a plain dot product.
 //!
 //! Error mapping (reflected in [`ClipError`]):
 //! - 422 `decode_failed` / `empty_text` → permanent: ImageApi marks the
 //!   row failed or surfaces the empty-query error to the search caller.
 //! - 503 `cuda_oom` / `engine_unavailable` → defer-and-retry: no marker.
 //! - Any other 5xx / network error → defer.
 use anyhow::{Context, Result};
 use base64::Engine;
 use reqwest::Client;
 use serde::{Deserialize, Serialize};
 use std::time::Duration;
 #[derive(Debug, Clone, Serialize)]
 pub struct EncodeImageMeta {
    pub content_hash: String,
    pub library_id: i32,
    pub rel_path: String,
 }
 #[derive(Debug, Clone, Deserialize)]
 #[allow(dead_code)] // duration_ms logged by the backfill drain
 pub struct EncodeResponse {
    pub model_version: String,
    pub embedding_dim: i32,
    pub duration_ms: i64,
    /// base64 of `embedding_dim * 4` bytes (LE float32). ImageApi stores
    /// the decoded bytes verbatim as a BLOB.
    pub embedding: String,
 }
 impl EncodeResponse {
    /// Decode the wire-format embedding back into raw bytes for storage.
    /// Validates the buffer is `embedding_dim * 4` bytes long so a
    /// malformed response surfaces here rather than as a downstream
    /// silent length mismatch.
    pub fn decode_embedding(&self) -> Result<Vec<u8>> {
        let bytes = base64::engine::general_purpose::STANDARD
            .decode(self.embedding.as_bytes())
            .context("clip embedding base64 decode")?;
        let expected = (self.embedding_dim as usize) * 4;
        if bytes.len() != expected {
            anyhow::bail!(
                "clip embedding wrong size: got {} bytes, expected {} ({} * 4)",
                bytes.len(),
                expected,
                self.embedding_dim
            );
        }
        Ok(bytes)
    }
 }
 #[derive(Debug, Clone, Deserialize)]
 #[allow(dead_code)] // load_error consumed by future health probe
 pub struct ClipHealth {
    pub loaded: bool,
    pub device: String,
    pub model_version: String,
    pub embedding_dim: i32,
    #[serde(default)]
    pub load_error: Option<String>,
 }
 #[derive(Debug)]
 pub enum ClipError {
    /// Apollo refused for a reason that won't change on retry (decode
    /// failure on /encode_image, empty text on /encode_text).
    Permanent(anyhow::Error),
    /// Apollo couldn't process this turn but might next time (CUDA OOM,
    /// engine not loaded, network hiccup).
    Transient(anyhow::Error),
    /// Feature is disabled (no `APOLLO_CLIP_API_BASE_URL` /
    /// `APOLLO_API_BASE_URL`).
    Disabled,
 }
 impl std::fmt::Display for ClipError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            ClipError::Permanent(e) => write!(f, "permanent: {e}"),
            ClipError::Transient(e) => write!(f, "transient: {e}"),
            ClipError::Disabled => write!(f, "clip client disabled"),
        }
    }
 }
 impl std::error::Error for ClipError {}
 #[derive(Clone)]
 pub struct ClipClient {
    client: Client,
    base_url: Option<String>,
 }
 impl ClipClient {
    pub fn new(base_url: Option<String>) -> Self {
        let timeout_secs = std::env::var("CLIP_REQUEST_TIMEOUT_SEC")
            .ok()
            .and_then(|s| s.parse::<u64>().ok())
            .unwrap_or(60);
        let client = Client::builder()
            .timeout(Duration::from_secs(timeout_secs))
            .build()
            .expect("reqwest client build");
        Self {
            client,
            base_url: base_url.map(|u| u.trim_end_matches('/').to_string()),
        }
    }
    /// Read both standard env vars. `APOLLO_CLIP_API_BASE_URL` wins;
    /// fallback to `APOLLO_API_BASE_URL`. Both unset → disabled.
    pub fn from_env() -> Self {
        let base = std::env::var("APOLLO_CLIP_API_BASE_URL")
            .ok()
            .filter(|s| !s.trim().is_empty())
            .or_else(|| {
                std::env::var("APOLLO_API_BASE_URL")
                    .ok()
                    .filter(|s| !s.trim().is_empty())
            });
        Self::new(base)
    }
    pub fn is_enabled(&self) -> bool {
        self.base_url.is_some()
    }
    /// Encode an image to a 768-d (ViT-L/14) or 512-d (ViT-B/32)
    /// L2-normalized embedding. Used by the backfill drain.
    pub async fn encode_image(
        &self,
        bytes: Vec<u8>,
        meta: EncodeImageMeta,
    ) -> std::result::Result<EncodeResponse, ClipError> {
        let Some(base) = self.base_url.as_deref() else {
            return Err(ClipError::Disabled);
        };
        let url = format!("{}/api/internal/clip/encode_image", base);
        let meta_json = serde_json::to_string(&meta)
            .map_err(|e| ClipError::Permanent(anyhow::anyhow!("meta serialize: {e}")))?;
        let form = reqwest::multipart::Form::new()
            .text("meta", meta_json)
            .part(
                "file",
                reqwest::multipart::Part::bytes(bytes)
                    .file_name(meta.rel_path.clone())
                    .mime_str("application/octet-stream")
                    .unwrap_or_else(|_| reqwest::multipart::Part::bytes(Vec::new())),
            );
        self.send_multipart(&url, form).await
    }
    /// Encode a natural-language query to an embedding. Used by the
    /// search route to rank stored image embeddings by cosine sim.
    pub async fn encode_text(&self, text: &str) -> std::result::Result<EncodeResponse, ClipError> {
        let Some(base) = self.base_url.as_deref() else {
            return Err(ClipError::Disabled);
        };
        let url = format!("{}/api/internal/clip/encode_text", base);
        let body = serde_json::json!({ "text": text });
        let resp = match self.client.post(&url).json(&body).send().await {
            Ok(r) => r,
            Err(e) if e.is_timeout() || e.is_connect() => {
                log::warn!("clip encode_text network error to {url}: {e}");
                return Err(ClipError::Transient(anyhow::anyhow!(
                    "clip client network: {e}"
                )));
            }
            Err(e) => {
                log::warn!("clip encode_text request error to {url}: {e}");
                return Err(ClipError::Transient(anyhow::anyhow!(
                    "clip client request: {e}"
                )));
            }
        };
        let status = resp.status();
        if status.is_success() {
            let body: EncodeResponse = resp
                .json()
                .await
                .map_err(|e| ClipError::Transient(anyhow::anyhow!("clip response decode: {e}")))?;
            return Ok(body);
        }
        let body_text = resp.text().await.unwrap_or_default();
        log::warn!("clip encode_text HTTP {status} from {url}: {body_text}");
        Err(classify_error_response(status.as_u16(), &body_text))
    }
    /// Engine reachability + device/model report. Used as a startup
    /// sanity check from the probe binary and (later) the backlog drain.
    #[allow(dead_code)] // consumed by probe + drain
    pub async fn health(&self) -> Result<ClipHealth> {
        let base = self.base_url.as_deref().context("clip client disabled")?;
        let url = format!("{}/api/internal/clip/health", base);
        let resp = self.client.get(&url).send().await?.error_for_status()?;
        let body: ClipHealth = resp.json().await?;
        Ok(body)
    }
    async fn send_multipart(
        &self,
        url: &str,
        form: reqwest::multipart::Form,
    ) -> std::result::Result<EncodeResponse, ClipError> {
        let resp = match self.client.post(url).multipart(form).send().await {
            Ok(r) => r,
            Err(e) if e.is_timeout() || e.is_connect() => {
                return Err(ClipError::Transient(anyhow::anyhow!(
                    "clip client network: {e}"
                )));
            }
            Err(e) => {
                return Err(ClipError::Transient(anyhow::anyhow!(
                    "clip client request: {e}"
                )));
            }
        };
        let status = resp.status();
        if status.is_success() {
            let body: EncodeResponse = resp
                .json()
                .await
                .map_err(|e| ClipError::Transient(anyhow::anyhow!("clip response decode: {e}")))?;
            return Ok(body);
        }
        let body_text = resp.text().await.unwrap_or_default();
        Err(classify_error_response(status.as_u16(), &body_text))
    }
 }
 /// Pulled out as a pure function so the marker-row contract is unit-
 /// testable without spinning up an HTTP server. Matches the shape used
 /// by face_client::classify_error_response so future retry policies
 /// can share code.
 fn classify_error_response(status: u16, body_text: &str) -> ClipError {
    let detail_code = serde_json::from_str::<serde_json::Value>(body_text)
        .ok()
        .and_then(|v| {
            v.get("detail")
                .and_then(|d| d.as_str().map(str::to_string))
                .or_else(|| {
                    v.get("detail")
                        .and_then(|d| d.get("code"))
                        .and_then(|c| c.as_str())
                        .map(str::to_string)
                })
        })
        .unwrap_or_default();
    if status == 422 {
        return ClipError::Permanent(anyhow::anyhow!(
            "clip {} {}: {}",
            status,
            detail_code,
            body_text
        ));
    }
    if status == 503 {
        return ClipError::Transient(anyhow::anyhow!(
            "clip {} {}: {}",
            status,
            detail_code,
            body_text
        ));
    }
    // 408 / 413 / 429 are operator-fixable infra issues; defer.
    if matches!(status, 408 | 413 | 429) {
        return ClipError::Transient(anyhow::anyhow!(
            "clip {} {}: {}",
            status,
            detail_code,
            body_text
        ));
    }
    if (400..500).contains(&status) {
        ClipError::Permanent(anyhow::anyhow!(
            "clip {} {}: {}",
            status,
            detail_code,
            body_text
        ))
    } else {
        ClipError::Transient(anyhow::anyhow!(
            "clip {} {}: {}",
            status,
            detail_code,
            body_text
        ))
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    fn is_permanent(e: &ClipError) -> bool {
        matches!(e, ClipError::Permanent(_))
    }
    fn is_transient(e: &ClipError) -> bool {
        matches!(e, ClipError::Transient(_))
    }
    #[test]
    fn classify_422_decode_failed_is_permanent() {
        assert!(is_permanent(&classify_error_response(
            422,
            r#"{"detail":"decode_failed: bad bytes"}"#
        )));
    }
    #[test]
    fn classify_422_empty_text_is_permanent() {
        assert!(is_permanent(&classify_error_response(
            422,
            r#"{"detail":"empty_text"}"#
        )));
    }
    #[test]
    fn classify_503_cuda_oom_is_transient() {
        assert!(is_transient(&classify_error_response(
            503,
            r#"{"detail":{"code":"cuda_oom","error":"out of memory"}}"#,
        )));
    }
    #[test]
    fn classify_5xx_is_transient_other_4xx_is_permanent() {
        assert!(is_transient(&classify_error_response(500, "")));
        assert!(is_permanent(&classify_error_response(404, "{}")));
    }
    #[test]
    fn classify_infra_4xx_is_transient() {
        assert!(is_transient(&classify_error_response(408, "")));
        assert!(is_transient(&classify_error_response(413, "<html>")));
        assert!(is_transient(&classify_error_response(429, "{}")));
    }
    #[test]
    fn decode_embedding_size_mismatch_errors() {
        // dim=4 says we expect 16 bytes (4 floats × 4 bytes). Encode 8.
        use base64::Engine;
        let resp = EncodeResponse {
            model_version: "ViT-L/14".into(),
            embedding_dim: 4,
            duration_ms: 0,
            embedding: base64::engine::general_purpose::STANDARD.encode([0u8; 8]),
        };
        assert!(resp.decode_embedding().is_err());
    }
    #[test]
    fn decode_embedding_round_trip() {
        use base64::Engine;
        let bytes: Vec<u8> = (0..16).collect();
        let resp = EncodeResponse {
            model_version: "ViT-L/14".into(),
            embedding_dim: 4,
            duration_ms: 0,
            embedding: base64::engine::general_purpose::STANDARD.encode(&bytes),
        };
        assert_eq!(resp.decode_embedding().unwrap(), bytes);
    }
 }
@@ -1,88 +0,0 @@
 // GPU lease — in-process coordination for llama-swap model contention.
 //
 // llama-swap runs the heavyweight models (chat / vision / Chatterbox TTS) as
 // a mutually-exclusive set on one GPU (matrix DSL `(q27 | … | tts) & e`): a
 // request for a non-resident model is HELD by llama-swap until the resident
 // model's in-flight requests drain, then the models swap. That hold counts
 // against the *holder's* reqwest timeout — measured live: a queued TTS burned
 // 77s of its budget behind a single LLM turn, and an LLM request behind a
 // running synthesis waited the entire remaining synth. Uncoordinated
 // cross-model traffic therefore times out instead of queueing.
 //
 // The lease moves that wait into this process, BEFORE the HTTP request is
 // sent and before its timeout starts:
 // - chat/vision requests (the LLM-side slots) share the READ lease;
 // - TTS synthesis and voice-library ops (anything that spins Chatterbox up
 //   and evicts the LLM) take the WRITE lease;
 // - embeddings take NO lease: the `embed` slot is in llama-swap's
 //   always-resident group (the `& e` term) and never participates in a swap,
 //   so leasing it would only stall searches behind a queued synthesis.
 //
 // tokio's RwLock is fair (FIFO, write-preferring): a queued TTS gets the GPU
 // right after the current LLM request drains, and later LLM requests queue
 // behind it — bounded waits in both directions, no starvation, no timeout
 // budget burned while waiting.
 //
 // RULES: hold a lease for exactly one HTTP request (for streaming, the
 // stream's lifetime) and NEVER acquire one while already holding one — once a
 // writer is queued, new read acquisitions block, so nested acquisition can
 // deadlock.
 use std::sync::LazyLock;
 use std::time::Instant;
 use tokio::sync::{RwLock, RwLockReadGuard, RwLockWriteGuard};
 static GPU_LEASE: LazyLock<RwLock<()>> = LazyLock::new(|| RwLock::new(()));
 /// Waits longer than this are logged — they mean a cross-model swap was
 /// avoided and quantify what the request *would* have burned of its timeout.
 const SLOW_WAIT_LOG_SECS: f64 = 2.0;
 /// Shared lease for LLM-side requests (chat / vision slots).
 pub async fn llm_lease() -> RwLockReadGuard<'static, ()> {
    let started = Instant::now();
    let guard = GPU_LEASE.read().await;
    log_slow_wait("llm", started);
    guard
 }
 /// Exclusive lease for TTS-side requests (speech synthesis + voice-library
 /// ops that spin up Chatterbox).
 pub async fn tts_lease() -> RwLockWriteGuard<'static, ()> {
    let started = Instant::now();
    let guard = GPU_LEASE.write().await;
    log_slow_wait("tts", started);
    guard
 }
 fn log_slow_wait(kind: &str, started: Instant) {
    let waited = started.elapsed().as_secs_f64();
    if waited > SLOW_WAIT_LOG_SECS {
        log::info!("GPU lease ({kind}): waited {waited:.1}s for the other model class to drain");
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    // One sequential test, not several: the lease is a single global, so
    // parallel tests interleaving reads and writes on it can hit the very
    // nested-acquisition deadlock the module comment warns about.
    #[tokio::test]
    async fn write_lease_excludes_readers_then_reads_share() {
        let w = tts_lease().await;
        // A reader must not acquire while the writer is held.
        let pending = tokio::spawn(async { drop(llm_lease().await) });
        tokio::task::yield_now().await;
        assert!(!pending.is_finished());
        drop(w);
        pending.await.expect("reader acquires after writer drops");
        // With no writer queued, read leases are shared.
        let a = llm_lease().await;
        let b = llm_lease().await;
        drop(a);
        drop(b);
    }
 }
@@ -170,621 +170,3 @@ pub struct ModelCapabilities {
    pub has_vision: bool,
    pub has_tool_calling: bool,
 }
 /// A single node in the conversation tree. Wraps ChatMessage with structural
 /// metadata so the tree can be persisted as JSON in `training_messages`.
 #[derive(Serialize, Deserialize, Clone, Debug)]
 pub struct StoredChatNode {
    pub id: u64,
    #[serde(skip_serializing_if = "Option::is_none")]
    pub parent_id: Option<u64>,
    pub message: ChatMessage,
 }
 /// Top-level container for tree-based chat history. Stored as JSON in the
 /// `training_messages` column. Backward-compatible reader detects whether
 /// the stored value is a flat array (old format) or this object (new format).
 #[derive(Serialize, Deserialize, Clone, Debug)]
 pub struct ChatHistoryStore {
    pub nodes: Vec<StoredChatNode>,
    pub active_leaf_id: u64,
 }
 impl ChatHistoryStore {
    /// Create an empty store. Test-only: production stores are always built
    /// from `from_flat_array` or deserialized from `training_messages`.
    #[cfg(test)]
    pub fn empty() -> Self {
        Self {
            nodes: Vec::new(),
            active_leaf_id: 0,
        }
    }
    /// Convert a flat `Vec<ChatMessage>` (old format) into a tree.
    /// Each message becomes a node chained by parent_id.
    pub fn from_flat_array(messages: Vec<ChatMessage>) -> Self {
        let mut nodes = Vec::with_capacity(messages.len());
        let mut prev_id: Option<u64> = None;
        for (i, msg) in messages.into_iter().enumerate() {
            let id = (i + 1) as u64;
            nodes.push(StoredChatNode {
                id,
                parent_id: prev_id,
                message: msg,
            });
            prev_id = Some(id);
        }
        let active_leaf_id = prev_id.unwrap_or(0);
        Self {
            nodes,
            active_leaf_id,
        }
    }
    /// Allocate the next node ID (max existing + 1, or 1 if empty).
    pub fn next_id(&self) -> u64 {
        self.nodes.iter().map(|n| n.id).max().unwrap_or(0) + 1
    }
    /// Append a new node as a child of the given parent. Returns the new node's id.
    pub fn append_node(&mut self, parent_id: Option<u64>, message: ChatMessage) -> u64 {
        let id = self.next_id();
        self.nodes.push(StoredChatNode {
            id,
            parent_id,
            message,
        });
        id
    }
    /// Traverse from root to the given leaf_id, returning the path in order.
    /// Returns None if the leaf_id doesn't exist.
    pub fn path_to_leaf(&self, leaf_id: u64) -> Option<Vec<&StoredChatNode>> {
        // Build a map from id to node for fast lookup.
        let node_map: std::collections::HashMap<u64, usize> = self
            .nodes
            .iter()
            .enumerate()
            .map(|(i, n)| (n.id, i))
            .collect();
        let &_leaf_idx = node_map.get(&leaf_id)?;
        let mut path_indices = Vec::new();
        let mut current_id = leaf_id;
        while let Some(&idx) = node_map.get(&current_id) {
            path_indices.push(idx);
            match self.nodes[idx].parent_id {
                Some(pid) => current_id = pid,
                None => break,
            }
        }
        path_indices.reverse();
        Some(path_indices.into_iter().map(|i| &self.nodes[i]).collect())
    }
    /// All direct children of a given node, ordered by id (chronological).
    pub fn children_of(&self, node_id: u64) -> Vec<&StoredChatNode> {
        self.nodes
            .iter()
            .filter(|n| n.parent_id == Some(node_id))
            .collect()
    }
    /// For a given node, return (position, total) where:
    /// - total = number of children of the parent of this node (i.e., siblings + self)
    /// - position = 1-based rank of this node among siblings, ordered by id
    ///
    /// Returns None if the node has no parent (root) or the parent has 0 or 1 child.
    pub fn fork_at_node(&self, node_id: u64) -> Option<(usize, usize)> {
        let node = self.nodes.iter().find(|n| n.id == node_id)?;
        let parent_id = node.parent_id?;
        let children = self.children_of(parent_id);
        let total = children.len();
        if total <= 1 {
            return None;
        }
        let position = children.iter().position(|c| c.id == node_id)? + 1;
        Some((position, total))
    }
    /// All leaf node IDs within the subtree rooted at `node_id` (the node
    /// itself when it has no children). Empty when the node doesn't exist.
    pub fn descendant_leaves(&self, node_id: u64) -> Vec<u64> {
        if !self.nodes.iter().any(|n| n.id == node_id) {
            return Vec::new();
        }
        let mut result = Vec::new();
        let mut stack = vec![node_id];
        while let Some(id) = stack.pop() {
            let children = self.children_of(id);
            if children.is_empty() {
                result.push(id);
            } else {
                stack.extend(children.iter().map(|c| c.id));
            }
        }
        result
    }
    /// Branch options at a divergence point: one entry per direct child of
    /// `fork_node_id`, ordered by id (chronological — matches the stable
    /// position rank shown in "X/Y" fork chips).
    ///
    /// Each entry's `id` is the leaf to switch to for that branch: the
    /// `preferred_leaf` when it lives in that child's subtree (so the
    /// caller's current view maps onto its own option and can be filtered
    /// out client-side), otherwise the most recent (max-id) descendant leaf.
    ///
    /// Snippets preview the first visible message that *differs* between
    /// the branches, not blindly the first child: Rewind & Regenerate
    /// resends the identical question, so the first child is often shared
    /// across siblings and would make every option read the same.
    pub fn branch_options_at(&self, fork_node_id: u64, preferred_leaf: u64) -> Vec<BranchLeafInfo> {
        struct Candidate {
            rep: u64,
            message_count: usize,
            /// User/assistant contents after the divergence, tool
            /// scaffolding (tool results, empty dispatch turns) skipped.
            visible: Vec<String>,
        }
        let candidates: Vec<Candidate> = self
            .children_of(fork_node_id)
            .iter()
            .map(|child| {
                let leaves = self.descendant_leaves(child.id);
                let rep = if leaves.contains(&preferred_leaf) {
                    preferred_leaf
                } else {
                    leaves.iter().copied().max().unwrap_or(child.id)
                };
                let path = self.path_to_leaf(rep).unwrap_or_default();
                let message_count = path.len();
                let visible: Vec<String> = path
                    .iter()
                    .skip_while(|n| n.id != child.id)
                    .filter(|n| {
                        (n.message.role == "user" || n.message.role == "assistant")
                            && !n.message.content.trim().is_empty()
                    })
                    .map(|n| n.message.content.clone())
                    .collect();
                Candidate {
                    rep,
                    message_count,
                    visible,
                }
            })
            .collect();
        // Walk forward past positions where every branch has the same
        // content; stop at the first divergence (or when any branch runs
        // out — a length difference is itself distinguishing).
        let mut snippet_idx = 0usize;
        loop {
            let contents: Vec<Option<&String>> = candidates
                .iter()
                .map(|c| c.visible.get(snippet_idx))
                .collect();
            let all_present_and_equal =
                contents.iter().all(|c| c.is_some()) && contents.windows(2).all(|w| w[0] == w[1]);
            if !all_present_and_equal {
                break;
            }
            snippet_idx += 1;
        }
        candidates
            .into_iter()
            .enumerate()
            .map(|(i, c)| {
                // A branch exhausted before the divergence index falls back
                // to its own last message so it still shows *something*.
                let snippet = c
                    .visible
                    .get(snippet_idx)
                    .or_else(|| c.visible.last())
                    .map(|s| s.chars().take(100).collect::<String>())
                    .unwrap_or_default();
                BranchLeafInfo {
                    id: c.rep,
                    snippet,
                    message_count: c.message_count,
                    position: Some(i + 1),
                }
            })
            .collect()
    }
    /// All leaf node IDs (nodes with no children).
    pub fn leaves(&self) -> Vec<u64> {
        let child_ids: std::collections::HashSet<u64> =
            self.nodes.iter().filter_map(|n| n.parent_id).collect();
        self.nodes
            .iter()
            .filter(|n| !child_ids.contains(&n.id))
            .map(|n| n.id)
            .collect()
    }
    /// The parent of the given node, if any. Used to re-anchor a rewind
    /// that discards every rendered message onto the seed node above them.
    pub fn parent_of(&self, node_id: u64) -> Option<&StoredChatNode> {
        let node = self.nodes.iter().find(|n| n.id == node_id)?;
        let parent_id = node.parent_id?;
        self.nodes.iter().find(|n| n.id == parent_id)
    }
    /// Return all leaf nodes with metadata: id, snippet (first 100 chars of
    /// the first user-visible message after the divergence point), and
    /// message_count (number of nodes in the path from root to leaf).
    pub fn leaves_with_info(&self) -> Vec<BranchLeafInfo> {
        let leaf_ids = self.leaves();
        leaf_ids
            .into_iter()
            .map(|leaf_id| {
                let path = self.path_to_leaf(leaf_id).unwrap_or_default();
                let message_count = path.len();
                // Snippet: first 100 chars of the first user/assistant message
                // (skip system messages).
                let snippet = path
                    .iter()
                    .find(|n| n.message.role != "system")
                    .map(|n| n.message.content.chars().take(100).collect::<String>())
                    .unwrap_or_default();
                BranchLeafInfo {
                    id: leaf_id,
                    snippet,
                    message_count,
                    position: None,
                }
            })
            .collect()
    }
 }
 /// Metadata about a branch leaf, returned by the branches endpoint.
 #[derive(Serialize, Clone, Debug)]
 pub struct BranchLeafInfo {
    pub id: u64,
    #[serde(skip_serializing_if = "String::is_empty")]
    pub snippet: String,
    pub message_count: usize,
    /// 1-based sibling rank at the divergence point. Only present for
    /// scoped queries (`branch_options_at`); tree-wide leaf listings have
    /// no single divergence to rank against.
    #[serde(skip_serializing_if = "Option::is_none")]
    pub position: Option<usize>,
 }
 /// Strip a leading `<think>…</think>` reasoning block from model output.
 ///
 /// Thinking models sometimes emit chain-of-thought inside think tags before
 /// the real answer. Everything after the first `</think>` is the answer;
 /// when no tag is present — or the text after it is empty — the trimmed
 /// input is returned unchanged. Mirrors the behavior Ollama's
 /// `extract_final_answer` has applied to single-shot generation; shared here
 /// so the tool-calling final-content paths (agentic generation + chat) can
 /// apply the identical cleanup before parsing / persisting.
 pub fn strip_think_blocks(response: &str) -> String {
    let response = response.trim();
    if let Some(pos) = response.find("</think>") {
        let answer = response[pos + "</think>".len()..].trim();
        if !answer.is_empty() {
            return answer.to_string();
        }
    }
    response.to_string()
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    #[test]
    fn strip_think_blocks_removes_leading_think_block() {
        let raw = "<think>\nLet me reason about this.\n</think>\n\nTitle: A Day Out\n\nThe body.";
        assert_eq!(strip_think_blocks(raw), "Title: A Day Out\n\nThe body.");
    }
    #[test]
    fn strip_think_blocks_passes_through_plain_content() {
        assert_eq!(strip_think_blocks("  just an answer  "), "just an answer");
    }
    #[test]
    fn strip_think_blocks_keeps_content_when_answer_after_tag_is_empty() {
        // A think block with nothing after it: better to return the trimmed
        // original than an empty string (matches Ollama's fallback).
        let raw = "<think>only thoughts</think>";
        assert_eq!(strip_think_blocks(raw), raw);
    }
    #[test]
    fn strip_think_blocks_handles_unclosed_tag() {
        let raw = "<think>thinking forever";
        assert_eq!(strip_think_blocks(raw), raw);
    }
    // ── ChatHistoryStore tree traversal tests ──────────────────────────
    fn mk_msg(role: &str, content: &str) -> ChatMessage {
        match role {
            "system" => ChatMessage::system(content),
            "user" => ChatMessage::user(content),
            _ => ChatMessage {
                role: role.to_string(),
                content: content.to_string(),
                tool_calls: None,
                images: None,
            },
        }
    }
    #[test]
    fn store_from_flat_array_chains_parent_ids() {
        let msgs = vec![
            mk_msg("system", "sys"),
            mk_msg("user", "hello"),
            mk_msg("assistant", "hi there"),
        ];
        let store = ChatHistoryStore::from_flat_array(msgs);
        assert_eq!(store.nodes.len(), 3);
        assert_eq!(store.nodes[0].id, 1);
        assert_eq!(store.nodes[0].parent_id, None);
        assert_eq!(store.nodes[1].parent_id, Some(1));
        assert_eq!(store.nodes[2].parent_id, Some(2));
        assert_eq!(store.active_leaf_id, 3);
    }
    #[test]
    fn store_path_to_leaf_returns_ordered_path() {
        let mut store = ChatHistoryStore::empty();
        let n1 = store.append_node(None, mk_msg("user", "u1"));
        let n2 = store.append_node(Some(n1), mk_msg("assistant", "a1"));
        let n3 = store.append_node(Some(n2), mk_msg("user", "u2"));
        let path = store.path_to_leaf(n3).unwrap();
        assert_eq!(path.len(), 3);
        assert_eq!(path[0].id, n1);
        assert_eq!(path[1].id, n2);
        assert_eq!(path[2].id, n3);
    }
    #[test]
    fn store_path_to_leaf_returns_none_for_missing_id() {
        let store = ChatHistoryStore::empty();
        assert!(store.path_to_leaf(999).is_none());
    }
    #[test]
    fn store_children_of_returns_direct_children() {
        let mut store = ChatHistoryStore::empty();
        let root = store.append_node(None, mk_msg("user", "root"));
        let c1 = store.append_node(Some(root), mk_msg("assistant", "c1"));
        let c2 = store.append_node(Some(root), mk_msg("user", "c2"));
        let _d1 = store.append_node(Some(c1), mk_msg("assistant", "d1"));
        let children = store.children_of(root);
        assert_eq!(children.len(), 2);
        assert_eq!(children[0].id, c1);
        assert_eq!(children[1].id, c2);
    }
    #[test]
    fn store_fork_at_node_detects_fork() {
        let mut store = ChatHistoryStore::empty();
        let root = store.append_node(None, mk_msg("user", "root"));
        let c1 = store.append_node(Some(root), mk_msg("assistant", "c1"));
        let c2 = store.append_node(Some(root), mk_msg("user", "c2"));
        let c3 = store.append_node(Some(root), mk_msg("user", "c3"));
        // c1 is position 1 of 3 siblings
        assert_eq!(store.fork_at_node(c1), Some((1, 3)));
        assert_eq!(store.fork_at_node(c2), Some((2, 3)));
        assert_eq!(store.fork_at_node(c3), Some((3, 3)));
        // root has no parent, so no fork
        assert!(store.fork_at_node(root).is_none());
    }
    #[test]
    fn store_fork_at_node_returns_none_for_single_child() {
        let mut store = ChatHistoryStore::empty();
        let root = store.append_node(None, mk_msg("user", "root"));
        let _only = store.append_node(Some(root), mk_msg("assistant", "only"));
        assert!(store.fork_at_node(2).is_none());
    }
    #[test]
    fn store_leaves_returns_leaf_ids() {
        let mut store = ChatHistoryStore::empty();
        let root = store.append_node(None, mk_msg("user", "root"));
        let c1 = store.append_node(Some(root), mk_msg("assistant", "c1"));
        let c2 = store.append_node(Some(root), mk_msg("user", "c2"));
        let _d1 = store.append_node(Some(c1), mk_msg("assistant", "d1"));
        let leaves = store.leaves();
        assert_eq!(leaves.len(), 2);
        assert!(leaves.contains(&c2));
        assert!(leaves.contains(&_d1));
        assert!(!leaves.contains(&root));
        assert!(!leaves.contains(&c1));
    }
    #[test]
    fn store_parent_of_returns_parent() {
        let mut store = ChatHistoryStore::empty();
        let root = store.append_node(None, mk_msg("user", "root"));
        let child = store.append_node(Some(root), mk_msg("assistant", "child"));
        let parent = store.parent_of(child).unwrap();
        assert_eq!(parent.id, root);
        assert!(store.parent_of(root).is_none());
    }
    #[test]
    fn store_next_id_increments() {
        let mut store = ChatHistoryStore::empty();
        assert_eq!(store.next_id(), 1);
        store.append_node(None, mk_msg("user", "a"));
        assert_eq!(store.next_id(), 2);
        store.append_node(Some(1), mk_msg("assistant", "b"));
        assert_eq!(store.next_id(), 3);
    }
    #[test]
    fn store_descendant_leaves_walks_subtree() {
        let mut store = ChatHistoryStore::empty();
        let root = store.append_node(None, mk_msg("user", "root"));
        let c1 = store.append_node(Some(root), mk_msg("assistant", "c1"));
        let c2 = store.append_node(Some(root), mk_msg("user", "c2"));
        let d1 = store.append_node(Some(c1), mk_msg("assistant", "d1"));
        let d2 = store.append_node(Some(c1), mk_msg("assistant", "d2"));
        // c1's subtree has two leaves; c2 is its own leaf; root sees all.
        let mut c1_leaves = store.descendant_leaves(c1);
        c1_leaves.sort_unstable();
        assert_eq!(c1_leaves, vec![d1, d2]);
        assert_eq!(store.descendant_leaves(c2), vec![c2]);
        let mut all = store.descendant_leaves(root);
        all.sort_unstable();
        assert_eq!(all, vec![c2, d1, d2]);
        assert!(store.descendant_leaves(999).is_empty());
    }
    #[test]
    fn branch_options_ranks_children_and_prefers_viewing_leaf() {
        let mut store = ChatHistoryStore::empty();
        let root = store.append_node(None, mk_msg("user", "root"));
        // First branch: two leaves (older + newer).
        let c1 = store.append_node(Some(root), mk_msg("assistant", "reply one"));
        let old_leaf = store.append_node(Some(c1), mk_msg("user", "old follow-up"));
        let new_leaf = store.append_node(Some(c1), mk_msg("user", "new follow-up"));
        // Second branch: single leaf.
        let c2 = store.append_node(Some(root), mk_msg("assistant", "reply two"));
        // Viewing the *older* leaf of branch 1: its option must anchor to
        // that exact leaf, not the max-id one, so the client's
        // "hide the branch I'm on" filter matches.
        let opts = store.branch_options_at(root, old_leaf);
        assert_eq!(opts.len(), 2);
        assert_eq!(opts[0].id, old_leaf);
        assert_eq!(opts[0].position, Some(1));
        assert_eq!(opts[0].snippet, "reply one");
        assert_eq!(opts[1].id, c2);
        assert_eq!(opts[1].position, Some(2));
        // Viewing branch 2: branch 1 falls back to its most recent leaf.
        let opts = store.branch_options_at(root, c2);
        assert_eq!(opts[0].id, new_leaf);
        assert_eq!(opts[1].id, c2);
    }
    #[test]
    fn branch_options_snippet_skips_shared_resent_question() {
        // Rewind & Regenerate resends the identical question, so both
        // branches' first child matches — the preview must advance to the
        // first *differing* message (the assistant replies) or every
        // option would read the same.
        let mut store = ChatHistoryStore::empty();
        let root = store.append_node(None, mk_msg("assistant", "prior reply"));
        let q1 = store.append_node(Some(root), mk_msg("user", "same question"));
        let _a1 = store.append_node(Some(q1), mk_msg("assistant", "first answer"));
        let q2 = store.append_node(Some(root), mk_msg("user", "same question"));
        let a2 = store.append_node(Some(q2), mk_msg("assistant", "second answer"));
        let opts = store.branch_options_at(root, a2);
        assert_eq!(opts.len(), 2);
        assert_eq!(opts[0].snippet, "first answer");
        assert_eq!(opts[1].snippet, "second answer");
    }
    #[test]
    fn branch_options_snippet_falls_back_when_branch_exhausted() {
        // One branch is just the shared question (no reply yet); the other
        // continues past it. The shorter branch falls back to its own last
        // message rather than showing an empty preview.
        let mut store = ChatHistoryStore::empty();
        let root = store.append_node(None, mk_msg("assistant", "prior reply"));
        let _q1 = store.append_node(Some(root), mk_msg("user", "same question"));
        let q2 = store.append_node(Some(root), mk_msg("user", "same question"));
        let a2 = store.append_node(Some(q2), mk_msg("assistant", "second answer"));
        let opts = store.branch_options_at(root, a2);
        assert_eq!(opts[0].snippet, "same question");
        assert_eq!(opts[1].snippet, "second answer");
    }
    #[test]
    fn branch_options_snippet_skips_tool_scaffolding() {
        let mut store = ChatHistoryStore::empty();
        let root = store.append_node(None, mk_msg("user", "root"));
        // Branch whose diverging child is an empty tool-dispatch turn.
        let dispatch = store.append_node(
            Some(root),
            ChatMessage {
                role: "assistant".to_string(),
                content: String::new(),
                tool_calls: None,
                images: None,
            },
        );
        let tool = store.append_node(Some(dispatch), mk_msg("tool", "raw tool output"));
        let reply = store.append_node(Some(tool), mk_msg("assistant", "final answer"));
        let _other = store.append_node(Some(root), mk_msg("assistant", "direct reply"));
        let opts = store.branch_options_at(root, reply);
        assert_eq!(opts[0].snippet, "final answer");
        assert_eq!(opts[1].snippet, "direct reply");
    }
    #[test]
    fn store_append_node_sets_parent() {
        let mut store = ChatHistoryStore::empty();
        let parent = store.append_node(None, mk_msg("user", "parent"));
        let child = store.append_node(Some(parent), mk_msg("assistant", "child"));
        assert_eq!(store.nodes.len(), 2);
        assert_eq!(store.nodes[1].parent_id, Some(parent));
        assert_eq!(store.nodes[1].id, child);
    }
    #[test]
    fn branch_options_handles_nested_forks() {
        // Regression: verify sub-fork display when there are multiple forks.
        // Structure:
        //   root (user "q1")
        //   ├── a1 (assistant "answer 1")
        //   └── a2 (assistant "answer 2")
        //       ├── q2 (user "q2")
        //       │   ├── r1 (assistant "reply 1")
        //       │   └── r2 (assistant "reply 2")  <- active leaf
        //
        // branch_options_at(root) should return [a1, a2] (2 branches)
        // branch_options_at(q2) should return [r1, r2] (2 sub-branches)
        let mut store = ChatHistoryStore::empty();
        let root = store.append_node(None, mk_msg("user", "q1"));
        let a1 = store.append_node(Some(root), mk_msg("assistant", "answer 1"));
        let a2 = store.append_node(Some(root), mk_msg("assistant", "answer 2"));
        let q2 = store.append_node(Some(a2), mk_msg("user", "q2"));
        let r1 = store.append_node(Some(q2), mk_msg("assistant", "reply 1"));
        let r2 = store.append_node(Some(q2), mk_msg("assistant", "reply 2"));
        store.active_leaf_id = r2;
        // Root fork: should show [a1, a2's subtree]
        let opts = store.branch_options_at(root, r2);
        assert_eq!(opts.len(), 2);
        assert_eq!(opts[0].id, a1);
        assert_eq!(opts[0].position, Some(1));
        assert_eq!(opts[0].snippet, "answer 1");
        // a2's subtree includes r2, so the rep should be r2 (preferred leaf)
        assert_eq!(opts[1].id, r2);
        assert_eq!(opts[1].position, Some(2));
        assert_eq!(opts[1].snippet, "answer 2");
        // Sub-fork at q2: should show [r1, r2]
        let opts = store.branch_options_at(q2, r2);
        assert_eq!(opts.len(), 2);
        assert_eq!(opts[0].id, r1);
        assert_eq!(opts[0].position, Some(1));
        assert_eq!(opts[0].snippet, "reply 1");
        assert_eq!(opts[1].id, r2);
        assert_eq!(opts[1].position, Some(2));
        assert_eq!(opts[1].snippet, "reply 2");
    }
 }
@@ -1,88 +0,0 @@
 //! Bundle of the local LLM pair (Ollama + optional llama-swap) with the
 //! `LLM_BACKEND` dispatch baked in.
 //!
 //! Exists because passing the pair around as loose values invited the same
 //! bug three times: import/backfill tooling embedded corpora via
 //! `OllamaClient` directly while the query side dispatched through
 //! `embed_one`, so flipping `LLM_BACKEND=llamacpp` silently split queries
 //! and corpus into different vector spaces. Anything that writes or reads
 //! embeddings should go through this type (or `embed_one`/`embed_many`),
 //! never a concrete client.
 //!
 //! Deliberately knows nothing about chat policy — hybrid/OpenRouter routing
 //! is request-scoped and stays in `ResolvedBackend`. This is only the
 //! local stack: embeddings and offline single-shot generation.
 // Constructed by binaries, not the server — dead code from main.rs's view.
 #![allow(dead_code)]
 use std::sync::Arc;
 use anyhow::Result;
 use super::llamacpp::LlamaCppClient;
 use super::llm_client::LlmClient;
 use super::ollama::{EMBEDDING_MODEL, OllamaClient};
 #[derive(Clone)]
 pub struct LocalLlm {
    ollama: OllamaClient,
    llamacpp: Option<Arc<LlamaCppClient>>,
 }
 impl LocalLlm {
    pub fn new(ollama: OllamaClient, llamacpp: Option<Arc<LlamaCppClient>>) -> Self {
        Self { ollama, llamacpp }
    }
    /// Construct from the canonical env wiring shared with `AppState`.
    pub fn from_env() -> Self {
        Self::new(
            crate::state::build_ollama_from_env(),
            crate::state::build_llamacpp_from_env(),
        )
    }
    /// Embed a search query (applies `EMBED_QUERY_PREFIX`). Callers must
    /// pick query vs document — retrieval models treat the two sides
    /// differently and an unmarked embed invites prefix-mismatch bugs.
    pub async fn embed_query(&self, text: &str) -> Result<Vec<f32>> {
        super::embed_query(&self.ollama, self.llamacpp.as_deref(), text).await
    }
    /// Embed corpus text (applies `EMBED_DOCUMENT_PREFIX`).
    pub async fn embed_document(&self, text: &str) -> Result<Vec<f32>> {
        super::embed_document(&self.ollama, self.llamacpp.as_deref(), text).await
    }
    /// Single-shot local text generation via the `LLM_BACKEND`-selected
    /// client (offline tooling; chat turns belong to `ResolvedBackend`).
    pub async fn generate(&self, prompt: &str, system: Option<&str>) -> Result<String> {
        if super::local_backend_is_llamacpp() {
            if let Some(lc) = self.llamacpp.as_deref() {
                return <LlamaCppClient as LlmClient>::generate(lc, prompt, system, None).await;
            }
            anyhow::bail!(
                "LLM_BACKEND=llamacpp but LlamaCppClient is unconfigured — \
                 set LLAMA_SWAP_URL or switch to LLM_BACKEND=ollama"
            );
        }
        self.ollama.generate(prompt, system).await
    }
    /// Label identifying which backend + model produces embeddings right
    /// now. Store it alongside vectors (`model_version` columns) so a
    /// backend flip is detectable in the data, not just in env history.
    pub fn embedding_model_version(&self) -> String {
        if super::local_backend_is_llamacpp() {
            let slot = self
                .llamacpp
                .as_deref()
                .map(|c| c.embedding_model.as_str())
                .unwrap_or("embed");
            format!("llama-swap:{}", slot)
        } else {
            EMBEDDING_MODEL.to_string()
        }
    }
 }
@@ -1,23 +1,13 @@
 pub mod apollo_client;
 pub mod backend;
 pub mod clip_client;
 pub mod daily_summary_job;
 pub mod face_client;
 pub mod gpu;
 pub mod handlers;
 pub mod insight_chat;
 pub mod insight_generator;
 pub mod llamacpp;
 pub mod llm_client;
 pub mod local_llm;
 pub mod nl_query;
 pub mod ollama;
 pub mod openrouter;
 pub mod persona_chat;
 pub mod pronunciation;
 pub mod sms_client;
 pub mod tts;
 pub mod turn_registry;
 // strip_summary_boilerplate is used by binaries (test_daily_summary), not the library
 #[allow(unused_imports)]
@@ -26,35 +16,18 @@ pub use daily_summary_job::{
    generate_daily_summaries, strip_summary_boilerplate,
 };
 pub use handlers::{
-    cancel_generation_handler, cancel_turn_handler, chat_branches_handler, chat_history_handler,
+    chat_history_handler, chat_rewind_handler, chat_stream_handler, chat_turn_handler,
    chat_rewind_handler, chat_stream_handler, chat_switch_branch_handler, chat_turn_handler,
    delete_insight_handler, export_training_data_handler, generate_agentic_insight_handler,
-    generate_insight_handler, generation_status_handler, get_all_insights_handler,
+    generate_insight_handler, get_all_insights_handler, get_available_models_handler,
-    get_available_models_handler, get_insight_handler, get_insight_history_handler,
+    get_insight_handler, get_openrouter_models_handler, rate_insight_handler,
    get_openrouter_models_handler, rate_insight_handler, turn_async_handler, turn_replay_handler,
 };
 pub use insight_generator::InsightGenerator;
 pub use llamacpp::LlamaCppClient;
 #[allow(unused_imports)]
 pub use llm_client::{
    ChatMessage, LlmClient, ModelCapabilities, Tool, ToolCall, ToolCallFunction, ToolFunction,
 };
 pub use persona_chat::{
    persona_chat_branches_handler, persona_chat_conversations_handler,
    persona_chat_create_conversation_handler, persona_chat_delete_conversation_handler,
    persona_chat_history_handler, persona_chat_rewind_handler, persona_chat_switch_branch_handler,
    persona_chat_turn_handler, persona_turn_cancel_handler, persona_turn_replay_handler,
 };
 // LocalLlm is constructed by binaries (reembed_embeddings, importers), not the server
 #[allow(unused_imports)]
 pub use local_llm::LocalLlm;
 pub use ollama::{EMBEDDING_MODEL, OllamaClient};
 pub use sms_client::{SmsApiClient, SmsMessage};
 pub use tts::{
    cancel_speech_job_handler, create_speech_job_handler, create_voice_from_library_handler,
    create_voice_upload_handler, delete_voice_handler, list_voices_handler,
    speech_job_status_handler, tts_speech_handler,
 };
 /// Display name used for the user in message transcripts and first-person
 /// prompt text. Reads the `USER_NAME` env var; defaults to `"Me"`. Models
@@ -64,153 +37,3 @@ pub use tts::{
 pub fn user_display_name() -> String {
    std::env::var("USER_NAME").unwrap_or_else(|_| "Me".to_string())
 }
 /// One switch for the "local" LLM stack: when `LLM_BACKEND=llamacpp` is
 /// set, chat / vision describe / embeddings all route through llama-swap
 /// instead of Ollama. Any other value (including unset, the default) is
 /// Ollama. This is intentionally global — embeddings must be drawn from
 /// a single source or similarity search across the index breaks (mixed
 /// vector spaces, possibly mixed dims). The `backend=hybrid` per-request
 /// override remains orthogonal: it always sends chat to OpenRouter, and
 /// uses `LLM_BACKEND` for the describe-then-inline vision pass.
 pub fn local_backend_is_llamacpp() -> bool {
    matches!(
        std::env::var("LLM_BACKEND")
            .ok()
            .as_deref()
            .map(|s| s.trim().to_lowercase())
            .as_deref(),
        Some("llamacpp")
    )
 }
 /// Expected embedding dimensionality, env-overridable via `EMBEDDING_DIM`
 /// (default 768, nomic-embed-text). Every store/query dim check reads this —
 /// swapping to a different-dim model (e.g. Qwen3-Embedding-0.6B at 1024) is
 /// then a config flip plus a `reembed_embeddings` run, not a code change.
 /// Cached for the process lifetime; a flip requires a restart anyway since
 /// the corpus must be re-embedded with it.
 pub fn embedding_dim() -> usize {
    static DIM: std::sync::OnceLock<usize> = std::sync::OnceLock::new();
    *DIM.get_or_init(|| {
        std::env::var("EMBEDDING_DIM")
            .ok()
            .and_then(|v| v.parse().ok())
            .unwrap_or(768)
    })
 }
 /// Read an embedding prefix from the environment. `.env` values can't hold
 /// real newlines, so a literal `\n` in the value is expanded — Qwen3-style
 /// query instructions need one ("Instruct: ...\nQuery: ").
 fn embed_prefix(key: &str) -> String {
    std::env::var(key)
        .map(|v| v.replace("\\n", "\n"))
        .unwrap_or_default()
 }
 /// Embed a search query. Applies `EMBED_QUERY_PREFIX` (default empty) —
 /// retrieval models distinguish query-side from document-side text:
 /// nomic v1.5 wants `search_query: `, Qwen3-Embedding wants
 /// `Instruct: <task>\nQuery: `. Must pair with the document prefix the
 /// corpus was embedded with or similarity degrades.
 pub async fn embed_query(
    ollama: &OllamaClient,
    llamacpp: Option<&LlamaCppClient>,
    text: &str,
 ) -> anyhow::Result<Vec<f32>> {
    let prefixed = format!("{}{}", embed_prefix("EMBED_QUERY_PREFIX"), text);
    embed_one(ollama, llamacpp, &prefixed).await
 }
 /// Embed corpus text (the stored side of retrieval). Applies
 /// `EMBED_DOCUMENT_PREFIX` (default empty; nomic v1.5 wants
 /// `search_document: `, Qwen3-Embedding wants none).
 pub async fn embed_document(
    ollama: &OllamaClient,
    llamacpp: Option<&LlamaCppClient>,
    text: &str,
 ) -> anyhow::Result<Vec<f32>> {
    let prefixed = format!("{}{}", embed_prefix("EMBED_DOCUMENT_PREFIX"), text);
    embed_one(ollama, llamacpp, &prefixed).await
 }
 /// Embed a batch of strings via the configured local backend. Routes
 /// through llama-swap when `LLM_BACKEND=llamacpp` (and a client is
 /// configured), else Ollama. See [`local_backend_is_llamacpp`] for the
 /// rationale on consistency.
 pub async fn embed_many(
    ollama: &OllamaClient,
    llamacpp: Option<&LlamaCppClient>,
    texts: &[&str],
 ) -> anyhow::Result<Vec<Vec<f32>>> {
    if local_backend_is_llamacpp() {
        if let Some(lc) = llamacpp {
            return <LlamaCppClient as LlmClient>::generate_embeddings(lc, texts).await;
        }
        anyhow::bail!(
            "LLM_BACKEND=llamacpp but LlamaCppClient is unconfigured — \
             set LLAMA_SWAP_URL or switch to LLM_BACKEND=ollama"
        );
    }
    ollama.generate_embeddings(texts).await
 }
 /// Embed one string via the configured local backend. Single-text
 /// convenience over [`embed_many`].
 pub async fn embed_one(
    ollama: &OllamaClient,
    llamacpp: Option<&LlamaCppClient>,
    text: &str,
 ) -> anyhow::Result<Vec<f32>> {
    let mut vecs = embed_many(ollama, llamacpp, &[text]).await?;
    vecs.pop()
        .ok_or_else(|| anyhow::anyhow!("embedding backend returned no embeddings"))
 }
 #[cfg(test)]
 mod env_dispatch_tests {
    use super::*;
    /// Env vars are process-global, and the test harness runs in parallel —
    /// without this lock the `LLM_BACKEND` tests race each other and flake.
    static ENV_LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(());
    fn with_env<F: FnOnce()>(key: &str, val: Option<&str>, f: F) {
        let _guard = ENV_LOCK.lock().unwrap_or_else(|p| p.into_inner());
        let prev = std::env::var(key).ok();
        match val {
            Some(v) => unsafe { std::env::set_var(key, v) },
            None => unsafe { std::env::remove_var(key) },
        }
        f();
        match prev {
            Some(v) => unsafe { std::env::set_var(key, v) },
            None => unsafe { std::env::remove_var(key) },
        }
    }
    #[test]
    fn llm_backend_defaults_to_ollama() {
        with_env("LLM_BACKEND", None, || {
            assert!(!local_backend_is_llamacpp());
        });
    }
    #[test]
    fn llm_backend_llamacpp_case_insensitive() {
        with_env("LLM_BACKEND", Some("LlamaCpp"), || {
            assert!(local_backend_is_llamacpp());
        });
        with_env("LLM_BACKEND", Some("  llamacpp "), || {
            assert!(local_backend_is_llamacpp());
        });
    }
    #[test]
    fn llm_backend_unknown_value_is_ollama() {
        with_env("LLM_BACKEND", Some("vllm"), || {
            assert!(!local_backend_is_llamacpp());
        });
    }
 }
@@ -1,408 +0,0 @@
 //! Natural-language → structured-query translation for unified photo search.
 //!
 //! The unified search endpoint (`/photos/search/unified`, Phase 2) needs to
 //! turn a free-text query like *"sunset photos in Italy from last summer"*
 //! into the structured filter the existing `/photos` engine understands plus
 //! a semantic term for CLIP ranking. That translation is a single grounded
 //! LLM call, isolated here so it can be unit-tested without a network or the
 //! full `InsightGenerator`.
 //!
 //! Two-stage design:
 //!  1. The LLM emits a [`RawNlQuery`] — references are by *name* (tags) and
 //!     dates as ISO strings, never numeric ids it could hallucinate.
 //!  2. [`resolve_raw_query`] maps names against the real tag vocabulary and
 //!     converts ISO dates to unix seconds, producing a [`StructuredQuery`].
 //!     A tag the model invents that isn't in the vocab is surfaced in
 //!     `unmatched_tags` (the caller folds it back into the semantic term)
 //!     rather than silently dropped — this is the anti-noise guard.
 //!
 //! Geocoding of `place` and person filtering are intentionally *not* handled
 //! here: `place` stays as text for the caller to forward-geocode (async, see
 //! `geo::forward_geocode`), and person filtering is deferred until a
 //! person→photos resolver exists.
 use crate::ai::llm_client::{ChatMessage, LlmClient, Tool, strip_think_blocks};
 use anyhow::{Result, anyhow};
 use serde::{Deserialize, Serialize};
 /// Raw query object as emitted by the LLM. Tag references are by name
 /// (resolved against the real vocab in Rust); dates are ISO `YYYY-MM-DD`.
 /// Every field is optional so a partial / minimal model response still
 /// deserializes.
 #[derive(Debug, Clone, Default, Deserialize, PartialEq)]
 pub struct RawNlQuery {
    /// Visual/scene description handed to CLIP for ranking. The descriptive
    /// remainder after structured filters are peeled off.
    #[serde(default)]
    pub semantic: Option<String>,
    /// Tag names the photos must have. Matched case-insensitively against
    /// the supplied vocabulary; non-matches land in `unmatched_tags`.
    #[serde(default)]
    pub tags: Vec<String>,
    /// Tag names the photos must NOT have.
    #[serde(default)]
    pub exclude_tags: Vec<String>,
    #[serde(default)]
    pub camera_make: Option<String>,
    #[serde(default)]
    pub camera_model: Option<String>,
    #[serde(default)]
    pub lens_model: Option<String>,
    /// Free-text place/location name to forward-geocode (e.g. "Italy").
    #[serde(default)]
    pub place: Option<String>,
    /// Inclusive start date, ISO `YYYY-MM-DD`.
    #[serde(default)]
    pub date_from: Option<String>,
    /// Inclusive end date, ISO `YYYY-MM-DD`.
    #[serde(default)]
    pub date_to: Option<String>,
    /// "photo" | "video" — normalized in [`resolve_raw_query`].
    #[serde(default)]
    pub media_type: Option<String>,
 }
 /// Resolved structured query: tag names mapped to ids against the real
 /// vocab, ISO dates converted to unix seconds. `place` stays as text for the
 /// caller to forward-geocode into a gps circle. Serializable so the endpoint
 /// can echo it back to the client as "this is how I read your query"
 /// (editable filter chips).
 #[derive(Debug, Clone, Default, PartialEq, Serialize)]
 pub struct StructuredQuery {
    pub semantic: Option<String>,
    pub tag_ids: Vec<i32>,
    pub exclude_tag_ids: Vec<i32>,
    /// Tag names the model produced that don't exist in the vocabulary.
    /// The caller folds these back into the semantic term so the concept
    /// isn't lost — and surfacing them keeps a hallucinated tag from
    /// silently filtering the whole library to nothing.
    pub unmatched_tags: Vec<String>,
    pub camera_make: Option<String>,
    pub camera_model: Option<String>,
    pub lens_model: Option<String>,
    /// Raw place name awaiting forward-geocoding by the caller.
    pub place: Option<String>,
    pub date_from: Option<i64>,
    pub date_to: Option<i64>,
    /// Normalized to "photo" | "video"; `None` means no media-type filter.
    pub media_type: Option<String>,
 }
 /// Convert an ISO `YYYY-MM-DD` date to a unix timestamp (seconds). With
 /// `end_of_day`, returns 23:59:59 of that day so a `date_to` filter is
 /// inclusive of the whole day; otherwise 00:00:00. Returns `None` for any
 /// unparseable input (the filter is simply omitted rather than erroring).
 pub fn iso_to_unix(date: &str, end_of_day: bool) -> Option<i64> {
    let d = chrono::NaiveDate::parse_from_str(date.trim(), "%Y-%m-%d").ok()?;
    let time = if end_of_day {
        chrono::NaiveTime::from_hms_opt(23, 59, 59)?
    } else {
        chrono::NaiveTime::from_hms_opt(0, 0, 0)?
    };
    Some(d.and_time(time).and_utc().timestamp())
 }
 /// Normalize a free-form media-type string to the engine's vocabulary.
 /// Anything that isn't clearly photo or video (including "all") yields
 /// `None` — no filter.
 fn normalize_media_type(raw: &str) -> Option<String> {
    match raw.trim().to_lowercase().as_str() {
        "photo" | "photos" | "image" | "images" | "picture" | "pictures" => {
            Some("photo".to_string())
        }
        "video" | "videos" | "movie" | "movies" | "clip" | "clips" => Some("video".to_string()),
        _ => None,
    }
 }
 /// Resolve a raw LLM query against the real tag vocabulary, producing the
 /// structured filter. Pure — no network, no LLM — so it carries the
 /// correctness-critical mapping logic under unit test.
 ///
 /// `tag_vocab` is `(tag_id, tag_name)` pairs (the shape `TagDao::get_all_tags`
 /// yields once the count is dropped). Matching is case-insensitive and exact
 /// on the trimmed name.
 pub fn resolve_raw_query(raw: RawNlQuery, tag_vocab: &[(i32, String)]) -> StructuredQuery {
    // Case-insensitive name → id lookup. Built once per call.
    let lookup: std::collections::HashMap<String, i32> = tag_vocab
        .iter()
        .map(|(id, name)| (name.trim().to_lowercase(), *id))
        .collect();
    let resolve_names = |names: &[String], ids: &mut Vec<i32>, unmatched: &mut Vec<String>| {
        for name in names {
            let key = name.trim().to_lowercase();
            if key.is_empty() {
                continue;
            }
            match lookup.get(&key) {
                Some(id) if !ids.contains(id) => ids.push(*id),
                Some(_) => {} // duplicate, already collected
                None => {
                    if !unmatched.iter().any(|u| u.eq_ignore_ascii_case(name)) {
                        unmatched.push(name.trim().to_string());
                    }
                }
            }
        }
    };
    let mut tag_ids = Vec::new();
    let mut unmatched_tags = Vec::new();
    resolve_names(&raw.tags, &mut tag_ids, &mut unmatched_tags);
    // Excluded tags that don't match a real tag are simply ignored — you
    // can't exclude a tag that doesn't exist, and folding them into
    // `semantic` would make no sense.
    let mut exclude_tag_ids = Vec::new();
    let mut exclude_unmatched = Vec::new();
    resolve_names(
        &raw.exclude_tags,
        &mut exclude_tag_ids,
        &mut exclude_unmatched,
    );
    let clean = |s: Option<String>| s.map(|v| v.trim().to_string()).filter(|v| !v.is_empty());
    StructuredQuery {
        semantic: clean(raw.semantic),
        tag_ids,
        exclude_tag_ids,
        unmatched_tags,
        camera_make: clean(raw.camera_make),
        camera_model: clean(raw.camera_model),
        lens_model: clean(raw.lens_model),
        place: clean(raw.place),
        date_from: raw.date_from.as_deref().and_then(|d| iso_to_unix(d, false)),
        date_to: raw.date_to.as_deref().and_then(|d| iso_to_unix(d, true)),
        media_type: raw.media_type.as_deref().and_then(normalize_media_type),
    }
 }
 /// Build the grounded system prompt. The model is told the current date (so
 /// "last summer" resolves) and the exact tag vocabulary (so it uses real
 /// tags or routes the concept to `semantic` instead of inventing one).
 fn build_system_prompt(tag_vocab: &[(i32, String)], today: chrono::NaiveDate) -> String {
    // Cap the vocab dump so a huge library doesn't blow the context window;
    // the most-used tags are the ones a query is likely to reference.
    const MAX_TAGS: usize = 400;
    let mut names: Vec<&str> = tag_vocab.iter().map(|(_, n)| n.as_str()).collect();
    names.sort_unstable();
    names.dedup();
    let shown = names.len().min(MAX_TAGS);
    let vocab = names[..shown].join(", ");
    let truncation = if names.len() > MAX_TAGS {
        format!(" (showing {MAX_TAGS} of {} tags)", names.len())
    } else {
        String::new()
    };
    format!(
        "You translate a user's natural-language photo-search request into a JSON \
 filter. Today's date is {today}. Respond with ONLY a JSON object, no prose, no \
 code fences.\n\n\
 Schema (all fields optional):\n\
 {{\n  \
 \"semantic\": string|null,        // visual scene/subject for image similarity search\n  \
 \"tags\": string[],               // ONLY names from the tag list below\n  \
 \"exclude_tags\": string[],       // ONLY names from the tag list below\n  \
 \"camera_make\": string|null,\n  \
 \"camera_model\": string|null,\n  \
 \"lens_model\": string|null,\n  \
 \"place\": string|null,           // a location name to look up (city, country, landmark)\n  \
 \"date_from\": \"YYYY-MM-DD\"|null,  // inclusive\n  \
 \"date_to\": \"YYYY-MM-DD\"|null,    // inclusive\n  \
 \"media_type\": \"photo\"|\"video\"|null\n\
 }}\n\n\
 Rules:\n\
 - Put descriptive/visual concepts (\"sunset\", \"crowded beach\", \"red car\") in \"semantic\".\n\
 - Only use \"tags\"/\"exclude_tags\" values that appear EXACTLY in the tag list. If a \
 concept isn't a listed tag, put it in \"semantic\" instead — never invent a tag.\n\
 - Resolve relative dates against today's date (\"last summer\", \"2023\", \"last month\").\n\
 - Put place/location names in \"place\" (not \"semantic\").\n\
 - Omit (use null / empty array) anything the request doesn't mention.\n\n\
 Available tags{truncation}: {vocab}"
    )
 }
 /// Extract the JSON object from a model response that may include a leading
 /// `<think>` block, code fences, or trailing prose. Strips the think block
 /// first (so reasoning that mentions braces can't fool the scan), then
 /// returns the substring from the first `{` to the last `}` inclusive — or
 /// the trimmed text if no braces are found (which then fails to parse with a
 /// clear error).
 fn extract_json(raw: &str) -> String {
    let s = strip_think_blocks(raw);
    let start = s.find('{');
    let end = s.rfind('}');
    match (start, end) {
        (Some(a), Some(b)) if b >= a => s[a..=b].to_string(),
        _ => s.trim().to_string(),
    }
 }
 /// Parse a model response string into a [`StructuredQuery`], resolving names
 /// against the vocab. Separated from the LLM call so it's unit-testable.
 pub fn parse_response(response: &str, tag_vocab: &[(i32, String)]) -> Result<StructuredQuery> {
    let json = extract_json(response);
    let raw: RawNlQuery = serde_json::from_str(&json)
        .map_err(|e| anyhow!("failed to parse NL query JSON: {e}; raw response: {response:?}"))?;
    Ok(resolve_raw_query(raw, tag_vocab))
 }
 /// Translate a natural-language query into a [`StructuredQuery`] via one
 /// grounded LLM call. The `client` is any configured backend (the unified
 /// endpoint passes the resolved chat backend); `tag_vocab` grounds the tag
 /// mapping; `today` anchors relative-date resolution.
 pub async fn translate_nl_query(
    client: &dyn LlmClient,
    nl: &str,
    tag_vocab: &[(i32, String)],
    today: chrono::NaiveDate,
 ) -> Result<StructuredQuery> {
    let system = build_system_prompt(tag_vocab, today);
    let messages = vec![ChatMessage::system(system), ChatMessage::user(nl)];
    let (msg, _, _) = client.chat_with_tools(messages, Vec::<Tool>::new()).await?;
    parse_response(&msg.content, tag_vocab)
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    fn vocab() -> Vec<(i32, String)> {
        vec![
            (1, "beach".to_string()),
            (2, "Sunset".to_string()), // mixed case to exercise case-insensitivity
            (3, "family".to_string()),
        ]
    }
    #[test]
    fn iso_to_unix_start_and_end_of_day() {
        // 2023-01-01 UTC midnight = 1672531200.
        assert_eq!(iso_to_unix("2023-01-01", false), Some(1_672_531_200));
        // End of that day is 86399 seconds later.
        assert_eq!(
            iso_to_unix("2023-01-01", true),
            Some(1_672_531_200 + 86_399)
        );
    }
    #[test]
    fn iso_to_unix_rejects_garbage() {
        assert_eq!(iso_to_unix("last summer", false), None);
        assert_eq!(iso_to_unix("2023-13-99", false), None);
        assert_eq!(iso_to_unix("", false), None);
    }
    #[test]
    fn resolve_matches_tags_case_insensitively() {
        let raw = RawNlQuery {
            tags: vec!["BEACH".to_string(), "sunset".to_string()],
            ..Default::default()
        };
        let q = resolve_raw_query(raw, &vocab());
        assert_eq!(q.tag_ids, vec![1, 2]);
        assert!(q.unmatched_tags.is_empty());
    }
    #[test]
    fn resolve_surfaces_unmatched_tags_not_silently_dropped() {
        // A hallucinated / non-vocab tag must be surfaced so the caller can
        // fold it into semantic — never silently used as a hard filter.
        let raw = RawNlQuery {
            tags: vec!["beach".to_string(), "golden hour".to_string()],
            ..Default::default()
        };
        let q = resolve_raw_query(raw, &vocab());
        assert_eq!(q.tag_ids, vec![1]);
        assert_eq!(q.unmatched_tags, vec!["golden hour".to_string()]);
    }
    #[test]
    fn resolve_dedups_repeated_tags() {
        let raw = RawNlQuery {
            tags: vec![
                "beach".to_string(),
                "Beach".to_string(),
                "beach".to_string(),
            ],
            ..Default::default()
        };
        let q = resolve_raw_query(raw, &vocab());
        assert_eq!(q.tag_ids, vec![1]);
    }
    #[test]
    fn resolve_normalizes_media_type_and_dates() {
        let raw = RawNlQuery {
            media_type: Some("Videos".to_string()),
            date_from: Some("2023-06-01".to_string()),
            date_to: Some("2023-06-30".to_string()),
            ..Default::default()
        };
        let q = resolve_raw_query(raw, &vocab());
        assert_eq!(q.media_type.as_deref(), Some("video"));
        assert_eq!(q.date_from, iso_to_unix("2023-06-01", false));
        assert_eq!(q.date_to, iso_to_unix("2023-06-30", true));
    }
    #[test]
    fn resolve_media_type_all_is_no_filter() {
        let raw = RawNlQuery {
            media_type: Some("all".to_string()),
            ..Default::default()
        };
        assert_eq!(resolve_raw_query(raw, &vocab()).media_type, None);
    }
    #[test]
    fn resolve_trims_and_empties_to_none() {
        let raw = RawNlQuery {
            semantic: Some("   ".to_string()),
            camera_make: Some("  Fujifilm  ".to_string()),
            place: Some("".to_string()),
            ..Default::default()
        };
        let q = resolve_raw_query(raw, &vocab());
        assert_eq!(q.semantic, None);
        assert_eq!(q.camera_make.as_deref(), Some("Fujifilm"));
        assert_eq!(q.place, None);
    }
    #[test]
    fn parse_response_handles_code_fences_and_prose() {
        let resp = "Here is the filter:\n```json\n{\"semantic\":\"sunset\",\"tags\":[\"beach\"]}\n```\nDone.";
        let q = parse_response(resp, &vocab()).expect("parse");
        assert_eq!(q.semantic.as_deref(), Some("sunset"));
        assert_eq!(q.tag_ids, vec![1]);
    }
    #[test]
    fn parse_response_handles_think_block_then_json() {
        let resp = "<think>user wants beach sunsets</think>{\"tags\":[\"beach\",\"sunset\"]}";
        let q = parse_response(resp, &vocab()).expect("parse");
        assert_eq!(q.tag_ids, vec![1, 2]);
    }
    #[test]
    fn parse_response_errors_on_non_json() {
        assert!(parse_response("I cannot help with that.", &vocab()).is_err());
    }
    #[test]
    fn build_system_prompt_includes_date_and_vocab() {
        let today = chrono::NaiveDate::from_ymd_opt(2026, 6, 14).unwrap();
        let prompt = build_system_prompt(&vocab(), today);
        assert!(
            prompt.contains("2026-06-14"),
            "prompt should state today's date"
        );
        assert!(prompt.contains("beach"), "prompt should list the vocab");
        assert!(
            prompt.contains("never invent a tag"),
            "prompt should warn against inventing tags"
        );
    }
 }
@@ -360,7 +360,18 @@ impl OllamaClient {
    /// Extract final answer from thinking model output
    /// Handles <think>...</think> tags and takes everything after
    fn extract_final_answer(&self, response: &str) -> String {
-        crate::ai::llm_client::strip_think_blocks(response)
+        let response = response.trim();
        // Look for </think> tag and take everything after it
        if let Some(pos) = response.find("</think>") {
            let answer = response[pos + 8..].trim();
            if !answer.is_empty() {
                return answer.to_string();
            }
        }
        // Fallback: return the whole response trimmed
        response.to_string()
    }
    async fn try_generate(
@@ -413,7 +424,10 @@ impl OllamaClient {
        self.generate_with_images(prompt, system, None).await
    }
-    #[allow(dead_code)]
+    /// Variant of `generate` that sets Ollama's top-level `think: false`.
    /// Used by latency-sensitive callers like the rerank pass, where the
    /// task has nothing to reason about and chain-of-thought tokens are
    /// wasted wall time. Server-side no-op on non-reasoning models.
    pub async fn generate_no_think(&self, prompt: &str, system: Option<&str>) -> Result<String> {
        self.generate_with_options(prompt, system, None, Some(false))
            .await
@@ -548,16 +562,7 @@ Capture the key moment or theme. Return ONLY the title, nothing else."#,
        let title = self
            .generate_with_images(&prompt, Some(system), None)
            .await?;
-        // Models decorate despite "Return ONLY the title": quotes, bold
+        Ok(title.trim().trim_matches('"').to_string())
        // markers, sometimes a "Title:" label.
        use crate::ai::insight_generator::strip_title_markdown;
        let cleaned = strip_title_markdown(title.trim());
        let cleaned = cleaned
            .strip_prefix("Title:")
            .or_else(|| cleaned.strip_prefix("title:"))
            .map(strip_title_markdown)
            .unwrap_or(cleaned);
        Ok(cleaned.to_string())
    }
    /// Generate a summary for a single photo based on its context
@@ -844,14 +849,11 @@ Analyze the image and use specific details from both the visual content and the
                            if !chunk.message.role.is_empty() {
                                role = chunk.message.role;
                            }
-                            // Ollama ≥0.8 can stream tool_calls incrementally
+                            // Ollama only attaches tool_calls on the final chunk.
                            // across chunks (older servers attach them all to
                            // one chunk) — append rather than overwrite so
                            // calls from earlier chunks survive.
                            if let Some(tcs) = chunk.message.tool_calls
                                && !tcs.is_empty()
                            {
-                                append_streamed_tool_calls(&mut tool_calls, tcs);
+                                tool_calls = Some(tcs);
                            }
                            if chunk.done {
                                prompt_eval_count = chunk.prompt_eval_count;
@@ -1055,14 +1057,13 @@ Analyze the image and use specific details from both the visual content and the
            }
        };
-        // Validate embedding dimensions (EMBEDDING_DIM; 768 for nomic-embed-text:v1.5)
+        // Validate embedding dimensions (should be 768 for nomic-embed-text:v1.5)
        for (i, embedding) in embeddings.iter().enumerate() {
-            if embedding.len() != crate::ai::embedding_dim() {
+            if embedding.len() != 768 {
                log::warn!(
-                    "Unexpected embedding dimensions for item {}: {} (expected {})",
+                    "Unexpected embedding dimensions for item {}: {} (expected 768)",
                    i,
-                    embedding.len(),
+                    embedding.len()
                    crate::ai::embedding_dim()
                );
            }
        }
@@ -1331,20 +1332,8 @@ struct OllamaEmbedResponse {
    embeddings: Vec<Vec<f32>>,
 }
 /// Accumulate tool calls streamed across NDJSON chunks. Ollama ≥0.8 may
 /// emit each tool call on its own chunk; replacing the accumulator on every
 /// chunk would keep only the last call, so extend instead.
 fn append_streamed_tool_calls(
    acc: &mut Option<Vec<crate::ai::llm_client::ToolCall>>,
    new: Vec<crate::ai::llm_client::ToolCall>,
 ) {
    acc.get_or_insert_with(Vec::new).extend(new);
 }
 #[cfg(test)]
 mod tests {
    use super::append_streamed_tool_calls;
    use crate::ai::llm_client::{ToolCall, ToolCallFunction};
    #[test]
    fn generate_photo_description_prompt_is_concise() {
@@ -1355,38 +1344,4 @@ mod tests {
                      Focus on the people, location, and activity.";
        assert!(prompt.len() < 200, "Prompt should be concise");
    }
    fn call(name: &str) -> ToolCall {
        ToolCall {
            id: None,
            function: ToolCallFunction {
                name: name.to_string(),
                arguments: serde_json::json!({}),
            },
        }
    }
    #[test]
    fn streamed_tool_calls_across_chunks_accumulate() {
        // Two tool calls arriving in two separate stream chunks must BOTH
        // survive assembly — the old `tool_calls = Some(tcs)` kept only the
        // last chunk's calls.
        let mut acc: Option<Vec<ToolCall>> = None;
        append_streamed_tool_calls(&mut acc, vec![call("get_sms_messages")]);
        append_streamed_tool_calls(&mut acc, vec![call("reverse_geocode")]);
        let calls = acc.expect("tool calls accumulated");
        assert_eq!(calls.len(), 2);
        assert_eq!(calls[0].function.name, "get_sms_messages");
        assert_eq!(calls[1].function.name, "reverse_geocode");
    }
    #[test]
    fn streamed_tool_calls_single_chunk_batch_kept_intact() {
        // Older Ollama servers attach all calls to one chunk — unchanged.
        let mut acc: Option<Vec<ToolCall>> = None;
        append_streamed_tool_calls(&mut acc, vec![call("a"), call("b")]);
        let calls = acc.expect("tool calls accumulated");
        assert_eq!(calls.len(), 2);
    }
 }
@@ -1,282 +0,0 @@
 // User-configurable pronunciation overrides for TTS. Chatterbox mispronounces
 // place names ("Worcester"), initialisms ("WSL"), and clipped abbreviations
 // ("blvd"), so we rewrite them to phonetic spellings before synthesis.
 //
 // The map lives in a JSON file on the server — a flat object of
 // `"written form": "spoken form"` pairs, e.g.:
 //
 //   {
 //     "Worcester": "Wuster",
 //     "WSL": "W S L",
 //     "blvd": "boulevard",
 //     "Dr.": "Doctor"
 //   }
 //
 // Path comes from `TTS_PRONUNCIATIONS_PATH` (default `tts_pronunciations.json`
 // in the working directory). A missing file simply disables the feature. The
 // file is re-read whenever its mtime changes, so edits apply to the next
 // synthesis without a restart; a malformed edit keeps the last good map and
 // logs the parse error instead of silently dropping all overrides.
 //
 // Matching rules:
 // - Whole words only — `cat` never rewrites `category`. (Boundaries are only
 //   asserted next to word characters, so keys like `Dr.` still work.)
 // - Smartcase: an all-lowercase key matches case-insensitively; a key with
 //   any uppercase matches exactly. That lets `worcester` catch every casing
 //   while `US` (the country) leaves the pronoun `us` alone.
 // - Longer keys win over shorter ones (`New York Times` before `New York`).
 use regex::Regex;
 use std::collections::HashMap;
 use std::path::Path;
 use std::sync::{Arc, LazyLock, Mutex as StdMutex};
 use std::time::SystemTime;
 /// A compiled pronunciation map: one alternation regex over every key plus
 /// the lookup tables the replacement closure resolves matches against.
 #[derive(Default)]
 struct CompiledMap {
    /// `None` when the map is empty — apply() is then a no-op.
    regex: Option<Regex>,
    /// Case-sensitive entries, keyed verbatim.
    exact: HashMap<String, String>,
    /// Case-insensitive entries, keyed lowercased.
    folded: HashMap<String, String>,
 }
 impl CompiledMap {
    fn from_entries(entries: &HashMap<String, String>) -> Self {
        let mut keys: Vec<&str> = entries
            .keys()
            .map(|k| k.as_str())
            .filter(|k| !k.trim().is_empty())
            .collect();
        if keys.is_empty() {
            return Self::default();
        }
        // Longest key first so overlapping entries prefer the more specific
        // one (regex alternation is first-match-wins, not longest-match).
        keys.sort_by(|a, b| b.len().cmp(&a.len()).then(a.cmp(b)));
        let mut exact = HashMap::new();
        let mut folded = HashMap::new();
        let alternatives: Vec<String> = keys
            .iter()
            .map(|key| {
                let escaped = regex::escape(key);
                // Only assert a word boundary where the key edge is a word
                // character — `\b` adjacent to punctuation (e.g. the dot in
                // `Dr.`) would otherwise never match.
                let lead = if key
                    .chars()
                    .next()
                    .is_some_and(|c| c.is_alphanumeric() || c == '_')
                {
                    r"\b"
                } else {
                    ""
                };
                let trail = if key
                    .chars()
                    .last()
                    .is_some_and(|c| c.is_alphanumeric() || c == '_')
                {
                    r"\b"
                } else {
                    ""
                };
                let case_sensitive = key.chars().any(|c| c.is_uppercase());
                if case_sensitive {
                    exact.insert(key.to_string(), entries[*key].clone());
                    format!("{lead}{escaped}{trail}")
                } else {
                    folded.insert(key.to_lowercase(), entries[*key].clone());
                    format!("{lead}(?i:{escaped}){trail}")
                }
            })
            .collect();
        // Escaped fixed strings can't produce an invalid pattern; if one ever
        // does, treat the whole map as empty rather than panicking a handler.
        let pattern = alternatives.join("|");
        let regex = match Regex::new(&pattern) {
            Ok(r) => Some(r),
            Err(e) => {
                log::error!("pronunciation map failed to compile: {e}");
                None
            }
        };
        Self {
            regex,
            exact,
            folded,
        }
    }
    fn apply(&self, text: &str) -> String {
        let Some(re) = &self.regex else {
            return text.to_string();
        };
        re.replace_all(text, |caps: &regex::Captures| {
            let m = &caps[0];
            self.exact
                .get(m)
                .or_else(|| self.folded.get(&m.to_lowercase()))
                .cloned()
                // Unreachable in practice — every alternative came from one
                // of the two maps — but never drop the user's text.
                .unwrap_or_else(|| m.to_string())
        })
        .into_owned()
    }
 }
 struct CacheEntry {
    mtime: Option<SystemTime>,
    compiled: Arc<CompiledMap>,
 }
 static CACHE: LazyLock<StdMutex<Option<CacheEntry>>> = LazyLock::new(|| StdMutex::new(None));
 fn config_path() -> String {
    std::env::var("TTS_PRONUNCIATIONS_PATH")
        .ok()
        .map(|s| s.trim().to_string())
        .filter(|s| !s.is_empty())
        .unwrap_or_else(|| "tts_pronunciations.json".to_string())
 }
 /// Load the compiled map, re-reading the file only when its mtime changed
 /// since the last call (or it appeared/disappeared). Synthesis is serialized
 /// on a single GPU permit, so a stat per call is noise.
 fn current_map() -> Arc<CompiledMap> {
    let path_s = config_path();
    let path = Path::new(&path_s);
    let mtime = std::fs::metadata(path).and_then(|m| m.modified()).ok();
    let mut cache = CACHE.lock().unwrap();
    if let Some(entry) = cache.as_ref()
        && entry.mtime == mtime
    {
        return entry.compiled.clone();
    }
    let compiled = match mtime {
        None => Arc::new(CompiledMap::default()), // no file → no overrides
        Some(_) => match std::fs::read_to_string(path)
            .map_err(anyhow::Error::from)
            .and_then(|s| Ok(serde_json::from_str::<HashMap<String, String>>(&s)?))
        {
            Ok(entries) => {
                log::info!(
                    "loaded {} pronunciation override(s) from {path_s}",
                    entries.len()
                );
                Arc::new(CompiledMap::from_entries(&entries))
            }
            Err(e) => {
                log::error!("failed to load pronunciation map {path_s}: {e}");
                // Keep serving the previous map rather than regressing to
                // none mid-edit; still record the new mtime so the error
                // logs once per bad save, not once per synthesis.
                cache
                    .as_ref()
                    .map(|c| c.compiled.clone())
                    .unwrap_or_default()
            }
        },
    };
    *cache = Some(CacheEntry {
        mtime,
        compiled: compiled.clone(),
    });
    compiled
 }
 /// Rewrite configured words/abbreviations to their phonetic spellings.
 /// Call on cleaned (post-markdown-strip) text, right before synthesis.
 pub fn apply_pronunciations(text: &str) -> String {
    current_map().apply(text)
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    fn compile(pairs: &[(&str, &str)]) -> CompiledMap {
        let entries = pairs
            .iter()
            .map(|(k, v)| (k.to_string(), v.to_string()))
            .collect();
        CompiledMap::from_entries(&entries)
    }
    #[test]
    fn empty_map_is_a_noop() {
        let m = compile(&[]);
        assert_eq!(m.apply("nothing changes"), "nothing changes");
    }
    #[test]
    fn replaces_whole_words_only() {
        let m = compile(&[("cat", "kitty")]);
        assert_eq!(m.apply("the cat sat"), "the kitty sat");
        // No substring rewrites.
        assert_eq!(m.apply("the category"), "the category");
        assert_eq!(m.apply("concatenate"), "concatenate");
    }
    #[test]
    fn lowercase_keys_match_any_casing() {
        let m = compile(&[("worcester", "Wuster")]);
        assert_eq!(m.apply("Worcester is nice"), "Wuster is nice");
        assert_eq!(m.apply("in WORCESTER today"), "in Wuster today");
        assert_eq!(m.apply("worcester sauce"), "Wuster sauce");
    }
    #[test]
    fn uppercase_keys_match_case_sensitively() {
        let m = compile(&[("US", "U S")]);
        assert_eq!(m.apply("the US economy"), "the U S economy");
        // The pronoun survives.
        assert_eq!(m.apply("join us today"), "join us today");
    }
    #[test]
    fn keys_with_punctuation_work() {
        // `\b` is only asserted next to word characters, so the trailing dot
        // doesn't break matching.
        let m = compile(&[("Dr.", "Doctor"), ("blvd", "boulevard")]);
        assert_eq!(
            m.apply("Dr. Smith on Sunset blvd"),
            "Doctor Smith on Sunset boulevard"
        );
    }
    #[test]
    fn longer_keys_win_over_shorter() {
        let m = compile(&[("new york", "Noo York"), ("new york times", "the Times")]);
        assert_eq!(m.apply("read the new york times"), "read the the Times");
        assert_eq!(m.apply("visit new york soon"), "visit Noo York soon");
    }
    #[test]
    fn multiple_occurrences_all_rewrite() {
        let m = compile(&[("wsl", "W S L")]);
        assert_eq!(m.apply("WSL and wsl and Wsl"), "W S L and W S L and W S L");
    }
    #[test]
    fn replacement_text_is_verbatim() {
        // Replacements aren't re-scanned — a value containing another key
        // doesn't cascade.
        let m = compile(&[("a1", "b2"), ("b2", "c3")]);
        assert_eq!(m.apply("a1"), "b2");
    }
    #[test]
    fn blank_keys_are_ignored() {
        let m = compile(&[("", "x"), ("  ", "y"), ("ok", "fine")]);
        assert_eq!(m.apply("ok then"), "fine then");
    }
 }
@@ -20,36 +20,34 @@ impl SmsApiClient {
        }
    }
-    /// Compute a `[start, end]` unix-second window of `2 * radius_days`
+    /// Fetch messages for a specific contact within ±`days_radius` days of
-    /// centered on `center_ts`. `radius_days < 1` is clamped to 1 to avoid
+    /// the given timestamp (defaults to ±4 days when `None`). Falls back to
-    /// degenerate zero-width windows.
+    /// all contacts if no messages are found for the specified contact.
-    pub(crate) fn window_for_radius(center_ts: i64, radius_days: i64) -> (i64, i64) {
+    /// Messages are sorted by proximity to the center timestamp.
        let r = radius_days.max(1);
        let span = r * 86400;
        (center_ts - span, center_ts + span)
    }
    /// Fetch messages for a specific contact within ±`radius_days` of the
    /// given timestamp. Falls back to all contacts when no messages found
    /// for the named contact. Sorted by proximity to the center timestamp.
    pub async fn fetch_messages_for_contact(
        &self,
        contact: Option<&str>,
        center_timestamp: i64,
-        radius_days: i64,
+        days_radius: Option<i64>,
    ) -> Result<Vec<SmsMessage>> {
-        let effective_radius = radius_days.max(1);
+        use chrono::Duration;
        let (start_ts, end_ts) = Self::window_for_radius(center_timestamp, radius_days);
        let radius = days_radius.unwrap_or(4).clamp(1, 30);
        let center_dt = chrono::DateTime::from_timestamp(center_timestamp, 0)
            .ok_or_else(|| anyhow::anyhow!("Invalid timestamp"))?;
        let start_dt = center_dt - Duration::days(radius);
        let end_dt = center_dt + Duration::days(radius);
        let start_ts = start_dt.timestamp();
        let end_ts = end_dt.timestamp();
        // If contact specified, try fetching for that contact first
        if let Some(contact_name) = contact {
            log::info!(
                "Fetching SMS for contact: {} (±{} days from {})",
                contact_name,
-                effective_radius,
+                radius,
                center_dt.format("%Y-%m-%d %H:%M:%S")
            );
            let messages = self
@@ -74,7 +72,7 @@ impl SmsApiClient {
        // Fallback to all contacts
        log::info!(
            "Fetching all SMS messages (±{} days from {})",
-            effective_radius,
+            radius,
            center_dt.format("%Y-%m-%d %H:%M:%S")
        );
        self.fetch_messages(start_ts, end_ts, None, Some(center_timestamp))
@@ -257,48 +255,23 @@ impl SmsApiClient {
    }
    /// Search message bodies via the Django side's FTS5 / semantic / hybrid
-    /// endpoint. `params.mode` selects the ranking strategy:
+    /// endpoint. `mode` selects the ranking strategy:
    ///   - "fts5"     keyword-only, supports phrase / prefix / boolean / NEAR
    ///   - "semantic" embedding similarity
    ///   - "hybrid"   both merged via reciprocal rank fusion (recommended)
    ///
    /// All of `contact_id`, `date_from` / `date_to` (unix seconds), `is_mms`,
    /// `has_media`, and `offset` are pushed to SMS-API server-side so the
    /// filtered+paginated result set is exact rather than a client-side
    /// over-fetch.
    pub async fn search_messages(
        &self,
        query: &str,
-        params: &SmsSearchParams<'_>,
+        mode: &str,
        limit: usize,
    ) -> Result<Vec<SmsSearchHit>> {
-        let mut url = format!(
+        let url = format!(
            "{}/api/messages/search/?q={}&mode={}&limit={}",
            self.base_url,
            urlencoding::encode(query),
-            urlencoding::encode(params.mode),
+            urlencoding::encode(mode),
-            params.limit,
+            limit
        );
        if let Some(cid) = params.contact_id {
            url.push_str(&format!("&contact_id={}", cid));
        }
        if let Some(ref c) = params.contact {
            url.push_str(&format!("&contact={}", urlencoding::encode(c)));
        }
        if let Some(off) = params.offset {
            url.push_str(&format!("&offset={}", off));
        }
        if let Some(from) = params.date_from {
            url.push_str(&format!("&date_from={}", from));
        }
        if let Some(to) = params.date_to {
            url.push_str(&format!("&date_to={}", to));
        }
        if let Some(is_mms) = params.is_mms {
            url.push_str(&format!("&is_mms={}", is_mms));
        }
        if let Some(has_media) = params.has_media {
            url.push_str(&format!("&has_media={}", has_media));
        }
        let mut request = self.client.get(&url);
        if let Some(token) = &self.token {
@@ -401,33 +374,6 @@ pub struct SmsSearchHit {
    /// Present for semantic / hybrid modes; absent for fts5.
    #[serde(default)]
    pub similarity_score: Option<f32>,
    /// SMS-API-generated excerpt around the match, wrapped in `<mark>` tags.
    /// For MMS messages that only matched via attachment text / filename
    /// (empty `body`), the snippet is the only meaningful preview.
    #[serde(default)]
    pub snippet: Option<String>,
 }
 /// Optional filter / paging knobs for [`SmsApiClient::search_messages`].
 /// All fields except `mode` and `limit` map 1:1 to the same-named SMS-API
 /// query params (added in the 2026-05 search-enhancements release).
 #[derive(Debug, Clone)]
 pub struct SmsSearchParams<'a> {
    pub mode: &'a str,
    pub limit: usize,
    pub contact_id: Option<i64>,
    /// Contact name (case-insensitive). Resolved to a numeric ID by the
    /// SMS-API server when `contact_id` is not set.
    pub contact: Option<String>,
    /// Unix-seconds inclusive lower bound on `date`.
    pub date_from: Option<i64>,
    /// Unix-seconds inclusive upper bound on `date`.
    pub date_to: Option<i64>,
    /// `Some(true)` = MMS only, `Some(false)` = SMS only, `None` = both.
    pub is_mms: Option<bool>,
    /// `Some(true)` = only messages with image/video/audio attachments.
    pub has_media: Option<bool>,
    pub offset: Option<usize>,
 }
 #[derive(Deserialize)]
@@ -437,29 +383,3 @@ struct SmsSearchResponse {
    #[serde(default)]
    search_method: String,
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    #[test]
    fn window_for_radius_produces_2n_day_span() {
        let center: i64 = 1_700_000_000;
        let (start, end) = SmsApiClient::window_for_radius(center, 7);
        assert_eq!(end - start, 14 * 86400);
        assert_eq!(start + 7 * 86400, center);
        assert_eq!(end - 7 * 86400, center);
    }
    #[test]
    fn window_for_radius_clamps_zero_to_one() {
        let (start, end) = SmsApiClient::window_for_radius(100_000, 0);
        assert_eq!(end - start, 2 * 86400);
    }
    #[test]
    fn window_for_radius_clamps_negative_to_one() {
        let (start, end) = SmsApiClient::window_for_radius(100_000, -7);
        assert_eq!(end - start, 2 * 86400);
    }
 }
@@ -1,854 +0,0 @@
 use crate::ai::insight_chat::ChatStreamEvent;
 use std::collections::HashMap;
 use std::sync::Arc;
 use std::sync::Mutex as StdMutex;
 use std::sync::atomic::{AtomicU32, Ordering};
 use std::time::Instant;
 use tokio::sync::{Mutex, Notify};
 use tokio::task::AbortHandle;
 /// Maximum number of events buffered per turn. Agentic turns typically
 /// produce ~120 events; 500 provides 4× headroom. When exceeded, oldest
 /// events are evicted from the front.
 const MAX_BUFFERED_EVENTS: usize = 500;
 /// Turn status codes used by `TurnEntry::status`.
 #[derive(Debug, Clone, Copy, PartialEq)]
 pub enum TurnStatus {
    Running = 0,
    Done = 1,
    Error = 2,
    Cancelled = 3,
 }
 impl From<u32> for TurnStatus {
    fn from(v: u32) -> Self {
        match v {
            0 => TurnStatus::Running,
            1 => TurnStatus::Done,
            2 => TurnStatus::Error,
            3 => TurnStatus::Cancelled,
            _ => TurnStatus::Running,
        }
    }
 }
 impl TurnStatus {
    pub fn as_str(&self) -> &'static str {
        match self {
            TurnStatus::Running => "running",
            TurnStatus::Done => "done",
            TurnStatus::Error => "error",
            TurnStatus::Cancelled => "cancelled",
        }
    }
 }
 /// Shared metadata about a turn, read by the SSE replay handler to emit
 /// the initial `turn_info` event and to decide whether to wait for new
 /// events or close immediately.
 #[derive(Debug, Clone)]
 pub struct TurnInfo {
    pub turn_id: String,
    pub file_path: String,
    pub library_id: i32,
    /// Persona id for persona-scoped turns. None for insight-scoped turns.
    pub persona_id: Option<String>,
    /// `"insight"` (file-anchored chat) | `"persona"` (open chat).
    pub scope: String,
    pub status: TurnStatus,
    pub total_events_pushed: u32,
    pub buffered_count: u32,
 }
 /// Result of reading events at or after an absolute `skip_before` index.
 #[derive(Debug)]
 pub enum ReplayOutcome {
    /// New events are available. `next_skip` is the absolute index to pass
    /// on the next read (i.e. one past the last event returned).
    Events {
        events: Vec<ChatStreamEvent>,
        next_skip: u32,
    },
    /// The reader is caught up to the live edge — no events past `skip_before`
    /// yet. `next_skip` is the current high-water mark.
    CaughtUp { next_skip: u32 },
    /// `skip_before` points below the buffer's base index: the requested
    /// events were evicted. Maps to HTTP 410 Gone.
    Gone,
 }
 /// Per-turn state shared between the agentic loop (writer) and all SSE
 /// replay connections (readers).
 pub struct TurnEntry {
    pub turn_id: String,
    /// Stable identity used to scope the turn. For insight chat this is
    /// `(file_path, library_id)`. For persona chat it is `(persona_id)`.
    /// The generic fields stay populated regardless of scope so the SSE
    /// `turn_info` frame can always surface a useful label — a future
    /// "list active turns" debugging surface won't need to discriminate.
    pub file_path: String,
    pub library_id: i32,
    /// Persona id for persona-scoped turns (`scope == "persona"`). None for
    /// the existing insight-scoped turns. The field is the cheapest way to
    /// let the SSE `turn_info` payload round-trip without a parallel map.
    pub persona_id: Option<String>,
    /// `"insight"` (default — file-anchored chat) | `"persona"` (open chat).
    /// Stable string so the SSE `turn_info` payload can carry it as-is.
    pub scope: String,
    /// Shared event buffer — multiple SSE connections can read independently.
    /// Each connection tracks its own `skip_before` offset.
    events: Mutex<Vec<ChatStreamEvent>>,
    /// Monotonic counter: total events pushed (may exceed events.len()
    /// due to eviction). Used for skip_before indexing.
    total_events_pushed: AtomicU32,
    /// The event index that this entry started with. Adjusts on eviction
    /// so that `skip_before` stays absolute across connections.
    base_index: AtomicU32,
    pub status: AtomicU32,
    /// Abort handle for the spawned agentic task, set once after spawn.
    /// Behind a std `Mutex` because the entry is shared via `Arc` and the
    /// handle is installed after the entry is already in the registry.
    abort_handle: StdMutex<Option<AbortHandle>>,
    pub created_at: Instant,
    notify: Arc<Notify>,
 }
 impl TurnEntry {
    pub fn new(turn_id: String, file_path: String, library_id: i32) -> Self {
        Self::with_scope(turn_id, file_path, library_id, None, "insight")
    }
    /// Persona-scoped constructor. `file_path` is unused by the persona
    /// chat loop but kept on the struct so the SSE `turn_info` frame has
    /// the same shape across scopes.
    pub fn new_persona(turn_id: String, persona_id: String, library_id: i32) -> Self {
        Self::with_scope(
            turn_id,
            String::new(),
            library_id,
            Some(persona_id),
            "persona",
        )
    }
    fn with_scope(
        turn_id: String,
        file_path: String,
        library_id: i32,
        persona_id: Option<String>,
        scope: &str,
    ) -> Self {
        Self {
            turn_id,
            file_path,
            library_id,
            persona_id,
            scope: scope.to_string(),
            events: Mutex::new(Vec::new()),
            total_events_pushed: AtomicU32::new(0),
            base_index: AtomicU32::new(0),
            status: AtomicU32::new(TurnStatus::Running as u32),
            abort_handle: StdMutex::new(None),
            created_at: Instant::now(),
            notify: Arc::new(Notify::new()),
        }
    }
    /// Install the abort handle for the spawned agentic task. Called once,
    /// right after the task is spawned.
    pub fn set_abort_handle(&self, handle: AbortHandle) {
        *self.abort_handle.lock().expect("abort_handle poisoned") = Some(handle);
    }
    /// Abort the spawned agentic task, if a handle was installed. Returns
    /// `true` if a task was aborted.
    pub fn abort(&self) -> bool {
        if let Some(handle) = self
            .abort_handle
            .lock()
            .expect("abort_handle poisoned")
            .take()
        {
            handle.abort();
            true
        } else {
            false
        }
    }
    /// Push an event into the buffer. Evicts oldest events if the buffer
    /// exceeds `MAX_BUFFERED_EVENTS`. Notifies all waiting SSE connections.
    pub async fn push_event(&self, event: ChatStreamEvent) {
        {
            let mut events = self.events.lock().await;
            // Evict oldest events if we've hit the cap.
            if events.len() >= MAX_BUFFERED_EVENTS {
                // Drop the oldest event to make room and advance the base
                // index so skip_before stays absolute across connections.
                events.remove(0);
                self.base_index.fetch_add(1, Ordering::Relaxed);
            }
            events.push(event);
            // Increment while holding the buffer lock so the counter stays in
            // lock-step with the buffer even if multiple writers ever exist.
            self.total_events_pushed.fetch_add(1, Ordering::Relaxed);
        }
        self.notify.notify_waiters();
    }
    /// Get a snapshot of turn metadata for the `turn_info` SSE event.
    pub async fn info(&self) -> TurnInfo {
        let events = self.events.lock().await;
        let buffered = events.len() as u32;
        let total = self.total_events_pushed.load(Ordering::Relaxed);
        drop(events);
        TurnInfo {
            turn_id: self.turn_id.clone(),
            file_path: self.file_path.clone(),
            library_id: self.library_id,
            persona_id: self.persona_id.clone(),
            scope: self.scope.clone(),
            status: self.status.load(Ordering::Relaxed).into(),
            total_events_pushed: total,
            buffered_count: buffered,
        }
    }
    /// Set the terminal status and notify all waiters.
    pub fn set_terminal_status(&self, status: TurnStatus) {
        self.status.store(status as u32, Ordering::Relaxed);
        self.notify.notify_waiters();
    }
    /// Read buffered events at or after absolute index `skip_before` without
    /// waiting. Distinguishes "evicted" (Gone) from "caught up" (no new
    /// events yet) — the previous boolean/`Option` API conflated the two.
    pub async fn replay_from(&self, skip_before: u32) -> ReplayOutcome {
        let events = self.events.lock().await;
        let base = self.base_index.load(Ordering::Relaxed);
        // The buffer holds absolute indices [base, base + len). A request
        // below `base` asked for events that have been evicted.
        if skip_before < base {
            return ReplayOutcome::Gone;
        }
        let offset = (skip_before - base) as usize;
        let next_skip = base + events.len() as u32;
        if offset >= events.len() {
            // Caught up to (or past) the live edge — nothing new yet.
            return ReplayOutcome::CaughtUp { next_skip };
        }
        ReplayOutcome::Events {
            events: events[offset..].to_vec(),
            next_skip,
        }
    }
    /// Wait for the next batch of events past `skip_before`, the turn to
    /// finish, or eviction. Returns:
    /// - `Events` when new events are available (drained before any terminal
    ///   signal so the final `Done`/`Error` is never dropped),
    /// - `CaughtUp` only when the turn has reached a terminal status and the
    ///   reader is fully drained (the caller should close the stream),
    /// - `Gone` when `skip_before` points into evicted territory.
    pub async fn next_batch(&self, skip_before: u32) -> ReplayOutcome {
        loop {
            // Register interest BEFORE inspecting state so a push/terminal that
            // races between our read and our await can't be lost (Notify's
            // `notify_waiters` does not store a permit).
            let notified = self.notify.notified();
            tokio::pin!(notified);
            notified.as_mut().enable();
            match self.replay_from(skip_before).await {
                ReplayOutcome::CaughtUp { next_skip } => {
                    // No new events. If the turn is finished, every event
                    // (including the terminal one) has already been drained
                    // above on a prior call, so signal the caller to close.
                    if !self.is_running() {
                        return ReplayOutcome::CaughtUp { next_skip };
                    }
                    // Still running — wait for the next push or terminal.
                }
                other => return other, // Events or Gone
            }
            notified.await;
        }
    }
    /// Check if this turn is still running.
    pub fn is_running(&self) -> bool {
        self.status.load(Ordering::Relaxed) == TurnStatus::Running as u32
    }
 }
 /// In-memory registry of all active chat turns. Injected into `AppState`
 /// and shared across all handlers.
 pub struct TurnRegistry {
    entries: Mutex<HashMap<String, Arc<TurnEntry>>>,
    timeout_secs: u64,
 }
 impl TurnRegistry {
    pub fn new(timeout_secs: u64) -> Self {
        Self {
            entries: Mutex::new(HashMap::new()),
            timeout_secs,
        }
    }
    /// Returns the cleanup timeout in seconds.
    pub fn timeout_secs(&self) -> u64 {
        self.timeout_secs
    }
    /// Insert a new turn entry. Returns the turn_id.
    pub async fn insert(&self, entry: Arc<TurnEntry>) -> String {
        let turn_id = entry.turn_id.clone();
        let mut entries = self.entries.lock().await;
        entries.insert(turn_id.clone(), entry);
        turn_id
    }
    /// Look up a turn by id. Returns None if not found or expired.
    pub async fn get(&self, turn_id: &str) -> Option<Arc<TurnEntry>> {
        let entries = self.entries.lock().await;
        entries.get(turn_id).cloned()
    }
    /// Clean up stale entries older than the timeout. Returns the count of
    /// entries removed.
    pub async fn cleanup_stale(&self) -> usize {
        let mut entries = self.entries.lock().await;
        let _now = Instant::now();
        let stale: Vec<String> = entries
            .iter()
            .filter(|(_, entry)| entry.created_at.elapsed().as_secs() > self.timeout_secs)
            .map(|(id, _)| id.clone())
            .collect();
        for id in &stale {
            entries.remove(id);
        }
        if !stale.is_empty() {
            log::info!(
                "TurnRegistry: cleaned up {} stale entries (timeout={}s)",
                stale.len(),
                self.timeout_secs
            );
        }
        stale.len()
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use crate::ai::insight_chat::ChatStreamEvent;
    use std::time::Duration;
    /// Unwrap the events from a `ReplayOutcome::Events`, panicking otherwise.
    fn events_of(outcome: ReplayOutcome) -> Vec<ChatStreamEvent> {
        match outcome {
            ReplayOutcome::Events { events, .. } => events,
            other => panic!("expected Events, got {other:?}"),
        }
    }
    // ── TurnStatus ──────────────────────────────────────────────────
    #[test]
    fn turn_status_from_u32_valid_values() {
        assert_eq!(TurnStatus::from(0), TurnStatus::Running);
        assert_eq!(TurnStatus::from(1), TurnStatus::Done);
        assert_eq!(TurnStatus::from(2), TurnStatus::Error);
        assert_eq!(TurnStatus::from(3), TurnStatus::Cancelled);
    }
    #[test]
    fn turn_status_from_u32_unknown_defaults_to_running() {
        assert_eq!(TurnStatus::from(4), TurnStatus::Running);
        assert_eq!(TurnStatus::from(u32::MAX), TurnStatus::Running);
    }
    #[test]
    fn turn_status_as_str() {
        assert_eq!(TurnStatus::Running.as_str(), "running");
        assert_eq!(TurnStatus::Done.as_str(), "done");
        assert_eq!(TurnStatus::Error.as_str(), "error");
        assert_eq!(TurnStatus::Cancelled.as_str(), "cancelled");
    }
    // ── TurnEntry ───────────────────────────────────────────────────
    #[tokio::test]
    async fn turn_entry_push_and_replay() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        entry
            .push_event(ChatStreamEvent::TextDelta("hello".to_string()))
            .await;
        entry
            .push_event(ChatStreamEvent::TextDelta(" world".to_string()))
            .await;
        let events = events_of(entry.replay_from(0).await);
        assert_eq!(events.len(), 2);
    }
    #[tokio::test]
    async fn turn_entry_replay_with_skip() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        for i in 0..5 {
            entry
                .push_event(ChatStreamEvent::TextDelta(format!("e{i}")))
                .await;
        }
        // skip_before=0 → all 5 events
        let all = events_of(entry.replay_from(0).await);
        assert_eq!(all.len(), 5);
        // skip_before=2 → events 2,3,4 (3 events)
        let skipped = events_of(entry.replay_from(2).await);
        assert_eq!(skipped.len(), 3);
        // skip_before=5 → caught up to the live edge (not Gone).
        assert!(matches!(
            entry.replay_from(5).await,
            ReplayOutcome::CaughtUp { next_skip: 5 }
        ));
    }
    #[tokio::test]
    async fn turn_entry_replay_empty_by_default() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        // Empty buffer with skip_before=0 → caught up (nothing to replay yet).
        assert!(matches!(
            entry.replay_from(0).await,
            ReplayOutcome::CaughtUp { next_skip: 0 }
        ));
    }
    #[tokio::test]
    async fn turn_entry_is_running_initially() {
        let entry = TurnEntry::new("t1".to_string(), "/photo.jpg".to_string(), 1);
        assert!(entry.is_running());
    }
    #[tokio::test]
    async fn turn_entry_set_terminal_status() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        assert!(entry.is_running());
        entry.set_terminal_status(TurnStatus::Done);
        assert!(!entry.is_running());
    }
    #[tokio::test]
    async fn turn_entry_info() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            42,
        ));
        entry
            .push_event(ChatStreamEvent::TextDelta("x".to_string()))
            .await;
        entry.set_terminal_status(TurnStatus::Done);
        let info = entry.info().await;
        assert_eq!(info.turn_id, "t1");
        assert_eq!(info.file_path, "/photo.jpg");
        assert_eq!(info.library_id, 42);
        assert_eq!(info.status, TurnStatus::Done);
        assert_eq!(info.total_events_pushed, 1);
        assert_eq!(info.buffered_count, 1);
    }
    #[tokio::test]
    async fn turn_entry_eviction_caps_buffer() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        // Push MAX_BUFFERED_EVENTS + 10 events.
        for i in 0..(MAX_BUFFERED_EVENTS + 10) {
            entry
                .push_event(ChatStreamEvent::TextDelta(format!("e{i}")))
                .await;
        }
        // Asking from absolute 0 after eviction is Gone (0-9 were dropped).
        assert!(matches!(entry.replay_from(0).await, ReplayOutcome::Gone));
        // Reading from the new base (10) returns the full capped buffer.
        let events = events_of(entry.replay_from(10).await);
        assert_eq!(events.len(), MAX_BUFFERED_EVENTS);
        // First event should be at index 10 (0-9 were evicted).
        if let ChatStreamEvent::TextDelta(s) = &events[0] {
            assert_eq!(s, "e10");
        } else {
            panic!("expected TextDelta");
        }
        // Last event should be at index MAX_BUFFERED_EVENTS + 9.
        if let ChatStreamEvent::TextDelta(s) = &events[events.len() - 1] {
            assert_eq!(s, &format!("e{}", MAX_BUFFERED_EVENTS + 9));
        } else {
            panic!("expected TextDelta");
        }
    }
    #[tokio::test]
    async fn turn_entry_replay_evicted_index_is_gone() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        // Push one past the cap so exactly one event (index 0) is evicted.
        for i in 0..=MAX_BUFFERED_EVENTS {
            entry
                .push_event(ChatStreamEvent::TextDelta(format!("e{i}")))
                .await;
        }
        // Base is now 1; asking from absolute 0 is evicted territory → Gone.
        assert!(matches!(entry.replay_from(0).await, ReplayOutcome::Gone));
        // skip_before = MAX_BUFFERED_EVENTS → last event only (index valid).
        let last = events_of(entry.replay_from(MAX_BUFFERED_EVENTS as u32).await);
        assert_eq!(last.len(), 1);
        // skip_before = MAX_BUFFERED_EVENTS + 1 → caught up to the live edge.
        assert!(matches!(
            entry.replay_from((MAX_BUFFERED_EVENTS + 1) as u32).await,
            ReplayOutcome::CaughtUp { .. }
        ));
    }
    // ── TurnRegistry ────────────────────────────────────────────────
    #[tokio::test]
    async fn turn_registry_insert_and_get() {
        let registry = TurnRegistry::new(300);
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        let id = registry.insert(entry).await;
        assert_eq!(id, "t1");
        let retrieved = registry.get("t1").await;
        assert!(retrieved.is_some());
        assert_eq!(retrieved.unwrap().turn_id, "t1");
    }
    #[tokio::test]
    async fn turn_registry_get_nonexistent_returns_none() {
        let registry = TurnRegistry::new(300);
        assert!(registry.get("nonexistent").await.is_none());
    }
    #[tokio::test]
    async fn turn_registry_cleanup_stale_removes_old_entries() {
        let registry = TurnRegistry::new(0);
        let mut entry = TurnEntry::new("t1".to_string(), "/photo.jpg".to_string(), 1);
        entry.created_at = Instant::now() - Duration::from_secs(1);
        registry.insert(Arc::new(entry)).await;
        let cleaned = registry.cleanup_stale().await;
        assert_eq!(cleaned, 1);
        assert!(registry.get("t1").await.is_none());
    }
    #[tokio::test]
    async fn turn_registry_cleanup_stale_preserves_recent() {
        let registry = TurnRegistry::new(3600); // 1 hour
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        registry.insert(entry).await;
        let cleaned = registry.cleanup_stale().await;
        assert_eq!(cleaned, 0);
        assert!(registry.get("t1").await.is_some());
    }
    #[tokio::test]
    async fn turn_registry_cleanup_stale_multiple() {
        let registry = TurnRegistry::new(0);
        for i in 0..5 {
            let mut entry = TurnEntry::new(format!("t{i}"), "/photo.jpg".to_string(), 1);
            entry.created_at = Instant::now() - Duration::from_secs(1);
            registry.insert(Arc::new(entry)).await;
        }
        let cleaned = registry.cleanup_stale().await;
        assert_eq!(cleaned, 5);
    }
    #[tokio::test]
    async fn turn_registry_timeout_secs() {
        let registry = TurnRegistry::new(600);
        assert_eq!(registry.timeout_secs(), 600);
    }
    // ── next_batch / live replay ────────────────────────────────────
    /// Drain a turn the way the SSE replay handler does: pull batches via
    /// `next_batch` until the turn is finished and fully drained.
    async fn drain_to_end(entry: Arc<TurnEntry>) -> Vec<ChatStreamEvent> {
        let mut out = Vec::new();
        let mut skip = 0u32;
        while let ReplayOutcome::Events { events, next_skip } = entry.next_batch(skip).await {
            out.extend(events);
            skip = next_skip;
        }
        out
    }
    fn is_terminal(ev: &ChatStreamEvent) -> bool {
        matches!(ev, ChatStreamEvent::Done { .. } | ChatStreamEvent::Error(_))
    }
    /// The core guarantee behind the replay rewrite: a reader waiting on
    /// `next_batch` always receives the terminal event, even though the
    /// writer flips status to terminal immediately after pushing it.
    #[tokio::test]
    async fn next_batch_always_delivers_terminal_event() {
        for _ in 0..50 {
            let entry = Arc::new(TurnEntry::new("t".into(), "/p.jpg".into(), 1));
            let writer = entry.clone();
            let w = tokio::spawn(async move {
                writer
                    .push_event(ChatStreamEvent::IterationStart { n: 1, max: 6 })
                    .await;
                writer
                    .push_event(ChatStreamEvent::TextDelta("hi".into()))
                    .await;
                // Push terminal then flip status with no await between — the
                // race that previously dropped the Done on the reader side.
                writer
                    .push_event(ChatStreamEvent::Done {
                        tool_calls_made: 0,
                        iterations_used: 1,
                        truncated: false,
                        prompt_tokens: None,
                        eval_tokens: None,
                        num_ctx: None,
                        amended_insight_id: None,
                        backend_used: "local".into(),
                        model_used: "m".into(),
                        cancelled: false,
                    })
                    .await;
                writer.set_terminal_status(TurnStatus::Done);
            });
            let events = drain_to_end(entry).await;
            w.await.unwrap();
            assert!(
                events.last().is_some_and(is_terminal),
                "terminal event missing; got {} events",
                events.len()
            );
            assert_eq!(events.len(), 3, "expected IterationStart, TextDelta, Done");
        }
    }
    /// A reader that connects before any event is pushed blocks in
    /// `next_batch` and then receives events as the writer produces them.
    #[tokio::test]
    async fn next_batch_waits_for_late_events() {
        let entry = Arc::new(TurnEntry::new("t".into(), "/p.jpg".into(), 1));
        let writer = entry.clone();
        tokio::spawn(async move {
            tokio::task::yield_now().await;
            writer
                .push_event(ChatStreamEvent::TextDelta("late".into()))
                .await;
            writer.set_terminal_status(TurnStatus::Done);
        });
        // First call blocks until the writer pushes, rather than returning
        // CaughtUp on the empty buffer of a running turn.
        match entry.next_batch(0).await {
            ReplayOutcome::Events { events, next_skip } => {
                assert_eq!(events.len(), 1);
                assert_eq!(next_skip, 1);
            }
            other => panic!("expected Events, got {other:?}"),
        }
    }
    #[tokio::test]
    async fn next_batch_closes_on_terminal_when_caught_up() {
        let entry = Arc::new(TurnEntry::new("t".into(), "/p.jpg".into(), 1));
        entry
            .push_event(ChatStreamEvent::TextDelta("x".into()))
            .await;
        entry.set_terminal_status(TurnStatus::Done);
        // Caught up (skip past the one buffered event) on a finished turn →
        // CaughtUp so the handler closes the stream rather than hanging.
        assert!(matches!(
            entry.next_batch(1).await,
            ReplayOutcome::CaughtUp { .. }
        ));
    }
    #[tokio::test]
    async fn next_batch_reports_gone_for_evicted_index() {
        let entry = Arc::new(TurnEntry::new("t".into(), "/p.jpg".into(), 1));
        for i in 0..=MAX_BUFFERED_EVENTS {
            entry
                .push_event(ChatStreamEvent::TextDelta(format!("e{i}")))
                .await;
        }
        // Index 0 was evicted (base advanced to 1).
        assert!(matches!(entry.next_batch(0).await, ReplayOutcome::Gone));
    }
    // ── abort handle (#1 cancellation) ──────────────────────────────
    #[tokio::test]
    async fn abort_handle_aborts_task_once() {
        let entry = Arc::new(TurnEntry::new("t".into(), "/p.jpg".into(), 1));
        // No handle installed yet → abort is a no-op.
        assert!(!entry.abort());
        let handle = tokio::spawn(async {
            // Long-lived task that only ends via abort.
            futures::future::pending::<()>().await;
        });
        entry.set_abort_handle(handle.abort_handle());
        assert!(entry.abort(), "first abort should fire");
        assert!(!entry.abort(), "handle is taken; second abort is a no-op");
        // The aborted task resolves to a cancellation JoinError.
        let join = handle.await;
        assert!(join.unwrap_err().is_cancelled());
    }
    #[tokio::test]
    async fn base_index_tracks_eviction() {
        let entry = Arc::new(TurnEntry::new("t".into(), "/p.jpg".into(), 1));
        for i in 0..(MAX_BUFFERED_EVENTS + 5) {
            entry
                .push_event(ChatStreamEvent::TextDelta(format!("e{i}")))
                .await;
        }
        let info = entry.info().await;
        // 5 events evicted; total keeps climbing, buffer stays capped.
        assert_eq!(info.total_events_pushed, (MAX_BUFFERED_EVENTS + 5) as u32);
        assert_eq!(info.buffered_count, MAX_BUFFERED_EVENTS as u32);
        // First live index is 5: reading from there yields the full buffer.
        let from_base = events_of(entry.replay_from(5).await);
        assert_eq!(from_base.len(), MAX_BUFFERED_EVENTS);
    }
    // ── Persona-scope regression guards ───────────────────────────
    //
    // The open-chat surface relies on `scope == "persona"` and the
    // optional `persona_id` surviving a round-trip through `info()`.
    // A future refactor that hard-codes "insight" or drops the field
    // would silently break the SSE `turn_info` payload — guard it.
    #[tokio::test]
    async fn turn_entry_supports_persona_scope_label() {
        let entry = Arc::new(TurnEntry::new_persona(
            "tp1".to_string(),
            "journal".to_string(),
            1,
        ));
        assert_eq!(entry.scope, "persona");
        assert_eq!(entry.persona_id.as_deref(), Some("journal"));
        let info = entry.info().await;
        assert_eq!(info.scope, "persona");
        assert_eq!(info.persona_id.as_deref(), Some("journal"));
        // File-anchored fields stay blank — the persona chat has no file.
        assert!(info.file_path.is_empty());
    }
    #[tokio::test]
    async fn turn_info_payload_for_persona_does_not_leak_file_path() {
        // Defensive: even if some future code path accidentally populates
        // file_path on a persona entry, the SSE `turn_info` payload
        // should not surface it to clients. Today file_path is always
        // empty for persona entries, so we assert that contract.
        let entry = Arc::new(TurnEntry::new_persona(
            "tp2".to_string(),
            "default".to_string(),
            1,
        ));
        let info = entry.info().await;
        let json = serde_json::to_string(&serde_json::json!({
            "turn_id": info.turn_id,
            "persona_id": info.persona_id,
            "scope": info.scope,
            "status": info.status.as_str(),
            "total_events_pushed": info.total_events_pushed,
            "buffered_count": info.buffered_count,
        }))
        .unwrap();
        // Privacy: a persona chat reply that quotes an SMS must not echo
        // the file path that message originally surfaced from.
        assert!(
            !json.contains("file_path"),
            "persona turn_info payload should not leak file_path: {json}"
        );
    }
    #[tokio::test]
    async fn insight_scope_defaults_when_using_legacy_constructor() {
        // The legacy `TurnEntry::new` constructor (used by the existing
        // file-anchored insight chat) must still report scope="insight"
        // so callers can discriminate without a parallel map.
        let entry = Arc::new(TurnEntry::new("t-insight".into(), "/p.jpg".into(), 1));
        let info = entry.info().await;
        assert_eq!(info.scope, "insight");
        assert!(info.persona_id.is_none());
    }
 }
@@ -1,796 +0,0 @@
 //! Per-tick drains the watcher runs alongside ingest.
 //!
 //! These passes were previously inlined in `main.rs`; they exist because
 //! a quick scan only walks recently-modified files, so any backlog of
 //! rows missing a `content_hash` / `date_taken` / face detection
 //! wouldn't otherwise drain except during the once-an-hour full scan.
 //! Each function is bounded per call by a `*_PER_TICK` env-var cap.
 use std::collections::HashMap;
 use std::path::PathBuf;
 use std::sync::{Arc, Mutex};
 use log::{debug, info, warn};
 use crate::content_hash;
 use crate::database::ExifDao;
 use crate::date_resolver;
 use crate::face_watch;
 use crate::faces;
 use crate::file_types;
 use crate::libraries;
 use crate::tags;
 /// Compute and persist content_hash for image_exif rows where it's NULL.
 ///
 /// Bounded per call by `FACE_HASH_BACKFILL_MAX_PER_TICK` (default 2000)
 /// so a watcher tick on a large legacy library doesn't block for hours
 /// blake3-ing every photo at once. Subsequent scans pick up the rest.
 /// For 50k+ libraries the dedicated `cargo run --bin backfill_hashes`
 /// is still faster (it doesn't fight a watcher loop for the DAO mutex).
 ///
 /// Drains unhashed image_exif rows by querying them directly, independent
 /// of the filesystem walk. Quick scans only walk recently-modified files,
 /// so a backlog of pre-existing unhashed rows never enters
 /// `process_new_files`'s candidate set — left alone, it would only drain
 /// on full scans (default once an hour). Calling this every tick keeps
 /// the face-detection backlog moving regardless.
 ///
 /// Returns the number of rows successfully backfilled this pass.
 pub fn backfill_unhashed_backlog(
    context: &opentelemetry::Context,
    library: &libraries::Library,
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
 ) -> usize {
    let cap: i64 = dotenv::var("FACE_HASH_BACKFILL_MAX_PER_TICK")
        .ok()
        .and_then(|s| s.parse().ok())
        .filter(|n: &i64| *n > 0)
        .unwrap_or(2000);
    // Fetch up to cap+1 rows so we can tell "more remain" without a
    // separate count query. Across libraries — there's no per-library
    // filter on get_rows_missing_hash today — but we only ever update
    // rows whose library_id matches the caller's library, so other
    // libraries' rows just get skipped here and picked up on the next
    // library's tick. Negligible cost given the cap.
    let rows: Vec<(i32, String)> = {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        dao.get_rows_missing_hash(context, cap + 1)
            .unwrap_or_default()
    };
    if rows.is_empty() {
        return 0;
    }
    let more_than_cap = rows.len() as i64 > cap;
    let base_path = std::path::Path::new(&library.root_path);
    let mut backfilled = 0usize;
    let mut errors = 0usize;
    let mut skipped_other_lib = 0usize;
    for (lib_id, rel_path) in rows.iter().take(cap as usize) {
        if *lib_id != library.id {
            skipped_other_lib += 1;
            continue;
        }
        let abs = base_path.join(rel_path);
        if !abs.exists() {
            // File walked away — the watcher's reconciliation pass will
            // remove the orphan exif row eventually.
            continue;
        }
        match content_hash::compute(&abs) {
            Ok(id) => {
                let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
                if let Err(e) = dao.backfill_content_hash(
                    context,
                    library.id,
                    rel_path,
                    &id.content_hash,
                    id.size_bytes,
                ) {
                    warn!(
                        "face_watch: backfill_content_hash failed for {}: {:?}",
                        rel_path, e
                    );
                    errors += 1;
                } else {
                    backfilled += 1;
                }
            }
            Err(e) => {
                debug!(
                    "face_watch: hash compute failed for {} ({:?})",
                    abs.display(),
                    e
                );
                errors += 1;
            }
        }
    }
    if backfilled > 0 || errors > 0 || more_than_cap {
        info!(
            "face_watch: backfill pass for library '{}': hashed {} ({} error(s), {} skipped to other libraries; {} cap, more_remain={})",
            library.name, backfilled, errors, skipped_other_lib, cap, more_than_cap
        );
    }
    backfilled
 }
 /// Drain image_exif rows whose `date_taken` was never resolved or was
 /// resolved by the weakest fallback (`fs_time`). Runs the canonical-date
 /// waterfall — exiftool batch (one subprocess for the whole tick's
 /// rows) → filename regex → earliest_fs_time — and persists each
 /// resolution with its source tag. Capped per tick by
 /// `DATE_BACKFILL_MAX_PER_TICK` (default 500) so a 14k-row library
 /// drains over a few quick-scan ticks without blocking the watcher.
 ///
 /// kamadak-exif is intentionally skipped here: the row already has a
 /// NULL date_taken because the ingest path's kamadak-exif call returned
 /// nothing, and re-running it would just produce the same answer.
 /// exiftool is the meaningful new attempt — it handles videos and
 /// MakerNote-hosted dates kamadak can't reach.
 pub fn backfill_missing_date_taken(
    context: &opentelemetry::Context,
    library: &libraries::Library,
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
 ) -> usize {
    let cap: i64 = dotenv::var("DATE_BACKFILL_MAX_PER_TICK")
        .ok()
        .and_then(|s| s.parse().ok())
        .filter(|n: &i64| *n > 0)
        .unwrap_or(500);
    let rows: Vec<(i32, String)> = {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        dao.get_rows_needing_date_backfill(context, library.id, cap + 1)
            .unwrap_or_default()
    };
    if rows.is_empty() {
        return 0;
    }
    let more_than_cap = rows.len() as i64 > cap;
    let base_path = std::path::Path::new(&library.root_path);
    // Build absolute paths and drop rows whose files no longer exist —
    // the missing-file scan in library_maintenance retires deleted rows
    // separately. Without this filter, NULL-date rows for missing files
    // would loop through the drain forever (no source can resolve them).
    let mut existing: Vec<(String, PathBuf)> = Vec::with_capacity(rows.len());
    for (_, rel_path) in rows.iter().take(cap as usize) {
        let abs = base_path.join(rel_path);
        if abs.exists() {
            existing.push((rel_path.clone(), abs));
        }
    }
    if existing.is_empty() {
        return 0;
    }
    // One exiftool subprocess for the whole batch; the resolver falls
    // through to filename / fs_time per file when exiftool can't supply
    // a date (or isn't installed at all).
    let paths: Vec<PathBuf> = existing.iter().map(|(_, p)| p.clone()).collect();
    let resolved = date_resolver::resolve_dates_batch(&paths, &HashMap::new());
    let mut backfilled = 0usize;
    let mut unresolved = 0usize;
    let mut by_source: HashMap<&'static str, usize> = HashMap::new();
    {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        for (rel_path, abs) in &existing {
            let Some(rd) = resolved.get(abs).copied() else {
                unresolved += 1;
                continue;
            };
            match dao.backfill_date_taken(
                context,
                library.id,
                rel_path,
                rd.timestamp,
                rd.source.as_str(),
            ) {
                Ok(()) => {
                    backfilled += 1;
                    *by_source.entry(rd.source.as_str()).or_insert(0) += 1;
                }
                Err(e) => {
                    warn!(
                        "date_backfill: update failed for lib {} {}: {:?}",
                        library.id, rel_path, e
                    );
                }
            }
        }
    }
    if backfilled > 0 || unresolved > 0 || more_than_cap {
        info!(
            "date_backfill: library '{}': resolved {} ({:?}), {} unresolved, cap={}, more_remain={}",
            library.name, backfilled, by_source, unresolved, cap, more_than_cap
        );
    }
    backfilled
 }
 /// Per-tick face-detection drain. Pulls a capped batch of hashed-but-
 /// unscanned image_exif rows directly via the FaceDao anti-join and
 /// hands them to the existing detection pass. Runs on every tick (not
 /// just full scans) so the backlog moves at quick-scan cadence.
 /// Per-tick CLIP encoding drain. Mirrors `process_face_backlog`: pull
 /// up to `CLIP_BACKLOG_MAX_PER_TICK` candidates with a known
 /// `content_hash` but no `clip_embedding`, hand them to
 /// `clip_watch::run_clip_encoding_pass` for parallel fan-out, and let
 /// that module write the result back via `backfill_clip_embedding`.
 ///
 /// Idempotent — a row stays in the candidate set until its embedding
 /// lands, so a transient failure (Apollo unreachable, CUDA OOM) just
 /// defers to the next tick. Permanent failures (un-decodable bytes)
 /// retry every tick at this point; future Branch may add a status
 /// column like face_detections has.
 pub fn process_clip_backlog(
    context: &opentelemetry::Context,
    library: &libraries::Library,
    clip_client: &crate::ai::clip_client::ClipClient,
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
    excluded_dirs: &[String],
 ) {
    if !clip_client.is_enabled() {
        return;
    }
    let cap: i64 = dotenv::var("CLIP_BACKLOG_MAX_PER_TICK")
        .ok()
        .and_then(|s| s.parse().ok())
        .filter(|n: &i64| *n > 0)
        .unwrap_or(32);
    let rows: Vec<(String, String)> = {
        let mut dao = exif_dao.lock().expect("exif dao");
        match dao.list_clip_unencoded_candidates(context, library.id, cap) {
            Ok(r) => r,
            Err(e) => {
                warn!(
                    "clip_watch: list_clip_unencoded_candidates failed for library '{}': {:?}",
                    library.name, e
                );
                return;
            }
        }
    };
    if rows.is_empty() {
        return;
    }
    info!(
        "clip_watch: backlog drain — encoding {} candidate(s) for library '{}' (cap={})",
        rows.len(),
        library.name,
        cap
    );
    let candidates: Vec<crate::clip_watch::ClipCandidate> = rows
        .into_iter()
        .map(
            |(rel_path, content_hash)| crate::clip_watch::ClipCandidate {
                rel_path,
                content_hash,
            },
        )
        .collect();
    crate::clip_watch::run_clip_encoding_pass(
        library,
        excluded_dirs,
        clip_client,
        Arc::clone(exif_dao),
        candidates,
    );
 }
 pub fn process_face_backlog(
    context: &opentelemetry::Context,
    library: &libraries::Library,
    face_client: &crate::ai::face_client::FaceClient,
    face_dao: &Arc<Mutex<Box<dyn faces::FaceDao>>>,
    tag_dao: &Arc<Mutex<Box<dyn tags::TagDao>>>,
    excluded_dirs: &[String],
 ) {
    let cap: i64 = dotenv::var("FACE_BACKLOG_MAX_PER_TICK")
        .ok()
        .and_then(|s| s.parse().ok())
        .filter(|n: &i64| *n > 0)
        .unwrap_or(64);
    let rows: Vec<(String, String)> = {
        let mut dao = face_dao.lock().expect("face dao");
        match dao.list_unscanned_candidates(context, library.id, cap) {
            Ok(r) => r,
            Err(e) => {
                warn!(
                    "face_watch: list_unscanned_candidates failed for library '{}': {:?}",
                    library.name, e
                );
                return;
            }
        }
    };
    if rows.is_empty() {
        return;
    }
    info!(
        "face_watch: backlog drain — running detection on {} candidate(s) for library '{}' (cap={})",
        rows.len(),
        library.name,
        cap
    );
    let candidates: Vec<face_watch::FaceCandidate> = rows
        .into_iter()
        .map(|(rel_path, content_hash)| face_watch::FaceCandidate {
            rel_path,
            content_hash,
        })
        .collect();
    face_watch::run_face_detection_pass(
        library,
        excluded_dirs,
        face_client,
        Arc::clone(face_dao),
        Arc::clone(tag_dao),
        candidates,
    );
 }
 /// Compute content_hash for any image rows the walker just touched
 /// whose stored EXIF row is still hash-less. Called from
 /// `process_new_files` so freshly-ingested files don't have to wait for
 /// the next standalone `backfill_unhashed_backlog` tick before face
 /// detection can key on their bytes.
 ///
 /// Cap is on **successes only**. An earlier version counted errors too,
 /// so a pocket of chronically-unhashable files at the front of the
 /// table (vanished mid-scan, permission denied, etc.) burned the budget
 /// every tick and the rest of the backlog never advanced.
 pub fn backfill_missing_content_hashes(
    context: &opentelemetry::Context,
    files: &[(PathBuf, String)],
    library: &libraries::Library,
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
 ) {
    let image_paths: Vec<String> = files
        .iter()
        .filter(|(p, _)| !file_types::is_video_file(p))
        .map(|(_, rel)| rel.clone())
        .collect();
    if image_paths.is_empty() {
        return;
    }
    let exif_records = {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        dao.get_exif_batch(context, Some(library.id), &image_paths)
            .unwrap_or_default()
    };
    // Cheap lookup back from rel_path → absolute file_path so
    // content_hash::compute can read the bytes.
    let path_by_rel: HashMap<String, &PathBuf> =
        files.iter().map(|(p, rel)| (rel.clone(), p)).collect();
    let cap: usize = dotenv::var("FACE_HASH_BACKFILL_MAX_PER_TICK")
        .ok()
        .and_then(|s| s.parse().ok())
        .filter(|n: &usize| *n > 0)
        .unwrap_or(2000);
    // Count the unhashed backlog up front so we can surface "still needs
    // backfill: N" in the log — without it, a face-scan that's stuck at
    // 44% looks stalled when really it's chipping through hashes.
    let unhashed_total = exif_records
        .iter()
        .filter(|r| r.content_hash.is_none())
        .count();
    let mut backfilled = 0usize;
    let mut errors = 0usize;
    for record in &exif_records {
        if backfilled >= cap {
            break;
        }
        if record.content_hash.is_some() {
            continue;
        }
        let Some(file_path) = path_by_rel.get(&record.file_path) else {
            // Walked file went missing between the directory scan and now;
            // next tick will retry naturally.
            continue;
        };
        match content_hash::compute(file_path) {
            Ok(id) => {
                let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
                if let Err(e) = dao.backfill_content_hash(
                    context,
                    library.id,
                    &record.file_path,
                    &id.content_hash,
                    id.size_bytes,
                ) {
                    warn!(
                        "face_watch: backfill_content_hash failed for {}: {:?}",
                        record.file_path, e
                    );
                    errors += 1;
                } else {
                    backfilled += 1;
                }
            }
            Err(e) => {
                debug!(
                    "face_watch: hash compute failed for {} ({:?})",
                    file_path.display(),
                    e
                );
                errors += 1;
            }
        }
    }
    // Always log when there's an unhashed backlog so an operator
    // looking at "scan stuck at 44%" can see backfill is running and
    // how much remains. Quiet only when there's nothing to do.
    if unhashed_total > 0 || backfilled > 0 || errors > 0 {
        let remaining = unhashed_total.saturating_sub(backfilled);
        info!(
            "face_watch: backfilled {}/{} content_hash for library '{}' ({} error(s); {} still need backfill; cap={})",
            backfilled, unhashed_total, library.name, errors, remaining, cap
        );
    }
 }
 /// Build the face-detection candidate list for a scan tick.
 ///
 /// Returns `(rel_path, content_hash)` for every image file that has a
 /// content_hash recorded in image_exif but no row in face_detections
 /// yet. Re-querying image_exif here picks up rows the EXIF write loop
 /// just inserted alongside any pre-existing rows the watcher walked
 /// over — covers both new uploads and the initial backlog scan.
 pub fn build_face_candidates(
    context: &opentelemetry::Context,
    library: &libraries::Library,
    files: &[(PathBuf, String)],
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
    face_dao: &Arc<Mutex<Box<dyn faces::FaceDao>>>,
 ) -> Vec<face_watch::FaceCandidate> {
    // Restrict to image files; videos aren't face-scanned in v1 (kamadak
    // doesn't even register them in image_exif).
    let image_paths: Vec<String> = files
        .iter()
        .filter(|(p, _)| !file_types::is_video_file(p))
        .map(|(_, rel)| rel.clone())
        .collect();
    if image_paths.is_empty() {
        return Vec::new();
    }
    let exif_records = {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        dao.get_exif_batch(context, Some(library.id), &image_paths)
            .unwrap_or_default()
    };
    // rel_path → content_hash (only rows with a hash; without one we have
    // nothing to key face data against).
    let mut hash_by_path: HashMap<String, String> = HashMap::with_capacity(exif_records.len());
    for record in exif_records {
        if let Some(h) = record.content_hash {
            hash_by_path.insert(record.file_path, h);
        }
    }
    let mut candidates = Vec::new();
    let mut dao = face_dao.lock().expect("face dao");
    for rel_path in image_paths {
        let Some(hash) = hash_by_path.get(&rel_path) else {
            continue;
        };
        match dao.already_scanned(context, hash) {
            Ok(true) => continue,
            Ok(false) => candidates.push(face_watch::FaceCandidate {
                rel_path,
                content_hash: hash.clone(),
            }),
            Err(e) => {
                warn!("face_watch: already_scanned errored for {}: {:?}", hash, e);
            }
        }
    }
    candidates
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use std::fs;
    use std::sync::{Arc, Mutex};
    use diesel::prelude::*;
    use tempfile::TempDir;
    use crate::database::models::{InsertImageExif, InsertLibrary};
    use crate::database::test::in_memory_db_connection;
    use crate::database::{ExifDao, SqliteExifDao, schema};
    use crate::faces::{FaceDao, SqliteFaceDao};
    use crate::libraries::Library;
    fn ctx() -> opentelemetry::Context {
        opentelemetry::Context::new()
    }
    /// Everything `setup` hands back to a test: tempdir, library, shared
    /// connection, and the two DAOs. Aliased to keep clippy's
    /// type-complexity lint satisfied.
    type SetupFixture = (
        TempDir,
        Library,
        Arc<Mutex<diesel::SqliteConnection>>,
        Arc<Mutex<Box<dyn ExifDao>>>,
        Arc<Mutex<Box<dyn FaceDao>>>,
    );
    /// Build a tempdir-backed library + DAOs sharing a single in-memory
    /// SQLite connection (so cross-table joins like
    /// `list_unscanned_candidates` see consistent state).
    fn setup() -> SetupFixture {
        let tmp = TempDir::new().expect("tempdir");
        let mut conn = in_memory_db_connection();
        // Migration seeds library id=1 with a placeholder root; rewrite it
        // to point at the tempdir so `<root>/<rel_path>` resolves to real
        // files this test creates.
        diesel::update(schema::libraries::table.filter(schema::libraries::id.eq(1)))
            .set(schema::libraries::root_path.eq(tmp.path().to_string_lossy().to_string()))
            .execute(&mut conn)
            .expect("rewrite library 1 root");
        // Add a second library so cross-library skip cases have somewhere
        // to put their rows.
        diesel::insert_into(schema::libraries::table)
            .values(InsertLibrary {
                name: "other",
                root_path: "/tmp/other-test-lib",
                created_at: 0,
                enabled: true,
                excluded_dirs: None,
            })
            .execute(&mut conn)
            .expect("seed second library");
        let library = Library {
            id: 1,
            name: "main".to_string(),
            root_path: tmp.path().to_string_lossy().to_string(),
            enabled: true,
            excluded_dirs: Vec::new(),
        };
        let shared = Arc::new(Mutex::new(conn));
        let exif_dao: Arc<Mutex<Box<dyn ExifDao>>> = Arc::new(Mutex::new(Box::new(
            SqliteExifDao::from_shared(Arc::clone(&shared)),
        )));
        let face_dao: Arc<Mutex<Box<dyn FaceDao>>> = Arc::new(Mutex::new(Box::new(
            SqliteFaceDao::from_connection(Arc::clone(&shared)),
        )));
        (tmp, library, shared, exif_dao, face_dao)
    }
    fn insert_exif(
        exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
        lib_id: i32,
        rel: &str,
        content_hash: Option<&str>,
    ) {
        let mut dao = exif_dao.lock().unwrap();
        dao.store_exif(
            &ctx(),
            InsertImageExif {
                library_id: lib_id,
                file_path: rel.to_string(),
                camera_make: None,
                camera_model: None,
                lens_model: None,
                width: None,
                height: None,
                orientation: None,
                gps_latitude: None,
                gps_longitude: None,
                gps_altitude: None,
                focal_length: None,
                aperture: None,
                shutter_speed: None,
                iso: None,
                date_taken: None,
                created_time: 0,
                last_modified: 0,
                content_hash: content_hash.map(|s| s.to_string()),
                size_bytes: None,
                phash_64: None,
                dhash_64: None,
                date_taken_source: None,
            },
        )
        .expect("insert");
    }
    fn write_image(root: &std::path::Path, rel: &str, bytes: &[u8]) {
        let abs = root.join(rel);
        if let Some(parent) = abs.parent() {
            fs::create_dir_all(parent).expect("mkdir");
        }
        fs::write(abs, bytes).expect("write file");
    }
    #[test]
    fn backfill_unhashed_backlog_hashes_missing_rows_in_this_library() {
        let (tmp, library, _conn, exif_dao, _face_dao) = setup();
        write_image(tmp.path(), "a.jpg", b"alpha-bytes");
        write_image(tmp.path(), "b.jpg", b"bravo-bytes");
        insert_exif(&exif_dao, 1, "a.jpg", None);
        insert_exif(&exif_dao, 1, "b.jpg", None);
        let backfilled = backfill_unhashed_backlog(&ctx(), &library, &exif_dao);
        assert_eq!(backfilled, 2);
        let mut dao = exif_dao.lock().unwrap();
        let rows = dao
            .get_exif_batch(&ctx(), Some(1), &["a.jpg".to_string(), "b.jpg".to_string()])
            .unwrap();
        assert_eq!(rows.len(), 2);
        for r in rows {
            assert!(
                r.content_hash.is_some(),
                "row {} should have a hash",
                r.file_path
            );
        }
    }
    #[test]
    fn backfill_unhashed_backlog_skips_other_libraries_and_missing_files() {
        let (tmp, library, _conn, exif_dao, _face_dao) = setup();
        write_image(tmp.path(), "exists.jpg", b"hello");
        // Row for this library whose file is missing on disk:
        insert_exif(&exif_dao, 1, "ghost.jpg", None);
        insert_exif(&exif_dao, 1, "exists.jpg", None);
        // Row in the other library — must be skipped (different lib_id).
        insert_exif(&exif_dao, 2, "other.jpg", None);
        let backfilled = backfill_unhashed_backlog(&ctx(), &library, &exif_dao);
        assert_eq!(backfilled, 1, "only the existing in-library file hashes");
        let mut dao = exif_dao.lock().unwrap();
        let other = dao
            .get_exif_batch(&ctx(), Some(2), &["other.jpg".to_string()])
            .unwrap();
        assert_eq!(other.len(), 1);
        assert!(
            other[0].content_hash.is_none(),
            "other-library row must remain unhashed"
        );
        let ghost = dao
            .get_exif_batch(&ctx(), Some(1), &["ghost.jpg".to_string()])
            .unwrap();
        assert_eq!(ghost.len(), 1);
        assert!(
            ghost[0].content_hash.is_none(),
            "missing-on-disk row stays unhashed (reconciliation removes it later)"
        );
    }
    #[test]
    fn backfill_unhashed_backlog_respects_per_tick_cap() {
        // Env-var-driven cap; the function reads it on every call, so we
        // can set it just for this test and unset before returning.
        // Serial guard: tests in the same binary may share env, but each
        // backfill call re-reads — and we only care that the cap shape
        // (success count <= cap, more_remain logged) holds.
        unsafe {
            std::env::set_var("FACE_HASH_BACKFILL_MAX_PER_TICK", "2");
        }
        let (tmp, library, _conn, exif_dao, _face_dao) = setup();
        for i in 0..5 {
            let rel = format!("img_{}.jpg", i);
            write_image(tmp.path(), &rel, format!("bytes-{}", i).as_bytes());
            insert_exif(&exif_dao, 1, &rel, None);
        }
        let backfilled = backfill_unhashed_backlog(&ctx(), &library, &exif_dao);
        assert_eq!(backfilled, 2, "cap=2 must bound the per-tick successes");
        unsafe {
            std::env::remove_var("FACE_HASH_BACKFILL_MAX_PER_TICK");
        }
    }
    #[test]
    fn backfill_missing_content_hashes_skips_videos_and_hashed_rows() {
        let (tmp, library, _conn, exif_dao, _face_dao) = setup();
        // Two image rows (one already hashed, one not), one video.
        write_image(tmp.path(), "fresh.jpg", b"fresh-pixels");
        write_image(tmp.path(), "already.jpg", b"already-pixels");
        write_image(tmp.path(), "clip.mp4", b"video-bytes");
        insert_exif(&exif_dao, 1, "fresh.jpg", None);
        insert_exif(&exif_dao, 1, "already.jpg", Some("pre-existing-hash"));
        insert_exif(&exif_dao, 1, "clip.mp4", None);
        let files: Vec<(PathBuf, String)> = vec![
            (tmp.path().join("fresh.jpg"), "fresh.jpg".to_string()),
            (tmp.path().join("already.jpg"), "already.jpg".to_string()),
            (tmp.path().join("clip.mp4"), "clip.mp4".to_string()),
        ];
        backfill_missing_content_hashes(&ctx(), &files, &library, &exif_dao);
        let mut dao = exif_dao.lock().unwrap();
        let rows = dao
            .get_exif_batch(
                &ctx(),
                Some(1),
                &[
                    "fresh.jpg".to_string(),
                    "already.jpg".to_string(),
                    "clip.mp4".to_string(),
                ],
            )
            .unwrap();
        let by_path: HashMap<String, Option<String>> = rows
            .into_iter()
            .map(|r| (r.file_path, r.content_hash))
            .collect();
        assert!(
            by_path["fresh.jpg"].is_some(),
            "fresh image must get a hash"
        );
        assert_eq!(
            by_path["already.jpg"].as_deref(),
            Some("pre-existing-hash"),
            "already-hashed image left untouched"
        );
        assert!(
            by_path["clip.mp4"].is_none(),
            "video skipped (not face-scanned, no hash needed via this path)"
        );
    }
    #[test]
    fn build_face_candidates_filters_videos_unhashed_and_already_scanned() {
        let (tmp, library, _conn, exif_dao, face_dao) = setup();
        // Seed image_exif with: hashed unscanned, hashed scanned, unhashed,
        // and a video. Files don't need to exist on disk — the function
        // doesn't read them, only the DB rows.
        insert_exif(&exif_dao, 1, "fresh.jpg", Some("hash-fresh"));
        insert_exif(&exif_dao, 1, "scanned.jpg", Some("hash-scanned"));
        insert_exif(&exif_dao, 1, "unhashed.jpg", None);
        insert_exif(&exif_dao, 1, "clip.mp4", Some("hash-video"));
        // Mark `scanned.jpg`'s hash as already detected.
        {
            let mut dao = face_dao.lock().unwrap();
            dao.mark_status(&ctx(), 1, "hash-scanned", "scanned.jpg", "no_faces", "test")
                .expect("mark scanned");
        }
        let files: Vec<(PathBuf, String)> = vec![
            (tmp.path().join("fresh.jpg"), "fresh.jpg".to_string()),
            (tmp.path().join("scanned.jpg"), "scanned.jpg".to_string()),
            (tmp.path().join("unhashed.jpg"), "unhashed.jpg".to_string()),
            (tmp.path().join("clip.mp4"), "clip.mp4".to_string()),
        ];
        let candidates = build_face_candidates(&ctx(), &library, &files, &exif_dao, &face_dao);
        assert_eq!(
            candidates.len(),
            1,
            "exactly fresh.jpg should be a candidate"
        );
        assert_eq!(candidates[0].rel_path, "fresh.jpg");
        assert_eq!(candidates[0].content_hash, "hash-fresh");
    }
 }
@@ -1,7 +1,7 @@
 use anyhow::{Context, Result};
 use chrono::Utc;
 use clap::Parser;
-use image_api::ai::LocalLlm;
+use image_api::ai::ollama::OllamaClient;
 use image_api::bin_progress;
 use image_api::database::calendar_dao::{InsertCalendarEvent, SqliteCalendarEventDao};
 use image_api::parsers::ical_parser::parse_ics_file;
@@ -44,10 +44,22 @@ async fn main() -> Result<()> {
    let context = opentelemetry::Context::current();
-    // LocalLlm dispatches per LLM_BACKEND, so embeddings written here land
+    let ollama = if args.generate_embeddings {
-    // in the same vector space the query side searches.
+        let primary_url = dotenv::var("OLLAMA_PRIMARY_URL")
-    let llm = if args.generate_embeddings {
+            .or_else(|_| dotenv::var("OLLAMA_URL"))
-        Some(LocalLlm::from_env())
+            .unwrap_or_else(|_| "http://localhost:11434".to_string());
        let fallback_url = dotenv::var("OLLAMA_FALLBACK_URL").ok();
        let primary_model = dotenv::var("OLLAMA_PRIMARY_MODEL")
            .or_else(|_| dotenv::var("OLLAMA_MODEL"))
            .unwrap_or_else(|_| "nomic-embed-text:v1.5".to_string());
        let fallback_model = dotenv::var("OLLAMA_FALLBACK_MODEL").ok();
        Some(OllamaClient::new(
            primary_url,
            fallback_url,
            primary_model,
            fallback_model,
        ))
    } else {
        None
    };
@@ -78,7 +90,7 @@ async fn main() -> Result<()> {
        }
        // Generate embedding if requested (blocking call)
-        let embedding = if let Some(ref llm) = llm {
+        let embedding = if let Some(ref ollama_client) = ollama {
            let text = format!(
                "{} {} {}",
                event.summary,
@@ -88,7 +100,7 @@ async fn main() -> Result<()> {
            match tokio::task::block_in_place(|| {
                tokio::runtime::Handle::current()
-                    .block_on(async { llm.embed_document(&text).await })
+                    .block_on(async { ollama_client.generate_embedding(&text).await })
            }) {
                Ok(emb) => Some(emb),
                Err(e) => {
@@ -1,7 +1,7 @@
 use anyhow::{Context, Result};
 use chrono::Utc;
 use clap::Parser;
-use image_api::ai::LocalLlm;
+use image_api::ai::ollama::OllamaClient;
 use image_api::bin_progress;
 use image_api::database::search_dao::{InsertSearchRecord, SqliteSearchHistoryDao};
 use image_api::parsers::search_html_parser::parse_search_html;
@@ -38,9 +38,16 @@ async fn main() -> Result<()> {
    info!("Found {} search records", searches.len());
-    // LocalLlm dispatches per LLM_BACKEND, so embeddings written here land
+    let primary_url = dotenv::var("OLLAMA_PRIMARY_URL")
-    // in the same vector space the query side searches.
+        .or_else(|_| dotenv::var("OLLAMA_URL"))
-    let llm = LocalLlm::from_env();
+        .unwrap_or_else(|_| "http://localhost:11434".to_string());
    let fallback_url = dotenv::var("OLLAMA_FALLBACK_URL").ok();
    let primary_model = dotenv::var("OLLAMA_PRIMARY_MODEL")
        .or_else(|_| dotenv::var("OLLAMA_MODEL"))
        .unwrap_or_else(|_| "nomic-embed-text:v1.5".to_string());
    let fallback_model = dotenv::var("OLLAMA_FALLBACK_MODEL").ok();
    let ollama = OllamaClient::new(primary_url, fallback_url, primary_model, fallback_model);
    let context = opentelemetry::Context::current();
    let mut inserted_count = 0usize;
@@ -60,11 +67,12 @@ async fn main() -> Result<()> {
        let pb_for_warn = pb.clone();
        let embeddings_result = tokio::task::spawn({
-            let llm = llm.clone();
+            let ollama_client = ollama.clone();
            async move {
                // Generate embeddings in parallel for the batch
                let mut embeddings = Vec::new();
                for query in &queries {
-                    match llm.embed_document(query).await {
+                    match ollama_client.generate_embedding(query).await {
                        Ok(emb) => embeddings.push(Some(emb)),
                        Err(e) => {
                            pb_for_warn.println(format!("embedding failed for '{}': {}", query, e));
@@ -14,7 +14,6 @@ use image_api::database::{
    SqliteInsightDao, SqliteKnowledgeDao, SqliteLocationHistoryDao, SqliteSearchHistoryDao,
    connect,
 };
 use image_api::faces::{FaceDao, SqliteFaceDao};
 use image_api::file_types::{IMAGE_EXTENSIONS, VIDEO_EXTENSIONS};
 use image_api::libraries::{self, Library};
 use image_api::tags::{SqliteTagDao, TagDao};
@@ -183,11 +182,6 @@ async fn main() -> anyhow::Result<()> {
        Arc::new(Mutex::new(Box::new(SqliteTagDao::default())));
    let knowledge_dao: Arc<Mutex<Box<dyn KnowledgeDao>>> =
        Arc::new(Mutex::new(Box::new(SqliteKnowledgeDao::new())));
    let face_dao: Arc<Mutex<Box<dyn FaceDao>>> =
        Arc::new(Mutex::new(Box::new(SqliteFaceDao::new())));
    let persona_dao: Arc<Mutex<Box<dyn image_api::database::PersonaDao>>> = Arc::new(Mutex::new(
        Box::new(image_api::database::SqlitePersonaDao::new()),
    ));
    // Pass the full library set so `resolve_full_path` probes every root,
    // even when --library restricts the walk. A rel_path shared across
@@ -195,7 +189,6 @@ async fn main() -> anyhow::Result<()> {
    let generator = InsightGenerator::new(
        ollama,
        None,
        None,
        sms_client,
        apollo_client,
        insight_dao.clone(),
@@ -205,9 +198,7 @@ async fn main() -> anyhow::Result<()> {
        location_dao,
        search_dao,
        tag_dao,
        face_dao,
        knowledge_dao,
        persona_dao,
        all_libs.clone(),
    );
@@ -336,13 +327,11 @@ async fn main() -> anyhow::Result<()> {
                args.top_p,
                args.top_k,
                args.min_p,
                None, // enable_thinking: leave model/template default
                args.max_iterations,
                None,
                Vec::new(),
                Vec::new(),
-                1, // operator user_id — populate_knowledge is single-user offline tool
+                false, // disable_writes — keep KB writes on for the population job
                "default".to_string(),
            )
            .await
        {
@@ -1,273 +0,0 @@
 //! Probe binary for CLIP semantic search.
 //!
 //! No DB writes. Walks a library's `image_exif` rows, encodes a sample
 //! via Apollo's `/encode_image`, encodes the user's --query via
 //! `/encode_text`, and prints the top-K most similar photos by cosine
 //! similarity so the operator can eyeball quality before committing to
 //! the persistence phase (column populated by backlog drain, search
 //! endpoint, UI).
 //!
 //! Usage:
 //!     cargo run --release --bin probe_clip_search -- \
 //!         --library 1 --limit 200 --query "a beach at sunset" --top 10
 //!
 //! Env: standard ImageApi `.env`. Requires either
 //! `APOLLO_CLIP_API_BASE_URL` or `APOLLO_API_BASE_URL` to be set.
 use std::path::{Path, PathBuf};
 use std::sync::{Arc, Mutex};
 use std::time::Instant;
 use clap::Parser;
 use log::{info, warn};
 use image_api::ai::clip_client::{ClipClient, ClipError, EncodeImageMeta};
 use image_api::database::{ExifDao, SqliteExifDao, connect};
 use image_api::exif;
 use image_api::file_types;
 use image_api::libraries::{self, Library};
 #[derive(Parser, Debug)]
 #[command(name = "probe_clip_search")]
 #[command(about = "Top-K CLIP semantic search over a sample of image_exif rows")]
 struct Args {
    /// Library id to sample from.
    #[arg(long)]
    library: i32,
    /// Max files to encode. CPU inference is slow (~1-3 s per photo at
    /// ViT-L/14); start small and grow once GPU is sorted.
    #[arg(long, default_value_t = 50)]
    limit: usize,
    /// Natural-language query. Empty triggers an error from Apollo.
    #[arg(long)]
    query: String,
    /// How many top results to print.
    #[arg(long, default_value_t = 10)]
    top: usize,
    /// Offset into the library's rel_path listing.
    #[arg(long, default_value_t = 0)]
    offset: i64,
    /// How many DB rows to scan before giving up on hitting the limit.
    #[arg(long, default_value_t = 5000)]
    max_scan: i64,
 }
 /// Same as `face_watch::read_image_bytes_for_detect` (which is pub(crate)).
 /// Inlined for the throwaway probe.
 fn read_image_bytes(path: &Path) -> std::io::Result<Vec<u8>> {
    if file_types::needs_ffmpeg_thumbnail(path)
        && let Some(preview) = exif::extract_embedded_jpeg_preview(path)
    {
        return Ok(preview);
    }
    std::fs::read(path)
 }
 /// Decode a base64'd LE float32 vector to a `Vec<f32>`.
 fn decode_f32_vec(b64: &str) -> anyhow::Result<Vec<f32>> {
    use base64::Engine;
    let bytes = base64::engine::general_purpose::STANDARD.decode(b64.as_bytes())?;
    if bytes.len() % 4 != 0 {
        anyhow::bail!("embedding byte length {} not divisible by 4", bytes.len());
    }
    let mut out = Vec::with_capacity(bytes.len() / 4);
    for chunk in bytes.chunks_exact(4) {
        out.push(f32::from_le_bytes([chunk[0], chunk[1], chunk[2], chunk[3]]));
    }
    Ok(out)
 }
 /// Plain dot product. Apollo L2-normalizes both sides, so this is cosine sim.
 fn dot(a: &[f32], b: &[f32]) -> f32 {
    a.iter().zip(b.iter()).map(|(x, y)| x * y).sum()
 }
 #[tokio::main]
 async fn main() -> anyhow::Result<()> {
    env_logger::init();
    dotenv::dotenv().ok();
    let args = Args::parse();
    if args.query.trim().is_empty() {
        anyhow::bail!("--query must not be empty");
    }
    let client = ClipClient::from_env();
    if !client.is_enabled() {
        anyhow::bail!(
            "ClipClient disabled: set APOLLO_CLIP_API_BASE_URL or APOLLO_API_BASE_URL in .env"
        );
    }
    match client.health().await {
        Ok(h) => info!(
            "clip engine: loaded={} device={} model={} dim={}",
            h.loaded, h.device, h.model_version, h.embedding_dim
        ),
        Err(e) => warn!("health probe failed (continuing): {e}"),
    }
    let mut seed_conn = connect();
    if let Some(base) = dotenv::var("BASE_PATH").ok().as_deref() {
        libraries::seed_or_patch_from_env(&mut seed_conn, base);
    }
    let libs = libraries::load_all(&mut seed_conn);
    drop(seed_conn);
    let lib: Library = libs
        .into_iter()
        .find(|l| l.id == args.library)
        .ok_or_else(|| anyhow::anyhow!("library id {} not found", args.library))?;
    info!(
        "probing library #{} ({}) at {}",
        lib.id, lib.name, lib.root_path
    );
    let dao: Arc<Mutex<Box<dyn ExifDao>>> = Arc::new(Mutex::new(Box::new(SqliteExifDao::new())));
    let ctx = opentelemetry::Context::new();
    // Encode the query up-front so the long image-encode loop doesn't
    // race a slow query encode. Fails fast on a misspelled query.
    let query_resp = client
        .encode_text(&args.query)
        .await
        .map_err(|e| anyhow::anyhow!("encode_text: {e}"))?;
    let query_vec = decode_f32_vec(&query_resp.embedding)?;
    info!(
        "query encoded ({}d, {}ms): {:?}",
        query_resp.embedding_dim, query_resp.duration_ms, args.query
    );
    // Page through (id, rel_path), filter to images on disk, encode up
    // to `limit`. Each encoded photo gets scored against the query and
    // kept in a top-K heap.
    const PAGE: i64 = 500;
    let mut offset = args.offset;
    let mut scanned: i64 = 0;
    let mut encoded = 0usize;
    let mut perm_fail = 0usize;
    let mut transient_fail = 0usize;
    let root = PathBuf::from(&lib.root_path);
    let started = Instant::now();
    // (similarity, rel_path) — we keep all scored results and sort at
    // the end. With limit≤few-hundred this is trivial.
    let mut scores: Vec<(f32, String)> = Vec::with_capacity(args.limit);
    'outer: loop {
        if scanned >= args.max_scan {
            warn!(
                "scan cap ({}) reached before hitting limit ({}); bump --max-scan to scan deeper",
                args.max_scan, args.limit
            );
            break;
        }
        let rows = {
            let mut guard = dao.lock().expect("dao lock");
            guard
                .list_rel_paths_for_library_page(&ctx, lib.id, PAGE, offset)
                .map_err(|e| anyhow::anyhow!("list rel_paths: {:?}", e))?
        };
        if rows.is_empty() {
            info!("no more rows after offset {}", offset);
            break;
        }
        offset += rows.len() as i64;
        scanned += rows.len() as i64;
        for (_id, rel_path) in rows {
            if encoded >= args.limit {
                break 'outer;
            }
            let abs = root.join(&rel_path);
            if !file_types::is_image_file(&abs) || !abs.exists() {
                continue;
            }
            let bytes = match read_image_bytes(&abs) {
                Ok(b) => b,
                Err(e) => {
                    warn!("read {rel_path}: {e}");
                    continue;
                }
            };
            let meta = EncodeImageMeta {
                content_hash: String::new(),
                library_id: lib.id,
                rel_path: rel_path.clone(),
            };
            let call_start = Instant::now();
            match client.encode_image(bytes, meta).await {
                Ok(resp) => {
                    encoded += 1;
                    let vec = match decode_f32_vec(&resp.embedding) {
                        Ok(v) => v,
                        Err(e) => {
                            warn!("decode {rel_path}: {e}");
                            continue;
                        }
                    };
                    if vec.len() != query_vec.len() {
                        warn!(
                            "dim mismatch for {rel_path}: image={} query={}",
                            vec.len(),
                            query_vec.len()
                        );
                        continue;
                    }
                    let sim = dot(&vec, &query_vec);
                    scores.push((sim, rel_path.clone()));
                    if encoded.is_multiple_of(10) {
                        info!(
                            "progress: {} encoded, {:.1}s elapsed",
                            encoded,
                            started.elapsed().as_secs_f32()
                        );
                    }
                    let _ = call_start;
                }
                Err(ClipError::Permanent(e)) => {
                    perm_fail += 1;
                    warn!("permanent encode failure for {rel_path}: {e}");
                }
                Err(ClipError::Transient(e)) => {
                    transient_fail += 1;
                    warn!("transient encode failure for {rel_path}: {e}");
                }
                Err(ClipError::Disabled) => {
                    anyhow::bail!("clip client became disabled mid-run; impossible");
                }
            }
        }
    }
    scores.sort_by(|a, b| b.0.partial_cmp(&a.0).unwrap_or(std::cmp::Ordering::Equal));
    let elapsed = started.elapsed();
    println!();
    println!(
        "── top {} for query: {:?} ──",
        args.top.min(scores.len()),
        args.query
    );
    for (i, (sim, path)) in scores.iter().take(args.top).enumerate() {
        println!("[{:>2}] sim={:.3}  {}", i + 1, sim, path);
    }
    println!();
    println!("── summary ─────────────────────────────────────");
    println!("query                : {:?}", args.query);
    println!("scanned rows         : {scanned}");
    println!("encoded photos       : {encoded}");
    println!("permanent failures   : {perm_fail}");
    println!("transient failures   : {transient_fail}");
    println!("elapsed              : {:.1}s", elapsed.as_secs_f32());
    if encoded > 0 {
        println!(
            "throughput           : {:.2} photos/s ({:.0}ms/photo avg)",
            encoded as f32 / elapsed.as_secs_f32().max(0.001),
            elapsed.as_millis() as f32 / encoded as f32
        );
    }
    Ok(())
 }
@@ -1,465 +0,0 @@
 //! Re-embed stored corpora through `LocalLlm`, i.e. the same
 //! `LLM_BACKEND` dispatch the query side uses. The original import /
 //! backfill tools always embedded via Ollama, so a deploy running
 //! `LLM_BACKEND=llamacpp` queries vector spaces the corpora may not live
 //! in. Three tables share the problem and are all covered here:
 //!
 //! - `daily_conversation_summaries` — re-embeds
 //!   `strip_summary_boilerplate(summary)` (what the original job fed the
 //!   embedder); also rewrites `model_version`.
 //! - `calendar_events` — re-embeds "summary description location" exactly
 //!   as `import_calendar` does; rows without an embedding are skipped (the
 //!   import only embeds under `--generate-embeddings`).
 //! - `search_history` — re-embeds the raw query text.
 //! - `entities` (knowledge graph) — re-embeds "name description" exactly as
 //!   `tool_store_entity` does; embedding-less rows are skipped (embedding
 //!   is best-effort at store time).
 //!
 //! Source text is untouched — only vectors are rewritten. The old↔new
 //! cosine report doubles as a diagnostic: ~1.0 means both backends already
 //! shared a space (re-embedding was a no-op); low values confirm the
 //! mismatch this tool exists to fix.
 use anyhow::{Context, Result};
 use clap::Parser;
 use diesel::prelude::*;
 use diesel::sql_query;
 use diesel::sqlite::SqliteConnection;
 use image_api::ai::{LocalLlm, strip_summary_boilerplate};
 use image_api::bin_progress;
 use std::env;
 #[derive(Parser, Debug)]
 #[command(author, version, about = "Re-embed stored corpora via the configured LLM_BACKEND", long_about = None)]
 struct Args {
    /// Comma-separated tables to process: summaries, calendar, search, entities
    #[arg(long, default_value = "summaries,calendar,search,entities")]
    tables: String,
    /// Only process the first N rows per table (smoke test)
    #[arg(long)]
    limit: Option<usize>,
    /// Compute embeddings and report old↔new similarity without writing
    #[arg(long, default_value_t = false)]
    dry_run: bool,
 }
 #[derive(QueryableByName)]
 struct SummaryRow {
    #[diesel(sql_type = diesel::sql_types::Integer)]
    id: i32,
    #[diesel(sql_type = diesel::sql_types::Text)]
    summary: String,
    #[diesel(sql_type = diesel::sql_types::Binary)]
    embedding: Vec<u8>,
    #[diesel(sql_type = diesel::sql_types::Text)]
    model_version: String,
 }
 #[derive(QueryableByName)]
 struct CalendarRow {
    #[diesel(sql_type = diesel::sql_types::Integer)]
    id: i32,
    #[diesel(sql_type = diesel::sql_types::Text)]
    summary: String,
    #[diesel(sql_type = diesel::sql_types::Nullable<diesel::sql_types::Text>)]
    description: Option<String>,
    #[diesel(sql_type = diesel::sql_types::Nullable<diesel::sql_types::Text>)]
    location: Option<String>,
    #[diesel(sql_type = diesel::sql_types::Binary)]
    embedding: Vec<u8>,
 }
 #[derive(QueryableByName)]
 struct SearchRow {
    #[diesel(sql_type = diesel::sql_types::BigInt)]
    id: i64,
    #[diesel(sql_type = diesel::sql_types::Text)]
    query: String,
    #[diesel(sql_type = diesel::sql_types::Binary)]
    embedding: Vec<u8>,
 }
 #[derive(QueryableByName)]
 struct EntityRow {
    #[diesel(sql_type = diesel::sql_types::Integer)]
    id: i32,
    #[diesel(sql_type = diesel::sql_types::Text)]
    name: String,
    #[diesel(sql_type = diesel::sql_types::Text)]
    description: String,
    #[diesel(sql_type = diesel::sql_types::Binary)]
    embedding: Vec<u8>,
 }
 /// One unit of re-embed work, normalized across tables.
 struct WorkItem {
    /// Row key, as i64 so both i32 ids and rowids fit.
    id: i64,
    /// Text fed to the embedder — must match what the original writer used.
    text: String,
    /// Existing vector bytes, for the old↔new similarity report.
    old_embedding: Vec<u8>,
 }
 fn deserialize_vector(bytes: &[u8]) -> Option<Vec<f32>> {
    if !bytes.len().is_multiple_of(4) {
        return None;
    }
    Some(
        bytes
            .chunks_exact(4)
            .map(|c| f32::from_le_bytes([c[0], c[1], c[2], c[3]]))
            .collect(),
    )
 }
 fn serialize_vector(vec: &[f32]) -> Vec<u8> {
    vec.iter().flat_map(|f| f.to_le_bytes()).collect()
 }
 fn cosine_similarity(a: &[f32], b: &[f32]) -> f32 {
    if a.len() != b.len() {
        return 0.0;
    }
    let dot: f32 = a.iter().zip(b).map(|(x, y)| x * y).sum();
    let mag_a: f32 = a.iter().map(|x| x * x).sum::<f32>().sqrt();
    let mag_b: f32 = b.iter().map(|x| x * x).sum::<f32>().sqrt();
    if mag_a == 0.0 || mag_b == 0.0 {
        return 0.0;
    }
    dot / (mag_a * mag_b)
 }
 /// Embed `text`, halving it on "input too large" errors until it fits the
 /// server's physical batch (`--ubatch-size`). Mirrors the silent truncation
 /// Ollama applied when these corpora were first embedded — llama-server
 /// returns a 500 instead — except here it's surfaced via the returned flag.
 /// Returns `(embedding, truncated)`.
 async fn embed_with_truncation(llm: &LocalLlm, text: &str) -> Result<(Vec<f32>, bool)> {
    let mut text = text.to_string();
    let mut truncated = false;
    loop {
        match llm.embed_document(&text).await {
            Ok(emb) => return Ok((emb, truncated)),
            Err(e)
                if e.to_string().contains("too large to process") && text.chars().count() > 64 =>
            {
                let keep = text.chars().count() / 2;
                text = text.chars().take(keep).collect();
                truncated = true;
            }
            Err(e) => return Err(e),
        }
    }
 }
 /// Re-embed `items`, writing each new vector via `update`. Returns the
 /// old↔new cosines for the similarity report.
 async fn reembed_table(
    conn: &mut SqliteConnection,
    llm: &LocalLlm,
    label: &str,
    items: Vec<WorkItem>,
    dry_run: bool,
    update: impl Fn(&mut SqliteConnection, i64, Vec<u8>) -> Result<()>,
 ) -> Result<Vec<f32>> {
    println!("\n[{}] re-embedding {} rows...", label, items.len());
    let pb = bin_progress::determinate(items.len() as u64, format!("re-embedding {}", label));
    let mut sims: Vec<f32> = Vec::with_capacity(items.len());
    let mut updated = 0usize;
    let mut failed = 0usize;
    let mut truncated_count = 0usize;
    for item in &items {
        let new_emb = match embed_with_truncation(llm, &item.text).await {
            Ok((e, truncated)) => {
                if truncated {
                    truncated_count += 1;
                    pb.println(format!(
                        "⚠ {} id={}: input exceeded the embed server's batch size, \
                         truncated before embedding",
                        label, item.id
                    ));
                }
                e
            }
            Err(e) => {
                pb.inc(1);
                failed += 1;
                eprintln!("✗ {} id={}: {}", label, item.id, e);
                continue;
            }
        };
        // The whole pipeline (DAO checks, stored corpora) assumes
        // EMBEDDING_DIM dims. A mismatch means the active embed slot is not
        // serving the configured model — stop rather than corrupt the table.
        anyhow::ensure!(
            new_emb.len() == image_api::ai::embedding_dim(),
            "backend returned {}-dim embedding (expected {}) — '{}' does not \
             match the configured EMBEDDING_DIM",
            new_emb.len(),
            image_api::ai::embedding_dim(),
            llm.embedding_model_version()
        );
        if let Some(old_emb) = deserialize_vector(&item.old_embedding) {
            sims.push(cosine_similarity(&old_emb, &new_emb));
        }
        if !dry_run {
            update(conn, item.id, serialize_vector(&new_emb))
                .with_context(|| format!("updating {} id={}", label, item.id))?;
        }
        updated += 1;
        pb.inc(1);
    }
    pb.finish_and_clear();
    println!(
        "[{}] {} re-embedded ({} truncated), {} failed",
        label, updated, truncated_count, failed
    );
    Ok(sims)
 }
 fn report_similarity(label: &str, mut sims: Vec<f32>) {
    if sims.is_empty() {
        println!("[{}] no old↔new pairs to compare", label);
        return;
    }
    sims.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
    let mean: f32 = sims.iter().sum::<f32>() / sims.len() as f32;
    let median = sims[sims.len() / 2];
    println!(
        "[{}] old↔new cosine over identical text: min={:.3} median={:.3} mean={:.3} max={:.3}",
        label,
        sims.first().unwrap(),
        median,
        mean,
        sims.last().unwrap()
    );
    if median > 0.98 {
        println!(
            "[{}] → old and new backends agree (~same vector space); poor search \
             results are coming from something else (prefixes, thresholds, corpus).",
            label
        );
    } else if median > 0.9 {
        println!(
            "[{}] → same model family but measurably different vectors \
             (quantization / runtime drift); re-embedding was worthwhile.",
            label
        );
    } else {
        println!(
            "[{}] → vector-space mismatch confirmed — queries were searching a \
             different space than the corpus. This re-embed should fix it.",
            label
        );
    }
 }
 #[tokio::main]
 async fn main() -> Result<()> {
    dotenv::dotenv().ok();
    env_logger::init();
    let args = Args::parse();
    let tables: Vec<&str> = args.tables.split(',').map(|t| t.trim()).collect();
    for t in &tables {
        anyhow::ensure!(
            matches!(*t, "summaries" | "calendar" | "search" | "entities"),
            "unknown table '{}' — expected summaries, calendar, search, entities",
            t
        );
    }
    let database_url = env::var("DATABASE_URL").unwrap_or_else(|_| "auth.db".to_string());
    println!("Database: {}", database_url);
    let mut conn = SqliteConnection::establish(&database_url)
        .with_context(|| format!("connecting to {}", database_url))?;
    let llm = LocalLlm::from_env();
    let model_version = llm.embedding_model_version();
    println!("Embedding via '{}'", model_version);
    if args.dry_run {
        println!("DRY RUN — no rows will be written");
    }
    if tables.contains(&"summaries") {
        let mut rows: Vec<SummaryRow> = sql_query(
            "SELECT id, summary, embedding, model_version
             FROM daily_conversation_summaries ORDER BY date",
        )
        .load(&mut conn)
        .context("loading daily summaries")?;
        if let Some(limit) = args.limit {
            rows.truncate(limit);
        }
        if let Some(first) = rows.first() {
            println!(
                "\n[summaries] previous model_version '{}' → '{}'",
                first.model_version, model_version
            );
        }
        let items = rows
            .into_iter()
            .map(|r| WorkItem {
                id: r.id as i64,
                text: strip_summary_boilerplate(&r.summary),
                old_embedding: r.embedding,
            })
            .collect();
        let mv = model_version.clone();
        let sims = reembed_table(
            &mut conn,
            &llm,
            "summaries",
            items,
            args.dry_run,
            move |conn, id, emb| {
                sql_query(
                    "UPDATE daily_conversation_summaries
                     SET embedding = ?1, model_version = ?2 WHERE id = ?3",
                )
                .bind::<diesel::sql_types::Binary, _>(emb)
                .bind::<diesel::sql_types::Text, _>(&mv)
                .bind::<diesel::sql_types::Integer, _>(id as i32)
                .execute(conn)?;
                Ok(())
            },
        )
        .await?;
        report_similarity("summaries", sims);
    }
    if tables.contains(&"calendar") {
        let mut rows: Vec<CalendarRow> = sql_query(
            "SELECT id, summary, description, location, embedding
             FROM calendar_events WHERE embedding IS NOT NULL ORDER BY id",
        )
        .load(&mut conn)
        .context("loading calendar events")?;
        if let Some(limit) = args.limit {
            rows.truncate(limit);
        }
        let items = rows
            .into_iter()
            .map(|r| WorkItem {
                id: r.id as i64,
                // Same text construction as import_calendar.
                text: format!(
                    "{} {} {}",
                    r.summary,
                    r.description.as_deref().unwrap_or(""),
                    r.location.as_deref().unwrap_or("")
                ),
                old_embedding: r.embedding,
            })
            .collect();
        let sims = reembed_table(
            &mut conn,
            &llm,
            "calendar",
            items,
            args.dry_run,
            |conn, id, emb| {
                sql_query("UPDATE calendar_events SET embedding = ?1 WHERE id = ?2")
                    .bind::<diesel::sql_types::Binary, _>(emb)
                    .bind::<diesel::sql_types::Integer, _>(id as i32)
                    .execute(conn)?;
                Ok(())
            },
        )
        .await?;
        report_similarity("calendar", sims);
    }
    if tables.contains(&"search") {
        let mut rows: Vec<SearchRow> = sql_query(
            "SELECT rowid AS id, query, embedding
             FROM search_history ORDER BY rowid",
        )
        .load(&mut conn)
        .context("loading search history")?;
        if let Some(limit) = args.limit {
            rows.truncate(limit);
        }
        let items = rows
            .into_iter()
            .map(|r| WorkItem {
                id: r.id,
                text: r.query,
                old_embedding: r.embedding,
            })
            .collect();
        let sims = reembed_table(
            &mut conn,
            &llm,
            "search",
            items,
            args.dry_run,
            |conn, id, emb| {
                sql_query("UPDATE search_history SET embedding = ?1 WHERE rowid = ?2")
                    .bind::<diesel::sql_types::Binary, _>(emb)
                    .bind::<diesel::sql_types::BigInt, _>(id)
                    .execute(conn)?;
                Ok(())
            },
        )
        .await?;
        report_similarity("search", sims);
    }
    if tables.contains(&"entities") {
        let mut rows: Vec<EntityRow> = sql_query(
            "SELECT id, name, description, embedding
             FROM entities WHERE embedding IS NOT NULL ORDER BY id",
        )
        .load(&mut conn)
        .context("loading knowledge entities")?;
        if let Some(limit) = args.limit {
            rows.truncate(limit);
        }
        let items = rows
            .into_iter()
            .map(|r| WorkItem {
                id: r.id as i64,
                // Same text construction as tool_store_entity.
                text: format!("{} {}", r.name, r.description),
                old_embedding: r.embedding,
            })
            .collect();
        let sims = reembed_table(
            &mut conn,
            &llm,
            "entities",
            items,
            args.dry_run,
            |conn, id, emb| {
                sql_query("UPDATE entities SET embedding = ?1 WHERE id = ?2")
                    .bind::<diesel::sql_types::Binary, _>(emb)
                    .bind::<diesel::sql_types::Integer, _>(id as i32)
                    .execute(conn)?;
                Ok(())
            },
        )
        .await?;
        report_similarity("entities", sims);
    }
    println!(
        "\n{}",
        if args.dry_run {
            "Dry run complete"
        } else {
            "Done"
        }
    );
    Ok(())
 }
@@ -1,382 +0,0 @@
 //! `/photos/search?q=<text>` — CLIP semantic photo search.
 //!
 //! The route lives outside `files.rs` to keep that 1500+ line module
 //! focused on EXIF / tag listing. The flow is:
 //!
 //! 1. Parse query params (`q`, `limit`, `threshold`, optional `library`).
 //! 2. Call Apollo's `/api/internal/clip/encode_text` to get the query
 //!    vector (L2-normalized 768-d f32 for ViT-L/14).
 //! 3. Load every `(content_hash, clip_embedding)` for the scope from
 //!    `image_exif` via `ExifDao::list_clip_index`. ~28–43 MB for a 14k
 //!    library at ViT-L/14; loaded fresh per request — fast enough for
 //!    v1, optimize via an AppState cache later if needed.
 //! 4. Dot product (= cosine since both sides are L2-normalized), filter
 //!    above `threshold`, top-K by score.
 //! 5. Resolve each surviving hash back to a `(library_id, rel_path)` so
 //!    the frontend can render the photo / hand off to the carousel.
 //!
 //! Response shape is intentionally minimal — paths + score — so the
 //! frontend can reuse existing PhotoGrid rendering by joining against
 //! `/api/photos/match` (or calling `/image/metadata` lazily). Don't
 //! bake camera/EXIF metadata into this route; it would force a fan-out
 //! per result and balloon the response.
 use crate::AppState;
 use crate::ai::clip_client::ClipError;
 use crate::database::ExifDao;
 use actix_web::{HttpResponse, Result as ActixResult, web};
 use base64::Engine;
 use serde::{Deserialize, Serialize};
 use std::sync::Mutex;
 #[derive(Debug, Deserialize)]
 pub struct SearchQuery {
    /// Natural-language query. Required; empty triggers 400.
    pub q: String,
    /// Max results to return in this page. Capped to 200 server-side.
    /// Defaults to 20. Pair with `offset` for pagination.
    #[serde(default = "default_limit")]
    pub limit: usize,
    /// Zero-based offset into the sorted-and-filtered result set. The
    /// scoring loop still runs over the full embedding matrix on every
    /// page (cheap at personal-library scale — sub-100ms — and avoids
    /// stateful pagination cursors). Defaults to 0.
    #[serde(default)]
    pub offset: usize,
    /// Cosine-similarity floor below which results are dropped.
    /// 0.20 is the rough "this is plausibly relevant" line for OpenAI
    /// CLIP; tunable per call when sweeping. Defaults to 0.20.
    #[serde(default = "default_threshold")]
    pub threshold: f32,
    /// Optional single-library scope. Legacy param — new clients pass
    /// `library_ids` instead so multi-select scopes (Apollo's HUD library
    /// chips, FileViewer-React's library picker) actually filter. Kept
    /// for back-compat; `library_ids` wins when both are supplied.
    pub library: Option<i32>,
    /// Optional multi-library scope, comma-separated id list
    /// (`?library_ids=1,3`). Empty / omitted = every enabled library
    /// (the historical default). Apollo and FileViewer-React both send
    /// this when 2+ libraries are selected; the single-library case
    /// works through either param interchangeably.
    pub library_ids: Option<String>,
    /// Optional model-version filter. Defaults to the live engine's
    /// version (queried lazily). Forces a strict join so mid-flight
    /// model swaps can't mix geometries in a single response.
    #[serde(default)]
    pub model_version: Option<String>,
 }
 fn default_limit() -> usize {
    20
 }
 fn default_threshold() -> f32 {
    0.20
 }
 #[derive(Debug, Serialize)]
 pub struct SearchHit {
    pub library_id: i32,
    pub rel_path: String,
    pub content_hash: String,
    /// Cosine similarity in [-1, 1]. In practice OpenAI CLIP returns
    /// 0.10–0.40 for the typical photo library.
    pub score: f32,
 }
 #[derive(Debug, Serialize)]
 pub struct SearchResponse {
    pub query: String,
    pub model_version: String,
    pub threshold: f32,
    /// Total embeddings scored (= every photo in scope with a stored
    /// embedding). Same value across pages of the same query.
    pub considered: usize,
    /// Count of results above threshold, before pagination. Lets the
    /// client decide whether a "Load more" button is meaningful and
    /// stop fetching when ``offset + results.len() >= total_matching``.
    pub total_matching: usize,
    pub offset: usize,
    pub results: Vec<SearchHit>,
 }
 #[derive(Debug, Serialize)]
 struct SearchError {
    error: String,
 }
 /// Decode a stored `clip_embedding` BLOB back into a `Vec<f32>`. Returns
 /// `None` on malformed bytes — those rows get skipped rather than
 /// failing the whole query.
 fn decode_embedding(bytes: &[u8]) -> Option<Vec<f32>> {
    if bytes.is_empty() || !bytes.len().is_multiple_of(4) {
        return None;
    }
    let mut out = Vec::with_capacity(bytes.len() / 4);
    for chunk in bytes.chunks_exact(4) {
        out.push(f32::from_le_bytes([chunk[0], chunk[1], chunk[2], chunk[3]]));
    }
    Some(out)
 }
 #[inline]
 fn dot(a: &[f32], b: &[f32]) -> f32 {
    a.iter().zip(b.iter()).map(|(x, y)| x * y).sum()
 }
 /// Failure modes of [`score_photos`]. Carries enough to let each caller pick
 /// an appropriate HTTP status (the CLIP service being down is a 502, a
 /// disabled feature is a 503, a rejected query is a 400, a DB failure 500).
 pub enum ScoreError {
    /// CLIP search isn't configured at all (no Apollo endpoint).
    Disabled,
    /// The query was rejected by the encoder (client error).
    Rejected(String),
    /// The CLIP service is transiently unavailable (upstream error).
    Unavailable(String),
    /// The encoder returned an embedding we couldn't decode.
    MalformedEmbedding,
    /// A database / index load failure.
    Internal(String),
 }
 /// Result of scoring the whole library against a query embedding: the
 /// resolved model version, how many embeddings were considered, and every
 /// `(score, content_hash)` above threshold, sorted by descending score.
 /// Pagination and path resolution are the caller's job (see [`resolve_hits`])
 /// so this core can be reused for both the plain search endpoint and the
 /// unified endpoint (which filters by hash before paginating).
 pub struct ScoredPhotos {
    pub model_version: String,
    pub considered: usize,
    /// `(cosine_score, content_hash)` pairs, descending by score.
    pub hits: Vec<(f32, String)>,
 }
 /// Encode `q_text` via CLIP and score it against every stored embedding in
 /// the given library scope. Returns all matches above `threshold`, sorted by
 /// descending similarity. Pure of HTTP concerns so it's shared by
 /// `search_photos` and the unified search endpoint.
 pub async fn score_photos(
    state: &AppState,
    exif_dao: &Mutex<Box<dyn ExifDao>>,
    q_text: &str,
    library_ids: &[i32],
    threshold: f32,
    model_version: Option<&str>,
 ) -> Result<ScoredPhotos, ScoreError> {
    if !state.clip_client.is_enabled() {
        return Err(ScoreError::Disabled);
    }
    // 1. Encode the query text. Fast — Apollo's text encoder is ~50ms on CPU.
    let query_resp = match state.clip_client.encode_text(q_text).await {
        Ok(r) => r,
        Err(ClipError::Permanent(e)) => return Err(ScoreError::Rejected(e.to_string())),
        Err(ClipError::Transient(e)) => return Err(ScoreError::Unavailable(e.to_string())),
        Err(ClipError::Disabled) => return Err(ScoreError::Disabled),
    };
    // decode_embedding works on raw bytes; the wire format is b64.
    let query_bytes = base64::engine::general_purpose::STANDARD
        .decode(query_resp.embedding.as_bytes())
        .unwrap_or_default();
    let query_vec = decode_embedding(&query_bytes).ok_or(ScoreError::MalformedEmbedding)?;
    // 2. Pull the (hash, embedding) matrix under the dao lock, release
    // before scoring. The caller-supplied `model_version` (or the live
    // engine's) forces a strict join so a mid-flight model swap can't mix
    // geometries.
    let ctx = opentelemetry::Context::current();
    let rows: Vec<(String, Vec<u8>)> = {
        let mut dao = exif_dao.lock().expect("exif dao");
        dao.list_clip_index(
            &ctx,
            library_ids,
            model_version.or(Some(&query_resp.model_version)),
        )
        .map_err(|e| {
            log::warn!("clip_search: list_clip_index failed: {:?}", e);
            ScoreError::Internal("failed to load search index".into())
        })?
    };
    let considered = rows.len();
    // 3. Score. Keep all matches and sort at the end (~microseconds at 14k).
    let mut hits: Vec<(f32, String)> = Vec::with_capacity(considered);
    for (hash, blob) in rows {
        let Some(emb) = decode_embedding(&blob) else {
            continue;
        };
        if emb.len() != query_vec.len() {
            continue;
        }
        let sim = dot(&emb, &query_vec);
        if sim < threshold {
            continue;
        }
        hits.push((sim, hash));
    }
    hits.sort_by(|a, b| b.0.partial_cmp(&a.0).unwrap_or(std::cmp::Ordering::Equal));
    Ok(ScoredPhotos {
        model_version: query_resp.model_version,
        considered,
        hits,
    })
 }
 /// Resolve a page of `(score, content_hash)` pairs back to [`SearchHit`]s
 /// (each carrying `library_id` + `rel_path`). Hashes that no longer resolve
 /// to a row are skipped. Shared by both endpoints.
 pub fn resolve_hits(
    exif_dao: &Mutex<Box<dyn ExifDao>>,
    scored: &[(f32, String)],
 ) -> Vec<SearchHit> {
    if scored.is_empty() {
        return Vec::new();
    }
    let ctx = opentelemetry::Context::current();
    let hashes: Vec<String> = scored.iter().map(|(_, h)| h.clone()).collect();
    let mut dao = exif_dao.lock().expect("exif dao");
    let path_map = dao
        .get_rel_paths_for_hashes(&ctx, &hashes)
        .unwrap_or_else(|e| {
            log::warn!("clip_search: get_rel_paths_for_hashes failed: {:?}", e);
            std::collections::HashMap::new()
        });
    let mut results = Vec::with_capacity(scored.len());
    for (score, hash) in scored {
        let row = match dao.find_by_content_hash(&ctx, hash) {
            Ok(Some(r)) => r,
            Ok(None) => continue,
            Err(e) => {
                log::warn!("clip_search: find_by_content_hash failed for {hash}: {e:?}");
                continue;
            }
        };
        // Prefer get_rel_paths_for_hashes's first entry (shares image_exif's
        // natural order), falling back to the ImageExif row.
        let rel_path = path_map
            .get(hash)
            .and_then(|paths| paths.first().cloned())
            .unwrap_or(row.file_path);
        results.push(SearchHit {
            library_id: row.library_id,
            rel_path,
            content_hash: hash.clone(),
            score: *score,
        });
    }
    results
 }
 /// Parse the `library_ids` (multi) / `library` (single) scope params into a
 /// deduped id list. Empty = "every enabled library". Shared so the unified
 /// endpoint scopes CLIP identically.
 pub fn parse_library_scope(
    library_ids: Option<&str>,
    library: Option<i32>,
 ) -> Result<Vec<i32>, String> {
    if let Some(raw) = library_ids {
        let mut out: Vec<i32> = Vec::new();
        for piece in raw.split(',') {
            let trimmed = piece.trim();
            if trimmed.is_empty() {
                continue;
            }
            match trimmed.parse::<i32>() {
                Ok(id) => {
                    if !out.contains(&id) {
                        out.push(id);
                    }
                }
                Err(_) => return Err(format!("invalid library_ids entry: {trimmed:?}")),
            }
        }
        Ok(out)
    } else if let Some(id) = library {
        Ok(vec![id])
    } else {
        Ok(Vec::new())
    }
 }
 pub async fn search_photos(
    state: web::Data<AppState>,
    exif_dao: web::Data<Mutex<Box<dyn ExifDao>>>,
    query: web::Query<SearchQuery>,
 ) -> ActixResult<HttpResponse> {
    let q_text = query.q.trim().to_string();
    if q_text.is_empty() {
        return Ok(HttpResponse::BadRequest().json(SearchError {
            error: "query parameter `q` is required".into(),
        }));
    }
    let limit = query.limit.clamp(1, 200);
    let offset = query.offset;
    let threshold = query.threshold.clamp(-1.0, 1.0);
    let library_ids = match parse_library_scope(query.library_ids.as_deref(), query.library) {
        Ok(ids) => ids,
        Err(msg) => return Ok(HttpResponse::BadRequest().json(SearchError { error: msg })),
    };
    let scored = match score_photos(
        &state,
        &exif_dao,
        &q_text,
        &library_ids,
        threshold,
        query.model_version.as_deref(),
    )
    .await
    {
        Ok(s) => s,
        Err(e) => return Ok(score_error_response(e)),
    };
    let total_matching = scored.hits.len();
    // Pagination — slice the sorted list at `[offset, offset+limit)`. Offsets
    // past the end produce empty pages so "load more" stops naturally.
    let page: Vec<(f32, String)> = if offset >= total_matching {
        Vec::new()
    } else {
        let end = (offset + limit).min(total_matching);
        scored.hits[offset..end].to_vec()
    };
    let results = resolve_hits(&exif_dao, &page);
    Ok(HttpResponse::Ok().json(SearchResponse {
        query: q_text,
        model_version: scored.model_version,
        threshold,
        considered: scored.considered,
        total_matching,
        offset,
        results,
    }))
 }
 /// Map a [`ScoreError`] to the HTTP response `search_photos` historically
 /// returned for each failure mode. Reused by the unified endpoint.
 pub fn score_error_response(e: ScoreError) -> HttpResponse {
    match e {
        ScoreError::Disabled => HttpResponse::ServiceUnavailable().json(SearchError {
            error: "CLIP search is disabled (no Apollo CLIP endpoint configured)".into(),
        }),
        ScoreError::Rejected(msg) => HttpResponse::BadRequest().json(SearchError {
            error: format!("query rejected: {msg}"),
        }),
        ScoreError::Unavailable(msg) => HttpResponse::BadGateway().json(SearchError {
            error: format!("CLIP service unavailable: {msg}"),
        }),
        ScoreError::MalformedEmbedding => HttpResponse::BadGateway().json(SearchError {
            error: "CLIP service returned a malformed query embedding".into(),
        }),
        ScoreError::Internal(msg) => {
            HttpResponse::InternalServerError().json(SearchError { error: msg })
        }
    }
 }
@@ -1,246 +0,0 @@
 //! CLIP-encoding pass for the file watcher.
 //!
 //! `process_clip_backlog` in `backfill.rs` calls [`run_clip_encoding_pass`]
 //! with the page of candidates returned by
 //! `ExifDao::list_clip_unencoded_candidates`. We walk those, fan out K
 //! parallel encode calls to Apollo, and persist the resulting embeddings
 //! into `image_exif.clip_embedding` / `clip_model_version`.
 //!
 //! Unlike the face pipeline, CLIP has no marker rows — a permanent
 //! failure (un-decodable bytes) leaves the row's `clip_embedding` NULL
 //! and the drain will retry on the next tick. For personal-library
 //! scale this is fine; the per-tick cap bounds the wasted work, and
 //! `file_types::is_image_file` filters out videos / non-media client-
 //! side so most permanent failures are decoded-but-corrupt files (rare).
 //!
 //! The watcher thread isn't in any pre-existing async context, so we
 //! build a short-lived tokio runtime per pass and `block_on` the join
 //! of K encode futures. Concurrency knob: `CLIP_ENCODE_CONCURRENCY`
 //! (default 4 — lower than faces because Apollo's CLIP path doesn't
 //! release the GIL between preprocess and forward as cleanly).
 use crate::ai::clip_client::{ClipClient, ClipError, EncodeImageMeta};
 use crate::database::ExifDao;
 use crate::exif;
 use crate::file_types;
 use crate::libraries::Library;
 use crate::memories::PathExcluder;
 use log::{debug, info, warn};
 use std::path::Path;
 use std::sync::{Arc, Mutex};
 use tokio::sync::Semaphore;
 /// One file the watcher would like to CLIP-encode. Built from the DAO
 /// `list_clip_unencoded_candidates` result — needs the `content_hash`
 /// for traceability in Apollo's log lines, even though the embedding
 /// itself is keyed on `(library_id, rel_path)` for the back-write.
 #[derive(Debug, Clone)]
 pub struct ClipCandidate {
    pub rel_path: String,
    pub content_hash: String,
 }
 /// Synchronous entry point. Returns once every candidate has been
 /// processed (or definitively skipped). No-op when the client is
 /// disabled so the caller can call unconditionally.
 pub fn run_clip_encoding_pass(
    library: &Library,
    excluded_dirs: &[String],
    clip_client: &ClipClient,
    exif_dao: Arc<Mutex<Box<dyn ExifDao>>>,
    candidates: Vec<ClipCandidate>,
 ) {
    if !clip_client.is_enabled() {
        return;
    }
    if candidates.is_empty() {
        return;
    }
    let base = Path::new(&library.root_path);
    let filtered = filter_excluded(base, excluded_dirs, candidates, Some(&library.name));
    if filtered.is_empty() {
        return;
    }
    let concurrency: usize = std::env::var("CLIP_ENCODE_CONCURRENCY")
        .ok()
        .and_then(|s| s.parse().ok())
        .filter(|n: &usize| *n > 0)
        .unwrap_or(4);
    info!(
        "clip_watch: encoding {} candidate(s) for library '{}' (concurrency {})",
        filtered.len(),
        library.name,
        concurrency
    );
    let rt = match tokio::runtime::Builder::new_multi_thread()
        .worker_threads(2)
        .enable_all()
        .build()
    {
        Ok(rt) => rt,
        Err(e) => {
            warn!("clip_watch: failed to build tokio runtime: {e}");
            return;
        }
    };
    let library_id = library.id;
    let library_root = library.root_path.clone();
    rt.block_on(async move {
        let sem = Arc::new(Semaphore::new(concurrency));
        let mut handles = Vec::with_capacity(filtered.len());
        for cand in filtered {
            let permit_sem = sem.clone();
            let clip_client = clip_client.clone();
            let exif_dao = exif_dao.clone();
            let library_root = library_root.clone();
            handles.push(tokio::spawn(async move {
                let _permit = permit_sem.acquire().await.expect("clip semaphore");
                process_one(library_id, &library_root, cand, &clip_client, exif_dao).await;
            }));
        }
        for h in handles {
            let _ = h.await;
        }
    });
 }
 async fn process_one(
    library_id: i32,
    library_root: &str,
    cand: ClipCandidate,
    clip_client: &ClipClient,
    exif_dao: Arc<Mutex<Box<dyn ExifDao>>>,
 ) {
    let abs = Path::new(library_root).join(&cand.rel_path);
    let bytes = match read_image_bytes_for_encode(&abs) {
        Ok(b) => b,
        Err(e) => {
            // Same rationale as face_watch: don't mark — the file may
            // have been moved/renamed mid-scan; let the next pass retry.
            warn!(
                "clip_watch: read failed for {} (lib {}): {}",
                cand.rel_path, library_id, e
            );
            return;
        }
    };
    let meta = EncodeImageMeta {
        content_hash: cand.content_hash.clone(),
        library_id,
        rel_path: cand.rel_path.clone(),
    };
    let ctx = opentelemetry::Context::current();
    match clip_client.encode_image(bytes, meta).await {
        Ok(resp) => {
            let emb_bytes = match resp.decode_embedding() {
                Ok(b) => b,
                Err(e) => {
                    warn!("clip_watch: bad embedding for {}: {:?}", cand.rel_path, e);
                    return;
                }
            };
            let mut dao = exif_dao.lock().expect("exif dao");
            if let Err(e) = dao.backfill_clip_embedding(
                &ctx,
                library_id,
                &cand.rel_path,
                &emb_bytes,
                &resp.model_version,
            ) {
                warn!(
                    "clip_watch: backfill_clip_embedding failed for {}: {:?}",
                    cand.rel_path, e
                );
                return;
            }
            debug!(
                "clip_watch: {} → dim={} ({}ms, {})",
                cand.rel_path, resp.embedding_dim, resp.duration_ms, resp.model_version
            );
        }
        Err(ClipError::Permanent(e)) => {
            // No marker — the row sits with NULL embedding and the drain
            // retries next pass. For personal-library scale the cost of
            // re-attempting permanently-broken files is bounded by the
            // per-tick cap. If this becomes a recurring noise source,
            // add a `clip_status` column with `failed` semantics like
            // face_detections has.
            warn!(
                "clip_watch: permanent failure on {} (will retry next pass): {}",
                cand.rel_path, e
            );
        }
        Err(ClipError::Transient(e)) => {
            debug!(
                "clip_watch: transient on {}: {} (will retry next pass)",
                cand.rel_path, e
            );
        }
        Err(ClipError::Disabled) => {
            // Defensive — the entry-point already checked is_enabled().
        }
    }
 }
 /// Drop candidates whose paths land in an excluded dir or whose
 /// extension isn't an image. Mirrors `face_watch::filter_excluded` so
 /// the two backlogs stay shape-consistent. Library name is passed
 /// purely for the log line that surfaces an exclusion hit.
 pub fn filter_excluded(
    base: &Path,
    excluded_dirs: &[String],
    candidates: Vec<ClipCandidate>,
    library_name: Option<&str>,
 ) -> Vec<ClipCandidate> {
    let excluder = if excluded_dirs.is_empty() {
        None
    } else {
        Some(PathExcluder::new(base, excluded_dirs))
    };
    candidates
        .into_iter()
        .filter(|c| {
            let abs = base.join(&c.rel_path);
            if !file_types::is_image_file(&abs) {
                debug!(
                    "clip_watch: skipping non-image '{}' (lib {})",
                    c.rel_path,
                    library_name.unwrap_or("<unknown>")
                );
                return false;
            }
            if let Some(ex) = excluder.as_ref()
                && ex.is_excluded(&abs)
            {
                debug!(
                    "clip_watch: skipping excluded '{}' (lib {})",
                    c.rel_path,
                    library_name.unwrap_or("<unknown>")
                );
                return false;
            }
            true
        })
        .collect()
 }
 /// Read image bytes for CLIP encoding. Same logic as
 /// `face_watch::read_image_bytes_for_detect` — RAW / HEIC files don't
 /// decode in Apollo's PIL pipeline, so we pull the embedded JPEG
 /// preview the thumbnail pipeline already extracts. Plain JPEG / PNG /
 /// WebP go through a direct read.
 pub fn read_image_bytes_for_encode(path: &Path) -> std::io::Result<Vec<u8>> {
    if file_types::needs_ffmpeg_thumbnail(path)
        && let Some(preview) = exif::extract_embedded_jpeg_preview(path)
    {
        return Ok(preview);
    }
    std::fs::read(path)
 }
@@ -50,32 +50,14 @@ pub fn thumbnail_path(thumbs_dir: &Path, hash: &str) -> PathBuf {
    thumbs_dir.join(shard).join(format!("{}.jpg", hash))
 }
 /// Hash-keyed large-preview path: `<thumbs_dir>/_large/<hash[..2]>/<hash>.jpg`.
 /// Kept under the same root as 200px thumbs so deployments don't need a
 /// second env var, but namespaced under `_large/` so the existing 200px
 /// shards don't collide with the larger derivative.
 pub fn large_preview_path(thumbs_dir: &Path, hash: &str) -> PathBuf {
    let shard = shard_prefix(hash);
    thumbs_dir
        .join("_large")
        .join(shard)
        .join(format!("{}.jpg", hash))
 }
 /// Hash-keyed xlarge-preview path: `<thumbs_dir>/_xlarge/<hash[..2]>/<hash>.jpg`.
 pub fn xlarge_preview_path(thumbs_dir: &Path, hash: &str) -> PathBuf {
    let shard = shard_prefix(hash);
    thumbs_dir
        .join("_xlarge")
        .join(shard)
        .join(format!("{}.jpg", hash))
 }
 /// Hash-keyed HLS output directory: `<video_dir>/<hash[..2]>/<hash>/`.
 /// The playlist lives at `playlist.m3u8` inside this directory and its
-/// segments are co-located so HLS relative references Just Work. See
+/// segments are co-located so HLS relative references Just Work.
-/// [`crate::video::hls_paths`] for the filename constants and the
+///
-/// per-file helpers built on this dir.
+/// Allow-dead until Branch B/C rewires the HLS pipeline to use it; the
 /// helper lives here today so Branch A's path layout decisions stay
 /// adjacent to thumbnail/legacy ones.
 #[allow(dead_code)]
 pub fn hls_dir(video_dir: &Path, hash: &str) -> PathBuf {
    let shard = shard_prefix(hash);
    video_dir.join(shard).join(hash)
@@ -141,9 +123,6 @@ mod tests {
        let p = thumbnail_path(thumbs, "abcdef0123");
        assert_eq!(p, PathBuf::from("/tmp/thumbs/ab/abcdef0123.jpg"));
        let l = large_preview_path(thumbs, "abcdef0123");
        assert_eq!(l, PathBuf::from("/tmp/thumbs/_large/ab/abcdef0123.jpg"));
        let video = Path::new("/tmp/video");
        let d = hls_dir(video, "1234deadbeef");
        assert_eq!(d, PathBuf::from("/tmp/video/12/1234deadbeef"));
@@ -194,8 +194,6 @@ pub enum MediaType {
 #[serde(rename_all = "lowercase")]
 pub enum PhotoSize {
    Full,
    XLarge,
    Large,
    Thumb,
 }
@@ -222,12 +222,11 @@ impl CalendarEventDao for SqliteCalendarEventDao {
            // Validate embedding dimensions if provided
            if let Some(ref emb) = event.embedding
-                && emb.len() != crate::ai::embedding_dim()
+                && emb.len() != 768
            {
                return Err(anyhow::anyhow!(
-                    "Invalid embedding dimensions: {} (expected {})",
+                    "Invalid embedding dimensions: {} (expected 768)",
-                    emb.len(),
+                    emb.len()
                    crate::ai::embedding_dim()
                ));
            }
@@ -275,7 +274,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
                source_file: event.source_file,
            })
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn store_events_batch(
@@ -294,7 +293,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
                for event in events {
                    // Validate embedding if provided
                    if let Some(ref emb) = event.embedding
-                        && emb.len() != crate::ai::embedding_dim()
+                        && emb.len() != 768
                    {
                        log::warn!(
                            "Skipping event with invalid embedding dimensions: {}",
@@ -349,7 +348,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
            Ok(inserted)
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn find_events_in_range(
@@ -374,7 +373,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
            .map(|rows| rows.into_iter().map(|r| r.to_calendar_event()).collect())
            .map_err(|e| anyhow::anyhow!("Query error: {:?}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_similar_events(
@@ -386,11 +385,10 @@ impl CalendarEventDao for SqliteCalendarEventDao {
        trace_db_call(context, "query", "find_similar_events", |_span| {
            let mut conn = self.connection.lock().expect("Unable to get CalendarEventDao");
-            if query_embedding.len() != crate::ai::embedding_dim() {
+            if query_embedding.len() != 768 {
                return Err(anyhow::anyhow!(
-                    "Invalid query embedding dimensions: {} (expected {})",
+                    "Invalid query embedding dimensions: {} (expected 768)",
-                    query_embedding.len(),
+                    query_embedding.len()
                    crate::ai::embedding_dim()
                ));
            }
@@ -431,7 +429,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
            Ok(scored_events.into_iter().take(limit).map(|(_, event)| event).collect())
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_relevant_events_hybrid(
@@ -463,11 +461,10 @@ impl CalendarEventDao for SqliteCalendarEventDao {
            // Step 2: If query embedding provided, rank by semantic similarity
            if let Some(query_emb) = query_embedding {
-                if query_emb.len() != crate::ai::embedding_dim() {
+                if query_emb.len() != 768 {
                    return Err(anyhow::anyhow!(
-                        "Invalid query embedding dimensions: {} (expected {})",
+                        "Invalid query embedding dimensions: {} (expected 768)",
-                        query_emb.len(),
+                        query_emb.len()
                        crate::ai::embedding_dim()
                    ));
                }
@@ -503,7 +500,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
                Ok(events_in_range.into_iter().take(limit).map(|r| r.to_calendar_event()).collect())
            }
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn event_exists(
@@ -531,7 +528,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
            Ok(result.count > 0)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_event_count(&mut self, context: &opentelemetry::Context) -> Result<i64, DbError> {
@@ -554,6 +551,6 @@ impl CalendarEventDao for SqliteCalendarEventDao {
            Ok(result.count)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
 }
@@ -75,11 +75,6 @@ pub trait DailySummaryDao: Sync + Send {
        context: &opentelemetry::Context,
        contact: &str,
    ) -> Result<i64, DbError>;
    /// Cheap presence check — returns true iff at least one daily summary row
    /// exists. Used by gating logic that only needs "is the table empty?",
    /// avoiding a `COUNT(*)` full scan on large corpora.
    fn has_any_summaries(&mut self, context: &opentelemetry::Context) -> Result<bool, DbError>;
 }
 pub struct SqliteDailySummaryDao {
@@ -150,11 +145,10 @@ impl DailySummaryDao for SqliteDailySummaryDao {
                .expect("Unable to get DailySummaryDao");
            // Validate embedding dimensions
-            if summary.embedding.len() != crate::ai::embedding_dim() {
+            if summary.embedding.len() != 768 {
                return Err(anyhow::anyhow!(
-                    "Invalid embedding dimensions: {} (expected {})",
+                    "Invalid embedding dimensions: {} (expected 768)",
-                    summary.embedding.len(),
+                    summary.embedding.len()
                    crate::ai::embedding_dim()
                ));
            }
@@ -191,7 +185,7 @@ impl DailySummaryDao for SqliteDailySummaryDao {
                model_version: summary.model_version,
            })
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn find_similar_summaries(
@@ -203,11 +197,10 @@ impl DailySummaryDao for SqliteDailySummaryDao {
        trace_db_call(context, "query", "find_similar_summaries", |_span| {
            let mut conn = self.connection.lock().expect("Unable to get DailySummaryDao");
-            if query_embedding.len() != crate::ai::embedding_dim() {
+            if query_embedding.len() != 768 {
                return Err(anyhow::anyhow!(
-                    "Invalid query embedding dimensions: {} (expected {})",
+                    "Invalid query embedding dimensions: {} (expected 768)",
-                    query_embedding.len(),
+                    query_embedding.len()
                    crate::ai::embedding_dim()
                ));
            }
@@ -288,7 +281,7 @@ impl DailySummaryDao for SqliteDailySummaryDao {
            Ok(top_results)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_similar_summaries_with_time_weight(
@@ -301,11 +294,10 @@ impl DailySummaryDao for SqliteDailySummaryDao {
        trace_db_call(context, "query", "find_similar_summaries_with_time_weight", |_span| {
            let mut conn = self.connection.lock().expect("Unable to get DailySummaryDao");
-            if query_embedding.len() != crate::ai::embedding_dim() {
+            if query_embedding.len() != 768 {
                return Err(anyhow::anyhow!(
-                    "Invalid query embedding dimensions: {} (expected {})",
+                    "Invalid query embedding dimensions: {} (expected 768)",
-                    query_embedding.len(),
+                    query_embedding.len()
                    crate::ai::embedding_dim()
                ));
            }
@@ -411,7 +403,7 @@ impl DailySummaryDao for SqliteDailySummaryDao {
            Ok(top_results)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn summary_exists(
@@ -438,7 +430,7 @@ impl DailySummaryDao for SqliteDailySummaryDao {
            Ok(count > 0)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_summary_count(
@@ -460,31 +452,7 @@ impl DailySummaryDao for SqliteDailySummaryDao {
            .map(|r| r.count)
            .map_err(|e| anyhow::anyhow!("Count query error: {:?}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn has_any_summaries(&mut self, context: &opentelemetry::Context) -> Result<bool, DbError> {
        trace_db_call(context, "query", "has_any_summaries", |_span| {
            let mut conn = self
                .connection
                .lock()
                .expect("Unable to get DailySummaryDao");
            #[derive(QueryableByName)]
            struct ProbeResult {
                #[diesel(sql_type = diesel::sql_types::Integer)]
                #[allow(dead_code)]
                one: i32,
            }
            let rows: Vec<ProbeResult> =
                diesel::sql_query("SELECT 1 as one FROM daily_conversation_summaries LIMIT 1")
                    .load(conn.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Failed to probe daily summaries: {}", e))?;
            Ok(!rows.is_empty())
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
 }
@@ -1,681 +0,0 @@
 use diesel::prelude::*;
 use diesel::sqlite::SqliteConnection;
 use std::ops::DerefMut;
 use std::sync::{Arc, Mutex};
 use crate::database::models::{
    InsertInsightGenerationJob, InsightGenerationJob, InsightGenerationType, InsightJobStatus,
 };
 use crate::database::schema;
 use crate::database::{DbError, DbErrorKind, connect};
 use crate::otel::trace_db_call;
 /// Tracks async insight generation jobs. Each call to `create_job` inserts
 /// a new row; the application layer prevents concurrent running jobs by
 /// cancelling the old one before creating a new one.
 pub trait InsightGenerationJobDao: Sync + Send {
    /// Insert a new running job. Always creates a new row (no upsert).
    /// Cleans up terminal-state rows for the same key first.
    fn create_job(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
        generation_type: InsightGenerationType,
    ) -> Result<i32, DbError>;
    /// Mark a job as completed with the resulting insight id. Only updates
    /// if the job is still in "running" status (prevents overwriting a
    /// cancelled job with a late-completing task).
    fn complete_job(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
        insight_id: i32,
    ) -> Result<(), DbError>;
    /// Mark a job as failed with an error message. Only updates if the job
    /// is still in "running" status.
    fn fail_job(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
        error_message: &str,
    ) -> Result<(), DbError>;
    /// Cancel a specific job by id. Only updates if the job is still
    /// in "running" status. Returns true if a row was updated.
    fn cancel_job(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
    ) -> Result<bool, DbError>;
    /// Cancel all running jobs for a given file. Returns the number of
    /// jobs cancelled.
    fn cancel_active_jobs(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
    ) -> Result<usize, DbError>;
    /// Find the latest running job for a given file. Returns None if no
    /// running job exists.
    fn get_active_job(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
    ) -> Result<Option<InsightGenerationJob>, DbError>;
    /// Find any job by id regardless of status.
    fn get_job_by_id(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
    ) -> Result<Option<InsightGenerationJob>, DbError>;
    /// Mark all jobs still in "running" status as "failed" with a recovery
    /// error message. Returns the number of jobs recovered.
    fn recover_orphaned_jobs(&mut self, context: &opentelemetry::Context)
    -> Result<usize, DbError>;
 }
 pub struct SqliteInsightGenerationJobDao {
    connection: Arc<Mutex<SqliteConnection>>,
 }
 impl Default for SqliteInsightGenerationJobDao {
    fn default() -> Self {
        Self::new()
    }
 }
 impl SqliteInsightGenerationJobDao {
    pub fn new() -> Self {
        Self {
            connection: Arc::new(Mutex::new(connect())),
        }
    }
    #[cfg(test)]
    pub fn from_connection(conn: Arc<Mutex<SqliteConnection>>) -> Self {
        Self { connection: conn }
    }
 }
 impl InsightGenerationJobDao for SqliteInsightGenerationJobDao {
    fn create_job(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
        generation_type: InsightGenerationType,
    ) -> Result<i32, DbError> {
        trace_db_call(context, "insert", "create_job", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            let now = std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .expect("Time went backwards")
                .as_secs() as i64;
            let new_job = InsertInsightGenerationJob {
                library_id,
                path: file_path.to_string(),
                gen_type: generation_type.to_string(),
                status: InsightJobStatus::Running.to_string(),
                started_at: now,
            };
            diesel::insert_into(dsl::insight_generation_jobs)
                .values(&new_job)
                .execute(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Failed to insert job: {}", e))?;
            dsl::insight_generation_jobs
                .filter(
                    dsl::library_id
                        .eq(library_id)
                        .and(dsl::file_path.eq(file_path))
                        .and(dsl::generation_type.eq(generation_type.as_str()))
                        .and(dsl::status.eq(InsightJobStatus::Running.as_str())),
                )
                .select(dsl::id)
                .order(dsl::id.desc())
                .first::<i32>(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Failed to get job id: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn complete_job(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
        insight_id: i32,
    ) -> Result<(), DbError> {
        trace_db_call(context, "update", "complete_job", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            let now = std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .expect("Time went backwards")
                .as_secs() as i64;
            // Only update if still running — prevents cancelled job from
            // being overwritten by a late-completing task.
            diesel::update(
                dsl::insight_generation_jobs.filter(
                    dsl::id
                        .eq(job_id)
                        .and(dsl::status.eq(InsightJobStatus::Running.as_str())),
                ),
            )
            .set((
                dsl::status.eq(InsightJobStatus::Completed.as_str()),
                dsl::completed_at.eq(Some(now)),
                dsl::result_insight_id.eq(Some(insight_id)),
            ))
            .execute(connection.deref_mut())
            .map(|_| ())
            .map_err(|e| anyhow::anyhow!("Failed to complete job: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn fail_job(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
        error_message: &str,
    ) -> Result<(), DbError> {
        trace_db_call(context, "update", "fail_job", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            let now = std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .expect("Time went backwards")
                .as_secs() as i64;
            // Only update if still running.
            diesel::update(
                dsl::insight_generation_jobs.filter(
                    dsl::id
                        .eq(job_id)
                        .and(dsl::status.eq(InsightJobStatus::Running.as_str())),
                ),
            )
            .set((
                dsl::status.eq(InsightJobStatus::Failed.as_str()),
                dsl::completed_at.eq(Some(now)),
                dsl::error_message.eq(Some(error_message.to_string())),
            ))
            .execute(connection.deref_mut())
            .map(|_| ())
            .map_err(|e| anyhow::anyhow!("Failed to fail job: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn cancel_job(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
    ) -> Result<bool, DbError> {
        trace_db_call(context, "update", "cancel_job", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            let now = std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .expect("Time went backwards")
                .as_secs() as i64;
            let rows = diesel::update(
                dsl::insight_generation_jobs.filter(
                    dsl::id
                        .eq(job_id)
                        .and(dsl::status.eq(InsightJobStatus::Running.as_str())),
                ),
            )
            .set((
                dsl::status.eq(InsightJobStatus::Cancelled.as_str()),
                dsl::completed_at.eq(Some(now)),
                dsl::error_message.eq(Some("cancelled by user".to_string())),
            ))
            .execute(connection.deref_mut())
            .map_err(|e| anyhow::anyhow!("Failed to cancel job: {}", e))?;
            Ok(rows > 0)
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn cancel_active_jobs(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
    ) -> Result<usize, DbError> {
        trace_db_call(context, "update", "cancel_active_jobs", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            let now = std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .expect("Time went backwards")
                .as_secs() as i64;
            let rows = diesel::update(
                dsl::insight_generation_jobs.filter(
                    dsl::library_id
                        .eq(library_id)
                        .and(dsl::file_path.eq(file_path))
                        .and(dsl::status.eq(InsightJobStatus::Running.as_str())),
                ),
            )
            .set((
                dsl::status.eq(InsightJobStatus::Cancelled.as_str()),
                dsl::completed_at.eq(Some(now)),
                dsl::error_message.eq(Some("cancelled by newer request".to_string())),
            ))
            .execute(connection.deref_mut())
            .map_err(|e| anyhow::anyhow!("Failed to cancel active jobs: {}", e))?;
            Ok(rows)
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn get_active_job(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
    ) -> Result<Option<InsightGenerationJob>, DbError> {
        trace_db_call(context, "query", "get_active_job", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            dsl::insight_generation_jobs
                .filter(
                    dsl::library_id
                        .eq(library_id)
                        .and(dsl::file_path.eq(file_path))
                        .and(dsl::status.eq(InsightJobStatus::Running.as_str())),
                )
                .order(dsl::id.desc())
                .first::<InsightGenerationJob>(connection.deref_mut())
                .optional()
                .map_err(|e| anyhow::anyhow!("Failed to get active job: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn get_job_by_id(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
    ) -> Result<Option<InsightGenerationJob>, DbError> {
        trace_db_call(context, "query", "get_job_by_id", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            dsl::insight_generation_jobs
                .filter(dsl::id.eq(job_id))
                .first::<InsightGenerationJob>(connection.deref_mut())
                .optional()
                .map_err(|e| anyhow::anyhow!("Failed to get job: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn recover_orphaned_jobs(
        &mut self,
        context: &opentelemetry::Context,
    ) -> Result<usize, DbError> {
        trace_db_call(context, "update", "recover_orphaned_jobs", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            let now = std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .expect("Time went backwards")
                .as_secs() as i64;
            let rows = diesel::update(
                dsl::insight_generation_jobs
                    .filter(dsl::status.eq(InsightJobStatus::Running.as_str())),
            )
            .set((
                dsl::status.eq(InsightJobStatus::Failed.as_str()),
                dsl::completed_at.eq(Some(now)),
                dsl::error_message.eq(Some("server crashed while running".to_string())),
            ))
            .execute(connection.deref_mut())
            .map_err(|e| anyhow::anyhow!("Failed to recover orphaned jobs: {}", e))?;
            Ok(rows)
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use diesel::Connection;
    use diesel_migrations::{EmbeddedMigrations, MigrationHarness, embed_migrations};
    const DB_MIGRATIONS: EmbeddedMigrations = embed_migrations!();
    fn setup_dao() -> SqliteInsightGenerationJobDao {
        let mut conn = SqliteConnection::establish(":memory:")
            .expect("Unable to create in-memory db connection");
        conn.run_pending_migrations(DB_MIGRATIONS)
            .expect("Failure running DB migrations");
        SqliteInsightGenerationJobDao::from_connection(Arc::new(Mutex::new(conn)))
    }
    fn ctx() -> opentelemetry::Context {
        opentelemetry::Context::new()
    }
    #[test]
    fn create_job_inserts_new_row() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id_1 = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        let job_id_2 = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        assert_ne!(job_id_1, job_id_2, "each create_job call inserts a new row");
    }
    #[test]
    fn complete_job_sets_result() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        dao.complete_job(&ctx, job_id, 42).unwrap();
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(job.status, InsightJobStatus::Completed.as_str());
        assert_eq!(job.result_insight_id, Some(42));
        assert!(job.completed_at.is_some());
    }
    #[test]
    fn fail_job_sets_error() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Agentic)
            .unwrap();
        dao.fail_job(&ctx, job_id, "model timeout").unwrap();
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(job.status, InsightJobStatus::Failed.as_str());
        assert_eq!(job.error_message.as_deref(), Some("model timeout"));
        assert!(job.completed_at.is_some());
    }
    #[test]
    fn get_active_job_returns_none_when_completed() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        // Job is running
        let active = dao.get_active_job(&ctx, 1, "photos/test.jpg").unwrap();
        assert!(active.is_some());
        assert_eq!(active.unwrap().id, job_id);
        // Complete it
        dao.complete_job(&ctx, job_id, 1).unwrap();
        // No longer active
        let active = dao.get_active_job(&ctx, 1, "photos/test.jpg").unwrap();
        assert!(active.is_none());
    }
    #[test]
    fn cancel_active_jobs() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        let cancelled = dao.cancel_active_jobs(&ctx, 1, "photos/test.jpg").unwrap();
        assert_eq!(cancelled, 1, "should cancel 1 running job");
        // Job is no longer active
        let active = dao.get_active_job(&ctx, 1, "photos/test.jpg").unwrap();
        assert!(active.is_none());
        // Job exists with cancelled status
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(job.status, InsightJobStatus::Cancelled.as_str());
        // Cancelling again returns 0 (nothing to cancel)
        let cancelled2 = dao.cancel_active_jobs(&ctx, 1, "photos/test.jpg").unwrap();
        assert_eq!(cancelled2, 0, "should return 0 when no running job");
    }
    #[test]
    fn get_active_job_scoped_by_library() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id_1 = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        let job_id_2 = dao
            .create_job(&ctx, 2, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        assert_ne!(
            job_id_1, job_id_2,
            "different libraries should have separate jobs"
        );
        // Complete lib1's job
        dao.complete_job(&ctx, job_id_1, 1).unwrap();
        // lib1 has no active job
        let active1 = dao.get_active_job(&ctx, 1, "photos/test.jpg").unwrap();
        assert!(active1.is_none());
        // lib2 still has active job
        let active2 = dao.get_active_job(&ctx, 2, "photos/test.jpg").unwrap();
        assert!(active2.is_some());
        assert_eq!(active2.unwrap().id, job_id_2);
    }
    #[test]
    fn get_job_by_id_finds_any_status() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        // Find while running
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(job.status, InsightJobStatus::Running.as_str());
        // Complete it
        dao.complete_job(&ctx, job_id, 99).unwrap();
        // Still findable
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(job.status, InsightJobStatus::Completed.as_str());
        assert_eq!(job.result_insight_id, Some(99));
    }
    #[test]
    fn recover_orphaned_jobs() {
        let mut dao = setup_dao();
        let ctx = ctx();
        // Create two running jobs
        let job_id_1 = dao
            .create_job(&ctx, 1, "photos/a.jpg", InsightGenerationType::Standard)
            .unwrap();
        let job_id_2 = dao
            .create_job(&ctx, 1, "photos/b.jpg", InsightGenerationType::Agentic)
            .unwrap();
        // Complete one
        dao.complete_job(&ctx, job_id_1, 1).unwrap();
        // Recover should only affect the running job
        let recovered = dao.recover_orphaned_jobs(&ctx).unwrap();
        assert_eq!(recovered, 1, "should recover exactly 1 running job");
        // job_id_1 is still completed
        let job1 = dao.get_job_by_id(&ctx, job_id_1).unwrap().unwrap();
        assert_eq!(job1.status, InsightJobStatus::Completed.as_str());
        // job_id_2 is now failed with recovery message
        let job2 = dao.get_job_by_id(&ctx, job_id_2).unwrap().unwrap();
        assert_eq!(job2.status, InsightJobStatus::Failed.as_str());
        assert_eq!(
            job2.error_message.as_deref(),
            Some("server crashed while running")
        );
        // Second recovery is a no-op
        let recovered2 = dao.recover_orphaned_jobs(&ctx).unwrap();
        assert_eq!(recovered2, 0, "no running jobs remain");
    }
    #[test]
    fn complete_job_noop_when_cancelled() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        dao.cancel_job(&ctx, job_id).unwrap();
        // Late-completing task tries to mark as completed — should be a no-op
        dao.complete_job(&ctx, job_id, 42).unwrap();
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(
            job.status,
            InsightJobStatus::Cancelled.as_str(),
            "cancelled status must not be overwritten by late complete"
        );
        assert_eq!(
            job.result_insight_id, None,
            "insight_id must stay None when complete is a no-op"
        );
    }
    #[test]
    fn fail_job_noop_when_cancelled() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Agentic)
            .unwrap();
        dao.cancel_job(&ctx, job_id).unwrap();
        // Late-failing task tries to mark as failed — should be a no-op
        dao.fail_job(&ctx, job_id, "timeout after 120s").unwrap();
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(
            job.status,
            InsightJobStatus::Cancelled.as_str(),
            "cancelled status must not be overwritten by late fail"
        );
        assert_eq!(
            job.error_message.as_deref(),
            Some("cancelled by user"),
            "error_message must reflect the cancel, not the late fail"
        );
    }
    #[test]
    fn cancel_job_by_id() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        let cancelled = dao.cancel_job(&ctx, job_id).unwrap();
        assert!(cancelled, "should cancel running job");
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(job.status, InsightJobStatus::Cancelled.as_str());
        assert!(job.completed_at.is_some());
        // Cancelling again is a no-op
        let cancelled2 = dao.cancel_job(&ctx, job_id).unwrap();
        assert!(!cancelled2, "already cancelled job should return false");
    }
 }
@@ -21,22 +21,6 @@ pub trait InsightDao: Sync + Send {
        file_path: &str,
    ) -> Result<Option<PhotoInsight>, DbError>;
    /// Library-scoped variant of `get_insight`. The default `get_insight`
    /// finds any `is_current=true` row matching `file_path` across
    /// libraries — fine for the photo-grid metadata fetch (cross-library
    /// merge), wrong for the chat path: a regenerate on lib1 flips lib1's
    /// row to `is_current=false` and inserts a new lib1 row, but
    /// lib2's untouched `is_current=true` row for the same rel_path
    /// would still satisfy the path-only query and shadow the regen on
    /// the next history fetch. Always pass a library_id when you have
    /// one (chat / insight write paths always do).
    fn get_current_insight_for_library(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
    ) -> Result<Option<PhotoInsight>, DbError>;
    /// Return the most recent current insight whose rel_path is one of
    /// `paths`. Used for content-hash sharing: the caller expands a
    /// single file into all rel_paths with the same content_hash, then
@@ -47,6 +31,7 @@ pub trait InsightDao: Sync + Send {
        paths: &[String],
    ) -> Result<Option<PhotoInsight>, DbError>;
    #[allow(dead_code)]
    fn get_insight_history(
        &mut self,
        context: &opentelemetry::Context,
@@ -81,17 +66,6 @@ pub trait InsightDao: Sync + Send {
        approved: bool,
    ) -> Result<(), DbError>;
    /// Rate a specific insight version by primary key, regardless of
    /// `is_current`. Used by the per-file history view to approve/reject
    /// previously generated (superseded) versions, which the path-based
    /// `rate_insight` (current row only) cannot reach.
    fn rate_insight_by_id(
        &mut self,
        context: &opentelemetry::Context,
        insight_id: i32,
        approved: bool,
    ) -> Result<(), DbError>;
    fn get_approved_insights(
        &mut self,
        context: &opentelemetry::Context,
@@ -100,15 +74,13 @@ pub trait InsightDao: Sync + Send {
    /// Replace the `training_messages` JSON blob on the current row for
    /// `(library_id, rel_path)`. Used by chat-turn append mode to persist
    /// the extended conversation without inserting a new insight version.
    /// Returns the number of rows affected (0 if no current row matched,
    /// indicating a concurrent regenerate/reconcile flipped `is_current`).
    fn update_training_messages(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
        training_messages_json: &str,
-    ) -> Result<usize, DbError>;
+    ) -> Result<(), DbError>;
 }
 pub struct SqliteInsightDao {
@@ -171,13 +143,13 @@ impl InsightDao for SqliteInsightDao {
            )
            .set(is_current.eq(false))
            .execute(connection.deref_mut())
-            .map_err(|e| anyhow::anyhow!("Failed to flip is_current: {}", e))?;
+            .map_err(|_| anyhow::anyhow!("Update is_current error"))?;
            // Insert the new insight as current
            diesel::insert_into(photo_insights)
                .values(&insight)
                .execute(connection.deref_mut())
-                .map_err(|e| anyhow::anyhow!("Failed to insert insight: {}", e))?;
+                .map_err(|_| anyhow::anyhow!("Insert error"))?;
            // Retrieve the inserted record (is_current = true)
            photo_insights
@@ -185,12 +157,9 @@ impl InsightDao for SqliteInsightDao {
                .filter(rel_path.eq(&insight.file_path))
                .filter(is_current.eq(true))
                .first::<PhotoInsight>(connection.deref_mut())
-                .map_err(|e| anyhow::anyhow!("Failed to retrieve inserted insight: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
        .map_err(|e| {
            log::error!("store_insight failed: {}", e);
            DbError::new(DbErrorKind::InsertError)
        })
        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn get_insight(
@@ -208,36 +177,9 @@ impl InsightDao for SqliteInsightDao {
                .filter(is_current.eq(true))
                .first::<PhotoInsight>(connection.deref_mut())
                .optional()
-                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_current_insight_for_library(
        &mut self,
        context: &opentelemetry::Context,
        lib_id: i32,
        path: &str,
    ) -> Result<Option<PhotoInsight>, DbError> {
        trace_db_call(
            context,
            "query",
            "get_current_insight_for_library",
            |_span| {
                use schema::photo_insights::dsl::*;
                let mut connection = self.connection.lock().expect("Unable to get InsightDao");
                photo_insights
                    .filter(library_id.eq(lib_id))
                    .filter(rel_path.eq(path))
                    .filter(is_current.eq(true))
                    .first::<PhotoInsight>(connection.deref_mut())
                    .optional()
                    .map_err(|e| anyhow::anyhow!("Query error: {}", e))
            },
        )
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn get_insight_for_paths(
@@ -259,9 +201,9 @@ impl InsightDao for SqliteInsightDao {
                .order(generated_at.desc())
                .first::<PhotoInsight>(connection.deref_mut())
                .optional()
-                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_insight_history(
@@ -278,9 +220,9 @@ impl InsightDao for SqliteInsightDao {
                .filter(rel_path.eq(path))
                .order(generated_at.desc())
                .load::<PhotoInsight>(connection.deref_mut())
-                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_insight_by_id(
@@ -297,9 +239,9 @@ impl InsightDao for SqliteInsightDao {
                .find(insight_id)
                .first::<PhotoInsight>(connection.deref_mut())
                .optional()
-                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn delete_insight(
@@ -315,9 +257,9 @@ impl InsightDao for SqliteInsightDao {
            diesel::delete(photo_insights.filter(rel_path.eq(path)))
                .execute(connection.deref_mut())
                .map(|_| ())
-                .map_err(|e| anyhow::anyhow!("Delete error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Delete error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_all_insights(
@@ -333,9 +275,9 @@ impl InsightDao for SqliteInsightDao {
                .filter(is_current.eq(true))
                .order(generated_at.desc())
                .load::<PhotoInsight>(connection.deref_mut())
-                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn rate_insight(
@@ -357,29 +299,9 @@ impl InsightDao for SqliteInsightDao {
            .set(approved.eq(Some(is_approved)))
            .execute(connection.deref_mut())
            .map(|_| ())
-            .map_err(|e| anyhow::anyhow!("Update error: {}", e))
+            .map_err(|_| anyhow::anyhow!("Update error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
+        .map_err(|_| DbError::new(DbErrorKind::UpdateError))
    }
    fn rate_insight_by_id(
        &mut self,
        context: &opentelemetry::Context,
        target_id: i32,
        is_approved: bool,
    ) -> Result<(), DbError> {
        trace_db_call(context, "update", "rate_insight_by_id", |_span| {
            use schema::photo_insights::dsl::*;
            let mut connection = self.connection.lock().expect("Unable to get InsightDao");
            diesel::update(photo_insights.find(target_id))
                .set(approved.eq(Some(is_approved)))
                .execute(connection.deref_mut())
                .map(|_| ())
                .map_err(|e| anyhow::anyhow!("Update error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn get_approved_insights(
@@ -396,9 +318,9 @@ impl InsightDao for SqliteInsightDao {
                .filter(training_messages.is_not_null())
                .order(generated_at.desc())
                .load::<PhotoInsight>(connection.deref_mut())
-                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn update_training_messages(
@@ -407,7 +329,7 @@ impl InsightDao for SqliteInsightDao {
        lib_id: i32,
        path: &str,
        training_messages_json: &str,
-    ) -> Result<usize, DbError> {
+    ) -> Result<(), DbError> {
        trace_db_call(context, "update", "update_training_messages", |_span| {
            use schema::photo_insights::dsl::*;
@@ -421,95 +343,9 @@ impl InsightDao for SqliteInsightDao {
            )
            .set(training_messages.eq(Some(training_messages_json.to_string())))
            .execute(connection.deref_mut())
-            .map_err(|e| anyhow::anyhow!("Update error: {}", e))
+            .map(|_| ())
            .map_err(|_| anyhow::anyhow!("Update error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
+        .map_err(|_| DbError::new(DbErrorKind::UpdateError))
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use crate::database::test::in_memory_db_connection;
    fn dao() -> SqliteInsightDao {
        let conn = Arc::new(Mutex::new(in_memory_db_connection()));
        SqliteInsightDao::from_connection(conn)
    }
    /// Build an insight insert with sensible defaults; tests override the
    /// fields they care about (path, generated_at, model).
    fn insert(path: &str, generated_at: i64, model: &str) -> InsertPhotoInsight {
        InsertPhotoInsight {
            library_id: 1,
            file_path: path.to_string(),
            title: format!("title for {model}"),
            summary: "summary".to_string(),
            generated_at,
            model_version: model.to_string(),
            is_current: true,
            training_messages: None,
            backend: "local".to_string(),
            fewshot_source_ids: None,
            content_hash: None,
            num_ctx: None,
            temperature: None,
            top_p: None,
            top_k: None,
            min_p: None,
            system_prompt: None,
            persona_id: None,
            prompt_eval_count: None,
            eval_count: None,
        }
    }
    #[test]
    fn get_insight_history_returns_all_versions_newest_first() {
        let cx = opentelemetry::Context::new();
        let mut dao = dao();
        // store_insight flips prior rows to is_current=false, so three
        // generations for the same path leave a 3-row history.
        dao.store_insight(&cx, insert("a.jpg", 100, "m1")).unwrap();
        dao.store_insight(&cx, insert("a.jpg", 200, "m2")).unwrap();
        dao.store_insight(&cx, insert("a.jpg", 300, "m3")).unwrap();
        // A different path must not leak into the history.
        dao.store_insight(&cx, insert("b.jpg", 250, "other"))
            .unwrap();
        let history = dao.get_insight_history(&cx, "a.jpg").unwrap();
        assert_eq!(history.len(), 3);
        assert_eq!(
            history.iter().map(|i| i.generated_at).collect::<Vec<_>>(),
            vec![300, 200, 100],
            "history should be newest-first"
        );
        // Exactly one version is current (the latest generation).
        let current: Vec<_> = history.iter().filter(|i| i.is_current).collect();
        assert_eq!(current.len(), 1);
        assert_eq!(current[0].generated_at, 300);
    }
    #[test]
    fn rate_insight_by_id_rates_only_the_targeted_version() {
        let cx = opentelemetry::Context::new();
        let mut dao = dao();
        dao.store_insight(&cx, insert("a.jpg", 100, "m1")).unwrap();
        dao.store_insight(&cx, insert("a.jpg", 200, "m2")).unwrap();
        // History is newest-first: [200 (current), 100 (superseded)].
        let history = dao.get_insight_history(&cx, "a.jpg").unwrap();
        let old_version = history.iter().find(|i| i.generated_at == 100).unwrap();
        assert!(!old_version.is_current);
        dao.rate_insight_by_id(&cx, old_version.id, true).unwrap();
        let history = dao.get_insight_history(&cx, "a.jpg").unwrap();
        let old = history.iter().find(|i| i.generated_at == 100).unwrap();
        let current = history.iter().find(|i| i.generated_at == 200).unwrap();
        assert_eq!(old.approved, Some(true), "targeted version is rated");
        assert_eq!(current.approved, None, "current version is untouched");
    }
 }
@@ -216,12 +216,11 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            // Validate embedding dimensions if provided (rare for location data)
            if let Some(ref emb) = location.embedding
-                && emb.len() != crate::ai::embedding_dim()
+                && emb.len() != 768
            {
                return Err(anyhow::anyhow!(
-                    "Invalid embedding dimensions: {} (expected {})",
+                    "Invalid embedding dimensions: {} (expected 768)",
-                    emb.len(),
+                    emb.len()
                    crate::ai::embedding_dim()
                ));
            }
@@ -274,7 +273,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
                source_file: location.source_file,
            })
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn store_locations_batch(
@@ -293,7 +292,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
                for location in locations {
                    // Validate embedding if provided (rare)
                    if let Some(ref emb) = location.embedding
-                        && emb.len() != crate::ai::embedding_dim()
+                        && emb.len() != 768
                    {
                        log::warn!(
                            "Skipping location with invalid embedding dimensions: {}",
@@ -351,7 +350,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            Ok(inserted)
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn find_nearest_location(
@@ -386,7 +385,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            Ok(results.into_iter().next().map(|r| r.to_location_record()))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_locations_in_range(
@@ -414,7 +413,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            .map(|rows| rows.into_iter().map(|r| r.to_location_record()).collect())
            .map_err(|e| anyhow::anyhow!("Query error: {:?}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_locations_near_point(
@@ -469,7 +468,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            Ok(filtered)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn location_exists(
@@ -503,7 +502,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            Ok(result.count > 0)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_location_count(&mut self, context: &opentelemetry::Context) -> Result<i64, DbError> {
@@ -526,6 +525,6 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            Ok(result.count)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
 }
@@ -1,76 +1,9 @@
 use crate::database::schema::{
-    entities, entity_facts, entity_photo_links, favorites, image_exif, insight_generation_jobs,
+    entities, entity_facts, entity_photo_links, favorites, image_exif, libraries, photo_insights,
-    libraries, personas, photo_insights, precomputed_reels, user_ai_prefs, users,
+    users, video_preview_clips,
    video_preview_clips,
 };
 use serde::Serialize;
 /// Possible statuses for an insight generation job.
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, FromSqlRow)]
 #[serde(rename_all = "snake_case")]
 pub enum InsightJobStatus {
    Running,
    Completed,
    Failed,
    Cancelled,
 }
 impl InsightJobStatus {
    pub fn as_str(&self) -> &'static str {
        match self {
            Self::Running => "running",
            Self::Completed => "completed",
            Self::Failed => "failed",
            Self::Cancelled => "cancelled",
        }
    }
    pub fn parse(s: &str) -> Self {
        match s {
            "running" => Self::Running,
            "completed" => Self::Completed,
            "failed" => Self::Failed,
            "cancelled" => Self::Cancelled,
            other => {
                log::warn!(
                    "Unknown InsightJobStatus value: {:?}, treating as failed",
                    other
                );
                Self::Failed
            }
        }
    }
 }
 impl std::fmt::Display for InsightJobStatus {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.write_str(self.as_str())
    }
 }
 /// Type of insight generation (standard vs agentic).
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
 #[serde(rename_all = "snake_case")]
 pub enum InsightGenerationType {
    Standard,
    Agentic,
 }
 impl InsightGenerationType {
    pub fn as_str(&self) -> &'static str {
        match self {
            Self::Standard => "standard",
            Self::Agentic => "agentic",
        }
    }
 }
 impl std::fmt::Display for InsightGenerationType {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.write_str(self.as_str())
    }
 }
 #[derive(Insertable)]
 #[diesel(table_name = users)]
 pub struct InsertUser<'a> {
@@ -181,18 +114,9 @@ pub struct ImageExif {
    /// Snapshot of the prior `date_taken_source` taken on first manual
    /// override. NULL when no override is active.
    pub original_date_taken_source: Option<String>,
    /// L2-normalized CLIP image embedding (raw little-endian float32 bytes;
    /// length depends on the model — 768×4 for ViT-L/14, 512×4 for ViT-B/32).
    /// NULL until Apollo's CLIP service has encoded this photo via the
    /// backfill drain. Used by `/photos/search` for semantic queries.
    pub clip_embedding: Option<Vec<u8>>,
    /// Which CLIP model produced `clip_embedding` (e.g. `"ViT-L/14"`). A
    /// swap of `APOLLO_CLIP_MODEL` re-eligibilizes rows whose stored
    /// version differs so the drain rebuilds them.
    pub clip_model_version: Option<String>,
 }
-#[derive(Insertable, Clone)]
+#[derive(Insertable)]
 #[diesel(table_name = photo_insights)]
 pub struct InsertPhotoInsight {
    pub library_id: i32,
@@ -219,15 +143,6 @@ pub struct InsertPhotoInsight {
    /// inserted before the hash is available stay null and the
    /// reconciliation pass backfills them.
    pub content_hash: Option<String>,
    pub num_ctx: Option<i32>,
    pub temperature: Option<f32>,
    pub top_p: Option<f32>,
    pub top_k: Option<i32>,
    pub min_p: Option<f32>,
    pub system_prompt: Option<String>,
    pub persona_id: Option<String>,
    pub prompt_eval_count: Option<i32>,
    pub eval_count: Option<i32>,
 }
 #[derive(Serialize, Queryable, Clone, Debug)]
@@ -247,15 +162,6 @@ pub struct PhotoInsight {
    pub backend: String,
    pub fewshot_source_ids: Option<String>,
    pub content_hash: Option<String>,
    pub num_ctx: Option<i32>,
    pub temperature: Option<f32>,
    pub top_p: Option<f32>,
    pub top_k: Option<i32>,
    pub min_p: Option<f32>,
    pub system_prompt: Option<String>,
    pub persona_id: Option<String>,
    pub prompt_eval_count: Option<i32>,
    pub eval_count: Option<i32>,
 }
 // --- Libraries ---
@@ -332,44 +238,6 @@ pub struct InsertEntityFact {
    pub confidence: f32,
    pub status: String,
    pub created_at: i64,
    /// Which persona authored this fact. Shared entities, persona-tagged
    /// facts: each persona accumulates its own voice over the same
    /// real-world referents. Defaults to `'default'` for legacy rows
    /// (see migration 2026-05-09-000000).
    pub persona_id: String,
    /// Author's user_id. Required for the composite FK to
    /// `personas(user_id, persona_id)` (migration 2026-05-10-000000) and
    /// for cross-user fact isolation: two users with the same 'default'
    /// persona must not see each other's facts. Always paired with
    /// `persona_id` — they're a unit.
    pub user_id: i32,
    /// Real-world period the fact is/was true (unix seconds). NULL on
    /// either side = unbounded — `valid_from IS NULL` reads as
    /// "always-true-back-to-the-beginning", `valid_until IS NULL` as
    /// "still-true-now-or-unknown". Distinguishes valid time from
    /// transaction time (`created_at` is when we recorded the fact,
    /// not when it was true in the world). See migration
    /// 2026-05-10-000100.
    pub valid_from: Option<i64>,
    pub valid_until: Option<i64>,
    /// Points at the entity_facts.id that replaced this one. Set by
    /// the supersede endpoint; status flips to 'superseded' in the
    /// same transaction. See migration 2026-05-10-000200.
    pub superseded_by: Option<i32>,
    /// Provenance for model audit — see migration 2026-05-10-000300.
    /// `created_by_model` is the LLM identifier (e.g. "qwen2.5:7b",
    /// "anthropic/claude-sonnet-4") or NULL for legacy / manual rows.
    /// `created_by_backend` is "local" / "hybrid" / "manual" / NULL.
    pub created_by_model: Option<String>,
    pub created_by_backend: Option<String>,
    /// Audit trail for mutations after creation — see migration
    /// 2026-05-10-000500. `last_modified_*` stamp on any update
    /// (status flip, valid-time edit, supersede, manual PATCH);
    /// `last_modified_at` is unix seconds. NULL on rows that have
    /// never been touched since creation.
    pub last_modified_by_model: Option<String>,
    pub last_modified_by_backend: Option<String>,
    pub last_modified_at: Option<i64>,
 }
 #[derive(Serialize, Queryable, Clone, Debug)]
@@ -384,16 +252,6 @@ pub struct EntityFact {
    pub confidence: f32,
    pub status: String,
    pub created_at: i64,
    pub persona_id: String,
    pub user_id: i32,
    pub valid_from: Option<i64>,
    pub valid_until: Option<i64>,
    pub superseded_by: Option<i32>,
    pub created_by_model: Option<String>,
    pub created_by_backend: Option<String>,
    pub last_modified_by_model: Option<String>,
    pub last_modified_by_backend: Option<String>,
    pub last_modified_at: Option<i64>,
 }
 #[derive(Insertable)]
@@ -416,45 +274,6 @@ pub struct EntityPhotoLink {
    pub role: String,
 }
 // --- Personas ---
 #[derive(Insertable)]
 #[diesel(table_name = personas)]
 pub struct InsertPersona<'a> {
    pub user_id: i32,
    pub persona_id: &'a str,
    pub name: &'a str,
    pub system_prompt: &'a str,
    pub is_built_in: bool,
    pub include_all_memories: bool,
    pub created_at: i64,
    pub updated_at: i64,
    /// "Strict mode" — agent reads only see facts with status =
    /// 'reviewed' (human-verified). Default false. See migration
    /// 2026-05-10-000400.
    pub reviewed_only_facts: bool,
    /// Gate for the agent's update_fact / supersede_fact tools.
    /// Default false — fresh personas let the agent create but not
    /// alter or replace. Operator opts in once a model has earned
    /// trust. See migration 2026-05-10-000500.
    pub allow_agent_corrections: bool,
 }
 #[derive(Serialize, Queryable, Clone, Debug)]
 pub struct Persona {
    pub id: i32,
    pub user_id: i32,
    pub persona_id: String,
    pub name: String,
    pub system_prompt: String,
    pub is_built_in: bool,
    pub include_all_memories: bool,
    pub created_at: i64,
    pub updated_at: i64,
    pub reviewed_only_facts: bool,
    pub allow_agent_corrections: bool,
 }
 #[derive(Insertable)]
 #[diesel(table_name = video_preview_clips)]
 pub struct InsertVideoPreviewClip {
@@ -479,83 +298,3 @@ pub struct VideoPreviewClip {
    pub created_at: String,
    pub updated_at: String,
 }
 #[derive(Insertable)]
 #[diesel(table_name = insight_generation_jobs)]
 pub struct InsertInsightGenerationJob {
    pub library_id: i32,
    #[diesel(column_name = file_path)]
    pub path: String,
    #[diesel(column_name = generation_type)]
    pub gen_type: String,
    pub status: String,
    pub started_at: i64,
 }
 #[derive(Queryable, Serialize, Clone, Debug)]
 pub struct InsightGenerationJob {
    pub id: i32,
    pub library_id: i32,
    #[diesel(column_name = file_path)]
    pub path: String,
    #[diesel(column_name = generation_type)]
    pub gen_type: String,
    pub status: String,
    pub started_at: i64,
    pub completed_at: Option<i64>,
    pub result_insight_id: Option<i32>,
    pub error_message: Option<String>,
 }
 // --- Precomputed reels -------------------------------------------------------
 #[derive(Insertable)]
 #[diesel(table_name = precomputed_reels)]
 pub struct InsertablePrecomputedReel {
    pub span: String,
    pub library_key: String,
    pub cache_key: String,
    pub output_path: String,
    pub title: String,
    pub media_count: i32,
    pub render_version: i32,
    pub tz_offset_minutes: i32,
    pub voice: Option<String>,
    pub generated_at: i64,
 }
 #[derive(Serialize, Queryable, Clone, Debug)]
 pub struct PrecomputedReel {
    pub id: i32,
    pub span: String,
    pub library_key: String,
    pub cache_key: String,
    pub output_path: String,
    pub title: String,
    pub media_count: i32,
    pub render_version: i32,
    pub tz_offset_minutes: i32,
    pub voice: Option<String>,
    pub generated_at: i64,
 }
 // --- User AI preferences (Section E) ----------------------------------------
 #[derive(Queryable, Insertable, Debug, Clone, serde::Deserialize, serde::Serialize)]
 #[diesel(table_name = user_ai_prefs)]
 pub struct UserAiPrefs {
    pub id: i32,
    pub voice: Option<String>,
    pub tz_offset_minutes: Option<i32>,
    pub library: Option<String>,
    pub updated_at: i64,
 }
 #[derive(Insertable, Debug, Clone, serde::Deserialize, serde::Serialize)]
 #[diesel(table_name = user_ai_prefs)]
 pub struct UpsertUserAiPrefs {
    pub voice: Option<String>,
    pub tz_offset_minutes: Option<i32>,
    pub library: Option<String>,
    pub updated_at: i64,
 }
@@ -1,935 +0,0 @@
 #![allow(dead_code)]
 use diesel::prelude::*;
 use diesel::sqlite::SqliteConnection;
 use std::ops::DerefMut;
 use std::sync::{Arc, Mutex};
 use crate::database::models::{InsertPersona, Persona};
 use crate::database::schema;
 use crate::database::{DbError, DbErrorKind, connect};
 use crate::otel::trace_db_call;
 /// One row of the persona-chat transcript. Lives in
 /// `persona_chat_conversations` (one row per `(user_id, persona_id)`).
 ///
 /// One persona conversation.
 ///
 /// `turn_count` is the cumulative number of assistant turns on the active
 /// branch — what the list screen shows as the length of a conversation. The
 /// transcript itself is the `messages_json` blob, holding a serialized
 /// `ChatHistoryStore` tree. `title` is a generated summary of the opening
 /// exchange, empty until the first turn completes.
 #[derive(Clone, Debug)]
 pub struct PersonaChatRow {
    pub conversation_id: String,
    pub persona_id: String,
    pub title: String,
    pub messages_json: String,
    pub turn_count: i32,
    pub created_at: i64,
    pub updated_at: i64,
 }
 /// Column tuple → `PersonaChatRow`. Shared by the single-row and list
 /// queries so their `select(...)` orders can never drift apart.
 fn persona_chat_row(
    (conversation_id, persona_id, title, messages_json, turn_count, created_at, updated_at): (
        String,
        String,
        String,
        String,
        i32,
        i64,
        i64,
    ),
 ) -> PersonaChatRow {
    PersonaChatRow {
        conversation_id,
        persona_id,
        title,
        messages_json,
        turn_count,
        created_at,
        updated_at,
    }
 }
 /// Patch shape for update_persona. None = leave field alone. Built-ins are
 /// allowed to flip `include_all_memories` but should reject name/prompt
 /// edits at the handler layer (built-in copy lives in the migration).
 pub struct PersonaPatch {
    pub name: Option<String>,
    pub system_prompt: Option<String>,
    pub include_all_memories: Option<bool>,
    pub reviewed_only_facts: Option<bool>,
    pub allow_agent_corrections: Option<bool>,
 }
 /// One row of a bulk migration upload. Fields named to match the JSON
 /// shape the mobile client uploads (`POST /personas/migrate`).
 pub struct ImportPersona {
    pub persona_id: String,
    pub name: String,
    pub system_prompt: String,
    pub is_built_in: bool,
    pub created_at: i64,
 }
 pub trait PersonaDao: Sync + Send {
    fn list_personas(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
    ) -> Result<Vec<Persona>, DbError>;
    fn get_persona(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        persona_id: &str,
    ) -> Result<Option<Persona>, DbError>;
    fn create_persona(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        persona_id: &str,
        name: &str,
        system_prompt: &str,
        is_built_in: bool,
        include_all_memories: bool,
    ) -> Result<Persona, DbError>;
    fn update_persona(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        persona_id: &str,
        patch: PersonaPatch,
    ) -> Result<Option<Persona>, DbError>;
    fn delete_persona(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        persona_id: &str,
    ) -> Result<bool, DbError>;
    /// Idempotent bulk import. INSERT OR IGNORE on (user_id, persona_id)
    /// — re-uploading the same set is a no-op. Returns the number of rows
    /// actually inserted (skipped duplicates don't count).
    fn bulk_import(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        personas: &[ImportPersona],
    ) -> Result<usize, DbError>;
    // ── Persona-chat (open chat with persona) persistence ───────────
    //
    // Keyed on an opaque `conversation_id` so one persona can hold several
    // separate conversations. `user_id` is checked on every read and write:
    // the id is a bearer token for a transcript, so ownership can never be
    // assumed from the id alone.
    /// Fetch one conversation. None when it doesn't exist or belongs to
    /// another user — the two are deliberately indistinguishable to callers.
    fn get_persona_chat(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        conversation_id: &str,
    ) -> Result<Option<PersonaChatRow>, DbError>;
    /// Every conversation this user has going, newest first. Backs the chat
    /// list screen, so it returns the transcript blob too — the snippet is
    /// derived from the tree rather than denormalized into its own column.
    fn list_persona_chats(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
    ) -> Result<Vec<PersonaChatRow>, DbError>;
    /// Start a new conversation with a persona, returning its id. Several
    /// conversations with the same persona are expected, so this never
    /// reuses an existing row.
    fn create_persona_chat(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        persona_id: &str,
        created_at: i64,
    ) -> Result<String, DbError>;
    /// Replace a conversation's transcript. Called once per completed turn
    /// with the full new `messages_json`; `turn_count` is the cumulative
    /// assistant-turn count on the resulting active branch. Returns the
    /// number of rows written — 0 means the conversation is gone or is not
    /// this user's.
    fn update_persona_chat(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        conversation_id: &str,
        messages_json: &str,
        turn_count: i32,
        updated_at: i64,
    ) -> Result<usize, DbError>;
    /// Name a conversation. Written once, after the first turn produces
    /// enough of an exchange to summarize.
    fn set_persona_chat_title(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        conversation_id: &str,
        title: &str,
    ) -> Result<(), DbError>;
    /// Delete a conversation outright. Backs both the list screen's delete
    /// affordance and the reset endpoint.
    fn delete_persona_chat(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        conversation_id: &str,
    ) -> Result<(), DbError>;
 }
 pub struct SqlitePersonaDao {
    connection: Arc<Mutex<SqliteConnection>>,
 }
 impl Default for SqlitePersonaDao {
    fn default() -> Self {
        Self::new()
    }
 }
 impl SqlitePersonaDao {
    pub fn new() -> Self {
        Self {
            connection: Arc::new(Mutex::new(connect())),
        }
    }
    pub fn from_connection(conn: Arc<Mutex<SqliteConnection>>) -> Self {
        Self { connection: conn }
    }
 }
 impl PersonaDao for SqlitePersonaDao {
    fn list_personas(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
    ) -> Result<Vec<Persona>, DbError> {
        trace_db_call(cx, "query", "list_personas", |_span| {
            use schema::personas::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            personas
                .filter(user_id.eq(uid))
                .order(created_at.asc())
                .load::<Persona>(conn.deref_mut())
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn get_persona(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        pid: &str,
    ) -> Result<Option<Persona>, DbError> {
        trace_db_call(cx, "query", "get_persona", |_span| {
            use schema::personas::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            personas
                .filter(user_id.eq(uid))
                .filter(persona_id.eq(pid))
                .first::<Persona>(conn.deref_mut())
                .optional()
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn create_persona(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        pid: &str,
        nm: &str,
        prompt: &str,
        builtin: bool,
        include_all: bool,
    ) -> Result<Persona, DbError> {
        trace_db_call(cx, "insert", "create_persona", |_span| {
            use schema::personas::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            let now = chrono::Utc::now().timestamp_millis();
            diesel::insert_into(personas)
                .values(InsertPersona {
                    user_id: uid,
                    persona_id: pid,
                    name: nm,
                    system_prompt: prompt,
                    is_built_in: builtin,
                    include_all_memories: include_all,
                    created_at: now,
                    updated_at: now,
                    reviewed_only_facts: false,
                    allow_agent_corrections: false,
                })
                .execute(conn.deref_mut())
                .map_err(|e| anyhow::anyhow!("Insert error: {}", e))?;
            personas
                .filter(user_id.eq(uid))
                .filter(persona_id.eq(pid))
                .first::<Persona>(conn.deref_mut())
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
    }
    fn update_persona(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        pid: &str,
        patch: PersonaPatch,
    ) -> Result<Option<Persona>, DbError> {
        trace_db_call(cx, "update", "update_persona", |_span| {
            use schema::personas::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            let now = chrono::Utc::now().timestamp_millis();
            // Apply each field as its own UPDATE — keeps types simple
            // (Diesel's tuple updates don't compose cleanly across optional
            // columns) and matches the pattern already in use for entities
            // (knowledge_dao.rs::update_entity).
            if let Some(ref new_name) = patch.name {
                diesel::update(personas.filter(user_id.eq(uid)).filter(persona_id.eq(pid)))
                    .set((name.eq(new_name), updated_at.eq(now)))
                    .execute(conn.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Update name error: {}", e))?;
            }
            if let Some(ref new_prompt) = patch.system_prompt {
                diesel::update(personas.filter(user_id.eq(uid)).filter(persona_id.eq(pid)))
                    .set((system_prompt.eq(new_prompt), updated_at.eq(now)))
                    .execute(conn.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Update prompt error: {}", e))?;
            }
            if let Some(new_include_all) = patch.include_all_memories {
                diesel::update(personas.filter(user_id.eq(uid)).filter(persona_id.eq(pid)))
                    .set((include_all_memories.eq(new_include_all), updated_at.eq(now)))
                    .execute(conn.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Update include_all error: {}", e))?;
            }
            if let Some(new_reviewed_only) = patch.reviewed_only_facts {
                diesel::update(personas.filter(user_id.eq(uid)).filter(persona_id.eq(pid)))
                    .set((
                        reviewed_only_facts.eq(new_reviewed_only),
                        updated_at.eq(now),
                    ))
                    .execute(conn.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Update reviewed_only_facts error: {}", e))?;
            }
            if let Some(new_allow_corrections) = patch.allow_agent_corrections {
                diesel::update(personas.filter(user_id.eq(uid)).filter(persona_id.eq(pid)))
                    .set((
                        allow_agent_corrections.eq(new_allow_corrections),
                        updated_at.eq(now),
                    ))
                    .execute(conn.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Update allow_agent_corrections error: {}", e))?;
            }
            personas
                .filter(user_id.eq(uid))
                .filter(persona_id.eq(pid))
                .first::<Persona>(conn.deref_mut())
                .optional()
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn delete_persona(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        pid: &str,
    ) -> Result<bool, DbError> {
        trace_db_call(cx, "delete", "delete_persona", |_span| {
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            // One transaction so a persona and its conversations can't get
            // out of step. There is no FK between the two tables, and the
            // chat list skips conversations whose persona is gone — so
            // without this the transcripts would linger invisibly forever.
            conn.deref_mut().transaction::<_, anyhow::Error, _>(|tx| {
                {
                    use schema::persona_chat_conversations::dsl::*;
                    diesel::delete(
                        persona_chat_conversations
                            .filter(user_id.eq(uid))
                            .filter(persona_id.eq(pid)),
                    )
                    .execute(tx)
                    .map_err(|e| anyhow::anyhow!("Delete error: {}", e))?;
                }
                use schema::personas::dsl::*;
                let n = diesel::delete(personas.filter(user_id.eq(uid)).filter(persona_id.eq(pid)))
                    .execute(tx)
                    .map_err(|e| anyhow::anyhow!("Delete error: {}", e))?;
                Ok(n > 0)
            })
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn bulk_import(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        rows: &[ImportPersona],
    ) -> Result<usize, DbError> {
        trace_db_call(cx, "insert", "bulk_import_personas", |_span| {
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            let now = chrono::Utc::now().timestamp_millis();
            let mut inserted = 0usize;
            // INSERT OR IGNORE on the (user_id, persona_id) UNIQUE so
            // re-running migrate is a no-op for personas already on the
            // server.
            for p in rows {
                let n = diesel::sql_query(
                    "INSERT OR IGNORE INTO personas (user_id, persona_id, name, system_prompt, \
                     is_built_in, include_all_memories, created_at, updated_at) \
                     VALUES (?, ?, ?, ?, ?, 0, ?, ?)",
                )
                .bind::<diesel::sql_types::Integer, _>(uid)
                .bind::<diesel::sql_types::Text, _>(&p.persona_id)
                .bind::<diesel::sql_types::Text, _>(&p.name)
                .bind::<diesel::sql_types::Text, _>(&p.system_prompt)
                .bind::<diesel::sql_types::Bool, _>(p.is_built_in)
                .bind::<diesel::sql_types::BigInt, _>(p.created_at)
                .bind::<diesel::sql_types::BigInt, _>(now)
                .execute(conn.deref_mut())
                .map_err(|e| anyhow::anyhow!("Insert error: {}", e))?;
                inserted += n;
            }
            Ok(inserted)
        })
        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
    }
    fn get_persona_chat(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        cid: &str,
    ) -> Result<Option<PersonaChatRow>, DbError> {
        trace_db_call(cx, "query", "get_persona_chat", |_span| {
            use schema::persona_chat_conversations::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            persona_chat_conversations
                .filter(conversation_id.eq(cid))
                // Scoped by user as well as id: the id is a bearer token for
                // someone's transcript, so it never grants access on its own.
                .filter(user_id.eq(uid))
                .select((
                    conversation_id,
                    persona_id,
                    title,
                    messages_json,
                    turn_count,
                    created_at,
                    updated_at,
                ))
                .first::<(String, String, String, String, i32, i64, i64)>(conn.deref_mut())
                .optional()
                .map(|opt| opt.map(persona_chat_row))
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn list_persona_chats(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
    ) -> Result<Vec<PersonaChatRow>, DbError> {
        trace_db_call(cx, "query", "list_persona_chats", |_span| {
            use schema::persona_chat_conversations::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            persona_chat_conversations
                .filter(user_id.eq(uid))
                .order(updated_at.desc())
                .select((
                    conversation_id,
                    persona_id,
                    title,
                    messages_json,
                    turn_count,
                    created_at,
                    updated_at,
                ))
                .load::<(String, String, String, String, i32, i64, i64)>(conn.deref_mut())
                .map(|rows| rows.into_iter().map(persona_chat_row).collect())
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn create_persona_chat(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        pid: &str,
        created: i64,
    ) -> Result<String, DbError> {
        trace_db_call(cx, "insert", "create_persona_chat", |_span| {
            use schema::persona_chat_conversations::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            let new_id = uuid::Uuid::new_v4().to_string();
            diesel::insert_into(persona_chat_conversations)
                .values((
                    conversation_id.eq(&new_id),
                    user_id.eq(uid),
                    persona_id.eq(pid),
                    title.eq(""),
                    // The empty tree; the first turn seeds it with the
                    // persona's system prompt.
                    messages_json.eq("[]"),
                    turn_count.eq(0),
                    created_at.eq(created),
                    updated_at.eq(created),
                ))
                .execute(conn.deref_mut())
                .map_err(|e| anyhow::anyhow!("Insert error: {}", e))?;
            Ok(new_id)
        })
        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
    }
    fn update_persona_chat(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        cid: &str,
        json: &str,
        count: i32,
        updated: i64,
    ) -> Result<usize, DbError> {
        trace_db_call(cx, "update", "update_persona_chat", |_span| {
            use schema::persona_chat_conversations::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            diesel::update(
                persona_chat_conversations
                    .filter(conversation_id.eq(cid))
                    .filter(user_id.eq(uid)),
            )
            .set((
                messages_json.eq(json),
                turn_count.eq(count),
                updated_at.eq(updated),
            ))
            .execute(conn.deref_mut())
            .map_err(|e| anyhow::anyhow!("Update error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn set_persona_chat_title(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        cid: &str,
        new_title: &str,
    ) -> Result<(), DbError> {
        trace_db_call(cx, "update", "set_persona_chat_title", |_span| {
            use schema::persona_chat_conversations::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            diesel::update(
                persona_chat_conversations
                    .filter(conversation_id.eq(cid))
                    .filter(user_id.eq(uid)),
            )
            .set(title.eq(new_title))
            .execute(conn.deref_mut())
            .map_err(|e| anyhow::anyhow!("Update error: {}", e))?;
            Ok(())
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn delete_persona_chat(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        cid: &str,
    ) -> Result<(), DbError> {
        trace_db_call(cx, "delete", "delete_persona_chat", |_span| {
            use schema::persona_chat_conversations::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            diesel::delete(
                persona_chat_conversations
                    .filter(conversation_id.eq(cid))
                    .filter(user_id.eq(uid)),
            )
            .execute(conn.deref_mut())
            .map_err(|e| anyhow::anyhow!("Delete error: {}", e))?;
            Ok(())
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use crate::database::test::in_memory_db_connection;
    fn dao_with_user(username: &str) -> (SqlitePersonaDao, i32) {
        use crate::database::schema::users::dsl as u;
        let conn = Arc::new(Mutex::new(in_memory_db_connection()));
        diesel::insert_into(u::users)
            .values((u::username.eq(username), u::password.eq("x")))
            .execute(conn.lock().unwrap().deref_mut())
            .unwrap();
        let user_id: i32 = u::users
            .filter(u::username.eq(username))
            .select(u::id)
            .first(conn.lock().unwrap().deref_mut())
            .unwrap();
        (SqlitePersonaDao::from_connection(conn), user_id)
    }
    #[test]
    fn create_and_list_round_trip() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("alice");
        // The migration seeds 3 built-ins for any existing user; alice
        // was created post-migration so she starts empty.
        let p = dao
            .create_persona(&cx, uid, "custom-1", "Custom A", "prompt A", false, false)
            .unwrap();
        assert_eq!(p.persona_id, "custom-1");
        assert_eq!(p.user_id, uid);
        assert!(!p.is_built_in);
        let list = dao.list_personas(&cx, uid).unwrap();
        assert_eq!(list.len(), 1);
        assert_eq!(list[0].persona_id, "custom-1");
    }
    #[test]
    fn unique_constraint_blocks_duplicate_persona_id() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("bob");
        dao.create_persona(&cx, uid, "x", "X", "p", false, false)
            .unwrap();
        let err = dao.create_persona(&cx, uid, "x", "X2", "p2", false, false);
        assert!(
            err.is_err(),
            "second insert with same persona_id should fail"
        );
    }
    #[test]
    fn bulk_import_is_idempotent() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("carol");
        let rows = vec![
            ImportPersona {
                persona_id: "custom-a".into(),
                name: "A".into(),
                system_prompt: "p1".into(),
                is_built_in: false,
                created_at: 1,
            },
            ImportPersona {
                persona_id: "custom-b".into(),
                name: "B".into(),
                system_prompt: "p2".into(),
                is_built_in: false,
                created_at: 2,
            },
        ];
        let first = dao.bulk_import(&cx, uid, &rows).unwrap();
        assert_eq!(first, 2);
        let second = dao.bulk_import(&cx, uid, &rows).unwrap();
        assert_eq!(second, 0, "re-import should insert nothing");
        assert_eq!(dao.list_personas(&cx, uid).unwrap().len(), 2);
    }
    #[test]
    fn dao_update_does_not_block_built_ins() {
        // Documenting contract: the DAO is intentionally permissive —
        // `update_persona` will apply name/system_prompt edits to ANY
        // row, including built-ins. The guard against editing built-in
        // identity (name + systemPrompt) lives in the HTTP handler
        // (src/personas.rs::update_persona). If you find yourself
        // wanting to add the guard here too, prefer that — defence in
        // depth — but keep this test passing so anyone who removes
        // the handler guard gets a failing call site, not silent data
        // corruption.
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("eve");
        dao.create_persona(&cx, uid, "default", "Default", "old", true, false)
            .unwrap();
        let updated = dao
            .update_persona(
                &cx,
                uid,
                "default",
                PersonaPatch {
                    name: Some("Renamed".into()),
                    system_prompt: Some("new prompt".into()),
                    include_all_memories: None,
                    reviewed_only_facts: None,
                    allow_agent_corrections: None,
                },
            )
            .unwrap()
            .unwrap();
        assert_eq!(updated.name, "Renamed");
        assert_eq!(updated.system_prompt, "new prompt");
        assert!(
            updated.is_built_in,
            "is_built_in flag should be unchanged by patch"
        );
    }
    #[test]
    fn update_toggles_include_all_memories() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("dan");
        dao.create_persona(&cx, uid, "j", "Journal", "p", true, false)
            .unwrap();
        let updated = dao
            .update_persona(
                &cx,
                uid,
                "j",
                PersonaPatch {
                    name: None,
                    system_prompt: None,
                    include_all_memories: Some(true),
                    reviewed_only_facts: None,
                    allow_agent_corrections: None,
                },
            )
            .unwrap()
            .unwrap();
        assert!(updated.include_all_memories);
    }
    // ── Persona-chat DAO tests ─────────────────────────────────────
    /// Second user, for the isolation tests.
    fn second_user(dao: &SqlitePersonaDao, username: &str) -> i32 {
        use crate::database::schema::users::dsl as u;
        let conn = dao.connection.clone();
        diesel::insert_into(u::users)
            .values((u::username.eq(username), u::password.eq("x")))
            .execute(conn.lock().unwrap().deref_mut())
            .unwrap();
        u::users
            .filter(u::username.eq(username))
            .select(u::id)
            .first(conn.lock().unwrap().deref_mut())
            .unwrap()
    }
    #[test]
    fn persona_chat_get_returns_none_for_an_unknown_conversation() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("p1");
        assert!(
            dao.get_persona_chat(&cx, uid, "no-such-id")
                .unwrap()
                .is_none()
        );
    }
    #[test]
    fn persona_chat_create_then_get_round_trip() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("p2");
        let cid = dao.create_persona_chat(&cx, uid, "journal", 100).unwrap();
        let row = dao.get_persona_chat(&cx, uid, &cid).unwrap().unwrap();
        assert_eq!(row.conversation_id, cid);
        assert_eq!(row.persona_id, "journal");
        assert_eq!(row.title, "", "unnamed until the first turn completes");
        assert_eq!(row.turn_count, 0);
        assert_eq!(row.created_at, 100);
        assert_eq!(row.updated_at, 100);
    }
    #[test]
    fn persona_chat_create_makes_a_separate_row_each_time() {
        // The whole point of the conversation_id key: one persona, several
        // independent threads.
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("p3");
        let first = dao.create_persona_chat(&cx, uid, "journal", 100).unwrap();
        let second = dao.create_persona_chat(&cx, uid, "journal", 200).unwrap();
        assert_ne!(first, second);
        assert_eq!(dao.list_persona_chats(&cx, uid).unwrap().len(), 2);
    }
    #[test]
    fn persona_chat_update_replaces_the_transcript() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("p4");
        let cid = dao.create_persona_chat(&cx, uid, "journal", 100).unwrap();
        let rows = dao
            .update_persona_chat(&cx, uid, &cid, "first", 1, 150)
            .unwrap();
        assert_eq!(rows, 1);
        let rows = dao
            .update_persona_chat(&cx, uid, &cid, "second", 2, 200)
            .unwrap();
        assert_eq!(rows, 1);
        let row = dao.get_persona_chat(&cx, uid, &cid).unwrap().unwrap();
        assert_eq!(row.messages_json, "second");
        assert_eq!(row.turn_count, 2);
        assert_eq!(row.updated_at, 200);
        assert_eq!(row.created_at, 100, "creation time is not disturbed");
    }
    #[test]
    fn persona_chat_update_reports_zero_rows_for_an_unknown_conversation() {
        // The turn loop treats 0 as "the conversation went away mid-turn"
        // rather than silently succeeding.
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("p5");
        let rows = dao
            .update_persona_chat(&cx, uid, "no-such-id", "x", 1, 1)
            .unwrap();
        assert_eq!(rows, 0);
    }
    #[test]
    fn persona_chat_set_title_names_the_conversation() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("p6");
        let cid = dao.create_persona_chat(&cx, uid, "journal", 100).unwrap();
        dao.set_persona_chat_title(&cx, uid, &cid, "June recap")
            .unwrap();
        let row = dao.get_persona_chat(&cx, uid, &cid).unwrap().unwrap();
        assert_eq!(row.title, "June recap");
    }
    #[test]
    fn persona_chat_list_orders_newest_first() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("p7");
        let older = dao.create_persona_chat(&cx, uid, "journal", 100).unwrap();
        let newer = dao.create_persona_chat(&cx, uid, "coach", 300).unwrap();
        let middle = dao.create_persona_chat(&cx, uid, "default", 200).unwrap();
        let ids: Vec<String> = dao
            .list_persona_chats(&cx, uid)
            .unwrap()
            .into_iter()
            .map(|r| r.conversation_id)
            .collect();
        assert_eq!(
            ids,
            vec![newer, middle, older],
            "ordered by updated_at descending so the list screen needs no re-sort"
        );
    }
    #[test]
    fn persona_chat_list_is_empty_for_a_user_who_has_never_chatted() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("p8");
        assert!(dao.list_persona_chats(&cx, uid).unwrap().is_empty());
    }
    #[test]
    fn persona_chat_delete_removes_the_conversation() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("p9");
        let cid = dao.create_persona_chat(&cx, uid, "journal", 100).unwrap();
        dao.delete_persona_chat(&cx, uid, &cid).unwrap();
        assert!(dao.get_persona_chat(&cx, uid, &cid).unwrap().is_none());
        assert!(dao.list_persona_chats(&cx, uid).unwrap().is_empty());
    }
    #[test]
    fn deleting_a_persona_takes_its_conversations_with_it() {
        // No FK between the tables, and the chat list hides conversations
        // whose persona is gone — so an orphan row would be an invisible,
        // unreachable transcript sitting in the database forever.
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("p-cascade");
        dao.create_persona(&cx, uid, "custom-1", "Custom", "prompt", false, false)
            .unwrap();
        dao.create_persona_chat(&cx, uid, "custom-1", 100).unwrap();
        dao.create_persona_chat(&cx, uid, "custom-1", 200).unwrap();
        let survivor = dao.create_persona_chat(&cx, uid, "other", 300).unwrap();
        assert!(dao.delete_persona(&cx, uid, "custom-1").unwrap());
        let remaining: Vec<String> = dao
            .list_persona_chats(&cx, uid)
            .unwrap()
            .into_iter()
            .map(|r| r.conversation_id)
            .collect();
        assert_eq!(
            remaining,
            vec![survivor],
            "only the untouched persona's conversation survives"
        );
    }
    #[test]
    fn persona_chat_reads_and_writes_are_scoped_to_the_owner() {
        // A conversation_id is a bearer token for someone's transcript, so
        // holding one must not grant another user access to it.
        let cx = opentelemetry::Context::new();
        let (mut dao, uid1) = dao_with_user("owner");
        let uid2 = second_user(&dao, "intruder");
        let cid = dao.create_persona_chat(&cx, uid1, "journal", 100).unwrap();
        dao.update_persona_chat(&cx, uid1, &cid, "private", 1, 100)
            .unwrap();
        assert!(
            dao.get_persona_chat(&cx, uid2, &cid).unwrap().is_none(),
            "another user cannot read it"
        );
        assert_eq!(
            dao.update_persona_chat(&cx, uid2, &cid, "tampered", 9, 999)
                .unwrap(),
            0,
            "another user cannot write it"
        );
        dao.delete_persona_chat(&cx, uid2, &cid).unwrap();
        let row = dao.get_persona_chat(&cx, uid1, &cid).unwrap().unwrap();
        assert_eq!(row.messages_json, "private", "and cannot delete it");
        assert!(dao.list_persona_chats(&cx, uid2).unwrap().is_empty());
    }
 }
@@ -1,439 +0,0 @@
 use diesel::prelude::*;
 use diesel::sqlite::SqliteConnection;
 use std::ops::DerefMut;
 use std::sync::{Arc, Mutex};
 use crate::database::models::{InsertablePrecomputedReel, PrecomputedReel};
 use crate::database::schema;
 use crate::database::{DbError, DbErrorKind, connect};
 use crate::otel::trace_db_call;
 /// Ledger for precomputed memory reels. The nightly agentic job writes a
 /// row after each successful render; the `GET /reels/precomputed` handler
 /// reads it to gate on freshness and serve the cached MP4.
 pub trait PrecomputedReelDao: Sync + Send {
    /// Insert a precomputed reel row. Returns the new row's id.
    /// Written by the nightly agentic job (Section D).
    #[allow(dead_code)]
    fn record_reel(
        &mut self,
        context: &opentelemetry::Context,
        row: &InsertablePrecomputedReel,
    ) -> Result<i32, DbError>;
    /// Find the latest precomputed reel for the given (span, library_key).
    fn latest_for(
        &mut self,
        context: &opentelemetry::Context,
        span: &str,
        library_key: &str,
    ) -> Result<Option<PrecomputedReel>, DbError>;
    /// Return true when a fresh precomputed reel exists for the given
    /// (span, library_key, render_version) that was generated at or after
    /// `min_generated_at`. Used as a fast existence gate before falling
    /// back to `latest_for` (avoids a second query path).
    fn exists_fresh(
        &mut self,
        context: &opentelemetry::Context,
        span: &str,
        library_key: &str,
        render_version: i32,
        min_generated_at: i64,
    ) -> Result<bool, DbError>;
    /// Delete all but the newest `keep` rows for (span, library_key), returning
    /// the deleted rows so the caller can unlink their output files. Used by the
    /// nightly job to retire superseded reels (e.g. yesterday's daily).
    #[allow(dead_code)]
    fn prune_superseded(
        &mut self,
        context: &opentelemetry::Context,
        span: &str,
        library_key: &str,
        keep: usize,
    ) -> Result<Vec<PrecomputedReel>, DbError>;
    /// Every cache_key currently in the ledger. Used by the on-disk cache sweep
    /// to protect files a ledger row still points at.
    #[allow(dead_code)]
    fn all_cache_keys(&mut self, context: &opentelemetry::Context) -> Result<Vec<String>, DbError>;
 }
 pub struct SqlitePrecomputedReelDao {
    connection: Arc<Mutex<SqliteConnection>>,
 }
 impl Default for SqlitePrecomputedReelDao {
    fn default() -> Self {
        Self::new()
    }
 }
 impl SqlitePrecomputedReelDao {
    pub fn new() -> Self {
        Self {
            connection: Arc::new(Mutex::new(connect())),
        }
    }
    #[cfg(test)]
    pub fn from_connection(conn: Arc<Mutex<SqliteConnection>>) -> Self {
        Self { connection: conn }
    }
 }
 impl PrecomputedReelDao for SqlitePrecomputedReelDao {
    fn record_reel(
        &mut self,
        context: &opentelemetry::Context,
        row: &InsertablePrecomputedReel,
    ) -> Result<i32, DbError> {
        trace_db_call(context, "insert", "record_reel", |_span| {
            use schema::precomputed_reels::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock PrecomputedReelDao");
            diesel::insert_into(dsl::precomputed_reels)
                .values(row)
                .execute(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Failed to insert reel: {}", e))?;
            dsl::precomputed_reels
                .order(dsl::id.desc())
                .select(dsl::id)
                .first::<i32>(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Failed to get reel id: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
    }
    fn latest_for(
        &mut self,
        context: &opentelemetry::Context,
        span: &str,
        library_key: &str,
    ) -> Result<Option<PrecomputedReel>, DbError> {
        trace_db_call(context, "query", "latest_for", |_span| {
            use schema::precomputed_reels::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock PrecomputedReelDao");
            dsl::precomputed_reels
                .filter(dsl::span.eq(span))
                .filter(dsl::library_key.eq(library_key))
                .order(dsl::generated_at.desc())
                .first::<PrecomputedReel>(connection.deref_mut())
                .optional()
                .map_err(|e| anyhow::anyhow!("Failed to get latest reel: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn exists_fresh(
        &mut self,
        context: &opentelemetry::Context,
        span: &str,
        library_key: &str,
        render_version: i32,
        min_generated_at: i64,
    ) -> Result<bool, DbError> {
        trace_db_call(context, "query", "exists_fresh", |_span| {
            use schema::precomputed_reels::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock PrecomputedReelDao");
            let count: i64 = dsl::precomputed_reels
                .filter(dsl::span.eq(span))
                .filter(dsl::library_key.eq(library_key))
                .filter(dsl::render_version.eq(render_version))
                .filter(dsl::generated_at.ge(min_generated_at))
                .count()
                .get_result(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Failed to check fresh reel: {}", e))?;
            Ok(count > 0)
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn prune_superseded(
        &mut self,
        context: &opentelemetry::Context,
        span: &str,
        library_key: &str,
        keep: usize,
    ) -> Result<Vec<PrecomputedReel>, DbError> {
        trace_db_call(context, "delete", "prune_superseded", |_span| {
            use schema::precomputed_reels::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock PrecomputedReelDao");
            // Newest first; everything past `keep` is superseded. The table
            // holds at most a handful of rows per (span, library), so loading
            // and slicing in Rust is cheaper than a correlated subquery.
            let mut rows: Vec<PrecomputedReel> = dsl::precomputed_reels
                .filter(dsl::span.eq(span))
                .filter(dsl::library_key.eq(library_key))
                .order(dsl::generated_at.desc())
                .load::<PrecomputedReel>(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Failed to load reels for prune: {}", e))?;
            let stale = rows.split_off(rows.len().min(keep));
            if !stale.is_empty() {
                let ids: Vec<i32> = stale.iter().map(|r| r.id).collect();
                diesel::delete(dsl::precomputed_reels.filter(dsl::id.eq_any(ids)))
                    .execute(connection.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Failed to delete superseded reels: {}", e))?;
            }
            Ok(stale)
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn all_cache_keys(&mut self, context: &opentelemetry::Context) -> Result<Vec<String>, DbError> {
        trace_db_call(context, "query", "all_cache_keys", |_span| {
            use schema::precomputed_reels::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock PrecomputedReelDao");
            dsl::precomputed_reels
                .select(dsl::cache_key)
                .load::<String>(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Failed to load cache keys: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use diesel::Connection;
    use diesel_migrations::{EmbeddedMigrations, MigrationHarness, embed_migrations};
    const DB_MIGRATIONS: EmbeddedMigrations = embed_migrations!();
    fn setup_dao() -> SqlitePrecomputedReelDao {
        let mut conn = SqliteConnection::establish(":memory:")
            .expect("Unable to create in-memory db connection");
        conn.run_pending_migrations(DB_MIGRATIONS)
            .expect("Failure running DB migrations");
        SqlitePrecomputedReelDao::from_connection(Arc::new(Mutex::new(conn)))
    }
    fn ctx() -> opentelemetry::Context {
        opentelemetry::Context::new()
    }
    fn sample_row() -> InsertablePrecomputedReel {
        InsertablePrecomputedReel {
            span: "day".to_string(),
            library_key: "1".to_string(),
            cache_key: "abc123".to_string(),
            output_path: "/tmp/reel.mp4".to_string(),
            title: "Test Reel".to_string(),
            media_count: 10,
            render_version: 1,
            tz_offset_minutes: 0,
            voice: Some("default".to_string()),
            generated_at: 1_000_000,
        }
    }
    #[test]
    fn record_reel_inserts_and_returns_id() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let row = sample_row();
        let id = dao.record_reel(&ctx, &row).unwrap();
        assert!(id > 0, "should return a positive id");
    }
    #[test]
    fn record_reel_returns_increasing_ids() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let row = sample_row();
        let id1 = dao.record_reel(&ctx, &row).unwrap();
        let id2 = dao.record_reel(&ctx, &row).unwrap();
        assert!(id2 > id1, "each insert should get a higher id");
    }
    #[test]
    fn latest_for_returns_latest() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let row1 = InsertablePrecomputedReel {
            generated_at: 1_000_000,
            ..sample_row()
        };
        let row2 = InsertablePrecomputedReel {
            generated_at: 2_000_000,
            ..sample_row()
        };
        dao.record_reel(&ctx, &row1).unwrap();
        dao.record_reel(&ctx, &row2).unwrap();
        let latest = dao.latest_for(&ctx, "day", "1").unwrap().unwrap();
        assert_eq!(latest.generated_at, 2_000_000);
    }
    #[test]
    fn latest_for_scoped_by_span_and_library() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let day_row = InsertablePrecomputedReel {
            span: "day".to_string(),
            library_key: "1".to_string(),
            generated_at: 1_000_000,
            ..sample_row()
        };
        let week_row = InsertablePrecomputedReel {
            span: "week".to_string(),
            library_key: "1".to_string(),
            generated_at: 2_000_000,
            ..sample_row()
        };
        dao.record_reel(&ctx, &day_row).unwrap();
        dao.record_reel(&ctx, &week_row).unwrap();
        let day_latest = dao.latest_for(&ctx, "day", "1").unwrap().unwrap();
        assert_eq!(day_latest.span, "day");
        let week_latest = dao.latest_for(&ctx, "week", "1").unwrap().unwrap();
        assert_eq!(week_latest.span, "week");
        // Different library returns None
        let missing = dao.latest_for(&ctx, "day", "99").unwrap();
        assert!(missing.is_none());
    }
    #[test]
    fn latest_for_returns_none_when_no_rows() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let result = dao.latest_for(&ctx, "day", "1").unwrap();
        assert!(result.is_none());
    }
    #[test]
    fn exists_fresh_returns_true_when_present() {
        let mut dao = setup_dao();
        let ctx = ctx();
        dao.record_reel(&ctx, &sample_row()).unwrap();
        let exists = dao.exists_fresh(&ctx, "day", "1", 1, 900_000).unwrap();
        assert!(exists, "should find the row we just inserted");
    }
    #[test]
    fn exists_fresh_returns_false_when_missing() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let exists = dao.exists_fresh(&ctx, "day", "1", 1, 900_000).unwrap();
        assert!(!exists, "should not find anything in empty table");
    }
    #[test]
    fn exists_fresh_respects_min_generated_at() {
        let mut dao = setup_dao();
        let ctx = ctx();
        dao.record_reel(&ctx, &sample_row()).unwrap();
        // Below the threshold — should exist
        let exists = dao.exists_fresh(&ctx, "day", "1", 1, 500_000).unwrap();
        assert!(exists);
        // Above the threshold — should not exist
        let exists = dao.exists_fresh(&ctx, "day", "1", 1, 2_000_000).unwrap();
        assert!(!exists);
    }
    #[test]
    fn exists_fresh_respects_render_version() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let row_v1 = InsertablePrecomputedReel {
            render_version: 1,
            ..sample_row()
        };
        dao.record_reel(&ctx, &row_v1).unwrap();
        assert!(dao.exists_fresh(&ctx, "day", "1", 1, 900_000).unwrap());
        assert!(!dao.exists_fresh(&ctx, "day", "1", 2, 900_000).unwrap());
    }
    #[test]
    fn prune_superseded_keeps_newest_and_returns_deleted() {
        let mut dao = setup_dao();
        let ctx = ctx();
        // Three day/lib1 reels at increasing timestamps, plus an unrelated one.
        for (i, key) in ["k1", "k2", "k3"].iter().enumerate() {
            dao.record_reel(
                &ctx,
                &InsertablePrecomputedReel {
                    cache_key: key.to_string(),
                    generated_at: 1_000_000 + i as i64 * 1000,
                    ..sample_row()
                },
            )
            .unwrap();
        }
        let other = InsertablePrecomputedReel {
            library_key: "2".to_string(),
            cache_key: "other".to_string(),
            ..sample_row()
        };
        dao.record_reel(&ctx, &other).unwrap();
        // Keep the newest 2 of (day, "1"); k1 (oldest) is superseded.
        let deleted = dao.prune_superseded(&ctx, "day", "1", 2).unwrap();
        assert_eq!(deleted.len(), 1);
        assert_eq!(deleted[0].cache_key, "k1");
        // The newest 2 survive; the other-library row is untouched.
        let keys = dao.all_cache_keys(&ctx).unwrap();
        assert_eq!(keys.len(), 3);
        assert!(keys.contains(&"k2".to_string()));
        assert!(keys.contains(&"k3".to_string()));
        assert!(keys.contains(&"other".to_string()));
        assert!(!keys.contains(&"k1".to_string()));
    }
    #[test]
    fn prune_superseded_noop_when_within_keep() {
        let mut dao = setup_dao();
        let ctx = ctx();
        dao.record_reel(&ctx, &sample_row()).unwrap();
        let deleted = dao.prune_superseded(&ctx, "day", "1", 2).unwrap();
        assert!(deleted.is_empty());
        assert_eq!(dao.all_cache_keys(&ctx).unwrap().len(), 1);
    }
 }
@@ -96,7 +96,7 @@ impl PreviewDao for SqlitePreviewDao {
                .map(|_| ())
                .map_err(|e| anyhow::anyhow!("Insert error: {}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn update_status(
@@ -126,7 +126,7 @@ impl PreviewDao for SqlitePreviewDao {
                .map(|_| ())
                .map_err(|e| anyhow::anyhow!("Update error: {}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
+        .map_err(|_| DbError::new(DbErrorKind::UpdateError))
    }
    fn get_preview(
@@ -148,7 +148,7 @@ impl PreviewDao for SqlitePreviewDao {
                Err(e) => Err(anyhow::anyhow!("Query error: {}", e)),
            }
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_previews_batch(
@@ -170,7 +170,7 @@ impl PreviewDao for SqlitePreviewDao {
                .load::<VideoPreviewClip>(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_by_status(
@@ -188,7 +188,7 @@ impl PreviewDao for SqlitePreviewDao {
                .load::<VideoPreviewClip>(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
 }
@@ -57,28 +57,30 @@ impl ReconcileStats {
 /// watcher tick. Errors are logged but never propagated; reconciliation
 /// is best-effort and a transient DB hiccup must not stall the watcher.
 pub fn run(conn: &mut SqliteConnection) -> ReconcileStats {
-    let stats = ReconcileStats {
+    let mut stats = ReconcileStats::default();
-        tagged_photo_hashes_filled: match backfill_tagged_photo_hashes(conn) {
+
    stats.tagged_photo_hashes_filled = match backfill_tagged_photo_hashes(conn) {
        Ok(n) => n,
        Err(e) => {
            warn!("reconcile: tagged_photo hash backfill failed: {:?}", e);
            0
        }
-        },
+    };
-        photo_insights_hashes_filled: match backfill_photo_insights_hashes(conn) {
+
    stats.photo_insights_hashes_filled = match backfill_photo_insights_hashes(conn) {
        Ok(n) => n,
        Err(e) => {
            warn!("reconcile: photo_insights hash backfill failed: {:?}", e);
            0
        }
-        },
+    };
-        photo_insights_demoted: match collapse_insight_currents(conn) {
+
    stats.photo_insights_demoted = match collapse_insight_currents(conn) {
        Ok(n) => n,
        Err(e) => {
            warn!("reconcile: photo_insights scalar merge failed: {:?}", e);
            0
        }
        },
    };
    if stats.changed() {
@@ -57,16 +57,6 @@ diesel::table! {
        confidence -> Float,
        status -> Text,
        created_at -> BigInt,
        persona_id -> Text,
        user_id -> Integer,
        valid_from -> Nullable<BigInt>,
        valid_until -> Nullable<BigInt>,
        superseded_by -> Nullable<Integer>,
        created_by_model -> Nullable<Text>,
        created_by_backend -> Nullable<Text>,
        last_modified_by_model -> Nullable<Text>,
        last_modified_by_backend -> Nullable<Text>,
        last_modified_at -> Nullable<BigInt>,
    }
 }
@@ -138,8 +128,6 @@ diesel::table! {
        date_taken_source -> Nullable<Text>,
        original_date_taken -> Nullable<BigInt>,
        original_date_taken_source -> Nullable<Text>,
        clip_embedding -> Nullable<Binary>,
        clip_model_version -> Nullable<Text>,
    }
 }
@@ -171,35 +159,6 @@ diesel::table! {
    }
 }
 diesel::table! {
    persona_chat_conversations (conversation_id) {
        conversation_id -> Text,
        user_id -> Integer,
        persona_id -> Text,
        title -> Text,
        messages_json -> Text,
        turn_count -> Integer,
        created_at -> BigInt,
        updated_at -> BigInt,
    }
 }
 diesel::table! {
    personas (id) {
        id -> Integer,
        user_id -> Integer,
        persona_id -> Text,
        name -> Text,
        system_prompt -> Text,
        is_built_in -> Bool,
        include_all_memories -> Bool,
        created_at -> BigInt,
        updated_at -> BigInt,
        reviewed_only_facts -> Bool,
        allow_agent_corrections -> Bool,
    }
 }
 diesel::table! {
    persons (id) {
        id -> Integer,
@@ -229,15 +188,6 @@ diesel::table! {
        backend -> Text,
        fewshot_source_ids -> Nullable<Text>,
        content_hash -> Nullable<Text>,
        num_ctx -> Nullable<Integer>,
        temperature -> Nullable<Float>,
        top_p -> Nullable<Float>,
        top_k -> Nullable<Integer>,
        min_p -> Nullable<Float>,
        system_prompt -> Nullable<Text>,
        persona_id -> Nullable<Text>,
        prompt_eval_count -> Nullable<Integer>,
        eval_count -> Nullable<Integer>,
    }
 }
@@ -279,16 +229,6 @@ diesel::table! {
    }
 }
 diesel::table! {
    user_ai_prefs (id) {
        id -> Integer,
        voice -> Nullable<Text>,
        tz_offset_minutes -> Nullable<Integer>,
        library -> Nullable<Text>,
        updated_at -> BigInt,
    }
 }
 diesel::table! {
    video_preview_clips (id) {
        id -> Integer,
@@ -303,44 +243,12 @@ diesel::table! {
    }
 }
 diesel::table! {
    insight_generation_jobs (id) {
        id -> Integer,
        library_id -> Integer,
        file_path -> Text,
        generation_type -> Text,
        status -> Text,
        started_at -> BigInt,
        completed_at -> Nullable<BigInt>,
        result_insight_id -> Nullable<Integer>,
        error_message -> Nullable<Text>,
    }
 }
 diesel::table! {
    precomputed_reels (id) {
        id -> Integer,
        span -> Text,
        library_key -> Text,
        cache_key -> Text,
        output_path -> Text,
        title -> Text,
        media_count -> Integer,
        render_version -> Integer,
        tz_offset_minutes -> Integer,
        voice -> Nullable<Text>,
        generated_at -> BigInt,
    }
 }
 diesel::joinable!(entity_facts -> photo_insights (source_insight_id));
 diesel::joinable!(entity_photo_links -> entities (entity_id));
 diesel::joinable!(entity_photo_links -> libraries (library_id));
 diesel::joinable!(face_detections -> libraries (library_id));
 diesel::joinable!(face_detections -> persons (person_id));
 diesel::joinable!(image_exif -> libraries (library_id));
 diesel::joinable!(insight_generation_jobs -> libraries (library_id));
 diesel::joinable!(personas -> users (user_id));
 diesel::joinable!(persons -> entities (entity_id));
 diesel::joinable!(photo_insights -> libraries (library_id));
 diesel::joinable!(tagged_photo -> tags (tag_id));
@@ -355,18 +263,13 @@ diesel::allow_tables_to_appear_in_same_query!(
    face_detections,
    favorites,
    image_exif,
    insight_generation_jobs,
    libraries,
    location_history,
    persona_chat_conversations,
    personas,
    persons,
    photo_insights,
    precomputed_reels,
    search_history,
    tagged_photo,
    tags,
    user_ai_prefs,
    users,
    video_preview_clips,
 );
@@ -189,11 +189,10 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
                .expect("Unable to get SearchHistoryDao");
            // Validate embedding dimensions (REQUIRED for searches)
-            if search.embedding.len() != crate::ai::embedding_dim() {
+            if search.embedding.len() != 768 {
                return Err(anyhow::anyhow!(
-                    "Invalid embedding dimensions: {} (expected {})",
+                    "Invalid embedding dimensions: {} (expected 768)",
-                    search.embedding.len(),
+                    search.embedding.len()
                    crate::ai::embedding_dim()
                ));
            }
@@ -228,7 +227,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
                source_file: search.source_file,
            })
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn store_searches_batch(
@@ -246,7 +245,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
            conn.transaction::<_, anyhow::Error, _>(|conn| {
                for search in searches {
                    // Validate embedding (REQUIRED)
-                    if search.embedding.len() != crate::ai::embedding_dim() {
+                    if search.embedding.len() != 768 {
                        log::warn!(
                            "Skipping search with invalid embedding dimensions: {}",
                            search.embedding.len()
@@ -284,7 +283,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
            Ok(inserted)
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn find_searches_in_range(
@@ -311,7 +310,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
            .map(|rows| rows.into_iter().map(|r| r.to_search_record()).collect())
            .map_err(|e| anyhow::anyhow!("Query error: {:?}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_similar_searches(
@@ -326,11 +325,10 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
                .lock()
                .expect("Unable to get SearchHistoryDao");
-            if query_embedding.len() != crate::ai::embedding_dim() {
+            if query_embedding.len() != 768 {
                return Err(anyhow::anyhow!(
-                    "Invalid query embedding dimensions: {} (expected {})",
+                    "Invalid query embedding dimensions: {} (expected 768)",
-                    query_embedding.len(),
+                    query_embedding.len()
                    crate::ai::embedding_dim()
                ));
            }
@@ -374,7 +372,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
                .map(|(_, search)| search)
                .collect())
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_relevant_searches_hybrid(
@@ -408,11 +406,10 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
            // Step 2: If query embedding provided, rank by semantic similarity
            if let Some(query_emb) = query_embedding {
-                if query_emb.len() != crate::ai::embedding_dim() {
+                if query_emb.len() != 768 {
                    return Err(anyhow::anyhow!(
-                        "Invalid query embedding dimensions: {} (expected {})",
+                        "Invalid query embedding dimensions: {} (expected 768)",
-                        query_emb.len(),
+                        query_emb.len()
                        crate::ai::embedding_dim()
                    ));
                }
@@ -462,7 +459,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
                    .collect())
            }
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn search_exists(
@@ -493,7 +490,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
            Ok(result.count > 0)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_search_count(&mut self, context: &opentelemetry::Context) -> Result<i64, DbError> {
@@ -516,6 +513,6 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
            Ok(result.count)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
 }
@@ -1,206 +0,0 @@
 use diesel::prelude::*;
 use diesel::sqlite::SqliteConnection;
 use std::ops::DerefMut;
 use std::sync::{Arc, Mutex};
 use crate::database::models::{UpsertUserAiPrefs, UserAiPrefs};
 use crate::database::schema;
 use crate::database::{DbError, DbErrorKind, connect};
 use crate::otel::trace_db_call;
 /// Generic single-row table that passively mirrors the latest client AI
 /// request parameters (voice, timezone, library). Read by the nightly
 /// pre-generation scheduler (Section D) to pick up user preferences.
 pub trait UserAiPrefsDao: Sync + Send {
    /// Read the single row; `None` when it hasn't been populated yet.
    fn get_prefs(
        &mut self,
        context: &opentelemetry::Context,
    ) -> Result<Option<UserAiPrefs>, DbError>;
    /// Upsert the single row (id is always 1).
    #[allow(dead_code)]
    fn upsert_prefs(
        &mut self,
        context: &opentelemetry::Context,
        prefs: &UpsertUserAiPrefs,
    ) -> Result<(), DbError>;
 }
 pub struct SqliteUserAiPrefsDao {
    connection: Arc<Mutex<SqliteConnection>>,
 }
 impl Default for SqliteUserAiPrefsDao {
    fn default() -> Self {
        Self::new()
    }
 }
 impl SqliteUserAiPrefsDao {
    pub fn new() -> Self {
        Self {
            connection: Arc::new(Mutex::new(connect())),
        }
    }
    #[cfg(test)]
    pub fn from_connection(conn: Arc<Mutex<SqliteConnection>>) -> Self {
        Self { connection: conn }
    }
 }
 impl UserAiPrefsDao for SqliteUserAiPrefsDao {
    fn get_prefs(
        &mut self,
        context: &opentelemetry::Context,
    ) -> Result<Option<UserAiPrefs>, DbError> {
        trace_db_call(context, "query", "get_prefs", |_span| {
            use schema::user_ai_prefs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock UserAiPrefsDao");
            dsl::user_ai_prefs
                .first::<UserAiPrefs>(connection.deref_mut())
                .optional()
                .map_err(|e| anyhow::anyhow!("Failed to get prefs: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn upsert_prefs(
        &mut self,
        context: &opentelemetry::Context,
        prefs: &UpsertUserAiPrefs,
    ) -> Result<(), DbError> {
        trace_db_call(context, "upsert", "upsert_prefs", |_span| {
            use schema::user_ai_prefs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock UserAiPrefsDao");
            // Single-row table (id=1): one atomic upsert. The explicit id=1
            // makes the conflict target deterministic so the second call
            // updates in place rather than tripping the CHECK(id=1) constraint,
            // and real insert errors surface instead of being swallowed into a
            // separate update branch. The columns are set explicitly (rather
            // than via AsChangeset) so a None field overwrites to NULL — the
            // row mirrors the latest request exactly, not a merge of past ones.
            diesel::insert_into(dsl::user_ai_prefs)
                .values((dsl::id.eq(1), prefs))
                .on_conflict(dsl::id)
                .do_update()
                .set((
                    dsl::voice.eq(&prefs.voice),
                    dsl::tz_offset_minutes.eq(&prefs.tz_offset_minutes),
                    dsl::library.eq(&prefs.library),
                    dsl::updated_at.eq(&prefs.updated_at),
                ))
                .execute(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Failed to upsert prefs: {}", e))?;
            Ok(())
        })
        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use diesel::Connection;
    use diesel_migrations::{EmbeddedMigrations, MigrationHarness, embed_migrations};
    const DB_MIGRATIONS: EmbeddedMigrations = embed_migrations!();
    fn setup_dao() -> SqliteUserAiPrefsDao {
        let mut conn = SqliteConnection::establish(":memory:")
            .expect("Unable to create in-memory db connection");
        conn.run_pending_migrations(DB_MIGRATIONS)
            .expect("Failure running DB migrations");
        SqliteUserAiPrefsDao::from_connection(Arc::new(Mutex::new(conn)))
    }
    fn ctx() -> opentelemetry::Context {
        opentelemetry::Context::new()
    }
    #[test]
    fn get_prefs_returns_none_when_empty() {
        let mut dao = setup_dao();
        let result = dao.get_prefs(&ctx()).unwrap();
        assert!(result.is_none());
    }
    #[test]
    fn upsert_prefs_inserts_row() {
        let mut dao = setup_dao();
        let now = 1_700_000_000i64;
        let prefs = UpsertUserAiPrefs {
            voice: Some("grandma".to_string()),
            tz_offset_minutes: Some(-480),
            library: Some("1".to_string()),
            updated_at: now,
        };
        dao.upsert_prefs(&ctx(), &prefs).unwrap();
        let row = dao.get_prefs(&ctx()).unwrap().unwrap();
        assert_eq!(row.id, 1);
        assert_eq!(row.voice, Some("grandma".to_string()));
        assert_eq!(row.tz_offset_minutes, Some(-480));
        assert_eq!(row.library, Some("1".to_string()));
        assert_eq!(row.updated_at, now);
    }
    #[test]
    fn upsert_prefs_replaces_existing() {
        let mut dao = setup_dao();
        let now1 = 1_700_000_000i64;
        let now2 = 1_800_000_000i64;
        let prefs1 = UpsertUserAiPrefs {
            voice: Some("grandma".to_string()),
            tz_offset_minutes: Some(-480),
            library: Some("1".to_string()),
            updated_at: now1,
        };
        dao.upsert_prefs(&ctx(), &prefs1).unwrap();
        let prefs2 = UpsertUserAiPrefs {
            voice: Some("dad".to_string()),
            tz_offset_minutes: Some(-300),
            library: None,
            updated_at: now2,
        };
        dao.upsert_prefs(&ctx(), &prefs2).unwrap();
        let row = dao.get_prefs(&ctx()).unwrap().unwrap();
        assert_eq!(row.voice, Some("dad".to_string()));
        assert_eq!(row.tz_offset_minutes, Some(-300));
        assert!(row.library.is_none());
        assert_eq!(row.updated_at, now2);
    }
    #[test]
    fn upsert_partial_fields() {
        let mut dao = setup_dao();
        let now = 1_700_000_000i64;
        let prefs = UpsertUserAiPrefs {
            voice: None,
            tz_offset_minutes: Some(-480),
            library: None,
            updated_at: now,
        };
        dao.upsert_prefs(&ctx(), &prefs).unwrap();
        let row = dao.get_prefs(&ctx()).unwrap().unwrap();
        assert_eq!(row.tz_offset_minutes, Some(-480));
        assert!(row.voice.is_none());
        assert!(row.library.is_none());
    }
 }
@@ -230,21 +230,12 @@ fn exiftool_available() -> bool {
 /// One-file exiftool invocation. Used by the upload + GPS-write paths,
 /// which deal with one file at a time. The batch path uses
 /// `exiftool_dates_batch` so we don't pay subprocess startup per row.
 ///
 /// Notably absent: `-fast` / `-fast2`. For QuickTime/MP4 files whose
 /// `moov` atom sits at the end (non-faststart, common for Snapchat
 /// exports and any MP4 muxed without `-movflags +faststart`), `-fast2`
 /// causes exiftool to skip the trailer and return no `CreateDate` /
 /// `MediaCreateDate`, dropping us to the `fs_time` fallback for files
 /// that actually have a real capture date. We pre-filter to files that
 /// kamadak-exif couldn't read, so the JPEG fast-path is already covered
 /// — paying full-scan cost on the residual is the right trade.
 fn exiftool_date_single(path: &Path) -> Option<i64> {
    if !exiftool_available() {
        return None;
    }
    let mut cmd = Command::new("exiftool");
-    cmd.arg("-j").arg("-q").arg("-d").arg("%s");
+    cmd.arg("-j").arg("-q").arg("-d").arg("%s").arg("-fast2");
    for tag in EXIFTOOL_DATE_TAGS {
        cmd.arg(format!("-{}", tag));
    }
@@ -270,10 +261,7 @@ fn exiftool_dates_batch(paths: &[&Path]) -> HashMap<PathBuf, i64> {
    }
    let mut cmd = Command::new("exiftool");
-    // No `-fast2` — see exiftool_date_single for the rationale (QuickTime
+    cmd.arg("-j").arg("-q").arg("-d").arg("%s").arg("-fast2");
    // moov-at-end files miss CreateDate / MediaCreateDate when the trailer
    // is skipped).
    cmd.arg("-j").arg("-q").arg("-d").arg("%s");
    for tag in EXIFTOOL_DATE_TAGS {
        cmd.arg(format!("-{}", tag));
    }
@@ -234,7 +234,7 @@ async fn list_exact_handler(
    let span = global_tracer().start_with_context("duplicates.list_exact", &context);
    let span_context = opentelemetry::Context::current_with_span(span);
-    let library_id = libraries::resolve_library_param_state(&app_state, query.library.as_deref())
+    let library_id = libraries::resolve_library_param(&app_state, query.library.as_deref())
        .ok()
        .flatten()
        .map(|l| l.id);
@@ -265,7 +265,7 @@ async fn list_perceptual_handler(
    let span = global_tracer().start_with_context("duplicates.list_perceptual", &context);
    let span_context = opentelemetry::Context::current_with_span(span);
-    let library_id = libraries::resolve_library_param_state(&app_state, query.library.as_deref())
+    let library_id = libraries::resolve_library_param(&app_state, query.library.as_deref())
        .ok()
        .flatten()
        .map(|l| l.id);
@@ -449,7 +449,7 @@ async fn list_folder_pairs_handler(
    let span = global_tracer().start_with_context("duplicates.list_folder_pairs", &context);
    let span_context = opentelemetry::Context::current_with_span(span);
-    let library_id = libraries::resolve_library_param_state(&app_state, query.library.as_deref())
+    let library_id = libraries::resolve_library_param(&app_state, query.library.as_deref())
        .ok()
        .flatten()
        .map(|l| l.id);
@@ -47,7 +47,7 @@ use std::sync::{Arc, Mutex};
 /// Visual identity. The optional `entity_id` bridges this person to an
 /// LLM-extracted knowledge-graph entity (textual side). Persons are NOT
 /// auto-bridged at creation — only when the user explicitly links them in
-/// the management UI.
+/// the management UI, or when bootstrap finds an exact-name match.
 #[derive(Serialize, Queryable, Clone, Debug)]
 pub struct Person {
    pub id: i32,
@@ -366,10 +366,6 @@ pub struct EmbeddingsQuery {
    pub limit: i64,
    #[serde(default)]
    pub offset: i64,
    /// Restrict to one person's faces. Used by the similar-unassigned
    /// suggester to fetch a centroid pool. When set, takes precedence
    /// over `unassigned` (the more specific filter wins).
    pub person_id: Option<i32>,
 }
 fn default_unassigned() -> bool {
@@ -433,7 +429,6 @@ pub trait FaceDao: Send + Sync {
        ctx: &opentelemetry::Context,
        library_id: Option<i32>,
        unassigned: bool,
        person_id: Option<i32>,
        limit: i64,
        offset: i64,
    ) -> anyhow::Result<Vec<(FaceDetectionRow, String)>>;
@@ -508,10 +503,6 @@ pub trait FaceDao: Send + Sync {
        into: i32,
    ) -> anyhow::Result<Person>;
    /// Cheap presence probe — returns true iff at least one face has been
    /// detected (excluding marker rows). Used by chat-tool gating.
    fn has_any_faces(&mut self, ctx: &opentelemetry::Context) -> anyhow::Result<bool>;
    /// Resolve `(library_id, rel_path)` → `content_hash` via image_exif.
    /// Returns None when the photo hasn't been EXIF-indexed yet (no row
    /// in image_exif) or when the row exists but content_hash is NULL.
@@ -868,7 +859,6 @@ impl FaceDao for SqliteFaceDao {
        ctx: &opentelemetry::Context,
        library_id: Option<i32>,
        unassigned: bool,
        person_id: Option<i32>,
        limit: i64,
        offset: i64,
    ) -> anyhow::Result<Vec<(FaceDetectionRow, String)>> {
@@ -882,13 +872,7 @@ impl FaceDao for SqliteFaceDao {
            if let Some(lib) = library_id {
                query = query.filter(face_detections::library_id.eq(lib));
            }
-            // person_id is the more specific filter — when both it and
+            if unassigned {
            // `unassigned` are supplied, prefer the explicit person id and
            // ignore the IS NULL constraint (which would always return
            // empty for an assigned person).
            if let Some(pid) = person_id {
                query = query.filter(face_detections::person_id.eq(pid));
            } else if unassigned {
                query = query.filter(face_detections::person_id.is_null());
            }
            let rows = query
@@ -1024,12 +1008,7 @@ impl FaceDao for SqliteFaceDao {
                if let Some(lib) = library_id {
                    q = q.filter(face_detections::library_id.eq(lib));
                }
-                q.select(
+                q.select(diesel::dsl::count_distinct(face_detections::content_hash))
                    #[allow(deprecated)]
                    {
                        diesel::dsl::count_distinct(face_detections::content_hash)
                    },
                )
                    .first(conn.deref_mut())
                    .with_context(|| "stats: scanned")?
            };
@@ -1040,12 +1019,7 @@ impl FaceDao for SqliteFaceDao {
                if let Some(lib) = library_id {
                    q = q.filter(face_detections::library_id.eq(lib));
                }
-                q.select(
+                q.select(diesel::dsl::count_distinct(face_detections::content_hash))
                    #[allow(deprecated)]
                    {
                        diesel::dsl::count_distinct(face_detections::content_hash)
                    },
                )
                    .first(conn.deref_mut())
                    .with_context(|| "stats: with_faces")?
            };
@@ -1056,12 +1030,7 @@ impl FaceDao for SqliteFaceDao {
                if let Some(lib) = library_id {
                    q = q.filter(face_detections::library_id.eq(lib));
                }
-                q.select(
+                q.select(diesel::dsl::count_distinct(face_detections::content_hash))
                    #[allow(deprecated)]
                    {
                        diesel::dsl::count_distinct(face_detections::content_hash)
                    },
                )
                    .first(conn.deref_mut())
                    .with_context(|| "stats: no_faces")?
            };
@@ -1072,12 +1041,7 @@ impl FaceDao for SqliteFaceDao {
                if let Some(lib) = library_id {
                    q = q.filter(face_detections::library_id.eq(lib));
                }
-                q.select(
+                q.select(diesel::dsl::count_distinct(face_detections::content_hash))
                    #[allow(deprecated)]
                    {
                        diesel::dsl::count_distinct(face_detections::content_hash)
                    },
                )
                    .first(conn.deref_mut())
                    .with_context(|| "stats: failed")?
            };
@@ -1468,19 +1432,6 @@ impl FaceDao for SqliteFaceDao {
        })
    }
    fn has_any_faces(&mut self, ctx: &opentelemetry::Context) -> anyhow::Result<bool> {
        let mut conn = self.connection.lock().expect("face dao lock");
        trace_db_call(ctx, "query", "has_any_faces", |_span| {
            face_detections::table
                .filter(face_detections::status.eq("detected"))
                .select(face_detections::id)
                .first::<i32>(conn.deref_mut())
                .optional()
                .map(|x| x.is_some())
                .with_context(|| "has_any_faces query")
        })
    }
    fn resolve_content_hash(
        &mut self,
        ctx: &opentelemetry::Context,
@@ -1708,10 +1659,18 @@ where
                .route(web::get().to(list_persons_handler::<D>))
                .route(web::post().to(create_person_handler::<D>)),
        )
        .service(
            web::resource("/persons/bootstrap")
                .route(web::post().to(bootstrap_persons_handler::<D>)),
        )
        .service(
            web::resource("/persons/ignore-bucket")
                .route(web::post().to(ignore_bucket_handler::<D>)),
        )
        .service(
            web::resource("/tags/people-bootstrap-candidates")
                .route(web::get().to(bootstrap_candidates_handler::<D>)),
        )
        .service(
            web::resource("/persons/{id}")
                .route(web::get().to(get_person_handler::<D>))
@@ -1726,6 +1685,340 @@ where
        )
 }
 // ── Bootstrap (Phase 4) ─────────────────────────────────────────────────────
 #[derive(Serialize, Debug, Clone)]
 pub struct BootstrapCandidate {
    /// Display name — most-frequent capitalization across the case-insensitive
    /// group, or simply the first one seen if it's a tie.
    pub name: String,
    /// Lowercased name; the stable key for grouping and the auto-bind path.
    pub normalized_name: String,
    /// Sum of `tagged_photo` counts across all capitalizations of this name.
    pub usage_count: i64,
    /// Heuristic suggestion; the UI defaults this to checked but the user
    /// confirms before [`bootstrap_persons_handler`] actually creates rows.
    pub looks_like_person: bool,
    /// True when a `persons` row already exists for this name (any case).
    /// The UI hides these — re-running bootstrap is idempotent so it's fine
    /// either way, but the noise isn't worth showing.
    pub already_exists: bool,
 }
 #[derive(Serialize, Debug)]
 pub struct BootstrapCandidatesResponse {
    pub candidates: Vec<BootstrapCandidate>,
 }
 #[derive(Deserialize, Debug)]
 pub struct BootstrapPersonsReq {
    pub names: Vec<String>,
 }
 #[derive(Serialize, Debug)]
 pub struct BootstrapPersonsResponse {
    pub created: Vec<Person>,
    pub skipped: Vec<BootstrapSkipped>,
 }
 #[derive(Serialize, Debug)]
 pub struct BootstrapSkipped {
    pub name: String,
    pub reason: String,
 }
 /// Hard filter for the bootstrap candidate list. Returns true if the tag
 /// could plausibly be a person name; returns false to drop it from the
 /// candidates entirely (not just leave looks_like_person=false).
 ///
 /// Rules — all required:
 /// - At least 3 characters after trimming. Two-letter tags ("AB", "OK")
 ///   are almost always abbreviations or markers, not names.
 /// - No emoji or symbol-class characters. SQL-side string sort already
 ///   surfaces those at the top of the tag list; filtering them keeps
 ///   the candidate UI focused on names rather than chart-junk.
 /// - No control characters or null bytes.
 pub(crate) fn is_plausible_name_token(raw: &str) -> bool {
    let trimmed = raw.trim();
    if trimmed.chars().count() < 3 {
        return false;
    }
    for c in trimmed.chars() {
        // Letter / mark / decimal-digit / connector-punctuation /
        // dash / apostrophe / period / whitespace are all plausible in a
        // name. Anything else (emoji, symbols, math operators, arrows,
        // box drawing, control codes) disqualifies the whole tag.
        if c.is_alphabetic()
            || c.is_whitespace()
            || matches!(c, '\'' | '-' | '.' | '_' | '\u{2019}')
        {
            continue;
        }
        if c.is_ascii_digit() {
            // Digits don't disqualify here — `looks_like_person` rejects
            // them later, but `is_plausible_name_token` is just about
            // "could this be in the candidate list at all?". A tag like
            // "Sarah2" stays as a candidate (display-flagged not-a-person
            // by looks_like_person) so the operator can still spot and
            // confirm it manually if it's an alias.
            continue;
        }
        return false;
    }
    true
 }
 /// Conservative "this tag *might* be a person name" heuristic. False
 /// negatives are fine — the operator confirms in the UI before any row
 /// is created. False positives are also fine for the same reason; the
 /// goal is just to default sensible candidates to checked.
 ///
 /// Rules:
 /// - 1–2 whitespace-separated words
 /// - Each word starts with an uppercase character
 /// - No digits anywhere (rejects "Trip 2018", "2024", etc.)
 /// - Single-word names not on a small denylist of common non-person
 ///   tags (cat, christmas, beach, ...). Two-word names skip the
 ///   denylist because a real two-word person name is the dominant
 ///   case ("Sarah Smith") and false-blocking it is worse than false-
 ///   accepting "Sunset Walk".
 pub(crate) fn looks_like_person(raw: &str) -> bool {
    let trimmed = raw.trim();
    if trimmed.is_empty() {
        return false;
    }
    let words: Vec<&str> = trimmed.split_whitespace().collect();
    if !(1..=2).contains(&words.len()) {
        return false;
    }
    for w in &words {
        let Some(first) = w.chars().next() else {
            return false;
        };
        if !first.is_uppercase() {
            return false;
        }
        if w.chars().any(|c| c.is_ascii_digit()) {
            return false;
        }
    }
    if words.len() == 1 {
        const DENY: &[&str] = &[
            // Pets / animals
            "cat",
            "dog",
            "kitten",
            "puppy",
            "bird",
            "fish",
            "pet",
            "pets",
            // Events / occasions
            "birthday",
            "christmas",
            "halloween",
            "easter",
            "thanksgiving",
            "wedding",
            "anniversary",
            "vacation",
            "holiday",
            "party",
            "trip",
            "graduation",
            "concert",
            // Places (generic)
            "home",
            "work",
            "beach",
            "park",
            "hotel",
            "restaurant",
            "office",
            "house",
            "garden",
            // Subjects / styles
            "food",
            "sunset",
            "sunrise",
            "landscape",
            "portrait",
            "selfie",
            "nature",
            "flowers",
            "flower",
            "snow",
            "rain",
            "sky",
            // Buckets
            "untagged",
            "favorites",
            "favourites",
            "misc",
            "other",
            "random",
        ];
        let lower = trimmed.to_lowercase();
        if DENY.iter().any(|w| *w == lower) {
            return false;
        }
    }
    true
 }
 async fn bootstrap_candidates_handler<D: FaceDao>(
    _: Claims,
    request: HttpRequest,
    face_dao: web::Data<Mutex<D>>,
    tag_dao: web::Data<Mutex<crate::tags::SqliteTagDao>>,
 ) -> impl Responder {
    use std::collections::HashMap;
    let context = extract_context_from_request(&request);
    let span = global_tracer().start_with_context("faces.bootstrap_candidates", &context);
    let span_context = opentelemetry::Context::current_with_span(span);
    // All tags + their counts. Path filter unused — bootstrap is library-wide.
    let tags_with_counts = {
        let mut td = tag_dao.lock().expect("tag dao lock");
        match crate::tags::TagDao::get_all_tags(&mut *td, &span_context, None) {
            Ok(t) => t,
            Err(e) => return HttpResponse::InternalServerError().body(format!("{:#}", e)),
        }
    };
    // Group by lowercase name. Pick the most-frequent capitalization
    // for the display name (ties broken by first-seen). Filter out
    // short tags and tags carrying non-name characters (emojis, symbols)
    // before grouping — they're noise no operator would tick, so showing
    // them just makes the candidate list harder to scan.
    struct Group {
        display: String,
        display_freq: i64,
        total_count: i64,
    }
    let mut groups: HashMap<String, Group> = HashMap::new();
    for (count, tag) in tags_with_counts {
        if !is_plausible_name_token(&tag.name) {
            continue;
        }
        let lower = tag.name.to_lowercase();
        let g = groups.entry(lower).or_insert_with(|| Group {
            display: tag.name.clone(),
            display_freq: 0,
            total_count: 0,
        });
        g.total_count += count;
        if count > g.display_freq {
            g.display = tag.name.clone();
            g.display_freq = count;
        }
    }
    // Cross-reference against existing persons (bulk one-query lookup).
    let lower_names: Vec<String> = groups.keys().cloned().collect();
    let existing = {
        let mut fd = face_dao.lock().expect("face dao lock");
        match fd.find_persons_by_names_ci(&span_context, &lower_names) {
            Ok(m) => m,
            Err(e) => return HttpResponse::InternalServerError().body(format!("{:#}", e)),
        }
    };
    let mut candidates: Vec<BootstrapCandidate> = groups
        .into_iter()
        .map(|(lower, g)| BootstrapCandidate {
            looks_like_person: looks_like_person(&g.display),
            already_exists: existing.contains_key(&lower),
            name: g.display,
            normalized_name: lower,
            usage_count: g.total_count,
        })
        .collect();
    // Sort: persons-first heuristic by descending count, then alphabetical.
    // Persons-likely candidates surface near the top so the user doesn't
    // scroll past dozens of "vacation"-style tags to find them.
    candidates.sort_by(|a, b| {
        b.looks_like_person
            .cmp(&a.looks_like_person)
            .then(b.usage_count.cmp(&a.usage_count))
            .then(a.normalized_name.cmp(&b.normalized_name))
    });
    HttpResponse::Ok().json(BootstrapCandidatesResponse { candidates })
 }
 async fn bootstrap_persons_handler<D: FaceDao>(
    _: Claims,
    request: HttpRequest,
    body: web::Json<BootstrapPersonsReq>,
    face_dao: web::Data<Mutex<D>>,
 ) -> impl Responder {
    let context = extract_context_from_request(&request);
    let span = global_tracer().start_with_context("faces.bootstrap_persons", &context);
    let span_context = opentelemetry::Context::current_with_span(span);
    let mut created: Vec<Person> = Vec::new();
    let mut skipped: Vec<BootstrapSkipped> = Vec::new();
    let mut dao = face_dao.lock().expect("face dao lock");
    // Pre-fetch the existing-name set so a duplicate request reports
    // "already exists" (skipped) rather than firing N inserts that all
    // 409 against the UNIQUE COLLATE NOCASE constraint.
    let lower_names: Vec<String> = body.names.iter().map(|n| n.to_lowercase()).collect();
    let existing = match dao.find_persons_by_names_ci(&span_context, &lower_names) {
        Ok(m) => m,
        Err(e) => return HttpResponse::InternalServerError().body(format!("{:#}", e)),
    };
    for name in &body.names {
        let trimmed = name.trim();
        if trimmed.is_empty() {
            skipped.push(BootstrapSkipped {
                name: name.clone(),
                reason: "empty name".into(),
            });
            continue;
        }
        let lower = trimmed.to_lowercase();
        if existing.contains_key(&lower) {
            skipped.push(BootstrapSkipped {
                name: trimmed.to_string(),
                reason: "person already exists".into(),
            });
            continue;
        }
        match dao.create_person(
            &span_context,
            &CreatePersonReq {
                name: trimmed.to_string(),
                notes: None,
                entity_id: None,
                is_ignored: false,
            },
            /*from_tag*/ true,
        ) {
            Ok(p) => created.push(p),
            Err(e) => {
                if is_unique_violation(&e) {
                    // Race with a concurrent create; treat as skipped.
                    skipped.push(BootstrapSkipped {
                        name: trimmed.to_string(),
                        reason: "person already exists".into(),
                    });
                } else {
                    skipped.push(BootstrapSkipped {
                        name: trimmed.to_string(),
                        reason: format!("{:#}", e),
                    });
                }
            }
        }
    }
    HttpResponse::Ok().json(BootstrapPersonsResponse { created, skipped })
 }
 // ── Stats / list ────────────────────────────────────────────────────────────
 #[derive(Deserialize)]
@@ -1755,7 +2048,7 @@ async fn stats_handler<D: FaceDao>(
    let span = global_tracer().start_with_context("faces.stats", &context);
    let span_context = opentelemetry::Context::current_with_span(span);
-    let library_id = libraries::resolve_library_param_state(&app_state, query.library.as_deref())
+    let library_id = libraries::resolve_library_param(&app_state, query.library.as_deref())
        .ok()
        .flatten()
        .map(|l| l.id);
@@ -1782,8 +2075,7 @@ async fn list_faces_handler<D: FaceDao>(
    let normalized_path = normalize_path(&query.path);
    // resolve_library_param returns Option<&Library>; clone so the result
    // is owned (matching the primary_library fallback's type).
-    let library: Library =
+    let library: Library = libraries::resolve_library_param(&app_state, query.library.as_deref())
        libraries::resolve_library_param_state(&app_state, query.library.as_deref())
        .ok()
        .flatten()
        .cloned()
@@ -1823,7 +2115,6 @@ async fn embeddings_handler<D: FaceDao>(
        &span_context,
        query.library,
        query.unassigned,
        query.person_id,
        limit,
        offset,
    )
@@ -1871,7 +2162,7 @@ async fn create_face_handler<D: FaceDao>(
    }
    let normalized_path = normalize_path(&body.path);
-    let library: Library = match libraries::resolve_library_param_state(
+    let library: Library = match libraries::resolve_library_param(
        &app_state,
        body.library.as_ref().map(|i| i.to_string()).as_deref(),
    ) {
@@ -2039,19 +2330,12 @@ async fn update_face_handler<D: FaceDao>(
            match dao.get_face(&span_context, id) {
                Ok(Some(r)) => r,
                Ok(None) => return HttpResponse::NotFound().finish(),
-                Err(e) => {
+                Err(e) => return HttpResponse::InternalServerError().body(e.to_string()),
                    warn!("PATCH /image/faces/{}: 500 — get_face failed: {:#}", id, e);
                    return HttpResponse::InternalServerError().body(e.to_string());
                }
            }
        };
        let library = match app_state.library_by_id(current.library_id) {
            Some(l) => l.clone(),
            None => {
                warn!(
                    "PATCH /image/faces/{}: 500 — face row references unknown library_id {}",
                    id, current.library_id
                );
                return HttpResponse::InternalServerError().body(format!(
                    "face row references unknown library_id {}",
                    current.library_id
@@ -2134,17 +2418,7 @@ async fn update_face_handler<D: FaceDao>(
    let mut dao = face_dao.lock().expect("face dao lock");
    let row = match dao.update_face(&span_context, id, person_patch, bbox_patch, new_embedding) {
        Ok(r) => r,
-        Err(e) => {
+        Err(e) => return HttpResponse::InternalServerError().body(e.to_string()),
            // The full anyhow chain (`{:#}`) shows the diesel cause behind
            // the short context string we surface in the response body —
            // SQLITE_BUSY here usually means another DAO's writer held the
            // lock past `busy_timeout` (5s), which is invisible in `{}`.
            warn!(
                "PATCH /image/faces/{}: 500 — update_face failed: {:#}",
                id, e
            );
            return HttpResponse::InternalServerError().body(e.to_string());
        }
    };
    // Hydrate person_name so the response shape matches GET /image/faces
    // — the carousel overlay does an optimistic replace on this row, and
@@ -2152,13 +2426,7 @@ async fn update_face_handler<D: FaceDao>(
    // VFD label off the bbox even though the assignment didn't change.
    match hydrate_face_with_person(&mut *dao, &span_context, row) {
        Ok(joined) => HttpResponse::Ok().json(joined),
-        Err(e) => {
+        Err(e) => HttpResponse::InternalServerError().body(e.to_string()),
            warn!(
                "PATCH /image/faces/{}: 500 — hydrate_face_with_person failed: {:#}",
                id, e
            );
            HttpResponse::InternalServerError().body(e.to_string())
        }
    }
 }
@@ -2193,7 +2461,7 @@ async fn list_persons_handler<D: FaceDao>(
    let span = global_tracer().start_with_context("persons.list", &context);
    let span_context = opentelemetry::Context::current_with_span(span);
-    let library_id = libraries::resolve_library_param_state(&app_state, query.library.as_deref())
+    let library_id = libraries::resolve_library_param(&app_state, query.library.as_deref())
        .ok()
        .flatten()
        .map(|l| l.id);
@@ -2346,7 +2614,7 @@ async fn person_faces_handler<D: FaceDao>(
    let context = extract_context_from_request(&request);
    let span = global_tracer().start_with_context("persons.faces", &context);
    let span_context = opentelemetry::Context::current_with_span(span);
-    let library_id = libraries::resolve_library_param_state(&app_state, query.library.as_deref())
+    let library_id = libraries::resolve_library_param(&app_state, query.library.as_deref())
        .ok()
        .flatten()
        .map(|l| l.id);
@@ -2511,7 +2779,77 @@ mod tests {
        );
    }
-    // ── Phase 4: cosine + DAO support ───────────────────────────────────
+    // ── Phase 4: bootstrap heuristic + cosine + DAO support ─────────────
    #[test]
    fn is_plausible_name_token_filters_short_and_emoji() {
        // Hard filter applied before grouping — emojis and tags shorter
        // than 3 chars never make it into the candidate list, regardless
        // of looks_like_person's later assessment.
        assert!(is_plausible_name_token("Cameron"));
        assert!(is_plausible_name_token("Sarah Smith"));
        assert!(is_plausible_name_token("O'Brien"));
        assert!(is_plausible_name_token("Jean-Luc"));
        assert!(is_plausible_name_token("St. James"));
        assert!(is_plausible_name_token("Renée"));
        assert!(is_plausible_name_token("José"));
        // Asian script names — the alphabetic/letter check covers any
        // script, not just Latin.
        assert!(is_plausible_name_token("田中太郎"));
        // Below the 3-character floor.
        assert!(!is_plausible_name_token(""));
        assert!(!is_plausible_name_token(" "));
        assert!(!is_plausible_name_token("Bo"));
        assert!(!is_plausible_name_token("AB"));
        // Trim before counting — surrounding whitespace doesn't count.
        assert!(!is_plausible_name_token("  AB  "));
        // Emoji / symbol classes get the whole tag dropped.
        assert!(!is_plausible_name_token("🐱cat"));
        assert!(!is_plausible_name_token("Heart ❤"));
        assert!(!is_plausible_name_token("📸Photo"));
        assert!(!is_plausible_name_token("→ Trip"));
        assert!(!is_plausible_name_token("★Vacation"));
        // Digits are kept (handled by looks_like_person, not here).
        assert!(is_plausible_name_token("Trip 2018"));
        assert!(is_plausible_name_token("2024"));
    }
    #[test]
    fn looks_like_person_accepts_typical_names() {
        assert!(looks_like_person("Cameron"));
        assert!(looks_like_person("Sarah Smith"));
        assert!(looks_like_person("Mary Jane"));
        // Non-ASCII title-cased single word still counts.
        assert!(looks_like_person("Renée"));
    }
    #[test]
    fn looks_like_person_rejects_obvious_non_people() {
        // Digits, lowercase, three-or-more words, denylist hits.
        assert!(!looks_like_person("2018"));
        assert!(!looks_like_person("Trip 2018"));
        assert!(!looks_like_person("trip"));
        assert!(!looks_like_person("Birthday Party Cake"));
        assert!(!looks_like_person("cat"));
        assert!(!looks_like_person("Cat")); // denied even when title-cased
        assert!(!looks_like_person("Christmas"));
        assert!(!looks_like_person("home"));
        assert!(!looks_like_person(""));
        assert!(!looks_like_person("   "));
    }
    #[test]
    fn looks_like_person_two_words_skips_denylist() {
        // Two-word names get a pass on the single-word denylist —
        // "Sunset Walk" is much more likely a real album than a person,
        // but false-accepting is fine because the operator confirms.
        // What matters is we don't false-reject "Sarah Smith".
        assert!(looks_like_person("Sunset Walk"));
        assert!(looks_like_person("Sarah Smith"));
    }
    #[test]
    fn cosine_similarity_known_vectors() {
@@ -2984,87 +3322,6 @@ mod tests {
        assert_eq!(faces[0].person_id, Some(alice.id));
    }
    #[test]
    fn list_embeddings_filters_by_person_id() {
        // Apollo's similar-unassigned suggester relies on this filter to
        // pull a single person's embeddings without paging the whole
        // detected set client-side. When person_id is set it must win
        // over `unassigned=true` (otherwise the IS NULL constraint would
        // always return an empty set for an assigned person).
        let mut dao = fresh_dao();
        diesel::sql_query(
            "INSERT OR IGNORE INTO libraries (id, name, root_path, created_at) \
             VALUES (1, 'main', '/tmp', 0)",
        )
        .execute(dao.connection.lock().unwrap().deref_mut())
        .expect("seed libraries");
        let alice = dao
            .create_person(
                &ctx(),
                &CreatePersonReq {
                    name: "Alice".into(),
                    notes: None,
                    entity_id: None,
                    is_ignored: false,
                },
                false,
            )
            .unwrap();
        let bob = dao
            .create_person(
                &ctx(),
                &CreatePersonReq {
                    name: "Bob".into(),
                    notes: None,
                    entity_id: None,
                    is_ignored: false,
                },
                false,
            )
            .unwrap();
        let mk_row = |hash: &str, person: Option<i32>| InsertFaceDetectionInput {
            library_id: 1,
            content_hash: hash.into(),
            rel_path: format!("{hash}.jpg"),
            bbox: Some((0.1, 0.1, 0.2, 0.2)),
            embedding: Some(vec![0u8; 2048]),
            confidence: Some(0.9),
            source: "auto".into(),
            person_id: person,
            status: "detected".into(),
            model_version: "buffalo_l".into(),
        };
        dao.store_detection(&ctx(), mk_row("a1", Some(alice.id)))
            .unwrap();
        dao.store_detection(&ctx(), mk_row("a2", Some(alice.id)))
            .unwrap();
        dao.store_detection(&ctx(), mk_row("b1", Some(bob.id)))
            .unwrap();
        dao.store_detection(&ctx(), mk_row("u1", None)).unwrap();
        // person_id=alice returns only alice's two faces — ignoring the
        // (default-true) `unassigned` filter, which would have selected
        // u1 only.
        let alice_rows = dao
            .list_embeddings(&ctx(), None, true, Some(alice.id), 100, 0)
            .unwrap();
        assert_eq!(alice_rows.len(), 2);
        assert!(
            alice_rows
                .iter()
                .all(|(r, _)| r.person_id == Some(alice.id))
        );
        // unassigned=true with no person_id behaves as before.
        let unassigned_rows = dao
            .list_embeddings(&ctx(), None, true, None, 100, 0)
            .unwrap();
        assert_eq!(unassigned_rows.len(), 1);
        assert_eq!(unassigned_rows[0].0.content_hash, "u1");
    }
    // ── crop_image_to_bbox ──────────────────────────────────────────────
    // Pure helper used by the manual face-create handler. Generate a tiny
    // image in memory, write it to a temp file, then exercise the bbox
@@ -53,7 +53,6 @@ pub fn walk_library_files(base_path: &Path, excluded_dirs: &[String]) -> Vec<Dir
 /// used by the watcher's quick-scan tick to skip the long tail. Files
 /// whose metadata can't be read are kept; the caller's batch EXIF lookup
 /// dedups against existing rows.
 #[allow(dead_code)]
 pub fn enumerate_indexable_files(
    base_path: &Path,
    excluded_dirs: &[String],
@@ -22,42 +22,8 @@ pub fn needs_ffmpeg_thumbnail(path: &Path) -> bool {
 /// Supported video file extensions
 pub const VIDEO_EXTENSIONS: &[&str] = &["mp4", "mov", "avi", "mkv"];
 /// Audio file extensions accepted as voice-clone references (TTS). Mirrors
 /// the formats Chatterbox can decode (wav/mp3/flac/m4a/aac/ogg).
 pub const AUDIO_EXTENSIONS: &[&str] = &["wav", "mp3", "flac", "m4a", "aac", "ogg", "oga", "opus"];
 /// Filenames that are filesystem metadata, not real media — exact
 /// basename match. Extend if a new platform sidecar appears (Windows
 /// Thumbs.db / desktop.ini live here too if those libraries land).
 const METADATA_FILENAMES: &[&str] = &[".DS_Store"];
 /// True if the basename is a filesystem metadata sidecar that should be
 /// invisible to every media predicate.
 ///
 /// macOS writes `._<name>` AppleDouble companions when copying to
 /// non-HFS volumes — each holds the extended attributes of `<name>`,
 /// NOT a copy of the bytes. Same extension as the real file, so a
 /// pure-extension match treats `._photo.jpg` as a JPEG, ships it to
 /// the decoder, and accumulates failed rows: face_detections
 /// `status='failed'`, clip_embedding `status='failed'`, plus a
 /// pointless `image_exif` row whose `content_hash` will be the hash
 /// of the metadata blob. The downstream noise (failed-row counts that
 /// never go to zero, 422 bursts to Apollo, evictor timer reset by
 /// those 422s) is the visible damage. `.DS_Store` is the per-directory
 /// version (Finder view state) — no extension, but cheap to guard
 /// here too in case some future predicate matches by content type.
 pub fn is_filesystem_metadata(path: &Path) -> bool {
    let Some(name) = path.file_name().and_then(|n| n.to_str()) else {
        return false;
    };
    name.starts_with("._") || METADATA_FILENAMES.contains(&name)
 }
 /// Check if a path has an image extension
 pub fn is_image_file(path: &Path) -> bool {
    if is_filesystem_metadata(path) {
        return false;
    }
    if let Some(ext) = path.extension().and_then(|e| e.to_str()) {
        let ext_lower = ext.to_lowercase();
        IMAGE_EXTENSIONS.contains(&ext_lower.as_str())
@@ -68,9 +34,6 @@ pub fn is_image_file(path: &Path) -> bool {
 /// Check if a path has a video extension
 pub fn is_video_file(path: &Path) -> bool {
    if is_filesystem_metadata(path) {
        return false;
    }
    if let Some(ext) = path.extension().and_then(|e| e.to_str()) {
        let ext_lower = ext.to_lowercase();
        VIDEO_EXTENSIONS.contains(&ext_lower.as_str())
@@ -79,19 +42,6 @@ pub fn is_video_file(path: &Path) -> bool {
    }
 }
 /// Check if a path has an audio extension (voice-clone references)
 pub fn is_audio_file(path: &Path) -> bool {
    if is_filesystem_metadata(path) {
        return false;
    }
    if let Some(ext) = path.extension().and_then(|e| e.to_str()) {
        let ext_lower = ext.to_lowercase();
        AUDIO_EXTENSIONS.contains(&ext_lower.as_str())
    } else {
        false
    }
 }
 /// Check if a path has a supported media extension (image or video)
 pub fn is_media_file(path: &Path) -> bool {
    is_image_file(path) || is_video_file(path)
@@ -148,46 +98,4 @@ mod tests {
        assert!(!is_media_file(Path::new("document.txt")));
        assert!(!is_media_file(Path::new("no_extension")));
    }
    #[test]
    fn test_apple_double_excluded_from_media() {
        // The bug-of-record: ImageApi was shipping macOS AppleDouble
        // sidecars to Apollo's CLIP/face decoders, accumulating failed
        // rows and pinning Apollo's eviction timer with the 422 burst.
        // Predicate-level guard means every downstream walker
        // (face_watch, backfill, clip_watch, watcher) inherits the fix
        // without touching their filters.
        assert!(!is_image_file(Path::new("._photo.jpg")));
        assert!(!is_image_file(Path::new("dir/._photo.JPG")));
        assert!(!is_image_file(Path::new("a/b/._DSC_2182-S.jpg")));
        assert!(!is_video_file(Path::new("._video.mp4")));
        assert!(!is_media_file(Path::new("._photo.png")));
        // A real file that merely starts with "_" (no leading dot) is
        // not AppleDouble — must NOT be filtered.
        assert!(is_image_file(Path::new("_photo.jpg")));
    }
    #[test]
    fn test_ds_store_excluded() {
        // Finder per-directory metadata. No image extension so
        // is_image_file would already say false; the guard makes the
        // predicate's *reason* explicit and covers a hypothetical
        // future caller matching by basename.
        assert!(!is_image_file(Path::new(".DS_Store")));
        assert!(!is_video_file(Path::new(".DS_Store")));
        assert!(!is_media_file(Path::new("some/dir/.DS_Store")));
        assert!(is_filesystem_metadata(Path::new(".DS_Store")));
        assert!(is_filesystem_metadata(Path::new("dir/.DS_Store")));
    }
    #[test]
    fn test_dotfiles_other_than_apple_double_are_unaffected() {
        // We deliberately scope to `._*` + the exact .DS_Store name —
        // not all dotfiles — because a user could plausibly name a
        // cover image `.cover.jpg` and we shouldn't silently drop it.
        // If that turns out to be wrong, broaden here; for now,
        // narrow + explicit > broad + surprising.
        assert!(is_image_file(Path::new(".cover.jpg")));
        assert!(!is_filesystem_metadata(Path::new(".cover.jpg")));
    }
 }
@@ -10,7 +10,6 @@ use std::path::{Path, PathBuf};
 use std::sync::Mutex;
 use std::time::SystemTime;
 use crate::AppState;
 use crate::data::{
    Claims, ExifBatchRequest, ExifBatchResponse, ExifSummary, FilesRequest, FilterMode, MediaType,
    PhotosResponse, SortType,
@@ -19,8 +18,8 @@ use crate::database::ExifDao;
 use crate::file_types;
 use crate::geo::{gps_bounding_box, haversine_distance};
 use crate::memories::extract_date_from_filename;
 use crate::thumbnails::create_thumbnails;
 use crate::utils::earliest_fs_time;
 use crate::{AppState, create_thumbnails};
 use actix_web::web::Data;
 use actix_web::{
    HttpRequest, HttpResponse,
@@ -275,8 +274,8 @@ pub async fn list_photos<TagD: TagDao, FS: FileSystemAccess>(
    // Resolve the optional library filter. Unknown values return 400. A
    // `None` result means "union across all libraries" and downstream
    // walks iterate every configured library root.
-    let library =
+    let library = match crate::libraries::resolve_library_param(&app_state, req.library.as_deref())
-        match crate::libraries::resolve_library_param_state(&app_state, req.library.as_deref()) {
+    {
        Ok(lib) => lib,
        Err(msg) => {
            log::warn!("Rejecting /photos request: {}", msg);
@@ -1238,7 +1237,7 @@ pub async fn list_exif_summary(
    // Resolve the library filter up front so a bad id/name 400s before we
    // ever take the DAO mutex. None == union across all libraries.
    let library_filter =
-        match crate::libraries::resolve_library_param_state(&app_state, req.library.as_deref()) {
+        match crate::libraries::resolve_library_param(&app_state, req.library.as_deref()) {
            Ok(lib) => lib.map(|l| l.id),
            Err(msg) => {
                span.set_status(Status::error(msg.clone()));
@@ -1511,8 +1510,6 @@ mod tests {
            date_taken_source,
            original_date_taken: None,
            original_date_taken_source: None,
            clip_embedding: None,
            clip_model_version: None,
        }
    }
@@ -1552,8 +1549,6 @@ mod tests {
                date_taken_source: data.date_taken_source.clone(),
                original_date_taken: None,
                original_date_taken_source: None,
                clip_embedding: None,
                clip_model_version: None,
            })
        }
@@ -1600,8 +1595,6 @@ mod tests {
                date_taken_source: data.date_taken_source.clone(),
                original_date_taken: None,
                original_date_taken_source: None,
                clip_embedding: None,
                clip_model_version: None,
            })
        }
@@ -1695,21 +1688,6 @@ mod tests {
            Ok(())
        }
        fn list_distinct_content_hashes(
            &mut self,
            _context: &opentelemetry::Context,
        ) -> Result<Vec<String>, DbError> {
            Ok(Vec::new())
        }
        fn list_paths_and_hashes_for_library(
            &mut self,
            _context: &opentelemetry::Context,
            _library_id: i32,
        ) -> Result<Vec<(String, Option<String>)>, DbError> {
            Ok(Vec::new())
        }
        fn get_rows_needing_date_backfill(
            &mut self,
            _context: &opentelemetry::Context,
@@ -1938,35 +1916,6 @@ mod tests {
        ) -> Result<(), DbError> {
            Ok(())
        }
        fn list_clip_unencoded_candidates(
            &mut self,
            _context: &opentelemetry::Context,
            _library_id: i32,
            _limit: i64,
        ) -> Result<Vec<(String, String)>, DbError> {
            Ok(Vec::new())
        }
        fn backfill_clip_embedding(
            &mut self,
            _context: &opentelemetry::Context,
            _library_id: i32,
            _rel_path: &str,
            _embedding: &[u8],
            _model_version: &str,
        ) -> Result<(), DbError> {
            Ok(())
        }
        fn list_clip_index(
            &mut self,
            _context: &opentelemetry::Context,
            _library_ids: &[i32],
            _model_version: Option<&str>,
        ) -> Result<Vec<(String, Vec<u8>)>, DbError> {
            Ok(Vec::new())
        }
    }
    mod api {
@@ -1,5 +1,4 @@
 /// Geographic calculation utilities for GPS-based search
 use serde::Deserialize;
 use std::f64;
 /// Calculate distance between two GPS coordinates using the Haversine formula.
@@ -62,140 +61,6 @@ pub fn gps_bounding_box(lat: f64, lon: f64, radius_km: f64) -> (f64, f64, f64, f
    )
 }
 /// A place resolved from a free-text query via forward geocoding.
 ///
 /// The filter pipeline searches a *circle* (`gps_lat`/`gps_lon`/
 /// `gps_radius_km`), but a place can be anything from a single address to
 /// a whole country. We collapse Nominatim's bounding box into the smallest
 /// circle that circumscribes it (see [`bbox_to_circle`]) so "Portland" and
 /// "Italy" both map onto the existing circle filter without a schema change.
 #[derive(Debug, Clone, PartialEq)]
 pub struct GeoPlace {
    /// Nominatim's canonical name for the match (e.g. "Italia").
    pub display_name: String,
    /// Centroid latitude in decimal degrees.
    pub lat: f64,
    /// Centroid longitude in decimal degrees.
    pub lon: f64,
    /// Radius (km) of a circle centred on the centroid that covers the
    /// matched area. Floored to [`MIN_PLACE_RADIUS_KM`] so a point result
    /// (whose bounding box is microscopic) still yields a usable circle.
    pub radius_km: f64,
 }
 /// Floor for a geocoded place's radius. Point results (a street address)
 /// come back with a near-zero bounding box; without a floor the circle
 /// filter would match nothing.
 pub const MIN_PLACE_RADIUS_KM: f64 = 0.5;
 /// Collapse a bounding box into the centroid + circumscribing radius.
 ///
 /// Input is Nominatim's `boundingbox` order: `(south_lat, north_lat,
 /// west_lon, east_lon)`. The radius is the *largest* great-circle distance
 /// from the centroid to any of the four corners, so the resulting circle
 /// fully covers the box. (The corners aren't equidistant on a sphere —
 /// longitude lines converge toward the poles, so the equator-facing edge's
 /// corners are farthest; taking the max guarantees coverage in either
 /// hemisphere.)
 ///
 /// Pure and exact (no flooring) so it can be unit-tested directly; callers
 /// apply [`MIN_PLACE_RADIUS_KM`] when turning the result into a filter.
 pub fn bbox_to_circle(south: f64, north: f64, west: f64, east: f64) -> (f64, f64, f64) {
    let center_lat = (south + north) / 2.0;
    let center_lon = (west + east) / 2.0;
    let radius_km = [(south, west), (south, east), (north, west), (north, east)]
        .iter()
        .map(|(clat, clon)| haversine_distance(center_lat, center_lon, *clat, *clon))
        .fold(0.0_f64, f64::max);
    (center_lat, center_lon, radius_km)
 }
 /// Raw Nominatim `/search` result. `lat`/`lon` arrive as strings and
 /// `boundingbox` as a 4-element string array `[south, north, west, east]`.
 #[derive(Deserialize)]
 struct NominatimSearchResult {
    lat: String,
    lon: String,
    display_name: String,
    boundingbox: Option<[String; 4]>,
 }
 /// Forward-geocode a free-text place name to a [`GeoPlace`] via the public
 /// OpenStreetMap Nominatim `/search` endpoint.
 ///
 /// Mirrors `InsightGenerator::reverse_geocode`'s error posture: any network,
 /// HTTP, or parse failure returns `None` rather than propagating, so a flaky
 /// geocoder degrades the query to "no location filter" instead of failing it.
 ///
 /// Nominatim's usage policy requires a `User-Agent` and rate-limits to ~1
 /// request/second; callers doing this interactively should cache results.
 pub async fn forward_geocode(query: &str) -> Option<GeoPlace> {
    let q = query.trim();
    if q.is_empty() {
        return None;
    }
    let client = reqwest::Client::new();
    let response = match client
        .get("https://nominatim.openstreetmap.org/search")
        .query(&[("format", "json"), ("limit", "1"), ("q", q)])
        .header("User-Agent", "ImageAPI/1.0") // Nominatim requires User-Agent
        .send()
        .await
    {
        Ok(resp) => resp,
        Err(e) => {
            log::warn!("Forward geocoding network error for {q:?}: {e}");
            return None;
        }
    };
    if !response.status().is_success() {
        log::warn!(
            "Forward geocoding HTTP error for {q:?}: {}",
            response.status()
        );
        return None;
    }
    let results: Vec<NominatimSearchResult> = match response.json().await {
        Ok(r) => r,
        Err(e) => {
            log::warn!("Forward geocoding JSON parse error for {q:?}: {e}");
            return None;
        }
    };
    let top = results.into_iter().next()?;
    let lat: f64 = top.lat.parse().ok()?;
    let lon: f64 = top.lon.parse().ok()?;
    // Prefer the bounding box (handles large places); fall back to a
    // point + floor radius when Nominatim omits it.
    let (center_lat, center_lon, radius_km) = match &top.boundingbox {
        Some([s, n, w, e]) => match (s.parse(), n.parse(), w.parse(), e.parse()) {
            (Ok(s), Ok(n), Ok(w), Ok(e)) => bbox_to_circle(s, n, w, e),
            _ => (lat, lon, 0.0),
        },
        None => (lat, lon, 0.0),
    };
    let place = GeoPlace {
        display_name: top.display_name,
        lat: center_lat,
        lon: center_lon,
        radius_km: radius_km.max(MIN_PLACE_RADIUS_KM),
    };
    log::info!(
        "Forward geocoded {q:?} -> {} ({:.4}, {:.4}, r={:.1}km)",
        place.display_name,
        place.lat,
        place.lon,
        place.radius_km
    );
    Some(place)
 }
 #[cfg(test)]
 mod tests {
    use super::*;
@@ -253,41 +118,4 @@ mod tests {
            distance
        );
    }
    #[test]
    fn test_bbox_to_circle_centroid() {
        // Symmetric box around (10, 20): centroid should land dead centre.
        let (lat, lon, radius) = bbox_to_circle(9.0, 11.0, 19.0, 21.0);
        assert!((lat - 10.0).abs() < 1e-9, "centroid lat, got {lat}");
        assert!((lon - 20.0).abs() < 1e-9, "centroid lon, got {lon}");
        assert!(radius > 0.0, "radius should be positive, got {radius}");
    }
    #[test]
    fn test_bbox_to_circle_covers_corner() {
        // The radius must reach every corner of the box. Verify the
        // centroid-to-corner distance equals the returned radius for all
        // four corners (they're symmetric, so all equal).
        let (south, north, west, east) = (40.0, 42.0, -74.0, -72.0);
        let (lat, lon, radius) = bbox_to_circle(south, north, west, east);
        for (clat, clon) in [(south, west), (south, east), (north, west), (north, east)] {
            let d = haversine_distance(lat, lon, clat, clon);
            assert!(
                d <= radius + 1e-6,
                "corner ({clat},{clon}) at {d}km should be within radius {radius}km"
            );
        }
    }
    #[test]
    fn test_bbox_to_circle_country_vs_city_scale() {
        // A country-sized box yields a far larger radius than a city-sized
        // one — confirming the bbox approach scales with place size.
        let (_, _, country) = bbox_to_circle(35.5, 47.1, 6.6, 18.5); // ~Italy
        let (_, _, city) = bbox_to_circle(45.4, 45.6, -122.8, -122.5); // ~Portland
        assert!(
            country > city * 10.0,
            "country radius {country}km should dwarf city radius {city}km"
        );
    }
 }
@@ -1,128 +0,0 @@
 //! User-favorites endpoints. Favorites are keyed on `(user_id, rel_path)`
 //! and shared across libraries — a favorite created in lib1 is visible
 //! under lib2 if the same rel_path resolves there too.
 use std::sync::Mutex;
 use actix_web::{
    HttpRequest, HttpResponse, Responder, delete, get, put,
    web::{self, Data},
 };
 use log::{error, info, warn};
 use opentelemetry::trace::{Span, Status, Tracer};
 use crate::data::{AddFavoriteRequest, Claims, PhotosResponse};
 use crate::database::{DbError, DbErrorKind, FavoriteDao};
 use crate::otel::{extract_context_from_request, global_tracer};
 #[get("image/favorites")]
 pub async fn favorites(
    claims: Claims,
    request: HttpRequest,
    favorites_dao: Data<Mutex<Box<dyn FavoriteDao>>>,
 ) -> impl Responder {
    let tracer = global_tracer();
    let context = extract_context_from_request(&request);
    let mut span = tracer.start_with_context("get favorites", &context);
    match web::block(move || {
        favorites_dao
            .lock()
            .expect("Unable to get FavoritesDao")
            .get_favorites(claims.sub.parse::<i32>().unwrap())
    })
    .await
    {
        Ok(Ok(favorites)) => {
            let favorites = favorites
                .into_iter()
                .map(|favorite| favorite.path)
                .collect::<Vec<String>>();
            span.set_status(Status::Ok);
            // Favorites are library-agnostic (shared by rel_path), so we
            // intentionally leave photo_libraries empty to signal "no badge".
            HttpResponse::Ok().json(PhotosResponse {
                photos: favorites,
                dirs: Vec::new(),
                photo_libraries: Vec::new(),
                total_count: None,
                has_more: None,
                next_offset: None,
            })
        }
        Ok(Err(e)) => {
            span.set_status(Status::error(format!("Error getting favorites: {:?}", e)));
            error!("Error getting favorites: {:?}", e);
            HttpResponse::InternalServerError().finish()
        }
        Err(_) => HttpResponse::InternalServerError().finish(),
    }
 }
 #[put("image/favorites")]
 pub async fn put_add_favorite(
    claims: Claims,
    body: web::Json<AddFavoriteRequest>,
    favorites_dao: Data<Mutex<Box<dyn FavoriteDao>>>,
 ) -> impl Responder {
    if let Ok(user_id) = claims.sub.parse::<i32>() {
        let path = body.path.clone();
        match web::block::<_, Result<usize, DbError>>(move || {
            favorites_dao
                .lock()
                .expect("Unable to get FavoritesDao")
                .add_favorite(user_id, &path)
        })
        .await
        {
            Ok(Err(e)) if e.kind == DbErrorKind::AlreadyExists => {
                warn!("Favorite: {} exists for user: {}", &body.path, user_id);
                HttpResponse::Ok()
            }
            Ok(Err(e)) => {
                error!("{:?} {}. for user: {}", e, body.path, user_id);
                HttpResponse::BadRequest()
            }
            Ok(Ok(_)) => {
                info!("Adding favorite \"{}\" for userid: {}", body.path, user_id);
                HttpResponse::Created()
            }
            Err(e) => {
                error!("Blocking error while inserting favorite: {:?}", e);
                HttpResponse::InternalServerError()
            }
        }
    } else {
        error!("Unable to parse sub as i32: {}", claims.sub);
        HttpResponse::BadRequest()
    }
 }
 #[delete("image/favorites")]
 pub async fn delete_favorite(
    claims: Claims,
    body: web::Query<AddFavoriteRequest>,
    favorites_dao: Data<Mutex<Box<dyn FavoriteDao>>>,
 ) -> impl Responder {
    if let Ok(user_id) = claims.sub.parse::<i32>() {
        let path = body.path.clone();
        web::block(move || {
            favorites_dao
                .lock()
                .expect("Unable to get favorites dao")
                .remove_favorite(user_id, path);
        })
        .await
        .unwrap();
        info!(
            "Removing favorite \"{}\" for userid: {}",
            body.path, user_id
        );
        HttpResponse::Ok()
    } else {
        error!("Unable to parse sub as i32: {}", claims.sub);
        HttpResponse::BadRequest()
    }
 }
@@ -1,9 +0,0 @@
 //! HTTP route handlers, grouped by domain.
 //!
 //! These were previously inlined in `main.rs`; moving them out keeps
 //! `main()` focused on startup wiring and makes each domain
 //! independently testable with `actix_web::test::init_service`.
 pub mod favorites;
 pub mod image;
 pub mod video;
@@ -1,880 +0,0 @@
 //! Video-related endpoints: HLS playlist generation, segment streaming,
 //! and the short-clip preview pipeline.
 use std::collections::HashMap;
 use std::path::PathBuf;
 use std::sync::Mutex;
 use actix_files::NamedFile;
 use actix_web::{
    HttpRequest, HttpResponse, Responder, get, post,
    web::{self, Data},
 };
 use log::{debug, error, info, warn};
 use opentelemetry::KeyValue;
 use opentelemetry::trace::{Span, Status, Tracer};
 use serde::Serialize;
 use crate::content_hash;
 use crate::data::{
    Claims, PreviewClipRequest, PreviewStatusItem, PreviewStatusRequest, PreviewStatusResponse,
    ThumbnailRequest,
 };
 use crate::database::{ExifDao, PreviewDao};
 use crate::files::is_valid_full_path;
 use crate::libraries;
 use crate::otel::{extract_context_from_request, global_tracer};
 use crate::state::AppState;
 use crate::video::actors::{
    GeneratePreviewClipMessage, QueueVideosMessage, VideoToQueue, probe_video_stream_meta,
 };
 use crate::video::hls_paths;
 /// Response body for `POST /video/generate`. Clients consume
 /// `playlist_url` (hash-keyed, stable across libraries and renames)
 /// and poll for readiness via the URL itself.
 #[derive(Serialize, Debug)]
 struct GenerateVideoResponse {
    /// Hash-keyed URL to the HLS playlist. Resolves to
    /// `$VIDEO_PATH/<shard>/<hash>/playlist.m3u8` server-side. Relative
    /// segment refs inside the playlist resolve correctly because the
    /// browser appends to this URL's path.
    playlist_url: String,
    /// blake3 content hash of the source video. Stable per byte content,
    /// so duplicate uploads / archive ingests share one set of HLS
    /// output.
    content_hash: String,
    /// `true` iff the playlist file is already on disk. `false` means a
    /// transcode was queued; clients should retry the URL after a short
    /// delay (or rely on HLS.js's own retry policy).
    ready: bool,
    /// Source-video frame rate in Hz, probed via ffprobe. `None` when the
    /// probe failed or ffprobe couldn't parse either rate field — clients
    /// fall back to their own default (typically 30) for frame stepping.
    #[serde(skip_serializing_if = "Option::is_none")]
    frame_rate: Option<f32>,
 }
 #[post("/video/generate")]
 pub async fn generate_video(
    _claims: Claims,
    request: HttpRequest,
    app_state: Data<AppState>,
    exif_dao: Data<std::sync::Mutex<Box<dyn ExifDao>>>,
    body: web::Json<ThumbnailRequest>,
 ) -> impl Responder {
    let tracer = global_tracer();
    let context = extract_context_from_request(&request);
    let mut span = tracer.start_with_context("generate_video", &context);
    let preferred_library =
        libraries::resolve_library_param_state(&app_state, body.library.as_deref())
            .ok()
            .flatten()
            .unwrap_or_else(|| app_state.primary_library());
    // Try the resolved library first, then fall back to any other library
    // that actually contains the file — handles union-mode requests where
    // the mobile client passes no library but the file lives in a
    // non-primary library. Track which library won so the DB lookup is
    // scoped correctly.
    let resolved = is_valid_full_path(&preferred_library.root_path, &body.path, false)
        .filter(|p| p.exists())
        .map(|p| (preferred_library.id, preferred_library.root_path.clone(), p))
        .or_else(|| {
            app_state.libraries.iter().find_map(|lib| {
                if lib.id == preferred_library.id {
                    return None;
                }
                is_valid_full_path(&lib.root_path, &body.path, false)
                    .filter(|p| p.exists())
                    .map(|p| (lib.id, lib.root_path.clone(), p))
            })
        });
    let Some((resolved_library_id, resolved_root, full_path)) = resolved else {
        span.set_status(Status::error(format!("invalid path {:?}", &body.path)));
        return HttpResponse::BadRequest().finish();
    };
    // Build the rel_path used to look up the row. Forward-slash normalized
    // so the lookup matches DB rows on Windows — see `rel_path_for_lookup`.
    let full_path_str = full_path.to_string_lossy().to_string();
    let rel_path = rel_path_for_lookup(&full_path_str, &resolved_root);
    // DB lookup first. Cheap and avoids re-reading the file off disk for
    // already-ingested videos.
    let hash_from_db: Option<String> = {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        match dao.get_exif_batch(
            &context,
            Some(resolved_library_id),
            std::slice::from_ref(&rel_path),
        ) {
            Ok(rows) => rows.into_iter().next().and_then(|r| r.content_hash),
            Err(e) => {
                warn!(
                    "exif_dao.get_exif_batch failed for {} (lib {}): {:?}",
                    rel_path, resolved_library_id, e
                );
                None
            }
        }
    };
    // Best-effort fallback: compute on-the-fly when the DB row hasn't
    // been written or is mid-backfill. Read-only — no library mutation.
    let content_hash_str = match hash_from_db {
        Some(h) => h,
        None => match content_hash::compute(&full_path) {
            Ok(id) => id.content_hash,
            Err(e) => {
                error!(
                    "Failed to compute content_hash for {}: {}",
                    full_path.display(),
                    e
                );
                span.set_status(Status::error(format!("hash compute failed: {}", e)));
                return HttpResponse::InternalServerError().finish();
            }
        },
    };
    let video_dir = std::path::Path::new(&app_state.video_path);
    let playlist_path = hls_paths::playlist_for_hash(video_dir, &content_hash_str);
    let sentinel_path = hls_paths::sentinel_for_hash(video_dir, &content_hash_str);
    let ready = playlist_path.exists();
    if !ready && !sentinel_path.exists() {
        // Kick off generation via the existing actor pipeline. Fire-and-
        // forget — the playlist appears at `playlist_path` once ffmpeg
        // + rename complete. The client polls the URL.
        info!(
            "/video/generate: queueing playlist for {} (hash={})",
            full_path.display(),
            &content_hash_str[..content_hash_str.len().min(16)]
        );
        app_state.playlist_manager.do_send(QueueVideosMessage {
            videos: vec![VideoToQueue {
                video_path: full_path.clone(),
                content_hash: content_hash_str.clone(),
            }],
        });
        span.add_event(
            "playlist_queued",
            vec![KeyValue::new("content_hash", content_hash_str.clone())],
        );
    } else if ready {
        span.add_event(
            "playlist_already_present",
            vec![KeyValue::new("content_hash", content_hash_str.clone())],
        );
    } else {
        // Sentinel present — past transcode attempt failed. Return the
        // URL anyway (it'll 404 / 5xx at fetch time) so the client gets
        // a deterministic answer. Operator must delete the sentinel to
        // force a retry.
        warn!(
            "/video/generate: unsupported sentinel present for {} (hash={}); not re-queueing",
            full_path.display(),
            &content_hash_str[..content_hash_str.len().min(16)]
        );
    }
    let playlist_url = format!(
        "/video/hls/{}/{}",
        content_hash_str,
        hls_paths::PLAYLIST_FILENAME
    );
    // Probe the source for frame rate so the mobile scrubber can step at
    // the right interval. Cheap (~tens of ms) and only runs once per video
    // open. Probe failures degrade silently — clients have a fallback.
    let frame_rate = probe_video_stream_meta(&full_path.to_string_lossy())
        .await
        .frame_rate;
    span.set_status(Status::Ok);
    HttpResponse::Ok().json(GenerateVideoResponse {
        playlist_url,
        content_hash: content_hash_str,
        ready,
        frame_rate,
    })
 }
 /// Serve HLS playlist or segment files under the hash-keyed layout
 /// `$VIDEO_PATH/<shard>/<hash>/<file>`. The matched `{file}` must be
 /// either `playlist.m3u8` or a `segment_NNN.ts` style segment; any other
 /// shape is 400'd to defend against operators stashing other content in
 /// the hash dir.
 #[get("/video/hls/{hash}/{file}")]
 pub async fn stream_hls_file(
    request: HttpRequest,
    _: Claims,
    path: web::Path<(String, String)>,
    app_state: Data<AppState>,
 ) -> impl Responder {
    let tracer = global_tracer();
    let context = extract_context_from_request(&request);
    let mut span = tracer.start_with_context("stream_hls_file", &context);
    let (hash, file) = path.into_inner();
    if !is_valid_hash(&hash) {
        span.set_status(Status::error("invalid hash"));
        return HttpResponse::BadRequest().body("invalid hash");
    }
    if !is_allowed_hls_filename(&file) {
        span.set_status(Status::error("invalid file"));
        return HttpResponse::BadRequest().body("invalid file");
    }
    let shard = &hash[..2];
    let file_path = PathBuf::from(&app_state.video_path)
        .join(shard)
        .join(&hash)
        .join(&file);
    // Path-traversal guard: canonicalize both sides and require the file
    // to live under `app_state.video_path`. `is_valid_hash` /
    // `is_allowed_hls_filename` already block dangerous strings, but
    // belt-and-suspenders here is cheap.
    let canonical_base = match std::fs::canonicalize(&app_state.video_path) {
        Ok(p) => p,
        Err(e) => {
            error!("Failed to canonicalize VIDEO_PATH: {:?}", e);
            span.set_status(Status::error("VIDEO_PATH not canonicalisable"));
            return HttpResponse::InternalServerError().finish();
        }
    };
    let canonical_file = match std::fs::canonicalize(&file_path) {
        Ok(p) => p,
        Err(_) => {
            debug!("HLS file not found: {}", file_path.display());
            span.set_status(Status::error("not found"));
            return HttpResponse::NotFound().finish();
        }
    };
    if !canonical_file.starts_with(&canonical_base) {
        warn!(
            "Path traversal attempt: {} resolved outside VIDEO_PATH",
            file_path.display()
        );
        span.set_status(Status::error("traversal"));
        return HttpResponse::Forbidden().finish();
    }
    match NamedFile::open(&canonical_file) {
        Ok(f) => {
            span.set_status(Status::Ok);
            f.into_response(&request)
        }
        Err(_) => {
            span.set_status(Status::error("not found"));
            HttpResponse::NotFound().finish()
        }
    }
 }
 /// 64 lowercase-or-upper hex chars. Strict so we don't accept arbitrary
 /// strings that might canonicalize into trouble.
 fn is_valid_hash(s: &str) -> bool {
    s.len() == 64 && s.bytes().all(|b| b.is_ascii_hexdigit())
 }
 /// Compute the forward-slash `rel_path` used to look up a video's
 /// `image_exif` row, from its absolute path string and the library root.
 ///
 /// Normalizing to forward slashes is essential on Windows: `file_scan`
 /// stores rel_paths forward-slash regardless of OS, but a raw strip of a
 /// backslash Windows path (`Z:\...\pic\Melissa\clip.mp4`) yields
 /// `Melissa\clip.mp4`. `get_exif_batch` does an exact match with no
 /// normalization, so the backslash form misses and the handler falls back
 /// to re-hashing the entire file on every request.
 fn rel_path_for_lookup(full_path_str: &str, resolved_root: &str) -> String {
    full_path_str
        .strip_prefix(resolved_root)
        .unwrap_or(full_path_str)
        .trim_start_matches(['/', '\\'])
        .replace('\\', "/")
 }
 /// Allowed file names inside a hash dir. `playlist.m3u8` plus segment
 /// files matching the `segment_NNN.ts` template that `PlaylistGenerator`
 /// writes via `hls_paths::SEGMENT_TEMPLATE`. Anything else (including
 /// `.tmp`, `.unsupported`, dotfiles) returns 400 — these are internal
 /// artifacts the client should never request.
 fn is_allowed_hls_filename(name: &str) -> bool {
    if name == hls_paths::PLAYLIST_FILENAME {
        return true;
    }
    if let Some(rest) = name.strip_prefix("segment_")
        && let Some(num) = rest.strip_suffix(".ts")
        && !num.is_empty()
        && num.bytes().all(|b| b.is_ascii_digit())
    {
        return true;
    }
    false
 }
 #[get("/video/preview")]
 pub async fn get_video_preview(
    _claims: Claims,
    request: HttpRequest,
    req: web::Query<PreviewClipRequest>,
    app_state: Data<AppState>,
    preview_dao: Data<Mutex<Box<dyn PreviewDao>>>,
 ) -> impl Responder {
    let tracer = global_tracer();
    let context = extract_context_from_request(&request);
    let mut span = tracer.start_with_context("get_video_preview", &context);
    // Validate path
    let full_path = match is_valid_full_path(&app_state.base_path, &req.path, true) {
        Some(path) => path,
        None => {
            span.set_status(Status::error("Invalid path"));
            return HttpResponse::BadRequest().json(serde_json::json!({"error": "Invalid path"}));
        }
    };
    let full_path_str = full_path.to_string_lossy().to_string();
    // Use relative path (from BASE_PATH) for DB storage, consistent with EXIF convention
    let relative_path = full_path_str
        .strip_prefix(&app_state.base_path)
        .unwrap_or(&full_path_str)
        .trim_start_matches(['/', '\\'])
        .to_string();
    // Check preview status in DB
    let preview = {
        let mut dao = preview_dao.lock().expect("Unable to lock PreviewDao");
        dao.get_preview(&context, &relative_path)
    };
    match preview {
        Ok(Some(clip)) => match clip.status.as_str() {
            "complete" => {
                let preview_path = PathBuf::from(&app_state.preview_clips_path)
                    .join(&relative_path)
                    .with_extension("mp4");
                match NamedFile::open(&preview_path) {
                    Ok(file) => {
                        span.set_status(Status::Ok);
                        file.into_response(&request)
                    }
                    Err(_) => {
                        // File missing on disk but DB says complete - reset and regenerate
                        let mut dao = preview_dao.lock().expect("Unable to lock PreviewDao");
                        let _ = dao.update_status(
                            &context,
                            &relative_path,
                            "pending",
                            None,
                            None,
                            None,
                        );
                        app_state
                            .preview_clip_generator
                            .do_send(GeneratePreviewClipMessage {
                                video_path: full_path_str,
                            });
                        span.set_status(Status::Ok);
                        HttpResponse::Accepted().json(serde_json::json!({
                            "status": "processing",
                            "path": req.path
                        }))
                    }
                }
            }
            "processing" => {
                span.set_status(Status::Ok);
                HttpResponse::Accepted().json(serde_json::json!({
                    "status": "processing",
                    "path": req.path
                }))
            }
            "failed" => {
                let error_msg = clip
                    .error_message
                    .unwrap_or_else(|| "Unknown error".to_string());
                span.set_status(Status::error(format!("Generation failed: {}", error_msg)));
                HttpResponse::InternalServerError().json(serde_json::json!({
                    "error": format!("Generation failed: {}", error_msg)
                }))
            }
            _ => {
                // pending or unknown status - trigger generation
                app_state
                    .preview_clip_generator
                    .do_send(GeneratePreviewClipMessage {
                        video_path: full_path_str,
                    });
                span.set_status(Status::Ok);
                HttpResponse::Accepted().json(serde_json::json!({
                    "status": "processing",
                    "path": req.path
                }))
            }
        },
        Ok(None) => {
            // No record exists - insert as pending and trigger generation
            {
                let mut dao = preview_dao.lock().expect("Unable to lock PreviewDao");
                let _ = dao.insert_preview(&context, &relative_path, "pending");
            }
            app_state
                .preview_clip_generator
                .do_send(GeneratePreviewClipMessage {
                    video_path: full_path_str,
                });
            span.set_status(Status::Ok);
            HttpResponse::Accepted().json(serde_json::json!({
                "status": "processing",
                "path": req.path
            }))
        }
        Err(_) => {
            span.set_status(Status::error("Database error"));
            HttpResponse::InternalServerError().json(serde_json::json!({"error": "Database error"}))
        }
    }
 }
 #[post("/video/preview/status")]
 pub async fn get_preview_status(
    _claims: Claims,
    request: HttpRequest,
    body: web::Json<PreviewStatusRequest>,
    app_state: Data<AppState>,
    preview_dao: Data<Mutex<Box<dyn PreviewDao>>>,
 ) -> impl Responder {
    let tracer = global_tracer();
    let context = extract_context_from_request(&request);
    let mut span = tracer.start_with_context("get_preview_status", &context);
    // Limit to 200 paths per request
    if body.paths.len() > 200 {
        span.set_status(Status::error("Too many paths"));
        return HttpResponse::BadRequest()
            .json(serde_json::json!({"error": "Maximum 200 paths per request"}));
    }
    let previews = {
        let mut dao = preview_dao.lock().expect("Unable to lock PreviewDao");
        dao.get_previews_batch(&context, &body.paths)
    };
    match previews {
        Ok(clips) => {
            // Build a map of file_path -> VideoPreviewClip for quick lookup
            let clip_map: HashMap<String, _> = clips
                .into_iter()
                .map(|clip| (clip.file_path.clone(), clip))
                .collect();
            let mut items: Vec<PreviewStatusItem> = Vec::with_capacity(body.paths.len());
            for path in &body.paths {
                if let Some(clip) = clip_map.get(path) {
                    // Re-queue generation for stale pending/failed records
                    if clip.status == "pending" || clip.status == "failed" {
                        let full_path = format!(
                            "{}/{}",
                            app_state.base_path.trim_end_matches(['/', '\\']),
                            path.trim_start_matches(['/', '\\'])
                        );
                        app_state
                            .preview_clip_generator
                            .do_send(GeneratePreviewClipMessage {
                                video_path: full_path,
                            });
                    }
                    items.push(PreviewStatusItem {
                        path: path.clone(),
                        status: clip.status.clone(),
                        preview_url: if clip.status == "complete" {
                            Some(format!("/video/preview?path={}", urlencoding::encode(path)))
                        } else {
                            None
                        },
                    });
                } else {
                    // No record exists — insert as pending and trigger generation
                    {
                        let mut dao = preview_dao.lock().expect("Unable to lock PreviewDao");
                        let _ = dao.insert_preview(&context, path, "pending");
                    }
                    // Build full path for ffmpeg (actor needs the absolute path for input)
                    let full_path = format!(
                        "{}/{}",
                        app_state.base_path.trim_end_matches(['/', '\\']),
                        path.trim_start_matches(['/', '\\'])
                    );
                    info!("Triggering preview generation for '{}'", path);
                    app_state
                        .preview_clip_generator
                        .do_send(GeneratePreviewClipMessage {
                            video_path: full_path,
                        });
                    items.push(PreviewStatusItem {
                        path: path.clone(),
                        status: "pending".to_string(),
                        preview_url: None,
                    });
                }
            }
            span.set_status(Status::Ok);
            HttpResponse::Ok().json(PreviewStatusResponse { previews: items })
        }
        Err(_) => {
            span.set_status(Status::error("Database error"));
            HttpResponse::InternalServerError().json(serde_json::json!({"error": "Database error"}))
        }
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use crate::data::Claims;
    use crate::database::PreviewDao;
    use crate::testhelpers::TestPreviewDao;
    use actix_web::App;
    #[test]
    fn is_valid_hash_requires_64_ascii_hex() {
        assert!(is_valid_hash(&"a".repeat(64)));
        assert!(is_valid_hash(&"F".repeat(64)));
        assert!(is_valid_hash(&format!("ab{}", "0".repeat(62))));
        assert!(!is_valid_hash(&"a".repeat(63)));
        assert!(!is_valid_hash(&"a".repeat(65)));
        // Anything outside the hex alphabet — including '/', '.', '..' —
        // is rejected up front so the path-traversal canonicalisation
        // never has to defend the boundary alone.
        assert!(!is_valid_hash(&format!("/{}", "a".repeat(63))));
        assert!(!is_valid_hash(&format!("..{}", "a".repeat(62))));
        assert!(!is_valid_hash(&"g".repeat(64)));
    }
    #[test]
    fn is_allowed_hls_filename_accepts_only_playlist_and_segments() {
        assert!(is_allowed_hls_filename("playlist.m3u8"));
        assert!(is_allowed_hls_filename("segment_000.ts"));
        assert!(is_allowed_hls_filename("segment_999.ts"));
        assert!(is_allowed_hls_filename("segment_0.ts"));
        // Internal artifacts the client should never request.
        assert!(!is_allowed_hls_filename("playlist.m3u8.tmp"));
        assert!(!is_allowed_hls_filename("playlist.unsupported"));
        // Traversal / path components — defence in depth alongside
        // the actix path matcher itself.
        assert!(!is_allowed_hls_filename(".."));
        assert!(!is_allowed_hls_filename("../etc/passwd"));
        assert!(!is_allowed_hls_filename("segment_abc.ts"));
        assert!(!is_allowed_hls_filename("segment_.ts"));
        assert!(!is_allowed_hls_filename(""));
    }
    #[test]
    fn rel_path_for_lookup_normalizes_windows_separators() {
        // Windows: backslash root + backslash full path. The stored row is
        // forward-slash (`Melissa/clip.mp4`), so without normalization the
        // lookup misses and the handler re-hashes the whole file.
        assert_eq!(
            rel_path_for_lookup(r"Z:\Media\pic\Melissa\clip.mp4", r"Z:\Media\pic"),
            "Melissa/clip.mp4"
        );
    }
    #[test]
    fn rel_path_for_lookup_handles_unix_separators() {
        assert_eq!(
            rel_path_for_lookup("/media/pic/Melissa/clip.mp4", "/media/pic"),
            "Melissa/clip.mp4"
        );
    }
    #[test]
    fn rel_path_for_lookup_file_at_root_has_no_separator() {
        // A file directly in the library root has no internal separator, so
        // the bug never manifested here — guard against a regression anyway.
        assert_eq!(
            rel_path_for_lookup(r"Z:\Media\pic\clip.mp4", r"Z:\Media\pic"),
            "clip.mp4"
        );
        assert_eq!(
            rel_path_for_lookup("/media/pic/clip.mp4", "/media/pic"),
            "clip.mp4"
        );
    }
    #[test]
    fn rel_path_for_lookup_strips_leading_separators() {
        // Both separator styles are trimmed from the front after the root
        // is stripped, regardless of which form the join produced.
        assert_eq!(
            rel_path_for_lookup(r"Z:\Media\pic\sub\a.mp4", r"Z:\Media\pic"),
            "sub/a.mp4"
        );
        assert_eq!(
            rel_path_for_lookup("/media/pic//sub/a.mp4", "/media/pic"),
            "sub/a.mp4"
        );
    }
    #[test]
    fn rel_path_for_lookup_falls_back_when_root_does_not_match() {
        // If the root doesn't prefix the path (e.g. a stale mount), we keep
        // the whole path but still normalize separators rather than panic.
        assert_eq!(
            rel_path_for_lookup(r"D:\other\Melissa\clip.mp4", r"Z:\Media\pic"),
            "D:/other/Melissa/clip.mp4"
        );
    }
    fn make_token() -> String {
        let claims = Claims::valid_user("1".to_string());
        jsonwebtoken::encode(
            &jsonwebtoken::Header::default(),
            &claims,
            &jsonwebtoken::EncodingKey::from_secret(b"test_key"),
        )
        .unwrap()
    }
    fn make_preview_dao(dao: TestPreviewDao) -> Data<Mutex<Box<dyn PreviewDao>>> {
        Data::new(Mutex::new(Box::new(dao) as Box<dyn PreviewDao>))
    }
    #[actix_rt::test]
    async fn test_get_preview_status_returns_pending_for_unknown() {
        let dao = TestPreviewDao::new();
        let preview_dao = make_preview_dao(dao);
        let app_state = Data::new(AppState::test_state());
        let token = make_token();
        let app = actix_web::test::init_service(
            App::new()
                .service(get_preview_status)
                .app_data(app_state)
                .app_data(preview_dao.clone()),
        )
        .await;
        let req = actix_web::test::TestRequest::post()
            .uri("/video/preview/status")
            .insert_header(("Authorization", format!("Bearer {}", token)))
            .set_json(serde_json::json!({"paths": ["photos/new_video.mp4"]}))
            .to_request();
        let resp = actix_web::test::call_service(&app, req).await;
        assert_eq!(resp.status(), 200);
        let body: serde_json::Value = actix_web::test::read_body_json(resp).await;
        let previews = body["previews"].as_array().unwrap();
        assert_eq!(previews.len(), 1);
        assert_eq!(previews[0]["status"], "pending");
        // Verify the DAO now has a pending record
        let mut dao_lock = preview_dao.lock().unwrap();
        let ctx = opentelemetry::Context::new();
        let clip = dao_lock.get_preview(&ctx, "photos/new_video.mp4").unwrap();
        assert!(clip.is_some());
        assert_eq!(clip.unwrap().status, "pending");
    }
    #[actix_rt::test]
    async fn test_get_preview_status_returns_complete_with_url() {
        let mut dao = TestPreviewDao::new();
        let ctx = opentelemetry::Context::new();
        dao.insert_preview(&ctx, "photos/done.mp4", "pending")
            .unwrap();
        dao.update_status(
            &ctx,
            "photos/done.mp4",
            "complete",
            Some(9.5),
            Some(500000),
            None,
        )
        .unwrap();
        let preview_dao = make_preview_dao(dao);
        let app_state = Data::new(AppState::test_state());
        let token = make_token();
        let app = actix_web::test::init_service(
            App::new()
                .service(get_preview_status)
                .app_data(app_state)
                .app_data(preview_dao),
        )
        .await;
        let req = actix_web::test::TestRequest::post()
            .uri("/video/preview/status")
            .insert_header(("Authorization", format!("Bearer {}", token)))
            .set_json(serde_json::json!({"paths": ["photos/done.mp4"]}))
            .to_request();
        let resp = actix_web::test::call_service(&app, req).await;
        assert_eq!(resp.status(), 200);
        let body: serde_json::Value = actix_web::test::read_body_json(resp).await;
        let previews = body["previews"].as_array().unwrap();
        assert_eq!(previews.len(), 1);
        assert_eq!(previews[0]["status"], "complete");
        assert!(
            previews[0]["preview_url"]
                .as_str()
                .unwrap()
                .contains("photos%2Fdone.mp4")
        );
    }
    #[actix_rt::test]
    async fn test_get_preview_status_rejects_over_200_paths() {
        let dao = TestPreviewDao::new();
        let preview_dao = make_preview_dao(dao);
        let app_state = Data::new(AppState::test_state());
        let token = make_token();
        let app = actix_web::test::init_service(
            App::new()
                .service(get_preview_status)
                .app_data(app_state)
                .app_data(preview_dao),
        )
        .await;
        let paths: Vec<String> = (0..201).map(|i| format!("video_{}.mp4", i)).collect();
        let req = actix_web::test::TestRequest::post()
            .uri("/video/preview/status")
            .insert_header(("Authorization", format!("Bearer {}", token)))
            .set_json(serde_json::json!({"paths": paths}))
            .to_request();
        let resp = actix_web::test::call_service(&app, req).await;
        assert_eq!(resp.status(), 400);
    }
    #[actix_rt::test]
    async fn test_get_preview_status_mixed_statuses() {
        let mut dao = TestPreviewDao::new();
        let ctx = opentelemetry::Context::new();
        dao.insert_preview(&ctx, "a.mp4", "pending").unwrap();
        dao.insert_preview(&ctx, "b.mp4", "pending").unwrap();
        dao.update_status(&ctx, "b.mp4", "complete", Some(10.0), Some(100000), None)
            .unwrap();
        let preview_dao = make_preview_dao(dao);
        let app_state = Data::new(AppState::test_state());
        let token = make_token();
        let app = actix_web::test::init_service(
            App::new()
                .service(get_preview_status)
                .app_data(app_state)
                .app_data(preview_dao),
        )
        .await;
        let req = actix_web::test::TestRequest::post()
            .uri("/video/preview/status")
            .insert_header(("Authorization", format!("Bearer {}", token)))
            .set_json(serde_json::json!({"paths": ["a.mp4", "b.mp4", "c.mp4"]}))
            .to_request();
        let resp = actix_web::test::call_service(&app, req).await;
        assert_eq!(resp.status(), 200);
        let body: serde_json::Value = actix_web::test::read_body_json(resp).await;
        let previews = body["previews"].as_array().unwrap();
        assert_eq!(previews.len(), 3);
        // a.mp4 is pending
        assert_eq!(previews[0]["path"], "a.mp4");
        assert_eq!(previews[0]["status"], "pending");
        // b.mp4 is complete with URL
        assert_eq!(previews[1]["path"], "b.mp4");
        assert_eq!(previews[1]["status"], "complete");
        assert!(previews[1]["preview_url"].is_string());
        // c.mp4 was not found — handler inserts pending
        assert_eq!(previews[2]["path"], "c.mp4");
        assert_eq!(previews[2]["status"], "pending");
    }
    /// Verifies that the status endpoint re-queues generation for stale
    /// "pending" and "failed" records (e.g., after a server restart or
    /// when clip files were deleted). The do_send to the actor exercises
    /// the re-queue code path; the actor runs against temp dirs so it
    /// won't panic.
    #[actix_rt::test]
    async fn test_get_preview_status_requeues_pending_and_failed() {
        let mut dao = TestPreviewDao::new();
        let ctx = opentelemetry::Context::new();
        // Simulate stale records left from a previous server run
        dao.insert_preview(&ctx, "stale/pending.mp4", "pending")
            .unwrap();
        dao.insert_preview(&ctx, "stale/failed.mp4", "pending")
            .unwrap();
        dao.update_status(
            &ctx,
            "stale/failed.mp4",
            "failed",
            None,
            None,
            Some("ffmpeg error"),
        )
        .unwrap();
        let preview_dao = make_preview_dao(dao);
        let app_state = Data::new(AppState::test_state());
        let token = make_token();
        let app = actix_web::test::init_service(
            App::new()
                .service(get_preview_status)
                .app_data(app_state)
                .app_data(preview_dao),
        )
        .await;
        let req = actix_web::test::TestRequest::post()
            .uri("/video/preview/status")
            .insert_header(("Authorization", format!("Bearer {}", token)))
            .set_json(serde_json::json!({
                "paths": ["stale/pending.mp4", "stale/failed.mp4"]
            }))
            .to_request();
        let resp = actix_web::test::call_service(&app, req).await;
        assert_eq!(resp.status(), 200);
        let body: serde_json::Value = actix_web::test::read_body_json(resp).await;
        let previews = body["previews"].as_array().unwrap();
        assert_eq!(previews.len(), 2);
        // Both records are returned with their current status
        assert_eq!(previews[0]["path"], "stale/pending.mp4");
        assert_eq!(previews[0]["status"], "pending");
        assert!(previews[0].get("preview_url").is_none());
        assert_eq!(previews[1]["path"], "stale/failed.mp4");
        assert_eq!(previews[1]["status"], "failed");
        assert!(previews[1].get("preview_url").is_none());
    }
 }
@@ -1,409 +0,0 @@
 //! Per-library HLS readiness: Prometheus gauges + `/hls/stats` endpoint.
 //!
 //! The new hash-keyed pipeline transcodes lazily — most of a freshly
 //! mounted library is "pending" for the first hour, and operators want
 //! a live read on "how much work is left, am I CPU-bound, do I need to
 //! bump `HLS_CONCURRENCY`." This module supplies both surfaces against
 //! the same compute path:
 //!
 //! - **Prometheus gauges** `imageserver_hls_videos_total{library}`,
 //!   `..._with_playlist{library}`, `..._pending{library}`,
 //!   `..._unsupported{library}`. Updated every watcher full-scan tick
 //!   and on every `/hls/stats` request, so the freshness matches
 //!   whichever surface the operator is watching.
 //!
 //! - **`GET /hls/stats`** returns a JSON snapshot of the same counts
 //!   plus a top-level cross-library aggregate. Claims-protected
 //!   (matches every other authenticated read in this crate).
 //!
 //! Cost is O(distinct video hashes per library), each row needing a
 //! single `stat()` on the playlist file. On a 100k-video library that's
 //! noticeable; on a typical home library (few thousand) it's noise.
 //! We call from explicit triggers only — never per-request from
 //! middleware — so the cost is bounded.
 use std::collections::HashSet;
 use std::path::Path;
 use std::sync::{Arc, Mutex};
 use actix_web::{HttpResponse, Responder, get, web};
 use lazy_static::lazy_static;
 use log::{info, warn};
 use prometheus::IntGaugeVec;
 use serde::Serialize;
 use crate::data::Claims;
 use crate::database::ExifDao;
 use crate::file_types;
 use crate::libraries::Library;
 use crate::state::AppState;
 use crate::video::hls_paths;
 lazy_static! {
    pub static ref HLS_VIDEOS_TOTAL: IntGaugeVec = IntGaugeVec::new(
        prometheus::Opts::new(
            "imageserver_hls_videos_total",
            "Distinct video content hashes per library known to image_exif",
        ),
        &["library"],
    )
    .expect("HLS_VIDEOS_TOTAL");
    pub static ref HLS_VIDEOS_WITH_PLAYLIST: IntGaugeVec = IntGaugeVec::new(
        prometheus::Opts::new(
            "imageserver_hls_videos_with_playlist",
            "Videos whose hash-keyed HLS playlist is already on disk",
        ),
        &["library"],
    )
    .expect("HLS_VIDEOS_WITH_PLAYLIST");
    pub static ref HLS_VIDEOS_PENDING: IntGaugeVec = IntGaugeVec::new(
        prometheus::Opts::new(
            "imageserver_hls_videos_pending",
            "Videos whose hash-keyed HLS playlist is not yet on disk",
        ),
        &["library"],
    )
    .expect("HLS_VIDEOS_PENDING");
    pub static ref HLS_VIDEOS_UNSUPPORTED: IntGaugeVec = IntGaugeVec::new(
        prometheus::Opts::new(
            "imageserver_hls_videos_unsupported",
            "Videos with an `.unsupported` sentinel — ffmpeg refused; \
             operator must delete to retry",
        ),
        &["library"],
    )
    .expect("HLS_VIDEOS_UNSUPPORTED");
 }
 /// Per-library HLS readiness snapshot.
 #[derive(Serialize, Debug, Clone, PartialEq, Eq)]
 pub struct HlsLibraryStats {
    pub library_id: i32,
    pub library: String,
    /// Distinct video content hashes (dedupes intra-library bytes-at-N-paths).
    pub total: usize,
    /// Of `total`, hashes whose `playlist.m3u8` is on disk.
    pub with_playlist: usize,
    /// Of `total`, hashes whose ffmpeg attempt left a `.unsupported`
    /// sentinel. Counted separately because they won't progress without
    /// operator intervention (delete the sentinel to retry).
    pub unsupported: usize,
    /// `total - (with_playlist + unsupported)` — videos awaiting transcode.
    pub pending: usize,
    /// Distinct rel_paths under this library that are video files but
    /// whose `image_exif.content_hash` is still NULL (mid-backfill).
    /// These don't yet count toward `total` because they're invisible
    /// to the hash-keyed pipeline; surfaced so the operator can see
    /// "hash backfill, then transcode" pipeline depth.
    pub hashless_videos: usize,
 }
 /// JSON response body for `GET /hls/stats`.
 #[derive(Serialize, Debug)]
 pub struct HlsStatsResponse {
    pub libraries: Vec<HlsLibraryStats>,
    pub total: usize,
    pub with_playlist: usize,
    pub pending: usize,
    pub unsupported: usize,
    pub hashless_videos: usize,
 }
 /// Compute current readiness per library and publish to Prometheus.
 /// Returns the same data so callers can serialise it. The publish step
 /// is idempotent on the gauge — old values get overwritten.
 pub fn compute_and_publish(
    libraries: &[Library],
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
    video_dir: &Path,
 ) -> Vec<HlsLibraryStats> {
    let ctx = opentelemetry::Context::new();
    let mut out = Vec::with_capacity(libraries.len());
    for lib in libraries {
        let stats = compute_for_library(&ctx, lib, exif_dao, video_dir);
        publish_gauges(&stats);
        out.push(stats);
    }
    out
 }
 fn publish_gauges(s: &HlsLibraryStats) {
    HLS_VIDEOS_TOTAL
        .with_label_values(&[s.library.as_str()])
        .set(s.total as i64);
    HLS_VIDEOS_WITH_PLAYLIST
        .with_label_values(&[s.library.as_str()])
        .set(s.with_playlist as i64);
    HLS_VIDEOS_PENDING
        .with_label_values(&[s.library.as_str()])
        .set(s.pending as i64);
    HLS_VIDEOS_UNSUPPORTED
        .with_label_values(&[s.library.as_str()])
        .set(s.unsupported as i64);
 }
 fn compute_for_library(
    ctx: &opentelemetry::Context,
    lib: &Library,
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
    video_dir: &Path,
 ) -> HlsLibraryStats {
    let rows = {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        match dao.list_paths_and_hashes_for_library(ctx, lib.id) {
            Ok(r) => r,
            Err(e) => {
                warn!(
                    "hls_stats: list_paths_and_hashes_for_library failed for lib {}: {:?}",
                    lib.id, e
                );
                Vec::new()
            }
        }
    };
    stats_from_rows(lib, &rows, video_dir)
 }
 /// Pure function — same compute as [`compute_for_library`] but works
 /// on caller-supplied rows. Split out so tests don't need a full
 /// `ExifDao` mock; the integration path is exercised through
 /// `compute_and_publish` against the real SQLite DAO at runtime.
 fn stats_from_rows(
    lib: &Library,
    rows: &[(String, Option<String>)],
    video_dir: &Path,
 ) -> HlsLibraryStats {
    let mut hashes: HashSet<String> = HashSet::new();
    let mut hashless_videos = 0usize;
    for (rel_path, hash_opt) in rows {
        if !file_types::is_video_file(Path::new(rel_path)) {
            continue;
        }
        match hash_opt {
            Some(h) => {
                hashes.insert(h.clone());
            }
            None => {
                hashless_videos += 1;
            }
        }
    }
    let mut with_playlist = 0usize;
    let mut unsupported = 0usize;
    for h in &hashes {
        if hls_paths::playlist_for_hash(video_dir, h).exists() {
            with_playlist += 1;
        } else if hls_paths::sentinel_for_hash(video_dir, h).exists() {
            unsupported += 1;
        }
    }
    let total = hashes.len();
    let pending = total.saturating_sub(with_playlist + unsupported);
    HlsLibraryStats {
        library_id: lib.id,
        library: lib.name.clone(),
        total,
        with_playlist,
        unsupported,
        pending,
        hashless_videos,
    }
 }
 /// Log a single info line summarising readiness across all libraries.
 /// Called by the watcher at the end of a full-scan tick so operators
 /// who tail the log see the headline number without scraping
 /// Prometheus.
 pub fn log_summary(stats: &[HlsLibraryStats]) {
    let total: usize = stats.iter().map(|s| s.total).sum();
    let with_playlist: usize = stats.iter().map(|s| s.with_playlist).sum();
    let pending: usize = stats.iter().map(|s| s.pending).sum();
    let unsupported: usize = stats.iter().map(|s| s.unsupported).sum();
    let hashless: usize = stats.iter().map(|s| s.hashless_videos).sum();
    let per_lib: Vec<String> = stats
        .iter()
        .map(|s| {
            format!(
                "{}={}/{} pending={} unsupported={} hashless={}",
                s.library, s.with_playlist, s.total, s.pending, s.unsupported, s.hashless_videos,
            )
        })
        .collect();
    info!(
        "HLS readiness: {}/{} playlists on disk, {} pending, {} unsupported, {} hashless videos | per-library: [{}]",
        with_playlist,
        total,
        pending,
        unsupported,
        hashless,
        per_lib.join(", "),
    );
 }
 #[get("/hls/stats")]
 pub async fn hls_stats_handler(
    _claims: Claims,
    app_state: web::Data<AppState>,
    exif_dao: web::Data<Mutex<Box<dyn ExifDao>>>,
 ) -> impl Responder {
    let libraries = app_state.libraries.clone();
    let video_dir = std::path::PathBuf::from(&app_state.video_path);
    let exif_dao = exif_dao.into_inner();
    // Synchronous file IO + DB query — run on a blocking pool so the
    // actix worker thread stays free for other requests.
    let stats =
        match web::block(move || compute_and_publish(&libraries, &exif_dao, &video_dir)).await {
            Ok(s) => s,
            Err(e) => {
                warn!("/hls/stats: blocking task failed: {:?}", e);
                Vec::new()
            }
        };
    let total: usize = stats.iter().map(|s| s.total).sum();
    let with_playlist: usize = stats.iter().map(|s| s.with_playlist).sum();
    let pending: usize = stats.iter().map(|s| s.pending).sum();
    let unsupported: usize = stats.iter().map(|s| s.unsupported).sum();
    let hashless_videos: usize = stats.iter().map(|s| s.hashless_videos).sum();
    HttpResponse::Ok().json(HlsStatsResponse {
        libraries: stats,
        total,
        with_playlist,
        pending,
        unsupported,
        hashless_videos,
    })
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use tempfile::tempdir;
    fn lib(id: i32, name: &str) -> Library {
        Library {
            id,
            name: name.into(),
            root_path: String::new(),
            enabled: true,
            excluded_dirs: Vec::new(),
        }
    }
    fn rows(vs: Vec<(&str, Option<&str>)>) -> Vec<(String, Option<String>)> {
        vs.into_iter()
            .map(|(p, h)| (p.to_string(), h.map(|s| s.to_string())))
            .collect()
    }
    fn touch(dir: &Path, rel: &str) {
        let p = dir.join(rel);
        std::fs::create_dir_all(p.parent().unwrap()).unwrap();
        std::fs::write(p, b"").unwrap();
    }
    #[test]
    fn videos_only_count_in_total() {
        let tmp = tempdir().unwrap();
        let r = rows(vec![
            ("photos/IMG.jpg", Some(&"a".repeat(64))), // image: ignored
            ("clip.mp4", Some(&"b".repeat(64))),
            ("vid.mov", Some(&"c".repeat(64))),
        ]);
        let stats = stats_from_rows(&lib(1, "main"), &r, tmp.path());
        assert_eq!(stats.total, 2);
        assert_eq!(stats.with_playlist, 0);
        assert_eq!(stats.pending, 2);
        assert_eq!(stats.unsupported, 0);
        assert_eq!(stats.hashless_videos, 0);
    }
    #[test]
    fn hash_dedup_collapses_duplicate_rel_paths() {
        let tmp = tempdir().unwrap();
        let r = rows(vec![
            ("a/clip.mp4", Some(&"a".repeat(64))),
            ("b/clip.mp4", Some(&"a".repeat(64))), // same bytes, dup
            ("other.mp4", Some(&"b".repeat(64))),
        ]);
        let stats = stats_from_rows(&lib(1, "main"), &r, tmp.path());
        assert_eq!(stats.total, 2, "duplicate hashes collapse");
    }
    #[test]
    fn playlist_existence_promotes_to_with_playlist() {
        let tmp = tempdir().unwrap();
        let hash = "a".repeat(64);
        touch(tmp.path(), &format!("aa/{}/playlist.m3u8", hash));
        let r = rows(vec![("clip.mp4", Some(&hash))]);
        let stats = stats_from_rows(&lib(1, "main"), &r, tmp.path());
        assert_eq!(stats.total, 1);
        assert_eq!(stats.with_playlist, 1);
        assert_eq!(stats.pending, 0);
    }
    #[test]
    fn sentinel_existence_promotes_to_unsupported() {
        let tmp = tempdir().unwrap();
        let hash = "b".repeat(64);
        touch(tmp.path(), &format!("bb/{}/playlist.unsupported", hash));
        let r = rows(vec![("clip.mov", Some(&hash))]);
        let stats = stats_from_rows(&lib(1, "main"), &r, tmp.path());
        assert_eq!(stats.total, 1);
        assert_eq!(stats.unsupported, 1);
        assert_eq!(stats.with_playlist, 0);
        assert_eq!(stats.pending, 0);
    }
    #[test]
    fn null_hash_videos_are_hashless_not_total() {
        let tmp = tempdir().unwrap();
        let r = rows(vec![
            ("clip.mp4", None),
            ("other.mp4", Some(&"a".repeat(64))),
        ]);
        let stats = stats_from_rows(&lib(1, "main"), &r, tmp.path());
        assert_eq!(stats.total, 1, "hashless row excluded from total");
        assert_eq!(stats.hashless_videos, 1);
    }
    #[test]
    fn publish_gauges_sets_per_library_value() {
        let s = HlsLibraryStats {
            library_id: 7,
            library: "test_publish_a".into(),
            total: 5,
            with_playlist: 2,
            pending: 3,
            unsupported: 0,
            hashless_videos: 0,
        };
        publish_gauges(&s);
        assert_eq!(
            HLS_VIDEOS_TOTAL
                .with_label_values(&["test_publish_a"])
                .get(),
            5
        );
        assert_eq!(
            HLS_VIDEOS_PENDING
                .with_label_values(&["test_publish_a"])
                .get(),
            3
        );
        assert_eq!(
            HLS_VIDEOS_WITH_PLAYLIST
                .with_label_values(&["test_publish_a"])
                .get(),
            2
        );
    }
 }
@@ -7,8 +7,6 @@ pub mod ai;
 pub mod auth;
 pub mod bin_progress;
 pub mod cleanup;
 pub mod clip_search;
 pub mod clip_watch;
 pub mod content_hash;
 pub mod data;
 pub mod database;
@@ -28,14 +26,11 @@ pub mod memories;
 pub mod otel;
 pub mod parsers;
 pub mod perceptual_hash;
 pub mod personas;
 pub mod service;
 pub mod state;
 pub mod tags;
 #[cfg(test)]
 pub mod testhelpers;
 pub mod thumbnails;
 pub mod unified_search;
 pub mod utils;
 pub mod video;
@@ -43,3 +38,20 @@ pub mod video;
 pub use data::{Claims, ThumbnailRequest};
 pub use database::{connect, schema};
 pub use state::AppState;
 // Stub functions for modules that reference main.rs
 // These are not used by cleanup_files binary
 use std::path::Path;
 use walkdir::DirEntry;
 pub fn create_thumbnails(_libs: &[libraries::Library], _excluded_dirs: &[String]) {
    // Stub - implemented in main.rs
 }
 pub fn update_media_counts(_media_dir: &Path, _excluded_dirs: &[String]) {
    // Stub - implemented in main.rs
 }
 pub fn is_video(entry: &DirEntry) -> bool {
    file_types::direntry_is_video(entry)
 }
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Cameron Cordes	2ff06413c6	ai: restructure agentic user message — facts up top + forcing gate Small models (~8B) were producing generic responses regardless of persona, and bailing out of the agentic loop on iteration 1. Two underlying causes: 1. Photo facts (date, location, contact, tags, visual) were buried between "Please analyze this photo" preamble and "Use the available tools" outro. Small models skim and miss them, which is why outputs weren't anchoring to the actual photo. 2. The user message ended with "write a detailed insight" — small models took the path of least resistance and just wrote, ignoring the soft "aim to use 5 tools" floor in the system prompt. Restructured the user message: - Leads with a "## This photo" bulleted block so the metadata is visible top-down. File path, date+source, contact, location+GPS, tags, and (in hybrid) the visual description are all bullets the model can't skim past. - Replaces the prose body with a numbered "## What to do" recipe: (1) recall_facts_for_photo + recall_entities, (2) ≥3 of the time-window tools, (3) write only after tool results, referencing specific facts. "Generic narration is not acceptable" is explicit. - Ends with a hard forcing line: "YOUR FIRST RESPONSE MUST BE A TOOL CALL. Do not output any final answer text until you have called at least 5 tools." Replaces the soft "aim to" floor with a directive small models actually follow. Tradeoff: big models also follow the recipe literally and may call 5 tools when 3 would do. Optimizing for the small-model floor first; soften once that's working. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 10:59:39 -04:00
Cameron Cordes	66ea8490ab	backfill_date_taken: surface the actual diesel error in warnings The DAO swallowed every diesel::update failure as a flat `anyhow!("Update error")`, then trace_db_call further reduced it to `DbError { kind: UpdateError }`. Operators saw "update failed for lib 2 Snapchat/foo.mp4: DbError { kind: UpdateError }" with no clue why (constraint violation? type mismatch? row vanished mid-flight? DB locked?). Two changes: - Preserve the diesel error in the anyhow chain along with the input params (lib, rel_path, date_taken, source) so the cause is visible. - Log the chain at warn-level inside the DAO before the trace wrapper collapses it to DbErrorKind::UpdateError, so the warning at the call site finally has something diagnosable next to it. - Treat zero-row updates as a debug-level "row likely retired by the missing-file scan" rather than a hard failure — that case is benign and shouldn't poison the drain's error tally. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 10:41:09 -04:00
Cameron Cordes	10ba706b39	ai: reframe iteration budget as capacity, not constraint Small models (~8B) were bailing out of the agentic loop after one or two tool calls under the previous "hard budget … stop when nearly exhausted" phrasing. They read that as a conservation directive and the "trivial photos may need fewer" clause gave them an easy out. Flipped both the agentic and chat-turn prompts to frame the budget as capacity to spend, with a soft floor (≥5 tool calls before writing) and an explicit reserve clause for the final reply. Big models will still deviate when warranted; small models follow the floor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 10:36:05 -04:00
Cameron Cordes	9071d05932	ai: insight tools audit — bug fixes, new tools, prompt structure Bug fixes: - get_sms_messages.days_radius is now actually honored (was hardcoded to ±4d in SmsApiClient::fetch_messages_for_contact). - describe_photo memoized for the lifetime of one agentic loop / one chat turn — re-running mid-loop produced conflicting visual descriptions in the transcript. Agentic user message: - Pre-resolve location via Apollo + Nominatim and emit one Location: line instead of bare GPS, mirroring the non-agentic flow. - Date now formats with weekday + canonical-date source so the model can hedge on fs_time-derived dates. - Hybrid mode visual block tells the model not to call describe_photo (the tool is already gated off in hybrid). System prompt structure: - custom_system_prompt now appends under an explicit "User overrides (these take precedence)" heading instead of prepending — so a custom voice/POV/format prompt actually beats the built-in defaults. - Numbered rules collapsed into bulleted "Tool-use guidance"; merged the contradictory "multiple tools BEFORE" / "after 5 calls" rules. - Chat budget annotation surfaces as its own ## heading. New tools: - recall_facts_for_entity(entity_id\|name) — facts for one entity without needing a photo path. Fills the "tell me about Sarah" chat case where recall_facts_for_photo doesn't apply. - find_photos_with_entity(entity_id\|name) — "when did I last see X / show me photos from the Tahoe trip" via entity_photo_links. - get_exif(file_path) — full EXIF row for any photo, for technical ("what camera was this on?") questions. Tools removed: - get_file_tags duplicated the inline Tags: line on the user message; exposing both gave models an excuse to "confirm" what they had. Tool descriptions tightened: - search_rag now correctly says "per-day, per-contact summaries" and explains the date is for time-decay weighting. - recall_entities warns about empty-filter dumps. - store_entity / store_fact document dedup return + snake_case predicate vocabulary. - reverse_geocode defers to the pre-resolved location and to get_personal_place_at for personal places. - get_current_datetime narrowed to time-since-photo use. Calendar / location: - get_calendar_events accepts query and embeds it for hybrid time + semantic ranking (was always passing None for the embedding). - get_location_history exposes limit; description tells the model there's no semantic ranking on this surface. New disable_writes flag: - POST /insights/generate/agentic and the chat endpoints accept disable_writes: bool. When true, drops store_entity / store_fact from the tool palette and rewrites the system prompt's knowledge- write line. Lets users explore alternate prompts (caption-style, third-person, haiku) without polluting the persistent KB. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-06 22:37:32 -04:00
		`@@ -1,2 +0,0 @@`
			`DROP INDEX IF EXISTS idx_entity_facts_superseded_by;`
			`ALTER TABLE entity_facts DROP COLUMN superseded_by;`
		`@@ -1 +0,0 @@`
			`ALTER TABLE personas DROP COLUMN reviewed_only_facts;`
		`@@ -1,2 +0,0 @@`
			`DROP INDEX IF EXISTS idx_precomputed_reels_span_library;`
			`DROP TABLE IF EXISTS precomputed_reels;`
		`@@ -1 +0,0 @@`
			`DROP TABLE IF EXISTS persona_chat_conversations;`