ai: restructure agentic user message — facts up top + forcing gate

Small models (~8B) were producing generic responses regardless of persona, and bailing out of the agentic loop on iteration 1. Two underlying causes: 1. Photo facts (date, location, contact, tags, visual) were buried between "Please analyze this photo" preamble and "Use the available tools" outro. Small models skim and miss them, which is why outputs weren't anchoring to the actual photo. 2. The user message ended with "write a detailed insight" — small models took the path of least resistance and just wrote, ignoring the soft "aim to use 5 tools" floor in the system prompt. Restructured the user message: - Leads with a "## This photo" bulleted block so the metadata is visible top-down. File path, date+source, contact, location+GPS, tags, and (in hybrid) the visual description are all bullets the model can't skim past. - Replaces the prose body with a numbered "## What to do" recipe: (1) recall_facts_for_photo + recall_entities, (2) ≥3 of the time-window tools, (3) write only after tool results, referencing specific facts. "Generic narration is not acceptable" is explicit. - Ends with a hard forcing line: "YOUR FIRST RESPONSE MUST BE A TOOL CALL. Do not output any final answer text until you have called at least 5 tools." Replaces the soft "aim to" floor with a directive small models actually follow. Tradeoff: big models also follow the recipe literally and may call 5 tools when 3 would do. Optimizing for the small-model floor first; soften once that's working. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
backfill_date_taken: surface the actual diesel error in warnings
2026-05-07 10:59:39 -04:00 · 2026-05-07 10:41:09 -04:00 · 2026-05-07 10:36:05 -04:00 · 2026-05-06 22:37:32 -04:00
93 changed files with 5883 additions and 22805 deletions
@@ -53,50 +53,11 @@ AGENTIC_CHAT_MAX_ITERATIONS=6
 # OPENROUTER_HTTP_REFERER=https://your-site.example
 # OPENROUTER_APP_TITLE=ImageApi
 # ── AI Insights — local backend switch ──────────────────────────────────
 # Picks which local LLM stack the server uses for chat, vision describe,
 # and embeddings. `ollama` (default) uses the OLLAMA_* settings above;
 # `llamacpp` uses the LLAMA_SWAP_* settings below. The switch is global
 # and applies to both `backend=local` and `backend=hybrid` (hybrid keeps
 # chat on OpenRouter but still uses this stack for the describe pass).
 # Don't flip mid-deploy without re-embedding existing index rows —
 # mixed vector spaces break similarity search.
 # LLM_BACKEND=ollama
 # ── AI Insights — llama.cpp / llama-swap (optional) ─────────────────────
 # Set LLAMA_SWAP_URL plus LLM_BACKEND=llamacpp to swap the local stack
 # off Ollama. Talks OpenAI-compatible /v1 to a llama-swap proxy fronting
 # per-slot llama-server instances. Chat models receive images directly
 # via content-parts (vision-capable models assumed); a separate vision
 # slot is used only by the describe_photo tool and describe-image utility.
 # LLAMA_SWAP_URL=http://localhost:9292/v1
 # LLAMA_SWAP_PRIMARY_MODEL=chat
 # Optional dedicated vision slot for describe_image. Defaults to
 # PRIMARY_MODEL so describe_photo works without extra config.
 # LLAMA_SWAP_VISION_MODEL=vision
 # LLAMA_SWAP_EMBEDDING_MODEL=embed
 # Comma-separated allowlist surfaced by /insights/models when
 # LLM_BACKEND=llamacpp. All report has_vision=true.
 # LLAMA_SWAP_ALLOWED_MODELS=chat,vision,embed
 # LLAMA_SWAP_REQUEST_TIMEOUT_SECONDS=180
 # ── Text-to-speech (optional, requires LLAMA_SWAP_URL) ───────────────────
 # TTS routes through the same llama-swap proxy (a Chatterbox model id), so it
 # only needs LLAMA_SWAP_URL — it does NOT require LLM_BACKEND=llamacpp.
 # Powers POST /tts/speech and the /tts/voices* endpoints (read-aloud insights
 # + voice cloning in the mobile app).
 # LLAMA_SWAP_TTS_MODEL=chatterbox        # TTS model id in config.yaml
 # LLAMA_SWAP_TTS_VOICE=m                 # default voice when a request omits one
 # LLAMA_SWAP_TTS_REF_SECONDS=30          # max voice-clone reference clip length (s)
 # LLAMA_SWAP_TTS_REQUEST_TIMEOUT_SECONDS=600   # synth timeout (long chunked text)
 # ── AI Insights — sibling services (optional) ───────────────────────────
-# Apollo (places, face inference, CLIP encoders). Single-Apollo deploys
+# Apollo (places + face inference). Single Apollo deploys typically set
-# typically set only APOLLO_API_BASE_URL and let the face + CLIP
+# only APOLLO_API_BASE_URL and let the face client fall back to it.
 # clients fall back to it.
 # APOLLO_API_BASE_URL=http://apollo.lan:8000
 # APOLLO_FACE_API_BASE_URL=http://apollo.lan:8000
 # APOLLO_CLIP_API_BASE_URL=http://apollo.lan:8000
 # SMS_API_URL=http://localhost:8000
 # SMS_API_TOKEN=
@@ -119,23 +80,6 @@ FACE_DETECT_TIMEOUT_SEC=60
 FACE_BACKLOG_MAX_PER_TICK=64
 FACE_HASH_BACKFILL_MAX_PER_TICK=2000
 # ── CLIP semantic photo search ──────────────────────────────────────────
 # ImageApi calls Apollo's /api/internal/clip/{encode_image,encode_text}
 # to populate per-photo embeddings during the watcher's backlog drain
 # and to encode user queries at /photos/search time. Disabled when
 # neither APOLLO_CLIP_API_BASE_URL nor APOLLO_API_BASE_URL is set.
 #
 # Per-watcher-tick cap on the encode drain. Default 32 ≈ ~1 photo/sec
 # on CPU, ~30 photos/sec on a single-GPU host (Apollo's threadpool
 # is 1 on CUDA, so concurrency is bounded server-side regardless of
 # our setting). Bump on a fresh deploy to clear the backlog faster.
 CLIP_BACKLOG_MAX_PER_TICK=32
 # Client-side parallel encode calls per drain pass. Apollo's GPU pool
 # serializes server-side; this just overlaps file-IO with inference.
 CLIP_ENCODE_CONCURRENCY=4
 # Per-encode HTTP timeout. CPU-only Apollo deploys may need higher.
 CLIP_REQUEST_TIMEOUT_SEC=60
 # ── RAG / search ────────────────────────────────────────────────────────
 # Set to `1` to enable cross-encoder reranking on /search results.
 SEARCH_RAG_RERANK=0
@@ -1,9 +0,0 @@
 # Normalize line endings in the repo to LF. Windows checkouts can still
 # present working-copy files as CRLF; this just keeps the committed history
 # stable so contributors on any OS don't see whitespace-only diffs every
 # time someone touches a file.
 * text=auto eol=lf
 # Migrations and SQL must be LF — SQLite parsers don't care, but diffing
 # is much cleaner with stable endings.
 *.sql text eol=lf
@@ -76,10 +76,7 @@ cargo run --bin cleanup_files -- --base-path /path/to/media --database-url ./dat
 ### Core Components
 **Layered Architecture:**
- **Startup wiring** (`main.rs`): only ~350 lines — env load, migrations, AppState, route registration, server bind. Background jobs are kicked off here but defined elsewhere.
+- **HTTP Layer** (`main.rs`): Route handlers for images, videos, metadata, tags, favorites, memories
 - **HTTP Layer** (`handlers/{image,video,favorites}.rs`, `files.rs`, `tags.rs`, `faces.rs`, `memories.rs`, `ai/handlers.rs`): the route handlers, grouped by domain.
 - **Background loops** (`watcher.rs`): the file-watcher tick (`watch_files`, `process_new_files`) and the orphaned-playlist cleanup (`cleanup_orphaned_playlists`). Per-tick drains are factored into `backfill.rs` (`backfill_unhashed_backlog`, `backfill_missing_date_taken`, `backfill_missing_content_hashes`, `process_face_backlog`, `build_face_candidates`).
 - **Thumbnails** (`thumbnails.rs`): generation pipeline + the `IMAGE_GAUGE` / `VIDEO_GAUGE` Prometheus metrics.
 - **Auth Layer** (`auth.rs`): JWT token validation, Claims extraction via FromRequest trait
 - **Service Layer** (`files.rs`, `exif.rs`, `memories.rs`): Business logic for file operations and EXIF extraction
 - **DAO Layer** (`database/mod.rs`): Trait-based data access (ExifDao, UserDao, FavoriteDao, TagDao)
@@ -395,24 +392,17 @@ under 2021, not 2014 — on the theory that EXIF is more reliable than
 import-named filenames. The reverse case (no EXIF, filename has a
 date) is unchanged.
-The `backfill_missing_date_taken` drain (`src/backfill.rs`) runs every
+The `backfill_missing_date_taken` drain (`src/main.rs`) runs every
-watcher tick alongside `backfill_unhashed_backlog` (also `src/backfill.rs`). It loads up to
+watcher tick alongside `backfill_unhashed_backlog`. It loads up to
 `DATE_BACKFILL_MAX_PER_TICK` rows (default 500) where
-`date_taken IS NULL` (backed by the `idx_image_exif_date_backfill`
+`date_taken IS NULL OR date_taken_source = 'fs_time'` (backed by the
-partial index), runs the waterfall batch via `resolve_dates_batch`,
+`idx_image_exif_date_backfill` partial index), runs the waterfall
-and writes results via the `backfill_date_taken` DAO method (touches
+batch via `resolve_dates_batch`, and writes results via the
-only `date_taken` + `date_taken_source` so EXIF / hash / perceptual
+`backfill_date_taken` DAO method (touches only `date_taken` +
-columns are preserved). Resolved rows — including the ones the
+`date_taken_source` so EXIF / hash / perceptual columns are
-waterfall could only resolve via `fs_time` — are not re-eligible:
+preserved). `filename`-sourced rows are intentionally not re-resolved
-the resolver is deterministic on file bytes + filename + fs metadata,
+— the regex is authoritative when it matches, and re-running exiftool
-so re-running on the same inputs lands on the same source every time.
+won't change the answer.
 An earlier version included `date_taken_source = 'fs_time'` in the
 eligibility predicate, but with `ORDER BY id ASC LIMIT 500` it spun on
 the same lowest-id rows in perpetuity and held the SQLite write lock
 long enough to starve face-PATCH writers (5s busy_timeout → 500). If
 a stronger tool comes online (exiftool install, new filename regex),
 re-resolve out-of-band rather than re-introducing the steady-state
 eligibility.
 `/memories` is a single SQL query against this column
 (`get_memories_in_window` in `src/database/mod.rs`), using
@@ -473,16 +463,10 @@ GET /memories?path=...&recursive=true
 POST /insights/generate              (non-agentic single-shot)
 POST /insights/generate/agentic      (tool-calling loop; body: { file_path, backend?, model?, ... })
 GET  /insights?path=...&library=...
-GET  /insights/models                (local-backend models + capabilities; Ollama OR llama-swap based on LLM_BACKEND)
+GET  /insights/models                (local Ollama models + capabilities)
 GET  /insights/openrouter/models     (curated OpenRouter allowlist)
 POST /insights/rate                  (thumbs up/down for training data)
 // Text-to-Speech (Chatterbox via llama-swap; needs LLAMA_SWAP_URL)
 POST /tts/speech                     (read-aloud: { text, voice?, ... } -> { audio_base64, format })
 GET  /tts/voices                     (Chatterbox voice library)
 POST /tts/voices/upload              (clone a voice from an uploaded clip; multipart)
 POST /tts/voices/from-library        (clone a voice from a library audio/video file)
 // Insight Chat Continuation
 POST /insights/chat                  (single-turn reply, non-streaming)
 POST /insights/chat/stream           (SSE: text / tool_call / tool_result / truncated / done)
@@ -520,9 +504,9 @@ ImageApi owns the face data; Apollo (sibling repo) hosts the insightface inferen
 **Why content_hash and not (library_id, rel_path):** ties face data to the bytes, not the path. A backup mount that copies files from the primary library naturally inherits the existing detections without re-running inference. This is the reference implementation of the multi-library data model — see "Multi-library data model" above.
-**File-watch hook** (`src/watcher.rs::process_new_files`): for each photo with a populated `content_hash`, check `FaceDao::already_scanned(hash)`; if not, send bytes (or embedded JPEG preview for RAW via `exif::extract_embedded_jpeg_preview`) to Apollo's `/api/internal/faces/detect`. K=`FACE_DETECT_CONCURRENCY` (default 8) parallel calls per scan tick; Apollo serializes them via its single-worker GPU pool. `face_watch.rs` is the Tokio orchestration layer.
+**File-watch hook** (`src/main.rs::process_new_files`): for each photo with a populated `content_hash`, check `FaceDao::already_scanned(hash)`; if not, send bytes (or embedded JPEG preview for RAW via `exif::extract_embedded_jpeg_preview`) to Apollo's `/api/internal/faces/detect`. K=`FACE_DETECT_CONCURRENCY` (default 8) parallel calls per scan tick; Apollo serializes them via its single-worker GPU pool. `face_watch.rs` is the Tokio orchestration layer.
-**Per-tick backlog drain** (`src/backfill.rs`): two passes that run on every watcher tick regardless of quick-vs-full scan:
+**Per-tick backlog drain** (also `src/main.rs`): two passes that run on every watcher tick regardless of quick-vs-full scan:
 - `backfill_unhashed_backlog` — populates `image_exif.content_hash` for photos that arrived before the hash field was retroactive. Capped by `FACE_HASH_BACKFILL_MAX_PER_TICK` (default 2000); errors don't burn the cap.
 - `process_face_backlog` — runs detection on photos that have a hash but no `face_detections` row. Capped by `FACE_BACKLOG_MAX_PER_TICK` (default 64). Selected via a SQL anti-join (`FaceDao::list_unscanned_candidates`); videos and EXCLUDED_DIRS paths filtered out client-side via `face_watch::filter_excluded` so they never reach Apollo.
@@ -537,8 +521,6 @@ ImageApi owns the face data; Apollo (sibling repo) hosts the insightface inferen
 Module map:
 - `src/faces.rs` — `FaceDao` trait + `SqliteFaceDao` impl, route handlers for `/faces/*`, `/image/faces/*`, `/persons/*`. Mirror of `tags.rs` layout.
 - `src/face_watch.rs` — Tokio orchestration for the file-watch detect pass; `filter_excluded` (PathExcluder + image-extension filter), `read_image_bytes_for_detect` (RAW preview fallback).
 - `src/backfill.rs` — per-tick drains (unhashed-hash, date_taken, face-backlog, etc.) called from `watcher::watch_files` and `watcher::process_new_files`.
 - `src/watcher.rs` — the watcher loop itself and `process_new_files` (file walk → EXIF write → face-candidate build).
 - `src/ai/face_client.rs` — HTTP client for Apollo's inference. Configured by `APOLLO_FACE_API_BASE_URL`, falls back to `APOLLO_API_BASE_URL`. Both unset → feature disabled, file-watch hook is a no-op.
 - `migrations/2026-04-29-000000_add_faces/` — schema.
@@ -637,45 +619,8 @@ OPENROUTER_EMBEDDING_MODEL=openai/text-embedding-3-small  # Optional, embeddings
 OPENROUTER_HTTP_REFERER=https://your-site.example    # Optional attribution header
 OPENROUTER_APP_TITLE=ImageApi                  # Optional attribution header
 # Local LLM backend switch. `ollama` (default) keeps the OLLAMA_* settings
 # above; `llamacpp` swaps the entire local stack (chat + vision describe +
 # embeddings) over to llama-swap. The switch is global and applies to
 # `backend=local` requests and to `backend=hybrid`'s describe pass (hybrid
 # chat still goes to OpenRouter). Don't flip mid-deploy without
 # re-embedding — mixed vector spaces break similarity search.
 LLM_BACKEND=ollama
 # llama.cpp / llama-swap (used when LLM_BACKEND=llamacpp). OpenAI-compatible
 # proxy hosting one or more llama-server processes. Chat models receive
 # images directly via content-parts (all models assumed vision-capable).
 LLAMA_SWAP_URL=http://localhost:9292/v1         # Required when LLM_BACKEND=llamacpp
 LLAMA_SWAP_PRIMARY_MODEL=chat                   # Chat slot id (matches config.yaml)
 LLAMA_SWAP_VISION_MODEL=                        # Dedicated vision slot for describe_image / describe_photo
                                                # tool. Defaults to PRIMARY_MODEL when unset.
 LLAMA_SWAP_EMBEDDING_MODEL=embed                # Embedding slot id
 LLAMA_SWAP_ALLOWED_MODELS=chat,coder            # Curated allowlist surfaced by GET /insights/models
                                                # when LLM_BACKEND=llamacpp. All report has_vision=true.
                                                # Empty = picker shows only the configured primary model.
 LLAMA_SWAP_REQUEST_TIMEOUT_SECONDS=180          # Per-request timeout; bump for slow CPU offload
 # Text-to-speech (Chatterbox served behind llama-swap). Only needs
 # LLAMA_SWAP_URL — independent of LLM_BACKEND. Powers /tts/speech (read-aloud)
 # and /tts/voices* (voice cloning). Reference audio is ffmpeg-normalized to WAV
 # server-side, so any source format works.
 LLAMA_SWAP_TTS_MODEL=chatterbox                # TTS model id in config.yaml (default: chatterbox)
 LLAMA_SWAP_TTS_VOICE=m                         # Default voice when /tts/speech omits one (optional)
 LLAMA_SWAP_TTS_REF_SECONDS=30                  # Max voice-clone reference clip length, seconds
                                               # (Chatterbox is zero-shot; ~10-20s clean ref is ideal)
 LLAMA_SWAP_TTS_REQUEST_TIMEOUT_SECONDS=600     # Per-request synth timeout (long chunked insights take
                                               # minutes); overrides the shared client timeout for /tts/speech
 # Insight Chat Continuation
 AGENTIC_CHAT_MAX_ITERATIONS=6                  # Cap on tool-calling iterations per chat turn (default 6)
 AGENTIC_CHAT_DEFAULT_NUM_CTX=32768             # Assumed context window for the history-truncation budget
                                               # when a chat request omits num_ctx (default 32768). Size to
                                               # the smallest context among the chat models actually served;
                                               # too small silently guts replayed history every turn (and
                                               # destroys llama.cpp KV-cache prefix reuse).
 ```
 **AI Insights Fallback Behavior:**
@@ -693,50 +638,10 @@ The `OllamaClient` provides methods to query available models:
 This allows runtime verification of model availability before generating insights.
 **Local backend switch (`LLM_BACKEND`):**
 One env var decides which "local" stack the server runs against — `ollama`
 (default) or `llamacpp`. It's global on purpose: chat, vision, and
 embeddings all route through the same backend, so the embedding-vector
 column in SQLite stays in one vector space. Don't flip mid-deploy without
 re-embedding the affected rows — similarity search will collapse.
 - `LLM_BACKEND=ollama`: chat, vision, and embeddings use Ollama. Vision
  capability is probed per-model via `/api/show`.
 - `LLM_BACKEND=llamacpp`: chat models receive images directly via OpenAI
  content-parts (all models assumed vision-capable). Embeddings hit the
  `embed` slot. A dedicated `LLAMA_SWAP_VISION_MODEL` slot (defaults to
  the chat model) handles `describe_image` for the `describe_photo` tool.
  Requires `LLAMA_SWAP_URL`.
 The per-request `backend=hybrid` override is orthogonal: it always sends
 chat to OpenRouter (text-only, images are pre-described and inlined), but
 the describe + embed passes still route through whichever `LLM_BACKEND`
 is configured.
 **Backend dispatch (`ResolvedBackend`):**
 `InsightGenerator::resolve_backend(kind, overrides)` is the single entry
 point that builds clients for a request. Returns a `ResolvedBackend` with
 two roles: `.chat()` (the agentic/chat client) and `.local()` (local-only
 utility calls: rerank, describe_image, embeddings). `BackendKind` is an
 enum (`Local` | `Hybrid`) replacing the stringly-typed `"local"` /
 `"hybrid"` labels. `SamplingOverrides` groups model/ctx/temp/top_p/top_k/
 min_p per-request overrides. All downstream code (`execute_tool`,
 `run_streaming_agentic_loop`, etc.) takes `&ResolvedBackend` rather than
 individual client references.
 `GET /insights/models` returns the local-backend models with capabilities
 in the same envelope shape regardless of `LLM_BACKEND`: Ollama servers
 when `ollama`, llama-swap slots (from `LLAMA_SWAP_ALLOWED_MODELS`) when
 `llamacpp`. No `/insights/llamacpp/models` — the picker reads a single
 endpoint.
 **Hybrid Backend (OpenRouter):**
 - Per-request opt-in via `backend=hybrid` on `POST /insights/generate/agentic`.
- Vision describe happens before the agentic loop; the description is inlined
+- Local Ollama still describes the image (vision); the description is inlined
-  into the chat prompt and the agentic loop runs on OpenRouter. Vision
+  into the chat prompt and the agentic loop runs on OpenRouter.
  routes through whichever `LLM_BACKEND` is configured.
 - `request.model` (if provided) overrides `OPENROUTER_DEFAULT_MODEL` for that
  call. The mobile picker reads from `OPENROUTER_ALLOWED_MODELS`.
 - No live capability precheck — the operator-curated allowlist is trusted.
@@ -744,15 +649,6 @@ endpoint.
 - `GET /insights/openrouter/models` returns `{ models, default_model, configured }`
  for client picker UIs.
 **Cross-replay matrix (chat continuation):**
 - `local → local` allowed (whether served by Ollama or llama-swap; that's
  a deploy-time decision, not a request-time one).
 - `hybrid → hybrid` allowed.
 - `hybrid → local` allowed (the inlined description replays as text).
 - `local → hybrid` rejected — the stored transcript has raw images in the
  first user message and OpenRouter providers don't accept that shape
  consistently. Regenerate the insight in hybrid mode instead.
 **Insight Chat Continuation:**
 After an agentic insight is generated, the full `Vec<ChatMessage>` transcript is
@@ -762,12 +658,7 @@ clients whether chat is available for a given insight.
 - `POST /insights/chat` runs one turn of the agentic loop against the replayed
  history. Body: `{ file_path, library?, user_message, model?, backend?, num_ctx?,
-  temperature?, top_p?, top_k?, min_p?, max_iterations?, system_prompt?, amend? }`.
+  temperature?, top_p?, top_k?, min_p?, max_iterations?, amend? }`.
  `system_prompt` is a per-turn override: in append mode (default) it's applied
  ephemerally — the original system message is restored before persistence so
  the stored transcript keeps its baked persona. In amend mode the override
  stays in place and becomes the new insight row's system message. Mirrors the
  internal `annotate_system_with_budget` swap-and-restore pattern.
 - `POST /insights/chat/stream` is the SSE variant — same request body, response
  is `text/event-stream` with events: `iteration_start`, `text` (delta), `tool_call`,
  `tool_result`, `truncated`, `done`, plus a server-emitted `error_message` on
@@ -799,17 +690,14 @@ Per-`(library_id, file_path)` async mutex (`AppState.insight_chat.chat_locks`)
 serialises concurrent turns on the same insight so the JSON blob doesn't race.
 Context management is a soft bound: if the serialized history exceeds
-`num_ctx - 2048` tokens (cheap 4-byte/token heuristic; `num_ctx` defaults
+`num_ctx - 2048` tokens (cheap 4-byte/token heuristic), the oldest
-to `AGENTIC_CHAT_DEFAULT_NUM_CTX`, 32768, when the request omits it), the
+assistant-tool_call + tool_result pairs are dropped until under budget. The
 oldest assistant-tool_call + tool_result pairs are dropped until under budget. The
 initial user message (with any images) and system prompt are always preserved.
 The `truncated` event / flag is surfaced to the client when a drop occurred.
 Configurable env:
 - `AGENTIC_CHAT_MAX_ITERATIONS` — cap on tool-calling iterations per turn
  (default 6). Per-request `max_iterations` is clamped to this cap.
 - `AGENTIC_CHAT_DEFAULT_NUM_CTX` — assumed context window for the truncation
  budget when the request omits `num_ctx` (default 32768).
 **Apollo Places integration (optional):**
@@ -2051,7 +2051,7 @@ dependencies = [
 [[package]]
 name = "image-api"
-version = "1.3.0"
+version = "1.1.0"
 dependencies = [
 "actix",
 "actix-cors",
@@ -2104,7 +2104,6 @@ dependencies = [
 "tokio",
 "tokio-util",
 "urlencoding",
 "uuid",
 "walkdir",
 "zerocopy",
 ]
@@ -4392,9 +4391,7 @@ version = "1.23.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "ddd74a9687298c6858e9b88ec8935ec45d22e8fd5e6394fa1bd4e99a87789c76"
 dependencies = [
 "getrandom 0.4.2",
 "js-sys",
 "serde_core",
 "wasm-bindgen",
 ]
@@ -1,6 +1,6 @@
 [package]
 name = "image-api"
-version = "1.3.0"
+version = "1.1.0"
 authors = ["Cameron Cordes <cameronc.dev@gmail.com>"]
 edition = "2024"
@@ -66,7 +66,6 @@ image_hasher = "3.0"
 bk-tree = "0.5"
 async-trait = "0.1"
 indicatif = "0.17"
 uuid = { version = "1.10", features = ["v4", "serde"] }
 # Windows lacks system sqlite3, so re-enable the bundled C build there.
 # Linux/macOS use the system library (faster builds, smaller binary).
@@ -147,34 +147,6 @@ so you can rewrite the saved summary from within chat.
 - `AGENTIC_CHAT_MAX_ITERATIONS` - Cap on tool-calling iterations per chat turn [default: `6`]
  - Per-request `max_iterations` (when sent by the client) is clamped to this cap
 #### Text-to-Speech (Optional)
 Reads insights aloud and manages cloned voices via a Chatterbox model served
 behind the same llama-swap proxy. Only requires `LLAMA_SWAP_URL` (the TTS client
 is built whenever that's set — independent of `LLM_BACKEND`). Endpoints:
 - `POST /tts/speech` — body `{ text, voice?, format?, exaggeration?, cfg_weight?,
  temperature? }`; returns `{ audio_base64, format }`. Input is cleaned
  server-side (markdown + emoji stripped) and the generation knobs are clamped
  to Chatterbox's ranges. Synthesis is serialized (one at a time — the upstream
  has no GPU lock of its own); a concurrent request gets a fast `429`.
 - `GET /tts/voices` — list the voice library.
 - `POST /tts/voices/upload` — multipart `voice_name` + `voice_file`; clone a
  voice from an uploaded clip (≤25 MB).
 - `POST /tts/voices/from-library` — body `{ voice_name, path, library? }`; clone
  from a library file (audio forwarded as-is; video has its audio extracted via
  ffmpeg).
 Env:
 - `LLAMA_SWAP_TTS_MODEL` - TTS model id in llama-swap's `config.yaml` [default: `chatterbox`]
 - `LLAMA_SWAP_TTS_VOICE` - default voice used when a `/tts/speech` request omits `voice` (optional)
 - `LLAMA_SWAP_TTS_REF_SECONDS` - max voice-clone reference clip length in seconds
  [default: `30`]. Reference audio is ffmpeg-normalized to mono 24 kHz WAV (so any
  source format works); Chatterbox is zero-shot, so a clean ~10–20s sample is the
  sweet spot — more rarely helps.
 - `LLAMA_SWAP_TTS_REQUEST_TIMEOUT_SECONDS` - per-request synthesis timeout in
  seconds [default: `600`]. Long insights are chunked + synthesized server-side
  and can take minutes; this is separate from (and overrides, for `/tts/speech`)
  the shared `LLAMA_SWAP_REQUEST_TIMEOUT_SECONDS`.
 #### Fallback Behavior
 - Primary server is tried first with 5-second connection timeout
 - On failure, automatically falls back to secondary server (if configured)
@@ -1,43 +0,0 @@
 -- Drop the persona-scoping column on entity_facts via the table-rebuild
 -- dance for SQLite-version portability (matches the pattern in
 -- 2026-04-20-000000_add_backend_to_insights/down.sql).
 DROP INDEX IF EXISTS idx_entity_facts_persona;
 CREATE TABLE entity_facts_backup AS
    SELECT id, subject_entity_id, predicate, object_entity_id, object_value,
           source_photo, source_insight_id, confidence, status, created_at
    FROM entity_facts;
 DROP TABLE entity_facts;
 CREATE TABLE entity_facts (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    subject_entity_id INTEGER NOT NULL,
    predicate TEXT NOT NULL,
    object_entity_id INTEGER,
    object_value TEXT,
    source_photo TEXT,
    source_insight_id INTEGER,
    confidence REAL NOT NULL DEFAULT 0.6,
    status TEXT NOT NULL DEFAULT 'active',
    created_at BIGINT NOT NULL,
    CONSTRAINT fk_ef_subject FOREIGN KEY (subject_entity_id) REFERENCES entities(id) ON DELETE CASCADE,
    CONSTRAINT fk_ef_object  FOREIGN KEY (object_entity_id)  REFERENCES entities(id) ON DELETE SET NULL,
    CONSTRAINT fk_ef_insight FOREIGN KEY (source_insight_id) REFERENCES photo_insights(id) ON DELETE SET NULL,
    CHECK (object_entity_id IS NOT NULL OR object_value IS NOT NULL)
 );
 INSERT INTO entity_facts
    SELECT id, subject_entity_id, predicate, object_entity_id, object_value,
           source_photo, source_insight_id, confidence, status, created_at
    FROM entity_facts_backup;
 DROP TABLE entity_facts_backup;
 CREATE INDEX idx_entity_facts_subject ON entity_facts(subject_entity_id);
 CREATE INDEX idx_entity_facts_predicate ON entity_facts(predicate);
 CREATE INDEX idx_entity_facts_status ON entity_facts(status);
 CREATE INDEX idx_entity_facts_source_photo ON entity_facts(source_photo);
 DROP INDEX IF EXISTS idx_personas_user;
 DROP TABLE IF EXISTS personas;
@@ -1,64 +0,0 @@
 -- Personas live server-side now (mobile previously stored them in
 -- AsyncStorage only). Each user gets the three built-ins seeded; custom
 -- personas land here too via POST /personas or POST /personas/migrate.
 --
 -- `entity_facts` gains a persona_id so each persona accumulates its own
 -- voice over a shared entity graph (entities themselves stay unscoped).
 -- Existing rows backfill to 'default' via the column DEFAULT — that
 -- becomes the historical baseline. The `include_all_memories` flag on
 -- personas lets any persona opt back into reading the full pool.
 CREATE TABLE personas (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    user_id INTEGER NOT NULL,
    persona_id TEXT NOT NULL,
    name TEXT NOT NULL,
    system_prompt TEXT NOT NULL,
    is_built_in BOOLEAN NOT NULL DEFAULT FALSE,
    include_all_memories BOOLEAN NOT NULL DEFAULT FALSE,
    created_at BIGINT NOT NULL,
    updated_at BIGINT NOT NULL,
    UNIQUE(user_id, persona_id),
    CONSTRAINT fk_personas_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
 );
 CREATE INDEX idx_personas_user ON personas(user_id);
 -- Seed built-ins for every existing user. System prompts copied verbatim
 -- from FileViewer-React/hooks/usePersonas.tsx so server and client agree
 -- on the canonical voice for each built-in.
 INSERT INTO personas (user_id, persona_id, name, system_prompt, is_built_in, created_at, updated_at)
 SELECT
    u.id,
    'default',
    'Default Assistant',
    'You are my long-term memory assistant. Use only the information provided. Do not invent details. Respond in 3–6 sentences in third person, leading with the most concrete moment from the photo and the surrounding context. Plain prose, no headings.',
    TRUE,
    strftime('%s', 'now') * 1000,
    strftime('%s', 'now') * 1000
 FROM users u
 UNION ALL
 SELECT
    u.id,
    'journal',
    'Personal Journal',
    'You are a personal journal writer. Write in first person, present tense, with warmth and reflection — focusing on emotions and meaningful moments. Use only the information provided; do not invent details. Aim for 4–8 sentences in a single flowing paragraph, no headings.',
    TRUE,
    strftime('%s', 'now') * 1000,
    strftime('%s', 'now') * 1000
 FROM users u
 UNION ALL
 SELECT
    u.id,
    'factual',
    'Factual Reporter',
    'You are a factual memory recorder. Be precise, objective, and concise. Lead with the date and place, then list what / when / who in 2–4 short sentences. Use only the information provided; if a detail is unknown, say so rather than guessing.',
    TRUE,
    strftime('%s', 'now') * 1000,
    strftime('%s', 'now') * 1000
 FROM users u;
 -- Persona scoping on facts only. Entities and entity_photo_links stay
 -- shared (real-world referents and shared photo ↔ entity associations).
 ALTER TABLE entity_facts ADD COLUMN persona_id TEXT NOT NULL DEFAULT 'default';
 CREATE INDEX idx_entity_facts_persona ON entity_facts(persona_id);
@@ -1,47 +0,0 @@
 -- Reverse 2026-05-10-000000_entity_facts_persona_fk: drop the
 -- composite FK and the user_id column via the same rebuild pattern.
 DROP INDEX IF EXISTS idx_entity_facts_user_persona;
 DROP INDEX IF EXISTS idx_entity_facts_persona;
 DROP INDEX IF EXISTS idx_entity_facts_source_photo;
 DROP INDEX IF EXISTS idx_entity_facts_status;
 DROP INDEX IF EXISTS idx_entity_facts_predicate;
 DROP INDEX IF EXISTS idx_entity_facts_subject;
 ALTER TABLE entity_facts RENAME TO entity_facts_old;
 CREATE TABLE entity_facts (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    subject_entity_id INTEGER NOT NULL,
    predicate TEXT NOT NULL,
    object_entity_id INTEGER,
    object_value TEXT,
    source_photo TEXT,
    source_insight_id INTEGER,
    confidence REAL NOT NULL DEFAULT 0.6,
    status TEXT NOT NULL DEFAULT 'active',
    created_at BIGINT NOT NULL,
    persona_id TEXT NOT NULL DEFAULT 'default',
    CONSTRAINT fk_ef_subject FOREIGN KEY (subject_entity_id) REFERENCES entities(id) ON DELETE CASCADE,
    CONSTRAINT fk_ef_object  FOREIGN KEY (object_entity_id)  REFERENCES entities(id) ON DELETE SET NULL,
    CONSTRAINT fk_ef_insight FOREIGN KEY (source_insight_id) REFERENCES photo_insights(id) ON DELETE SET NULL,
    CHECK (object_entity_id IS NOT NULL OR object_value IS NOT NULL)
 );
 INSERT INTO entity_facts
    (id, subject_entity_id, predicate, object_entity_id, object_value,
     source_photo, source_insight_id, confidence, status, created_at,
     persona_id)
 SELECT
    id, subject_entity_id, predicate, object_entity_id, object_value,
    source_photo, source_insight_id, confidence, status, created_at,
    persona_id
 FROM entity_facts_old;
 DROP TABLE entity_facts_old;
 CREATE INDEX idx_entity_facts_subject ON entity_facts(subject_entity_id);
 CREATE INDEX idx_entity_facts_predicate ON entity_facts(predicate);
 CREATE INDEX idx_entity_facts_status ON entity_facts(status);
 CREATE INDEX idx_entity_facts_source_photo ON entity_facts(source_photo);
 CREATE INDEX idx_entity_facts_persona ON entity_facts(persona_id);
@@ -1,82 +0,0 @@
 -- Add a real foreign key from entity_facts to personas. Until now,
 -- entity_facts.persona_id was a free-form string with no integrity
 -- guarantee — deleting a persona orphaned its facts, which then sat
 -- forever in the readable-only-via-PersonaFilter::All hive-mind view.
 --
 -- personas is keyed (user_id, persona_id) so the FK has to be
 -- composite. That requires entity_facts to carry user_id too, which
 -- has the side benefit of fixing multi-user fact leakage on the read
 -- path (without it, two users with the same 'default' persona would
 -- see each other's default-scoped facts).
 --
 -- SQLite can't ALTER TABLE to add an FK; the table-rebuild dance is
 -- the only way. Pattern matches 2026-05-09's down.sql and the older
 -- 2026-04-20-000000 migration.
 DROP INDEX IF EXISTS idx_entity_facts_subject;
 DROP INDEX IF EXISTS idx_entity_facts_predicate;
 DROP INDEX IF EXISTS idx_entity_facts_status;
 DROP INDEX IF EXISTS idx_entity_facts_source_photo;
 DROP INDEX IF EXISTS idx_entity_facts_persona;
 ALTER TABLE entity_facts RENAME TO entity_facts_old;
 CREATE TABLE entity_facts (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    subject_entity_id INTEGER NOT NULL,
    predicate TEXT NOT NULL,
    object_entity_id INTEGER,
    object_value TEXT,
    source_photo TEXT,
    source_insight_id INTEGER,
    confidence REAL NOT NULL DEFAULT 0.6,
    status TEXT NOT NULL DEFAULT 'active',
    created_at BIGINT NOT NULL,
    persona_id TEXT NOT NULL DEFAULT 'default',
    user_id INTEGER NOT NULL DEFAULT 1,
    CONSTRAINT fk_ef_subject FOREIGN KEY (subject_entity_id) REFERENCES entities(id) ON DELETE CASCADE,
    CONSTRAINT fk_ef_object  FOREIGN KEY (object_entity_id)  REFERENCES entities(id) ON DELETE SET NULL,
    CONSTRAINT fk_ef_insight FOREIGN KEY (source_insight_id) REFERENCES photo_insights(id) ON DELETE SET NULL,
    CONSTRAINT fk_ef_persona FOREIGN KEY (user_id, persona_id) REFERENCES personas(user_id, persona_id) ON DELETE CASCADE,
    CHECK (object_entity_id IS NOT NULL OR object_value IS NOT NULL)
 );
 -- Backfill: assign each legacy fact to the user that owns the matching
 -- persona. Built-ins are seeded per-user with the same persona_id
 -- string for everyone, so MIN(user_id) deterministically picks the
 -- earliest registered user (typically user 1, the operator). Custom
 -- persona_ids exist for at most one user, so MIN is also unique.
 -- Falls back to user_id=1 when no matching persona row exists; in that
 -- case the FK below would still fail, but legacy rows shouldn't be in
 -- that state because 2026-05-09 ADD COLUMN defaulted persona_id to
 -- 'default', which is seeded for every user.
 INSERT INTO entity_facts
    (id, subject_entity_id, predicate, object_entity_id, object_value,
     source_photo, source_insight_id, confidence, status, created_at,
     persona_id, user_id)
 SELECT
    old.id,
    old.subject_entity_id,
    old.predicate,
    old.object_entity_id,
    old.object_value,
    old.source_photo,
    old.source_insight_id,
    old.confidence,
    old.status,
    old.created_at,
    old.persona_id,
    COALESCE(
        (SELECT MIN(p.user_id) FROM personas p WHERE p.persona_id = old.persona_id),
        1
    )
 FROM entity_facts_old old;
 DROP TABLE entity_facts_old;
 CREATE INDEX idx_entity_facts_subject ON entity_facts(subject_entity_id);
 CREATE INDEX idx_entity_facts_predicate ON entity_facts(predicate);
 CREATE INDEX idx_entity_facts_status ON entity_facts(status);
 CREATE INDEX idx_entity_facts_source_photo ON entity_facts(source_photo);
 CREATE INDEX idx_entity_facts_persona ON entity_facts(persona_id);
 CREATE INDEX idx_entity_facts_user_persona ON entity_facts(user_id, persona_id);
@@ -1,5 +0,0 @@
 -- SQLite can drop columns since 3.35 (March 2021); embedded
 -- libsqlite3-sys is well past that. Drop in reverse insert order so
 -- a partial down still leaves the schema valid.
 ALTER TABLE entity_facts DROP COLUMN valid_until;
 ALTER TABLE entity_facts DROP COLUMN valid_from;
@@ -1,25 +0,0 @@
 -- Add valid-time columns to entity_facts.
 --
 -- entity_facts already has created_at — *transaction time*, the
 -- moment WE recorded the fact. That's not the same as the real-world
 -- period the fact was true. "Cameron is_in_relationship_with X" was
 -- only true during a window; recording it in 2026 doesn't make it
 -- true today. Without the distinction, every former relationship,
 -- former job, former address reads as currently-true.
 --
 -- Adding two BIGINT NULL columns: valid_from / valid_until (unix
 -- seconds). NULL means "unbounded on that side" — `valid_from IS
 -- NULL` reads as "always-true-back-to-the-beginning",
 -- `valid_until IS NULL` as "still-true-now-or-unknown". Both NULL =
 -- temporal validity unknown (current state of all legacy rows).
 --
 -- Conflict detection refines accordingly: same-predicate facts with
 -- different objects stop flagging when their intervals are disjoint
 -- ("lives_in NYC 2018-2020" and "lives_in SF 2020-present" are both
 -- valid, just at different times).
 ALTER TABLE entity_facts ADD COLUMN valid_from BIGINT;
 ALTER TABLE entity_facts ADD COLUMN valid_until BIGINT;
 -- Optional partial index for time-bounded scans. Skipped for now —
 -- conflict detection runs per-entity (small N) and doesn't need it.
@@ -1,2 +0,0 @@
 DROP INDEX IF EXISTS idx_entity_facts_superseded_by;
 ALTER TABLE entity_facts DROP COLUMN superseded_by;
@@ -1,31 +0,0 @@
 -- Add a supersession pointer to entity_facts.
 --
 -- Status alone is a one-way trapdoor: 'rejected' loses the link
 -- between the rejected fact and the one that replaced it. For
 -- evolving facts (Cameron's relationship, employer, address) the
 -- curator wants to *replace* a stale fact with a new one and keep
 -- the history readable: "from 2018 until 2022 this was true, then
 -- it became this other thing".
 --
 -- A nullable INTEGER column pointing at another entity_facts.id —
 -- no FK constraint because SQLite can't ALTER ADD COLUMN with REFs;
 -- the DAO's delete_fact clears dangling pointers in the same
 -- transaction as the parent delete to keep the column honest.
 --
 -- A status of 'superseded' on the old fact (alongside the existing
 -- active / reviewed / rejected) signals "replaced by a newer
 -- claim". Read paths already filter 'rejected' out of the active
 -- view; the curation UI will treat 'superseded' the same way for
 -- conflict detection so they don't keep flagging.
 --
 -- Pairs with the valid-time columns from 2026-05-10-000100: the
 -- supersede action auto-stamps the old fact's `valid_until` from
 -- the new fact's `valid_from`, closing the interval cleanly.
 ALTER TABLE entity_facts ADD COLUMN superseded_by INTEGER;
 -- Helpful index for "show me what superseded this fact" walks
 -- (rare today; cheap to add now while the table is small).
 CREATE INDEX idx_entity_facts_superseded_by
    ON entity_facts(superseded_by)
    WHERE superseded_by IS NOT NULL;
@@ -1,4 +0,0 @@
 DROP INDEX IF EXISTS idx_entity_facts_created_by_backend;
 DROP INDEX IF EXISTS idx_entity_facts_created_by_model;
 ALTER TABLE entity_facts DROP COLUMN created_by_backend;
 ALTER TABLE entity_facts DROP COLUMN created_by_model;
@@ -1,30 +0,0 @@
 -- Track which model + backend generated each fact so the curator
 -- can audit which configurations produce trustworthy knowledge.
 --
 -- photo_insights already carries `model_version` + `backend`, and
 -- entity_facts.source_insight_id links to it — but:
 --   1. source_insight_id is only set after an insight is stored
 --      (post-loop), so chat-continuation facts and facts whose insight
 --      was regenerated lose the link.
 --   2. JOINing for every read is more friction than just embedding the
 --      provenance on the fact row itself.
 --   3. Manual facts (POST /knowledge/facts) have no insight at all and
 --      need to record "manual" as their provenance.
 --
 -- Two nullable TEXT columns are enough for the audit use case: model
 -- (e.g. "qwen2.5:7b", "anthropic/claude-sonnet-4") and backend
 -- ("local", "hybrid", "manual"). Pre-existing rows leave both NULL —
 -- legacy facts predate this tracking and can't be back-filled
 -- reliably from training_messages without burning compute.
 ALTER TABLE entity_facts ADD COLUMN created_by_model TEXT;
 ALTER TABLE entity_facts ADD COLUMN created_by_backend TEXT;
 -- Indexes are cheap and useful for "show me all facts from model X"
 -- audit queries — partial so the legacy NULL rows don't bloat them.
 CREATE INDEX idx_entity_facts_created_by_model
    ON entity_facts(created_by_model)
    WHERE created_by_model IS NOT NULL;
 CREATE INDEX idx_entity_facts_created_by_backend
    ON entity_facts(created_by_backend)
    WHERE created_by_backend IS NOT NULL;
@@ -1 +0,0 @@
 ALTER TABLE personas DROP COLUMN reviewed_only_facts;
@@ -1,16 +0,0 @@
 -- Per-persona toggle: when true, agent reads only see facts whose
 -- status is exactly 'reviewed' (human-verified). When false (the
 -- default), agent reads see 'active' OR 'reviewed' — everything not
 -- rejected or superseded.
 --
 -- The mobile app surfaces this as "Strict mode" on the persona
 -- editor: useful when you want a persona's chat to be grounded
 -- exclusively on the curated subset, e.g. for tasks where
 -- hallucinated agent claims are particularly costly.
 --
 -- Note: this is separate from `include_all_memories` (which unions
 -- across personas for hive-mind reads). Reviewed-only operates on
 -- the status axis; include_all_memories operates on the persona-
 -- scope axis. They compose freely.
 ALTER TABLE personas ADD COLUMN reviewed_only_facts BOOLEAN NOT NULL DEFAULT 0;
@@ -1,5 +0,0 @@
 ALTER TABLE personas DROP COLUMN allow_agent_corrections;
 DROP INDEX IF EXISTS idx_entity_facts_last_modified_at;
 ALTER TABLE entity_facts DROP COLUMN last_modified_at;
 ALTER TABLE entity_facts DROP COLUMN last_modified_by_backend;
 ALTER TABLE entity_facts DROP COLUMN last_modified_by_model;
@@ -1,30 +0,0 @@
 -- Three coupled changes for agent self-correction safety:
 --
 -- 1. `entity_facts.last_modified_by_*` + `last_modified_at` track who
 --    most recently mutated each fact. `created_by_*` from migration
 --    2026-05-10-000300 records who first wrote the row; this records
 --    who last *changed* it. Separate columns so the create vs update
 --    audit is independently grep-able ("show me every fact gpt-5
 --    altered last week" stays a single index scan).
 --
 -- 2. `personas.allow_agent_corrections` is the gate for the new
 --    agent-side `update_fact` / `supersede_fact` tools. Default OFF —
 --    a fresh persona's agent can create but can't alter or replace.
 --    Operator opts in per-persona after the model has earned trust,
 --    typically via the strict-mode flow (curate, then ratchet up
 --    agent autonomy as confidence rises). Parallel in shape to
 --    `reviewed_only_facts` from 2026-05-10-000400; they compose.
 --
 -- 3. Index on `last_modified_at` (partial, NOT NULL) for the
 --    audit-feed reads in the curation UI ("show recent agent edits
 --    sorted newest first").
 ALTER TABLE entity_facts ADD COLUMN last_modified_by_model TEXT;
 ALTER TABLE entity_facts ADD COLUMN last_modified_by_backend TEXT;
 ALTER TABLE entity_facts ADD COLUMN last_modified_at BIGINT;
 CREATE INDEX idx_entity_facts_last_modified_at
    ON entity_facts(last_modified_at)
    WHERE last_modified_at IS NOT NULL;
 ALTER TABLE personas ADD COLUMN allow_agent_corrections BOOLEAN NOT NULL DEFAULT 0;
@@ -1,6 +0,0 @@
 -- Irreversible: we collapsed multiple raw entity_type strings to
 -- canonical forms and don't have a per-row record of the original.
 -- The down migration is intentionally a no-op (the rewritten values
 -- are still semantically correct), and the up migration is safe to
 -- re-run because every UPDATE is conditional on `!= canonical`.
 SELECT 1;
@@ -1,43 +0,0 @@
 -- Canonicalize `entities.entity_type` so legacy rows from before
 -- `normalize_entity_type` landed in upsert_entity stop polluting
 -- client-side filters. Mirrors the synonym map in
 -- `src/database/knowledge_dao.rs::normalize_entity_type`:
 --   person  ← person | people | human | individual | contact
 --   place   ← place  | location | venue | site | area | landmark
 --   event   ← event  | occasion | activity | celebration
 --   thing   ← thing  | object | item | product
 -- Types outside the synonym set (e.g. "friend", "family") are not
 -- recognized as canonical and get a lowercase+trim pass instead, so
 -- at minimum case variants collapse.
 --
 -- `UPDATE OR IGNORE` skips rows that would violate UNIQUE(name,
 -- entity_type) after the rewrite. Two rows like ("Sarah", "person")
 -- + ("Sarah", "Person") would otherwise collide — the duplicate
 -- survives unchanged so the curator can merge it via the curation
 -- UI rather than have the migration silently delete data.
 UPDATE OR IGNORE entities
 SET entity_type = 'person'
 WHERE LOWER(TRIM(entity_type)) IN ('person', 'people', 'human', 'individual', 'contact')
  AND entity_type != 'person';
 UPDATE OR IGNORE entities
 SET entity_type = 'place'
 WHERE LOWER(TRIM(entity_type)) IN ('place', 'location', 'venue', 'site', 'area', 'landmark')
  AND entity_type != 'place';
 UPDATE OR IGNORE entities
 SET entity_type = 'event'
 WHERE LOWER(TRIM(entity_type)) IN ('event', 'occasion', 'activity', 'celebration')
  AND entity_type != 'event';
 UPDATE OR IGNORE entities
 SET entity_type = 'thing'
 WHERE LOWER(TRIM(entity_type)) IN ('thing', 'object', 'item', 'product')
  AND entity_type != 'thing';
 -- Anything left ("Friend" vs "friend") gets a lowercase+trim sweep
 -- so at least case variants of the same custom type collapse.
 UPDATE OR IGNORE entities
 SET entity_type = LOWER(TRIM(entity_type))
 WHERE entity_type != LOWER(TRIM(entity_type));
@@ -1,5 +0,0 @@
 DROP INDEX IF EXISTS idx_image_exif_date_backfill;
 CREATE INDEX idx_image_exif_date_backfill
    ON image_exif (library_id, id)
    WHERE date_taken IS NULL OR date_taken_source = 'fs_time';
@@ -1,18 +0,0 @@
 -- Narrow the date-backfill partial index to NULL-only rows.
 --
 -- The original index (2026-05-06-000000_add_date_taken_source) also matched
 -- `date_taken_source = 'fs_time'` so the drain could "re-resolve weak
 -- entries when better tools become available." In practice the resolver
 -- is deterministic on file bytes + filename + fs metadata: a row that
 -- landed on fs_time once will land on fs_time again on every subsequent
 -- tick. With `ORDER BY id ASC LIMIT 500`, the drain spun on the same
 -- lowest-id fs_time rows in perpetuity, never advancing, while hammering
 -- the SQLite write lock once per row and starving other writers (face
 -- PATCHes were hitting busy_timeout and returning 500). Drop fs_time
 -- from the eligibility set; if exiftool / a new filename pattern ever
 -- comes online, a one-shot operator command can re-resolve.
 DROP INDEX IF EXISTS idx_image_exif_date_backfill;
 CREATE INDEX idx_image_exif_date_backfill
    ON image_exif (library_id, id)
    WHERE date_taken IS NULL;
@@ -1,3 +0,0 @@
 DROP INDEX IF EXISTS idx_image_exif_clip_backfill;
 ALTER TABLE image_exif DROP COLUMN clip_model_version;
 ALTER TABLE image_exif DROP COLUMN clip_embedding;
@@ -1,27 +0,0 @@
 -- CLIP semantic photo search: store a per-photo image embedding so
 -- text queries can rerank against the live library via cosine
 -- similarity. Apollo encodes the bytes via its CLIP service; ImageApi
 -- writes the resulting blob here.
 --
 -- `clip_embedding` is the raw little-endian float32 buffer of an
 -- L2-normalized vector (dim depends on the model — 768 bytes×4 for
 -- ViT-L/14, 512 bytes×4 for ViT-B/32). Apollo always returns the
 -- normalized form so the search-time dot product reduces to a plain
 -- cosine similarity.
 --
 -- `clip_model_version` echoes the upstream `APOLLO_CLIP_MODEL` (e.g.
 -- "ViT-L/14"). A model swap shouldn't silently mix geometries — the
 -- backfill drain will re-eligibilize rows whose stored model_version
 -- differs from the live engine's, and the search route refuses to
 -- mix rows from two model_versions in the same response.
 ALTER TABLE image_exif ADD COLUMN clip_embedding BLOB;
 ALTER TABLE image_exif ADD COLUMN clip_model_version TEXT;
 -- Partial index for the backfill drain. Mirrors the shape of
 -- `idx_image_exif_date_backfill`: candidate rows are those with a
 -- known content_hash (so we don't race the unhashed backlog) but no
 -- embedding yet. SELECT cost stays O(missing rows) instead of full
 -- table scan once the column is mostly populated.
 CREATE INDEX IF NOT EXISTS idx_image_exif_clip_backfill
    ON image_exif (id)
    WHERE clip_embedding IS NULL AND content_hash IS NOT NULL;
@@ -1,3 +0,0 @@
 DROP INDEX IF EXISTS idx_insight_gen_jobs_status_cleanup;
 DROP INDEX IF EXISTS idx_insight_gen_jobs_file;
 DROP TABLE IF EXISTS insight_generation_jobs;
@@ -1,23 +0,0 @@
 -- Track async insight generation jobs so the client can poll for
 -- completion after the server returns 202 Accepted. Each generation
 -- creates a new row; the application layer cancels prior running
 -- jobs before inserting.
 CREATE TABLE insight_generation_jobs (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    library_id INTEGER NOT NULL DEFAULT 1,
    file_path TEXT NOT NULL,
    generation_type TEXT NOT NULL,
    status TEXT NOT NULL DEFAULT 'running',
    started_at INTEGER NOT NULL,
    completed_at INTEGER,
    result_insight_id INTEGER,
    error_message TEXT
 );
 -- For the status endpoint: fast lookup by (library_id, file_path)
 CREATE INDEX idx_insight_gen_jobs_file
    ON insight_generation_jobs(library_id, file_path);
 -- For startup cleanup (future): prune old completed/failed jobs
 CREATE INDEX idx_insight_gen_jobs_status_cleanup
    ON insight_generation_jobs(status, started_at);
@@ -1,28 +0,0 @@
 -- Restore UNIQUE constraint
 CREATE TABLE insight_generation_jobs_new (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    library_id INTEGER NOT NULL DEFAULT 1,
    file_path TEXT NOT NULL,
    generation_type TEXT NOT NULL,
    status TEXT NOT NULL DEFAULT 'running',
    started_at INTEGER NOT NULL,
    completed_at INTEGER,
    result_insight_id INTEGER,
    error_message TEXT,
    UNIQUE(library_id, file_path, generation_type)
 );
 INSERT INTO insight_generation_jobs_new
    SELECT id, library_id, file_path, generation_type, status, started_at, completed_at, result_insight_id, error_message
    FROM insight_generation_jobs;
 DROP TABLE insight_generation_jobs;
 ALTER TABLE insight_generation_jobs_new RENAME TO insight_generation_jobs;
 CREATE INDEX idx_insight_gen_jobs_file
    ON insight_generation_jobs(library_id, file_path);
 CREATE INDEX idx_insight_gen_jobs_status_cleanup
    ON insight_generation_jobs(status, started_at);
@@ -1,30 +0,0 @@
 -- Remove UNIQUE(library_id, file_path, generation_type) constraint to allow
 -- multiple job rows per file. This enables proper cancel/regenerate semantics:
 -- a new job is always inserted on regenerate, and the old job is cancelled
 -- independently. The application layer prevents concurrent running jobs.
 CREATE TABLE insight_generation_jobs_new (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    library_id INTEGER NOT NULL DEFAULT 1,
    file_path TEXT NOT NULL,
    generation_type TEXT NOT NULL,
    status TEXT NOT NULL DEFAULT 'running',
    started_at INTEGER NOT NULL,
    completed_at INTEGER,
    result_insight_id INTEGER,
    error_message TEXT
 );
 INSERT INTO insight_generation_jobs_new
    SELECT id, library_id, file_path, generation_type, status, started_at, completed_at, result_insight_id, error_message
    FROM insight_generation_jobs;
 DROP TABLE insight_generation_jobs;
 ALTER TABLE insight_generation_jobs_new RENAME TO insight_generation_jobs;
 CREATE INDEX idx_insight_gen_jobs_file
    ON insight_generation_jobs(library_id, file_path);
 CREATE INDEX idx_insight_gen_jobs_status_cleanup
    ON insight_generation_jobs(status, started_at);
@@ -1,11 +0,0 @@
 -- SQLite doesn't support DROP COLUMN before 3.35.0; recreate the table
 -- without the new columns. This is only needed for rollback.
 CREATE TABLE photo_insights_old AS
    SELECT id, library_id, rel_path, title, summary, generated_at,
           model_version, is_current, training_messages, approved,
           backend, fewshot_source_ids, content_hash
    FROM photo_insights;
 DROP TABLE photo_insights;
 ALTER TABLE photo_insights_old RENAME TO photo_insights;
@@ -1,8 +0,0 @@
 -- Persist generation parameters on each insight row for auditing.
 ALTER TABLE photo_insights ADD COLUMN num_ctx INTEGER;
 ALTER TABLE photo_insights ADD COLUMN temperature REAL;
 ALTER TABLE photo_insights ADD COLUMN top_p REAL;
 ALTER TABLE photo_insights ADD COLUMN top_k INTEGER;
 ALTER TABLE photo_insights ADD COLUMN min_p REAL;
 ALTER TABLE photo_insights ADD COLUMN system_prompt TEXT;
 ALTER TABLE photo_insights ADD COLUMN persona_id TEXT;
@@ -1,13 +0,0 @@
 -- SQLite doesn't support DROP COLUMN before 3.35.0; recreate the table
 -- without the token-count columns. This is only needed for rollback.
 CREATE TABLE photo_insights_old AS
    SELECT id, library_id, rel_path, title, summary, generated_at,
           model_version, is_current, training_messages, approved,
           backend, fewshot_source_ids, content_hash,
           num_ctx, temperature, top_p, top_k, min_p,
           system_prompt, persona_id
    FROM photo_insights;
 DROP TABLE photo_insights;
 ALTER TABLE photo_insights_old RENAME TO photo_insights;
@@ -1,6 +0,0 @@
 -- Persist token usage on each insight row. Split from
 -- 2026-05-27-000002_add_insight_generation_params because that
 -- migration was already applied on some environments before these
 -- columns were added.
 ALTER TABLE photo_insights ADD COLUMN prompt_eval_count INTEGER;
 ALTER TABLE photo_insights ADD COLUMN eval_count INTEGER;
@@ -1,392 +0,0 @@
 # Insight Chat improvements — design
 **Date:** 2026-05-07
 **Branch:** `feature/insight-chat-improvements` (in both `ImageApi/` and `FileViewer-React/`)
 **Scope:** ImageApi photo-anchored insight + chat surface, plus the
 FileViewer-React client. Apollo's free/visit chat is **not** in this cycle.
 ## Problem
 Three concrete gaps in today's insight + chat surface:
 1. **Tool drift.** ImageApi exposes 13 tools to the LLM. Some are gated on
   `apollo_enabled` / `has_vision`, but several optional ones
   (`search_rag`, `get_calendar_events`, `get_location_history`) are
   registered unconditionally even when their backing tables are empty.
   Descriptions vary in quality and a couple have outright bugs.
 2. **Inconsistent / incomplete tool descriptions.** Tools like
   `search_messages` describe their selection rules but omit useful
   examples; `store_fact` doesn't show the `object_entity_id` vs
   `object_value` choice; `get_sms_messages` accepts a `days_radius`
   parameter that the backing client silently ignores. The LLM is being
   instructed against a slightly wrong reality.
 3. **System prompt fights the persona.** Today's generation prompt
   prepends the user's `custom_system_prompt` and then immediately asserts
   `"You are a personal photo memory assistant..."`. The user message
   demands `"a detailed insight with a title and summary"`. Both
   contradict whatever voice / shape / POV the persona just established.
   On chat continuation the persona is baked into the stored transcript at
   generation time and can't be changed without regenerating.
 ## Goals
 - Tool catalog is **representative** — every tool registered for a turn is
  backed by data the user actually has.
 - Tool descriptions are **concise but complete**, with examples for any
  tool whose param choice has multiple modes or non-obvious interactions.
 - Persona / system prompt is **authoritative** for voice, length, and
  shape — both at generation and during chat continuation.
 - Per-turn system prompt overrides on chat work without surprising
  side-effects on the stored transcript outside `amend` mode.
 ## Non-goals
 - Apollo backend / frontend changes. Separate cycle.
 - Refactoring the `generate_photo_title` post-hoc title flow. Already
  takes `custom_system_prompt`.
 - Tool consolidation (e.g. merging `search_messages` + `get_sms_messages`).
  Considered and deferred — keeps blast radius small.
 - Removing knowledge-memory tools (`recall_*` / `store_*`). Audit
  confirmed they have a live read path via `knowledge.rs` HTTP routes.
 - Persisting persona changes to the stored transcript outside `amend`
  mode. Deliberate — re-opens use the persona currently active in the
  client, not a sticky historical setting.
 ---
 ## Design
 ### A. System prompt — generation
 Today (`insight_generator.rs:3305–3326`):
 ```
 [custom_system_prompt if any] +
 "You are a personal photo memory assistant helping to reconstruct..." +
 {owner_id_note} +
 {fewshot_block} +
 "IMPORTANT INSTRUCTIONS:
 1. You MUST call multiple tools...
 2. When calling get_sms_messages and search_rag...
 3. Use recall_facts_for_photo...
 ...
 8. You have a hard budget of {max_iterations} iterations..."
 ```
 The first concatenation is the bug: `custom` claims one identity, the
 next line asserts another.
 **New structure** — two named blocks, in order:
 ```
 [Identity / voice / format block]    ← persona-controlled (or neutral default)
 [Procedural block]                   ← always identity-free
 ```
 **Identity block:**
 - When `custom_system_prompt` is supplied: use that string verbatim, no
  pre/append.
 - When not: a neutral default that doesn't fight a future persona.
  Working text: `"You are reconstructing a memory from a photo. Use the
  gathered context to write a thoughtful summary; you decide voice,
  length, and shape."`
 **Procedural block** — identity-free, always emitted:
 ```
 Tool-use guidance:
 - You have a budget of {max_iterations} tool-calling iterations.
 - Call tools to gather context BEFORE writing your final answer; don't
  answer after one or two calls.
 - When calling get_sms_messages or search_rag, make at least one call
  WITHOUT a contact filter — surrounding events matter even when a
  contact is known.
 - Use recall_facts_for_photo + recall_entities to load any prior
  knowledge about subjects in the photo.
 - When you identify people / places / events / things, use store_entity
  + store_fact to grow the persistent memory.
 - A tool returning no results is informative; continue with the others.
 {owner_id_note if applicable}
 {fewshot_block if applicable}
 ```
 Differences from today's "IMPORTANT INSTRUCTIONS" block: removed the
 "you are a personal photo memory assistant" framing and the explicit
 "at least 5 tool calls" floor (replaced with the softer "don't answer
 after one or two"). Few-shot stays — it's pattern-of-tool-use, not
 identity.
 ### B. User message — generation
 Today (line 3357):
 ```
 {visual_block}Please analyze this photo and gather any relevant context
 from the surrounding weeks.
 Photo file path: {file_path}
 Date taken: {date}
 {contact_info}
 {gps_info}
 {tags_info}
 Use the available tools to gather more context about this moment
 (messages, calendar events, location history, etc.), then write a
 detailed insight with a title and summary.
 ```
 Problems: the trailing line bakes in output shape ("title and
 summary"), and the title from the resulting response is **discarded
 anyway** — `generate_photo_title` (line 3494) regenerates the title
 post-hoc from the summary. So the prompt is constraining voice for no
 data-model benefit.
 **New payload** — context-only, no output prescription:
 ```
 {visual_block}Photo file path: {file_path}
 Date taken: {date}
 {contact_info}
 {gps_info}
 {tags_info}
 Gather context with the available tools, then respond.
 ```
 The persona owns shape. If a user wants "title-then-paragraph" output,
 their persona prompt says so.
 ### C. System prompt — chat continuation
 Add `system_prompt: Option<String>` to `ChatTurnRequest` (and to its
 HTTP wrapper `ChatTurnHttpRequest`). It carries through both the
 non-streaming `chat_turn` and the streaming `chat_turn_stream`.
 **Append mode (default, `amend=false`)** — ephemeral
 swap-and-restore, mirroring the existing `annotate_system_with_budget`
 pattern:
 1. Load stored transcript.
 2. If `system_prompt` is `Some(s)`:
   - If first message is a `system` role: stash original content,
     replace with `s`.
   - Else: prepend a synthetic ephemeral system message with `s` (note
     it's synthetic so the restore step pops it rather than rewriting).
 3. Run `annotate_system_with_budget` on top (existing per-turn budget
   note appends to whatever's there now).
 4. Run the agentic loop.
 5. **Before persistence**, restore the original system content (or pop
   the synthetic one). Run `restore_system_content` for the budget
   annotation as today.
 6. Save.
 Result: the model sees the override; the stored transcript is
 unchanged outside the model's actual reply.
 **Amend mode (`amend=true`)**:
 - If `system_prompt` is supplied: the override stays in place during
  the serialization for the new insight row. The new row's
  `training_messages` system message is the override. `is_current=false`
  flips on prior rows as today.
 - If not supplied: behaves as today (stored transcript's system message
  carries forward unchanged).
 ### D. FileViewer-React — client wiring
 `hooks/useInsightChat.tsx`:
 - `SendTurnOptions` gains `systemPromptOverride?: string | null`.
 - Inside `sendTurn`, before issuing the streaming POST:
  1. Read the active persona's `systemPrompt` from AsyncStorage
     (already loaded for generation flows — reuse the same accessor).
  2. If a one-shot `systemPromptOverride` is set, append as a suffix
     (`${persona}\n\n${override}`) so persona voice survives + override
     tweaks the turn.
  3. Include the resulting string as `system_prompt` on the request body.
 - No history-load change. The history endpoint still returns the stored
  transcript.
 `components/InsightChatModal.tsx`:
 - Add a small "Style note" composer affordance — a one-shot text input
  that, when filled, becomes the `systemPromptOverride` for the next
  send. Cleared after send.
 - The existing persona chip continues to open `PersonaManagerModal`.
 `hooks/usePersonas.tsx` and the bundled defaults:
 - Built-in `assistant` and `journal` prompts get audited and rewritten
  to **explicitly state voice / shape / length** — since the framework
  no longer guarantees a default shape, the persona must.
 ### E. Tool catalog — gating
 Widen `build_tool_definitions` from `(has_vision: bool, apollo_enabled:
 bool)` to a single `ToolGateOpts` struct:
 ```rust
 pub struct ToolGateOpts {
    pub has_vision: bool,
    pub apollo_enabled: bool,
    pub daily_summaries_present: bool,
    pub calendar_present: bool,
    pub location_history_present: bool,
 }
 ```
 The chat / generation services compute the three new fields lazily per
 turn via `SELECT 1 FROM <table> LIMIT 1` (cheap; cached for the turn's
 duration). Lazy because operators import data after launch and we don't
 want to require a restart for the LLM to discover its new capabilities.
 Per-tool gating:
 | Tool | Existing gate | New gate |
 |---|---|---|
 | `describe_photo` | `has_vision` | unchanged |
 | `get_personal_place_at` | `apollo_enabled` | unchanged |
 | `get_calendar_events` | none | `calendar_present` |
 | `get_location_history` | none | `location_history_present` |
 | `search_rag` | none | `daily_summaries_present` |
 All other tools always-on. (`get_sms_messages` and `search_messages`
 fail informatively if SMS-API is unreachable; not worth a startup probe
 since intermittent failures are the same shape.)
 ### F. Tool descriptions — convention
 Every description follows:
 1. One sentence: **what** + **when to call**.
 2. Param semantics worth knowing (units, ranges, mode behavior,
   precedence).
 3. **Example invocation** for tools with multiple modes, optional bands,
   or non-obvious parameter interactions.
 4. Cross-references when relevant: `prefer X when both apply`.
 Banned: all-caps section headers inside descriptions
 (`"CONTENT search"`, `"TIME-BASED fetch"`); persona-prescriptive language
 (`"you are a..."`); behavioral references to other tools by description
 rather than name.
 Tools getting examples: `search_messages`, `search_rag`, `store_fact`,
 `get_sms_messages`. Trivial tools (`get_current_datetime`,
 `reverse_geocode`, `get_file_tags`) skip the example.
 Sample (`search_messages`):
 > Search SMS/MMS message bodies. Modes: `fts5` (keyword + phrase + prefix
 > + AND/OR/NOT + NEAR proximity), `semantic` (embedding similarity,
 > requires generated embeddings), `hybrid` (RRF merge, recommended;
 > degrades to `fts5` when embeddings absent). Optional `start_ts` /
 > `end_ts` (real-UTC unix seconds) and `contact_id` filters. For pure
 > date / contact browsing without keywords, prefer `get_sms_messages`.
 >
 > Examples:
 > - `{query: "trader joe's"}` — phrase across all time.
 > - `{query: "dinner", contact_id: 42, start_ts: 1700000000, end_ts: 1700604800}`
 >   — keyword within a contact and a week.
 > - `{query: "NEAR(meeting work, 5)"}` — proximity search.
 ### G. SMS tool fixes
 #### `get_sms_messages` — honor `days_radius`
 Today: `sms_client::fetch_messages_for_contact(contact, center_ts)`
 hardcodes `Duration::days(4)` (lines 31–37). The tool accepts
 `days_radius` and silently ignores it.
 **Fix:** widen the signature to
 `fetch_messages_for_contact(contact, center_ts, days_radius)`. Tool
 plumbs through. Default 4 retained for back-compat.
 #### `search_messages` — add date and contact_id filters
 Today: ImageApi's `search_messages` only forwards `query`, `mode`,
 `limit` to SMS-API.
 **Fix:** add `start_ts`, `end_ts`, `contact_id` parameters.
 - `contact_id` forwards directly to SMS-API
  (`/api/messages/search/?contact_id=`).
 - `start_ts` / `end_ts` are not natively accepted by SMS-API's search
  endpoint. Apply client-side post-filter on the response (Apollo's
  pattern: `chat_tools.py:670–680`). Bump the SMS-API `limit` to a
  larger fetch pool when a date filter is supplied so in-window matches
  aren't lost to out-of-window FTS rank.
 ---
 ## Implementation sequencing
 Each step is independently mergeable.
 ### ImageApi PRs
 1. **Split system-prompt assembly + neutralize user message.** Two
   named blocks; user message context-only. Default identity string
   added. Tests: golden snapshots of the resulting `system_content`
   with and without `custom_system_prompt`.
 2. **`system_prompt` field on chat request + swap/restore + amend
   persistence.** Mirrors `annotate_system_with_budget` pattern. Tests:
   round-trip system content unchanged in append mode; persisted in
   amend mode.
 3. **`fetch_messages_for_contact` honors `days_radius`.** Tool wires
   the param through. Tests: window math at the client level.
 4. **`ToolGateOpts` + per-tool description rewrites.** Description
   text changes are the bulk of the diff but no behavior change beyond
   gating.
 ### FileViewer-React PR
 5. **Chat hook sends `system_prompt`; modal gets style-note input;
   built-in personas updated to specify shape.** The
   `useInsightChat.sendTurn` call site picks up the persona and
   includes it on every chat turn body. Style-note input is a one-shot
   suffix.
 ## Testing & verification
 **Automated:**
 - Unit (Rust): swap-and-restore round-trip preserves stored transcript.
 - Unit (Rust): amend mode persists override into new insight row.
 - Unit (Rust): `fetch_messages_for_contact(days_radius=N)` produces a
  window of `2N` days centered on `center_ts`.
 - Unit (Rust): `build_tool_definitions(opts)` excludes gated tools when
  the corresponding flag is false.
 **Manual:**
 - Run a chat turn against an existing insight without `system_prompt` →
  output unchanged from baseline.
 - Same insight, with override → output reflects new voice.
 - Re-open chat → original baked persona still authoritative (override
  was ephemeral).
 - Regenerate an insight with the journal persona → model's voice
  matches journal style; no "memory assistant" framing leaks through.
 - Toggle data presence (delete a row from `calendar_events`) → tool
  drops from the catalog on the next turn.
 ## Risks
 - **Default identity wording matters.** A too-neutral default ("Use the
  gathered context to write a summary") might produce flatter output
  than today's "personal photo memory assistant" framing for users
  who never set a persona. Mitigation: tune the default with a small
  set of test photos before merging.
 - **Persona-suffix style notes can contradict persona voice.** A user
  who picks `journal` (first person, warm) and adds the style note
  "respond in bullet points" will get a tonal collision. Acceptable —
  the user expressed a per-turn intent and we honor it. Document the
  composition rule in the persona-manager UI.
 - **Lazy data-presence probes add a per-turn `SELECT 1`.** Negligible
  on SQLite (sub-millisecond) but adds up across many turns. Cache the
  result for the turn's duration; don't re-probe per-tool.
 ## Open questions
 None blocking. Items deferred to a possible follow-up cycle:
 - Apollo parity for the same per-turn override pattern (already
  present; just needs RN client wiring on the photo path which is
  already proxy).
 - Tool consolidation (`search_messages` + `get_sms_messages` →
  single `search_messages` with optional date filter, Apollo-style).
  Considered and deferred — separate spec.
@@ -1,140 +0,0 @@
 use anyhow::{Result, anyhow};
 use crate::ai::llm_client::LlmClient;
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub enum BackendKind {
    Local,
    Hybrid,
 }
 impl BackendKind {
    pub fn parse(s: &str) -> Result<Self> {
        match s.trim().to_lowercase().as_str() {
            "local" | "" => Ok(Self::Local),
            "hybrid" => Ok(Self::Hybrid),
            other => Err(anyhow!(
                "unknown backend '{}'; expected 'local' or 'hybrid'",
                other
            )),
        }
    }
    pub fn as_str(&self) -> &'static str {
        match self {
            Self::Local => "local",
            Self::Hybrid => "hybrid",
        }
    }
 }
 impl std::fmt::Display for BackendKind {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.write_str(self.as_str())
    }
 }
 pub struct SamplingOverrides {
    pub model: Option<String>,
    pub num_ctx: Option<i32>,
    pub temperature: Option<f32>,
    pub top_p: Option<f32>,
    pub top_k: Option<i32>,
    pub min_p: Option<f32>,
 }
 impl SamplingOverrides {
    pub fn has_sampling(&self) -> bool {
        self.temperature.is_some()
            || self.top_p.is_some()
            || self.top_k.is_some()
            || self.min_p.is_some()
    }
 }
 pub struct ResolvedBackend {
    chat: Box<dyn LlmClient>,
    local: Box<dyn LlmClient>,
    pub kind: BackendKind,
    /// `true` when the chat model receives images directly (Ollama with
    /// vision, or llamacpp). `false` for hybrid where we describe-then-inline.
    pub images_inline: bool,
 }
 impl ResolvedBackend {
    pub fn new(
        chat: Box<dyn LlmClient>,
        local: Box<dyn LlmClient>,
        kind: BackendKind,
        images_inline: bool,
    ) -> Self {
        Self {
            chat,
            local,
            kind,
            images_inline,
        }
    }
    pub fn chat(&self) -> &dyn LlmClient {
        self.chat.as_ref()
    }
    pub fn local(&self) -> &dyn LlmClient {
        self.local.as_ref()
    }
    pub fn model(&self) -> &str {
        self.chat.primary_model()
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    #[test]
    fn parse_backend_kind() {
        assert_eq!(BackendKind::parse("local").unwrap(), BackendKind::Local);
        assert_eq!(BackendKind::parse("hybrid").unwrap(), BackendKind::Hybrid);
        assert_eq!(BackendKind::parse("  Local ").unwrap(), BackendKind::Local);
        assert_eq!(BackendKind::parse("HYBRID").unwrap(), BackendKind::Hybrid);
        assert_eq!(BackendKind::parse("").unwrap(), BackendKind::Local);
        assert!(BackendKind::parse("vllm").is_err());
    }
    #[test]
    fn backend_kind_as_str_roundtrips() {
        assert_eq!(
            BackendKind::parse(BackendKind::Local.as_str()).unwrap(),
            BackendKind::Local
        );
        assert_eq!(
            BackendKind::parse(BackendKind::Hybrid.as_str()).unwrap(),
            BackendKind::Hybrid
        );
    }
    #[test]
    fn sampling_overrides_has_sampling() {
        let empty = SamplingOverrides {
            model: None,
            num_ctx: None,
            temperature: None,
            top_p: None,
            top_k: None,
            min_p: None,
        };
        assert!(!empty.has_sampling());
        let with_temp = SamplingOverrides {
            model: None,
            num_ctx: Some(4096),
            temperature: Some(0.7),
            top_p: None,
            top_k: None,
            min_p: None,
        };
        assert!(with_temp.has_sampling());
    }
 }
@@ -1,392 +0,0 @@
 //! Thin async HTTP client for Apollo's `/api/internal/clip/*` endpoints.
 //!
 //! Apollo hosts the OpenAI CLIP inference service (ViT-L/14 by default,
 //! configurable via `APOLLO_CLIP_MODEL`). This client is the ImageApi side
 //! of the contract: shove image bytes through `/encode_image` to populate
 //! `image_exif.clip_embedding` during backfill, and call `/encode_text` to
 //! encode a user's natural-language query at search time. The actual
 //! cosine-similarity rerank runs locally in ImageApi.
 //!
 //! Mirrors `face_client.rs` / `tag_client.rs` shape: optional base URL
 //! (None = disabled — feature off, drain and search no-op), reqwest
 //! client with a generous timeout because GPU inference under a backlog
 //! can queue server-side (Apollo's threadpool is bounded to 1 worker on
 //! CUDA).
 //!
 //! Configured via `APOLLO_CLIP_API_BASE_URL`, falling back to
 //! `APOLLO_API_BASE_URL` when the dedicated var is unset (single-Apollo
 //! deploys are the common case).
 //!
 //! Wire format:
 //! - `/encode_image`: multipart/form-data with `file=<bytes>` and
 //!   `meta=<json>` (content_hash / library_id / rel_path for logging).
 //! - `/encode_text`: JSON `{"text": "<query>"}`.
 //!
 //! Both return `{model_version, embedding_dim, duration_ms, embedding}`
 //! where `embedding` is base64 of `dim×4` little-endian float32 bytes,
 //! L2-normalized so the rerank reduces to a plain dot product.
 //!
 //! Error mapping (reflected in [`ClipError`]):
 //! - 422 `decode_failed` / `empty_text` → permanent: ImageApi marks the
 //!   row failed or surfaces the empty-query error to the search caller.
 //! - 503 `cuda_oom` / `engine_unavailable` → defer-and-retry: no marker.
 //! - Any other 5xx / network error → defer.
 use anyhow::{Context, Result};
 use base64::Engine;
 use reqwest::Client;
 use serde::{Deserialize, Serialize};
 use std::time::Duration;
 #[derive(Debug, Clone, Serialize)]
 pub struct EncodeImageMeta {
    pub content_hash: String,
    pub library_id: i32,
    pub rel_path: String,
 }
 #[derive(Debug, Clone, Deserialize)]
 #[allow(dead_code)] // duration_ms logged by the backfill drain
 pub struct EncodeResponse {
    pub model_version: String,
    pub embedding_dim: i32,
    pub duration_ms: i64,
    /// base64 of `embedding_dim * 4` bytes (LE float32). ImageApi stores
    /// the decoded bytes verbatim as a BLOB.
    pub embedding: String,
 }
 impl EncodeResponse {
    /// Decode the wire-format embedding back into raw bytes for storage.
    /// Validates the buffer is `embedding_dim * 4` bytes long so a
    /// malformed response surfaces here rather than as a downstream
    /// silent length mismatch.
    pub fn decode_embedding(&self) -> Result<Vec<u8>> {
        let bytes = base64::engine::general_purpose::STANDARD
            .decode(self.embedding.as_bytes())
            .context("clip embedding base64 decode")?;
        let expected = (self.embedding_dim as usize) * 4;
        if bytes.len() != expected {
            anyhow::bail!(
                "clip embedding wrong size: got {} bytes, expected {} ({} * 4)",
                bytes.len(),
                expected,
                self.embedding_dim
            );
        }
        Ok(bytes)
    }
 }
 #[derive(Debug, Clone, Deserialize)]
 #[allow(dead_code)] // load_error consumed by future health probe
 pub struct ClipHealth {
    pub loaded: bool,
    pub device: String,
    pub model_version: String,
    pub embedding_dim: i32,
    #[serde(default)]
    pub load_error: Option<String>,
 }
 #[derive(Debug)]
 pub enum ClipError {
    /// Apollo refused for a reason that won't change on retry (decode
    /// failure on /encode_image, empty text on /encode_text).
    Permanent(anyhow::Error),
    /// Apollo couldn't process this turn but might next time (CUDA OOM,
    /// engine not loaded, network hiccup).
    Transient(anyhow::Error),
    /// Feature is disabled (no `APOLLO_CLIP_API_BASE_URL` /
    /// `APOLLO_API_BASE_URL`).
    Disabled,
 }
 impl std::fmt::Display for ClipError {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        match self {
            ClipError::Permanent(e) => write!(f, "permanent: {e}"),
            ClipError::Transient(e) => write!(f, "transient: {e}"),
            ClipError::Disabled => write!(f, "clip client disabled"),
        }
    }
 }
 impl std::error::Error for ClipError {}
 #[derive(Clone)]
 pub struct ClipClient {
    client: Client,
    base_url: Option<String>,
 }
 impl ClipClient {
    pub fn new(base_url: Option<String>) -> Self {
        let timeout_secs = std::env::var("CLIP_REQUEST_TIMEOUT_SEC")
            .ok()
            .and_then(|s| s.parse::<u64>().ok())
            .unwrap_or(60);
        let client = Client::builder()
            .timeout(Duration::from_secs(timeout_secs))
            .build()
            .expect("reqwest client build");
        Self {
            client,
            base_url: base_url.map(|u| u.trim_end_matches('/').to_string()),
        }
    }
    /// Read both standard env vars. `APOLLO_CLIP_API_BASE_URL` wins;
    /// fallback to `APOLLO_API_BASE_URL`. Both unset → disabled.
    pub fn from_env() -> Self {
        let base = std::env::var("APOLLO_CLIP_API_BASE_URL")
            .ok()
            .filter(|s| !s.trim().is_empty())
            .or_else(|| {
                std::env::var("APOLLO_API_BASE_URL")
                    .ok()
                    .filter(|s| !s.trim().is_empty())
            });
        Self::new(base)
    }
    pub fn is_enabled(&self) -> bool {
        self.base_url.is_some()
    }
    /// Encode an image to a 768-d (ViT-L/14) or 512-d (ViT-B/32)
    /// L2-normalized embedding. Used by the backfill drain.
    pub async fn encode_image(
        &self,
        bytes: Vec<u8>,
        meta: EncodeImageMeta,
    ) -> std::result::Result<EncodeResponse, ClipError> {
        let Some(base) = self.base_url.as_deref() else {
            return Err(ClipError::Disabled);
        };
        let url = format!("{}/api/internal/clip/encode_image", base);
        let meta_json = serde_json::to_string(&meta)
            .map_err(|e| ClipError::Permanent(anyhow::anyhow!("meta serialize: {e}")))?;
        let form = reqwest::multipart::Form::new()
            .text("meta", meta_json)
            .part(
                "file",
                reqwest::multipart::Part::bytes(bytes)
                    .file_name(meta.rel_path.clone())
                    .mime_str("application/octet-stream")
                    .unwrap_or_else(|_| reqwest::multipart::Part::bytes(Vec::new())),
            );
        self.send_multipart(&url, form).await
    }
    /// Encode a natural-language query to an embedding. Used by the
    /// search route to rank stored image embeddings by cosine sim.
    pub async fn encode_text(&self, text: &str) -> std::result::Result<EncodeResponse, ClipError> {
        let Some(base) = self.base_url.as_deref() else {
            return Err(ClipError::Disabled);
        };
        let url = format!("{}/api/internal/clip/encode_text", base);
        let body = serde_json::json!({ "text": text });
        let resp = match self.client.post(&url).json(&body).send().await {
            Ok(r) => r,
            Err(e) if e.is_timeout() || e.is_connect() => {
                return Err(ClipError::Transient(anyhow::anyhow!(
                    "clip client network: {e}"
                )));
            }
            Err(e) => {
                return Err(ClipError::Transient(anyhow::anyhow!(
                    "clip client request: {e}"
                )));
            }
        };
        let status = resp.status();
        if status.is_success() {
            let body: EncodeResponse = resp
                .json()
                .await
                .map_err(|e| ClipError::Transient(anyhow::anyhow!("clip response decode: {e}")))?;
            return Ok(body);
        }
        let body_text = resp.text().await.unwrap_or_default();
        Err(classify_error_response(status.as_u16(), &body_text))
    }
    /// Engine reachability + device/model report. Used as a startup
    /// sanity check from the probe binary and (later) the backlog drain.
    #[allow(dead_code)] // consumed by probe + drain
    pub async fn health(&self) -> Result<ClipHealth> {
        let base = self.base_url.as_deref().context("clip client disabled")?;
        let url = format!("{}/api/internal/clip/health", base);
        let resp = self.client.get(&url).send().await?.error_for_status()?;
        let body: ClipHealth = resp.json().await?;
        Ok(body)
    }
    async fn send_multipart(
        &self,
        url: &str,
        form: reqwest::multipart::Form,
    ) -> std::result::Result<EncodeResponse, ClipError> {
        let resp = match self.client.post(url).multipart(form).send().await {
            Ok(r) => r,
            Err(e) if e.is_timeout() || e.is_connect() => {
                return Err(ClipError::Transient(anyhow::anyhow!(
                    "clip client network: {e}"
                )));
            }
            Err(e) => {
                return Err(ClipError::Transient(anyhow::anyhow!(
                    "clip client request: {e}"
                )));
            }
        };
        let status = resp.status();
        if status.is_success() {
            let body: EncodeResponse = resp
                .json()
                .await
                .map_err(|e| ClipError::Transient(anyhow::anyhow!("clip response decode: {e}")))?;
            return Ok(body);
        }
        let body_text = resp.text().await.unwrap_or_default();
        Err(classify_error_response(status.as_u16(), &body_text))
    }
 }
 /// Pulled out as a pure function so the marker-row contract is unit-
 /// testable without spinning up an HTTP server. Matches the shape used
 /// by face_client::classify_error_response so future retry policies
 /// can share code.
 fn classify_error_response(status: u16, body_text: &str) -> ClipError {
    let detail_code = serde_json::from_str::<serde_json::Value>(body_text)
        .ok()
        .and_then(|v| {
            v.get("detail")
                .and_then(|d| d.as_str().map(str::to_string))
                .or_else(|| {
                    v.get("detail")
                        .and_then(|d| d.get("code"))
                        .and_then(|c| c.as_str())
                        .map(str::to_string)
                })
        })
        .unwrap_or_default();
    if status == 422 {
        return ClipError::Permanent(anyhow::anyhow!(
            "clip {} {}: {}",
            status,
            detail_code,
            body_text
        ));
    }
    if status == 503 {
        return ClipError::Transient(anyhow::anyhow!(
            "clip {} {}: {}",
            status,
            detail_code,
            body_text
        ));
    }
    // 408 / 413 / 429 are operator-fixable infra issues; defer.
    if matches!(status, 408 | 413 | 429) {
        return ClipError::Transient(anyhow::anyhow!(
            "clip {} {}: {}",
            status,
            detail_code,
            body_text
        ));
    }
    if (400..500).contains(&status) {
        ClipError::Permanent(anyhow::anyhow!(
            "clip {} {}: {}",
            status,
            detail_code,
            body_text
        ))
    } else {
        ClipError::Transient(anyhow::anyhow!(
            "clip {} {}: {}",
            status,
            detail_code,
            body_text
        ))
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    fn is_permanent(e: &ClipError) -> bool {
        matches!(e, ClipError::Permanent(_))
    }
    fn is_transient(e: &ClipError) -> bool {
        matches!(e, ClipError::Transient(_))
    }
    #[test]
    fn classify_422_decode_failed_is_permanent() {
        assert!(is_permanent(&classify_error_response(
            422,
            r#"{"detail":"decode_failed: bad bytes"}"#
        )));
    }
    #[test]
    fn classify_422_empty_text_is_permanent() {
        assert!(is_permanent(&classify_error_response(
            422,
            r#"{"detail":"empty_text"}"#
        )));
    }
    #[test]
    fn classify_503_cuda_oom_is_transient() {
        assert!(is_transient(&classify_error_response(
            503,
            r#"{"detail":{"code":"cuda_oom","error":"out of memory"}}"#,
        )));
    }
    #[test]
    fn classify_5xx_is_transient_other_4xx_is_permanent() {
        assert!(is_transient(&classify_error_response(500, "")));
        assert!(is_permanent(&classify_error_response(404, "{}")));
    }
    #[test]
    fn classify_infra_4xx_is_transient() {
        assert!(is_transient(&classify_error_response(408, "")));
        assert!(is_transient(&classify_error_response(413, "<html>")));
        assert!(is_transient(&classify_error_response(429, "{}")));
    }
    #[test]
    fn decode_embedding_size_mismatch_errors() {
        // dim=4 says we expect 16 bytes (4 floats × 4 bytes). Encode 8.
        use base64::Engine;
        let resp = EncodeResponse {
            model_version: "ViT-L/14".into(),
            embedding_dim: 4,
            duration_ms: 0,
            embedding: base64::engine::general_purpose::STANDARD.encode([0u8; 8]),
        };
        assert!(resp.decode_embedding().is_err());
    }
    #[test]
    fn decode_embedding_round_trip() {
        use base64::Engine;
        let bytes: Vec<u8> = (0..16).collect();
        let resp = EncodeResponse {
            model_version: "ViT-L/14".into(),
            embedding_dim: 4,
            duration_ms: 0,
            embedding: base64::engine::general_purpose::STANDARD.encode(&bytes),
        };
        assert_eq!(resp.decode_embedding().unwrap(), bytes);
    }
 }
@@ -170,55 +170,3 @@ pub struct ModelCapabilities {
    pub has_vision: bool,
    pub has_tool_calling: bool,
 }
 /// Strip a leading `<think>…</think>` reasoning block from model output.
 ///
 /// Thinking models sometimes emit chain-of-thought inside think tags before
 /// the real answer. Everything after the first `</think>` is the answer;
 /// when no tag is present — or the text after it is empty — the trimmed
 /// input is returned unchanged. Mirrors the behavior Ollama's
 /// `extract_final_answer` has applied to single-shot generation; shared here
 /// so the tool-calling final-content paths (agentic generation + chat) can
 /// apply the identical cleanup before parsing / persisting.
 pub fn strip_think_blocks(response: &str) -> String {
    let response = response.trim();
    if let Some(pos) = response.find("</think>") {
        let answer = response[pos + "</think>".len()..].trim();
        if !answer.is_empty() {
            return answer.to_string();
        }
    }
    response.to_string()
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    #[test]
    fn strip_think_blocks_removes_leading_think_block() {
        let raw = "<think>\nLet me reason about this.\n</think>\n\nTitle: A Day Out\n\nThe body.";
        assert_eq!(strip_think_blocks(raw), "Title: A Day Out\n\nThe body.");
    }
    #[test]
    fn strip_think_blocks_passes_through_plain_content() {
        assert_eq!(strip_think_blocks("  just an answer  "), "just an answer");
    }
    #[test]
    fn strip_think_blocks_keeps_content_when_answer_after_tag_is_empty() {
        // A think block with nothing after it: better to return the trimmed
        // original than an empty string (matches Ollama's fallback).
        let raw = "<think>only thoughts</think>";
        assert_eq!(strip_think_blocks(raw), raw);
    }
    #[test]
    fn strip_think_blocks_handles_unclosed_tag() {
        let raw = "<think>thinking forever";
        assert_eq!(strip_think_blocks(raw), raw);
    }
 }
@@ -1,18 +1,13 @@
 pub mod apollo_client;
 pub mod backend;
 pub mod clip_client;
 pub mod daily_summary_job;
 pub mod face_client;
 pub mod handlers;
 pub mod insight_chat;
 pub mod insight_generator;
 pub mod llamacpp;
 pub mod llm_client;
 pub mod ollama;
 pub mod openrouter;
 pub mod sms_client;
 pub mod tts;
 pub mod turn_registry;
 // strip_summary_boilerplate is used by binaries (test_daily_summary), not the library
 #[allow(unused_imports)]
@@ -21,25 +16,18 @@ pub use daily_summary_job::{
    generate_daily_summaries, strip_summary_boilerplate,
 };
 pub use handlers::{
-    cancel_generation_handler, cancel_turn_handler, chat_history_handler, chat_rewind_handler,
+    chat_history_handler, chat_rewind_handler, chat_stream_handler, chat_turn_handler,
-    chat_stream_handler, chat_turn_handler, delete_insight_handler, export_training_data_handler,
+    delete_insight_handler, export_training_data_handler, generate_agentic_insight_handler,
-    generate_agentic_insight_handler, generate_insight_handler, generation_status_handler,
+    generate_insight_handler, get_all_insights_handler, get_available_models_handler,
-    get_all_insights_handler, get_available_models_handler, get_insight_handler,
+    get_insight_handler, get_openrouter_models_handler, rate_insight_handler,
    get_insight_history_handler, get_openrouter_models_handler, rate_insight_handler,
    turn_async_handler, turn_replay_handler,
 };
 pub use insight_generator::InsightGenerator;
 pub use llamacpp::LlamaCppClient;
 #[allow(unused_imports)]
 pub use llm_client::{
    ChatMessage, LlmClient, ModelCapabilities, Tool, ToolCall, ToolCallFunction, ToolFunction,
 };
 pub use ollama::{EMBEDDING_MODEL, OllamaClient};
 pub use sms_client::{SmsApiClient, SmsMessage};
 pub use tts::{
    create_voice_from_library_handler, create_voice_upload_handler, list_voices_handler,
    tts_speech_handler,
 };
 /// Display name used for the user in message transcripts and first-person
 /// prompt text. Reads the `USER_NAME` env var; defaults to `"Me"`. Models
@@ -49,88 +37,3 @@ pub use tts::{
 pub fn user_display_name() -> String {
    std::env::var("USER_NAME").unwrap_or_else(|_| "Me".to_string())
 }
 /// One switch for the "local" LLM stack: when `LLM_BACKEND=llamacpp` is
 /// set, chat / vision describe / embeddings all route through llama-swap
 /// instead of Ollama. Any other value (including unset, the default) is
 /// Ollama. This is intentionally global — embeddings must be drawn from
 /// a single source or similarity search across the index breaks (mixed
 /// vector spaces, possibly mixed dims). The `backend=hybrid` per-request
 /// override remains orthogonal: it always sends chat to OpenRouter, and
 /// uses `LLM_BACKEND` for the describe-then-inline vision pass.
 pub fn local_backend_is_llamacpp() -> bool {
    matches!(
        std::env::var("LLM_BACKEND")
            .ok()
            .as_deref()
            .map(|s| s.trim().to_lowercase())
            .as_deref(),
        Some("llamacpp")
    )
 }
 /// Embed one string via the configured local backend. Routes through
 /// llama-swap when `LLM_BACKEND=llamacpp` (and a client is configured),
 /// else Ollama. Returns the single embedding vector. See
 /// [`local_backend_is_llamacpp`] for the rationale on consistency.
 pub async fn embed_one(
    ollama: &OllamaClient,
    llamacpp: Option<&LlamaCppClient>,
    text: &str,
 ) -> anyhow::Result<Vec<f32>> {
    if local_backend_is_llamacpp() {
        if let Some(lc) = llamacpp {
            let mut vecs = <LlamaCppClient as LlmClient>::generate_embeddings(lc, &[text]).await?;
            return vecs
                .pop()
                .ok_or_else(|| anyhow::anyhow!("llama-swap returned no embeddings"));
        }
        anyhow::bail!(
            "LLM_BACKEND=llamacpp but LlamaCppClient is unconfigured — \
             set LLAMA_SWAP_URL or switch to LLM_BACKEND=ollama"
        );
    }
    ollama.generate_embedding(text).await
 }
 #[cfg(test)]
 mod env_dispatch_tests {
    use super::*;
    fn with_env<F: FnOnce()>(key: &str, val: Option<&str>, f: F) {
        let prev = std::env::var(key).ok();
        match val {
            Some(v) => unsafe { std::env::set_var(key, v) },
            None => unsafe { std::env::remove_var(key) },
        }
        f();
        match prev {
            Some(v) => unsafe { std::env::set_var(key, v) },
            None => unsafe { std::env::remove_var(key) },
        }
    }
    #[test]
    fn llm_backend_defaults_to_ollama() {
        with_env("LLM_BACKEND", None, || {
            assert!(!local_backend_is_llamacpp());
        });
    }
    #[test]
    fn llm_backend_llamacpp_case_insensitive() {
        with_env("LLM_BACKEND", Some("LlamaCpp"), || {
            assert!(local_backend_is_llamacpp());
        });
        with_env("LLM_BACKEND", Some("  llamacpp "), || {
            assert!(local_backend_is_llamacpp());
        });
    }
    #[test]
    fn llm_backend_unknown_value_is_ollama() {
        with_env("LLM_BACKEND", Some("vllm"), || {
            assert!(!local_backend_is_llamacpp());
        });
    }
 }
@@ -360,7 +360,18 @@ impl OllamaClient {
    /// Extract final answer from thinking model output
    /// Handles <think>...</think> tags and takes everything after
    fn extract_final_answer(&self, response: &str) -> String {
-        crate::ai::llm_client::strip_think_blocks(response)
+        let response = response.trim();
        // Look for </think> tag and take everything after it
        if let Some(pos) = response.find("</think>") {
            let answer = response[pos + 8..].trim();
            if !answer.is_empty() {
                return answer.to_string();
            }
        }
        // Fallback: return the whole response trimmed
        response.to_string()
    }
    async fn try_generate(
@@ -413,7 +424,10 @@ impl OllamaClient {
        self.generate_with_images(prompt, system, None).await
    }
-    #[allow(dead_code)]
+    /// Variant of `generate` that sets Ollama's top-level `think: false`.
    /// Used by latency-sensitive callers like the rerank pass, where the
    /// task has nothing to reason about and chain-of-thought tokens are
    /// wasted wall time. Server-side no-op on non-reasoning models.
    pub async fn generate_no_think(&self, prompt: &str, system: Option<&str>) -> Result<String> {
        self.generate_with_options(prompt, system, None, Some(false))
            .await
@@ -835,14 +849,11 @@ Analyze the image and use specific details from both the visual content and the
                            if !chunk.message.role.is_empty() {
                                role = chunk.message.role;
                            }
-                            // Ollama ≥0.8 can stream tool_calls incrementally
+                            // Ollama only attaches tool_calls on the final chunk.
                            // across chunks (older servers attach them all to
                            // one chunk) — append rather than overwrite so
                            // calls from earlier chunks survive.
                            if let Some(tcs) = chunk.message.tool_calls
                                && !tcs.is_empty()
                            {
-                                append_streamed_tool_calls(&mut tool_calls, tcs);
+                                tool_calls = Some(tcs);
                            }
                            if chunk.done {
                                prompt_eval_count = chunk.prompt_eval_count;
@@ -1321,20 +1332,8 @@ struct OllamaEmbedResponse {
    embeddings: Vec<Vec<f32>>,
 }
 /// Accumulate tool calls streamed across NDJSON chunks. Ollama ≥0.8 may
 /// emit each tool call on its own chunk; replacing the accumulator on every
 /// chunk would keep only the last call, so extend instead.
 fn append_streamed_tool_calls(
    acc: &mut Option<Vec<crate::ai::llm_client::ToolCall>>,
    new: Vec<crate::ai::llm_client::ToolCall>,
 ) {
    acc.get_or_insert_with(Vec::new).extend(new);
 }
 #[cfg(test)]
 mod tests {
    use super::append_streamed_tool_calls;
    use crate::ai::llm_client::{ToolCall, ToolCallFunction};
    #[test]
    fn generate_photo_description_prompt_is_concise() {
@@ -1345,38 +1344,4 @@ mod tests {
                      Focus on the people, location, and activity.";
        assert!(prompt.len() < 200, "Prompt should be concise");
    }
    fn call(name: &str) -> ToolCall {
        ToolCall {
            id: None,
            function: ToolCallFunction {
                name: name.to_string(),
                arguments: serde_json::json!({}),
            },
        }
    }
    #[test]
    fn streamed_tool_calls_across_chunks_accumulate() {
        // Two tool calls arriving in two separate stream chunks must BOTH
        // survive assembly — the old `tool_calls = Some(tcs)` kept only the
        // last chunk's calls.
        let mut acc: Option<Vec<ToolCall>> = None;
        append_streamed_tool_calls(&mut acc, vec![call("get_sms_messages")]);
        append_streamed_tool_calls(&mut acc, vec![call("reverse_geocode")]);
        let calls = acc.expect("tool calls accumulated");
        assert_eq!(calls.len(), 2);
        assert_eq!(calls[0].function.name, "get_sms_messages");
        assert_eq!(calls[1].function.name, "reverse_geocode");
    }
    #[test]
    fn streamed_tool_calls_single_chunk_batch_kept_intact() {
        // Older Ollama servers attach all calls to one chunk — unchanged.
        let mut acc: Option<Vec<ToolCall>> = None;
        append_streamed_tool_calls(&mut acc, vec![call("a"), call("b")]);
        let calls = acc.expect("tool calls accumulated");
        assert_eq!(calls.len(), 2);
    }
 }
@@ -20,36 +20,34 @@ impl SmsApiClient {
        }
    }
-    /// Compute a `[start, end]` unix-second window of `2 * radius_days`
+    /// Fetch messages for a specific contact within ±`days_radius` days of
-    /// centered on `center_ts`. `radius_days < 1` is clamped to 1 to avoid
+    /// the given timestamp (defaults to ±4 days when `None`). Falls back to
-    /// degenerate zero-width windows.
+    /// all contacts if no messages are found for the specified contact.
-    pub(crate) fn window_for_radius(center_ts: i64, radius_days: i64) -> (i64, i64) {
+    /// Messages are sorted by proximity to the center timestamp.
        let r = radius_days.max(1);
        let span = r * 86400;
        (center_ts - span, center_ts + span)
    }
    /// Fetch messages for a specific contact within ±`radius_days` of the
    /// given timestamp. Falls back to all contacts when no messages found
    /// for the named contact. Sorted by proximity to the center timestamp.
    pub async fn fetch_messages_for_contact(
        &self,
        contact: Option<&str>,
        center_timestamp: i64,
-        radius_days: i64,
+        days_radius: Option<i64>,
    ) -> Result<Vec<SmsMessage>> {
-        let effective_radius = radius_days.max(1);
+        use chrono::Duration;
        let (start_ts, end_ts) = Self::window_for_radius(center_timestamp, radius_days);
        let radius = days_radius.unwrap_or(4).clamp(1, 30);
        let center_dt = chrono::DateTime::from_timestamp(center_timestamp, 0)
            .ok_or_else(|| anyhow::anyhow!("Invalid timestamp"))?;
        let start_dt = center_dt - Duration::days(radius);
        let end_dt = center_dt + Duration::days(radius);
        let start_ts = start_dt.timestamp();
        let end_ts = end_dt.timestamp();
        // If contact specified, try fetching for that contact first
        if let Some(contact_name) = contact {
            log::info!(
                "Fetching SMS for contact: {} (±{} days from {})",
                contact_name,
-                effective_radius,
+                radius,
                center_dt.format("%Y-%m-%d %H:%M:%S")
            );
            let messages = self
@@ -74,7 +72,7 @@ impl SmsApiClient {
        // Fallback to all contacts
        log::info!(
            "Fetching all SMS messages (±{} days from {})",
-            effective_radius,
+            radius,
            center_dt.format("%Y-%m-%d %H:%M:%S")
        );
        self.fetch_messages(start_ts, end_ts, None, Some(center_timestamp))
@@ -257,48 +255,23 @@ impl SmsApiClient {
    }
    /// Search message bodies via the Django side's FTS5 / semantic / hybrid
-    /// endpoint. `params.mode` selects the ranking strategy:
+    /// endpoint. `mode` selects the ranking strategy:
    ///   - "fts5"     keyword-only, supports phrase / prefix / boolean / NEAR
    ///   - "semantic" embedding similarity
    ///   - "hybrid"   both merged via reciprocal rank fusion (recommended)
    ///
    /// All of `contact_id`, `date_from` / `date_to` (unix seconds), `is_mms`,
    /// `has_media`, and `offset` are pushed to SMS-API server-side so the
    /// filtered+paginated result set is exact rather than a client-side
    /// over-fetch.
    pub async fn search_messages(
        &self,
        query: &str,
-        params: &SmsSearchParams<'_>,
+        mode: &str,
        limit: usize,
    ) -> Result<Vec<SmsSearchHit>> {
-        let mut url = format!(
+        let url = format!(
            "{}/api/messages/search/?q={}&mode={}&limit={}",
            self.base_url,
            urlencoding::encode(query),
-            urlencoding::encode(params.mode),
+            urlencoding::encode(mode),
-            params.limit,
+            limit
        );
        if let Some(cid) = params.contact_id {
            url.push_str(&format!("&contact_id={}", cid));
        }
        if let Some(ref c) = params.contact {
            url.push_str(&format!("&contact={}", urlencoding::encode(c)));
        }
        if let Some(off) = params.offset {
            url.push_str(&format!("&offset={}", off));
        }
        if let Some(from) = params.date_from {
            url.push_str(&format!("&date_from={}", from));
        }
        if let Some(to) = params.date_to {
            url.push_str(&format!("&date_to={}", to));
        }
        if let Some(is_mms) = params.is_mms {
            url.push_str(&format!("&is_mms={}", is_mms));
        }
        if let Some(has_media) = params.has_media {
            url.push_str(&format!("&has_media={}", has_media));
        }
        let mut request = self.client.get(&url);
        if let Some(token) = &self.token {
@@ -401,33 +374,6 @@ pub struct SmsSearchHit {
    /// Present for semantic / hybrid modes; absent for fts5.
    #[serde(default)]
    pub similarity_score: Option<f32>,
    /// SMS-API-generated excerpt around the match, wrapped in `<mark>` tags.
    /// For MMS messages that only matched via attachment text / filename
    /// (empty `body`), the snippet is the only meaningful preview.
    #[serde(default)]
    pub snippet: Option<String>,
 }
 /// Optional filter / paging knobs for [`SmsApiClient::search_messages`].
 /// All fields except `mode` and `limit` map 1:1 to the same-named SMS-API
 /// query params (added in the 2026-05 search-enhancements release).
 #[derive(Debug, Clone)]
 pub struct SmsSearchParams<'a> {
    pub mode: &'a str,
    pub limit: usize,
    pub contact_id: Option<i64>,
    /// Contact name (case-insensitive). Resolved to a numeric ID by the
    /// SMS-API server when `contact_id` is not set.
    pub contact: Option<String>,
    /// Unix-seconds inclusive lower bound on `date`.
    pub date_from: Option<i64>,
    /// Unix-seconds inclusive upper bound on `date`.
    pub date_to: Option<i64>,
    /// `Some(true)` = MMS only, `Some(false)` = SMS only, `None` = both.
    pub is_mms: Option<bool>,
    /// `Some(true)` = only messages with image/video/audio attachments.
    pub has_media: Option<bool>,
    pub offset: Option<usize>,
 }
 #[derive(Deserialize)]
@@ -437,29 +383,3 @@ struct SmsSearchResponse {
    #[serde(default)]
    search_method: String,
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    #[test]
    fn window_for_radius_produces_2n_day_span() {
        let center: i64 = 1_700_000_000;
        let (start, end) = SmsApiClient::window_for_radius(center, 7);
        assert_eq!(end - start, 14 * 86400);
        assert_eq!(start + 7 * 86400, center);
        assert_eq!(end - 7 * 86400, center);
    }
    #[test]
    fn window_for_radius_clamps_zero_to_one() {
        let (start, end) = SmsApiClient::window_for_radius(100_000, 0);
        assert_eq!(end - start, 2 * 86400);
    }
    #[test]
    fn window_for_radius_clamps_negative_to_one() {
        let (start, end) = SmsApiClient::window_for_radius(100_000, -7);
        assert_eq!(end - start, 2 * 86400);
    }
 }
@@ -1,580 +0,0 @@
 // TTS endpoints: proxy text-to-speech + voice-library management to the
 // Chatterbox server that sits behind llama-swap (via LlamaCppClient). Speech
 // synthesis returns audio as base64-in-JSON so the mobile app can play it as a
 // `data:` URI without a binary-fetch path. Voice cloning registers a named
 // voice from either an uploaded clip (device) or an existing library file
 // (audio read directly; video has its audio track extracted via ffmpeg).
 use actix_multipart::Multipart;
 use actix_web::{HttpRequest, HttpResponse, Responder, get, post, web};
 use anyhow::Context;
 use base64::Engine;
 use bytes::{BufMut, BytesMut};
 use futures::StreamExt;
 use opentelemetry::KeyValue;
 use opentelemetry::trace::{Span, Status, Tracer};
 use regex::Regex;
 use serde::{Deserialize, Serialize};
 use serde_json::json;
 use std::path::Path;
 use std::sync::LazyLock;
 use tokio::sync::Semaphore;
 use crate::data::Claims;
 use crate::file_types::{is_audio_file, is_video_file};
 use crate::files::is_valid_full_path;
 use crate::libraries;
 use crate::otel::{extract_context_from_request, global_tracer};
 use crate::state::AppState;
 /// Hard cap on an uploaded voice-reference clip. Chatterbox itself caps the
 /// payload (~60s clip); this is a defensive ceiling so a hostile/oversized
 /// upload can't balloon ImageApi memory before we ever forward it.
 const MAX_VOICE_UPLOAD_BYTES: usize = 25 * 1024 * 1024; // 25 MB
 /// Serialize speech synthesis: the Chatterbox server has no internal lock or
 /// queue, so concurrent requests contend on the single GPU and cascade into
 /// timeouts. One permit; when busy we fast-fail with 429 rather than queue —
 /// the app surfaces "busy" immediately, and typical jobs clear in well under a
 /// minute. (An abandoned upstream job can still occupy the GPU until it
 /// finishes — that's a wrapper limitation; the chunked-queue plan fixes it.)
 static TTS_PERMIT: LazyLock<Semaphore> = LazyLock::new(|| Semaphore::new(1));
 /// Sanitize a user-supplied voice name. The name is forwarded to Chatterbox
 /// where it becomes a filename in the voice-library directory, so we restrict
 /// it to a safe charset (alphanumerics, dash, underscore) — no path
 /// separators, dots, or whitespace — and bound its length. Returns `None`
 /// when nothing usable remains.
 fn sanitize_voice_name(raw: &str) -> Option<String> {
    let cleaned: String = raw
        .trim()
        .chars()
        .map(|c| {
            if c.is_ascii_alphanumeric() || c == '-' || c == '_' {
                c
            } else {
                '-'
            }
        })
        .collect();
    let cleaned = cleaned.trim_matches('-').to_string();
    if cleaned.is_empty() {
        return None;
    }
    Some(cleaned.chars().take(64).collect())
 }
 /// Optional default voice for synthesis when the request doesn't name one.
 /// Set `LLAMA_SWAP_TTS_VOICE=m` to read insights in a cloned voice by default.
 fn default_voice() -> Option<String> {
    std::env::var("LLAMA_SWAP_TTS_VOICE")
        .ok()
        .map(|s| s.trim().to_string())
        .filter(|s| !s.is_empty())
 }
 // Markdown / formatting strippers, compiled once. Insight text is markdown,
 // which TTS would otherwise read literally ("star star bold star star").
 static MD_IMAGE: LazyLock<Regex> = LazyLock::new(|| Regex::new(r"!\[([^\]]*)\]\([^)]*\)").unwrap());
 static MD_LINK: LazyLock<Regex> = LazyLock::new(|| Regex::new(r"\[([^\]]+)\]\([^)]*\)").unwrap());
 static MD_HEADING: LazyLock<Regex> =
    LazyLock::new(|| Regex::new(r"(?m)^\s{0,3}#{1,6}\s*").unwrap());
 static MD_BLOCKQUOTE: LazyLock<Regex> = LazyLock::new(|| Regex::new(r"(?m)^\s{0,3}>\s?").unwrap());
 static MD_LIST: LazyLock<Regex> =
    LazyLock::new(|| Regex::new(r"(?m)^\s{0,3}([-*+]|\d+\.)\s+").unwrap());
 static MD_EMPHASIS: LazyLock<Regex> = LazyLock::new(|| Regex::new(r"[*_`~]+").unwrap());
 static URL_RE: LazyLock<Regex> = LazyLock::new(|| Regex::new(r"https?://\S+").unwrap());
 static MULTISPACE: LazyLock<Regex> = LazyLock::new(|| Regex::new(r"[ \t]{2,}").unwrap());
 // Any run of 2+ newlines (incl. whitespace-only blank lines) collapses to ONE
 // newline: Chatterbox inserts a long pause (sometimes ~20s of silence) per
 // blank line, so paragraph breaks must reach it as a single line break at most.
 static MULTINEWLINE: LazyLock<Regex> = LazyLock::new(|| Regex::new(r"\n(?:[ \t]*\n)+").unwrap());
 /// True for emoji / pictographic symbols, which most TTS models either skip or
 /// mispronounce. Covers the main emoji blocks plus dingbats, misc-technical,
 /// variation selectors, and the ZWJ used to glue emoji sequences. We do NOT
 /// strip `[bracketed]` tags — non-turbo Chatterbox ignores them, and a future
 /// Turbo switch uses them as paralinguistic cues.
 fn is_emoji_like(c: char) -> bool {
    let u = c as u32;
    matches!(u,
        0x1F000..=0x1FAFF   // emoji, pictographs, supplemental symbols, flags
        | 0x2300..=0x23FF   // misc technical (⌚ ⏰ ⏳ …)
        | 0x2600..=0x27BF   // misc symbols + dingbats
        | 0x2B00..=0x2BFF   // misc symbols & arrows (★ ⬆ …)
        | 0xFE00..=0xFE0F   // variation selectors
        | 0x200D            // zero-width joiner
    )
 }
 /// Normalize insight text for speech: unwrap markdown links/images to their
 /// visible text, drop heading/list/blockquote/emphasis markers and URLs, strip
 /// emoji, and collapse whitespace. Centralized here so every caller (app,
 /// WebUI, curl) gets clean audio.
 fn clean_for_tts(input: &str) -> String {
    let s = MD_IMAGE.replace_all(input, "$1");
    let s = MD_LINK.replace_all(&s, "$1");
    let s = MD_HEADING.replace_all(&s, "");
    let s = MD_BLOCKQUOTE.replace_all(&s, "");
    let s = MD_LIST.replace_all(&s, "");
    let s = MD_EMPHASIS.replace_all(&s, "");
    let s = URL_RE.replace_all(&s, " ");
    let s: String = s.chars().filter(|c| !is_emoji_like(*c)).collect();
    let s = MULTISPACE.replace_all(&s, " ");
    let s = MULTINEWLINE.replace_all(&s, "\n");
    s.trim().to_string()
 }
 /// Decode an audio/video file to mono 24 kHz WAV via ffmpeg, returning the WAV
 /// bytes. Chatterbox validates the reference clip by file *extension* and
 /// rejects several formats (e.g. `.aac`, `.opus`), so we always normalize to
 /// WAV regardless of the source container. Capped at 30s — references only need
 /// a few seconds of clean speech.
 async fn run_ffmpeg_to_wav(input_path: &str) -> anyhow::Result<Vec<u8>> {
    let out = tempfile::Builder::new()
        .suffix(".wav")
        .tempfile()
        .context("creating temp wav")?;
    let out_s = out.path().to_string_lossy().to_string();
    // Cap the reference clip length. Chatterbox is zero-shot — a clean ~10–20s
    // sample is the sweet spot and more rarely helps — so we use the first N
    // seconds. Tune via LLAMA_SWAP_TTS_REF_SECONDS (default 30).
    let secs = std::env::var("LLAMA_SWAP_TTS_REF_SECONDS")
        .ok()
        .and_then(|s| s.trim().parse::<u32>().ok())
        .filter(|n| *n > 0)
        .unwrap_or(30)
        .to_string();
    let output = tokio::process::Command::new("ffmpeg")
        .args([
            "-y", "-i", input_path, "-vn", "-ac", "1", "-ar", "24000", "-t", &secs, "-f", "wav",
            &out_s,
        ])
        .output()
        .await
        .context("spawning ffmpeg")?;
    if !output.status.success() {
        anyhow::bail!("ffmpeg failed: {}", String::from_utf8_lossy(&output.stderr));
    }
    std::fs::read(&out_s).context("reading transcoded audio")
 }
 /// Normalize in-memory upload bytes to WAV: write to a temp file (keeping the
 /// source extension as an ffmpeg probe hint) then transcode.
 async fn transcode_bytes_to_wav(input: &[u8], src_ext: Option<&str>) -> anyhow::Result<Vec<u8>> {
    let suffix = src_ext
        .filter(|e| !e.is_empty())
        .map(|e| format!(".{e}"))
        .unwrap_or_else(|| ".bin".to_string());
    let in_tmp = tempfile::Builder::new()
        .suffix(&suffix)
        .tempfile()
        .context("creating temp input")?;
    std::fs::write(in_tmp.path(), input).context("writing temp input")?;
    run_ffmpeg_to_wav(&in_tmp.path().to_string_lossy()).await
 }
 #[derive(Debug, Deserialize)]
 pub struct TtsSpeechRequest {
    pub text: String,
    #[serde(default)]
    pub voice: Option<String>,
    /// Audio container, e.g. `"mp3"` (default) or `"wav"`.
    #[serde(default)]
    pub format: Option<String>,
    /// Chatterbox knobs (clamped server-side). exaggeration 0.25–2.0 (emotion),
    /// cfg_weight 0.0–1.0 (pace; ~0.3 for fast speakers, 0 to neutralize a
    /// reference accent), temperature 0.05–5.0 (randomness).
    #[serde(default)]
    pub exaggeration: Option<f32>,
    #[serde(default)]
    pub cfg_weight: Option<f32>,
    #[serde(default)]
    pub temperature: Option<f32>,
 }
 #[derive(Debug, Serialize)]
 pub struct TtsSpeechResponse {
    pub audio_base64: String,
    pub format: String,
 }
 /// POST /tts/speech — synthesize `text` (optionally in a named `voice`) and
 /// return base64-encoded audio for `data:` URI playback on the client.
 #[post("/tts/speech")]
 pub async fn tts_speech_handler(
    http_request: HttpRequest,
    _claims: Claims,
    req: web::Json<TtsSpeechRequest>,
    app_state: web::Data<AppState>,
 ) -> impl Responder {
    let parent_context = extract_context_from_request(&http_request);
    let mut span = global_tracer().start_with_context("http.tts.speech", &parent_context);
    let text = clean_for_tts(&req.text);
    if text.is_empty() {
        span.set_status(Status::error("text is required"));
        return HttpResponse::BadRequest().json(json!({ "error": "text is required" }));
    }
    let Some(client) = app_state.llamacpp.as_ref() else {
        span.set_status(Status::error("tts backend not configured"));
        return HttpResponse::ServiceUnavailable()
            .json(json!({ "error": "TTS backend not configured (set LLAMA_SWAP_URL)" }));
    };
    let format = req
        .format
        .as_deref()
        .filter(|s| !s.is_empty())
        .unwrap_or("mp3");
    let dv = default_voice();
    let voice = req
        .voice
        .as_deref()
        .filter(|s| !s.is_empty())
        .or(dv.as_deref());
    span.set_attribute(KeyValue::new("tts.model", client.tts_model.clone()));
    span.set_attribute(KeyValue::new("tts.format", format.to_string()));
    span.set_attribute(KeyValue::new("tts.has_voice", voice.is_some()));
    span.set_attribute(KeyValue::new("tts.text_len", text.len() as i64));
    // Clamp generation knobs to Chatterbox's documented ranges before forwarding.
    let exaggeration = req.exaggeration.map(|x| x.clamp(0.25, 2.0));
    let cfg_weight = req.cfg_weight.map(|x| x.clamp(0.0, 1.0));
    let temperature = req.temperature.map(|x| x.clamp(0.05, 5.0));
    // One synthesis at a time (see TTS_PERMIT) — fast-fail when busy.
    let Ok(_permit) = TTS_PERMIT.try_acquire() else {
        span.set_status(Status::error("tts busy"));
        return HttpResponse::TooManyRequests().json(json!({
            "error": "TTS is busy with another request — try again shortly"
        }));
    };
    match client
        .text_to_speech(&text, voice, format, exaggeration, cfg_weight, temperature)
        .await
    {
        Ok(bytes) => {
            span.set_attribute(KeyValue::new("tts.audio_bytes", bytes.len() as i64));
            span.set_status(Status::Ok);
            let audio_base64 = base64::engine::general_purpose::STANDARD.encode(&bytes);
            HttpResponse::Ok().json(TtsSpeechResponse {
                audio_base64,
                format: format.to_string(),
            })
        }
        Err(e) => {
            span.set_status(Status::error("tts synthesis failed"));
            log::error!("TTS synth failed: {:?}", e);
            HttpResponse::BadGateway().json(json!({ "error": format!("TTS failed: {e}") }))
        }
    }
 }
 /// GET /tts/voices — list the Chatterbox voice library (raw passthrough).
 #[get("/tts/voices")]
 pub async fn list_voices_handler(
    http_request: HttpRequest,
    _claims: Claims,
    app_state: web::Data<AppState>,
 ) -> impl Responder {
    let parent_context = extract_context_from_request(&http_request);
    let mut span = global_tracer().start_with_context("http.tts.voices.list", &parent_context);
    let Some(client) = app_state.llamacpp.as_ref() else {
        span.set_status(Status::error("tts backend not configured"));
        return HttpResponse::ServiceUnavailable()
            .json(json!({ "error": "TTS backend not configured" }));
    };
    match client.list_voices().await {
        Ok(v) => {
            span.set_status(Status::Ok);
            HttpResponse::Ok().json(v)
        }
        Err(e) => {
            span.set_status(Status::error("list_voices failed"));
            log::error!("list_voices failed: {:?}", e);
            HttpResponse::BadGateway().json(json!({ "error": format!("{e}") }))
        }
    }
 }
 /// POST /tts/voices/upload — register a cloned voice from an uploaded audio
 /// clip. Multipart fields: `voice_name` (text) + a file part (`voice_file`).
 #[post("/tts/voices/upload")]
 pub async fn create_voice_upload_handler(
    http_request: HttpRequest,
    _claims: Claims,
    mut payload: Multipart,
    app_state: web::Data<AppState>,
 ) -> impl Responder {
    let parent_context = extract_context_from_request(&http_request);
    let mut span = global_tracer().start_with_context("http.tts.voices.upload", &parent_context);
    let Some(client) = app_state.llamacpp.as_ref() else {
        span.set_status(Status::error("tts backend not configured"));
        return HttpResponse::ServiceUnavailable()
            .json(json!({ "error": "TTS backend not configured" }));
    };
    let mut voice_name: Option<String> = None;
    let mut file_bytes = BytesMut::new();
    let mut filename = "voice.wav".to_string();
    while let Some(Ok(mut part)) = payload.next().await {
        // Capture disposition fields up front so the immutable borrow ends
        // before we mutably stream the part body (mirrors handlers/image.rs).
        let (fname_opt, name_opt) = {
            let cd = part.content_disposition();
            (
                cd.and_then(|c| c.get_filename()).map(|s| s.to_string()),
                cd.and_then(|c| c.get_name()).map(|s| s.to_string()),
            )
        };
        if let Some(fname) = fname_opt {
            filename = fname;
            while let Some(Ok(data)) = part.next().await {
                if file_bytes.len() + data.len() > MAX_VOICE_UPLOAD_BYTES {
                    span.set_status(Status::error("voice clip exceeds limit"));
                    return HttpResponse::PayloadTooLarge()
                        .json(json!({ "error": "voice clip exceeds 25 MB" }));
                }
                file_bytes.put(data);
            }
        } else if name_opt.as_deref() == Some("voice_name") {
            let mut buf = BytesMut::new();
            while let Some(Ok(data)) = part.next().await {
                buf.put(data);
            }
            voice_name = Some(String::from_utf8_lossy(&buf).trim().to_string());
        } else {
            while let Some(Ok(_)) = part.next().await {}
        }
    }
    let Some(name) = voice_name.as_deref().and_then(sanitize_voice_name) else {
        span.set_status(Status::error("voice_name is required"));
        return HttpResponse::BadRequest()
            .json(json!({ "error": "voice_name is required (alphanumerics, - and _ only)" }));
    };
    if file_bytes.is_empty() {
        span.set_status(Status::error("voice_file is required"));
        return HttpResponse::BadRequest().json(json!({ "error": "voice_file is required" }));
    }
    span.set_attribute(KeyValue::new("tts.voice_name", name.clone()));
    span.set_attribute(KeyValue::new("tts.upload_bytes", file_bytes.len() as i64));
    // Normalize to WAV so any device format (e.g. .aac / .opus, which Chatterbox
    // rejects by extension) is accepted.
    let src_ext = Path::new(&filename).extension().and_then(|e| e.to_str());
    let wav = match transcode_bytes_to_wav(file_bytes.as_ref(), src_ext).await {
        Ok(w) => w,
        Err(e) => {
            span.set_status(Status::error("audio decode failed"));
            log::error!("voice upload transcode failed: {:?}", e);
            return HttpResponse::BadRequest()
                .json(json!({ "error": "couldn't decode that audio file" }));
        }
    };
    match client
        .create_voice(&name, wav, "reference.wav", "audio/wav")
        .await
    {
        Ok(v) => {
            span.set_status(Status::Ok);
            HttpResponse::Ok().json(v)
        }
        Err(e) => {
            span.set_status(Status::error("create_voice failed"));
            log::error!("create_voice (upload) failed: {:?}", e);
            HttpResponse::BadGateway().json(json!({ "error": format!("{e}") }))
        }
    }
 }
 #[derive(Debug, Deserialize)]
 pub struct CreateVoiceFromLibraryRequest {
    pub voice_name: String,
    /// Library-relative path to an audio or video file.
    pub path: String,
    #[serde(default)]
    pub library: Option<String>,
 }
 /// POST /tts/voices/from-library — register a cloned voice from a file already
 /// in a library. Audio and video alike are ffmpeg-normalized to a mono 24 kHz
 /// WAV reference clip (length capped by LLAMA_SWAP_TTS_REF_SECONDS).
 #[post("/tts/voices/from-library")]
 pub async fn create_voice_from_library_handler(
    http_request: HttpRequest,
    _claims: Claims,
    req: web::Json<CreateVoiceFromLibraryRequest>,
    app_state: web::Data<AppState>,
 ) -> impl Responder {
    let parent_context = extract_context_from_request(&http_request);
    let mut span =
        global_tracer().start_with_context("http.tts.voices.from_library", &parent_context);
    let Some(client) = app_state.llamacpp.as_ref() else {
        span.set_status(Status::error("tts backend not configured"));
        return HttpResponse::ServiceUnavailable()
            .json(json!({ "error": "TTS backend not configured" }));
    };
    let Some(voice_name) = sanitize_voice_name(&req.voice_name) else {
        span.set_status(Status::error("voice_name is required"));
        return HttpResponse::BadRequest()
            .json(json!({ "error": "voice_name is required (alphanumerics, - and _ only)" }));
    };
    let library = match libraries::resolve_library_param(&app_state, req.library.as_deref()) {
        Ok(Some(l)) => l,
        Ok(None) => app_state.primary_library(),
        Err(msg) => {
            span.set_status(Status::error("invalid library"));
            return HttpResponse::BadRequest().json(json!({ "error": msg }));
        }
    };
    // is_valid_full_path confines the path to the library root (no traversal).
    let abs = match is_valid_full_path(&library.root_path, &req.path, false) {
        Some(p) if p.exists() => p,
        _ => {
            span.set_status(Status::error("file not found"));
            return HttpResponse::NotFound().json(json!({ "error": "file not found in library" }));
        }
    };
    // Only real audio/video sources are valid voice references — refuse to
    // slurp arbitrary library files into memory / ffmpeg.
    if !is_audio_file(&abs) && !is_video_file(&abs) {
        span.set_status(Status::error("not an audio/video file"));
        return HttpResponse::BadRequest()
            .json(json!({ "error": "file is not an audio or video file" }));
    }
    span.set_attribute(KeyValue::new("tts.voice_name", voice_name.clone()));
    let wav = match prepare_reference_audio(&abs).await {
        Ok(b) => b,
        Err(e) => {
            span.set_status(Status::error("audio decode failed"));
            log::error!("voice reference prep failed for {:?}: {:?}", abs, e);
            return HttpResponse::BadRequest()
                .json(json!({ "error": "couldn't decode that file's audio" }));
        }
    };
    match client
        .create_voice(&voice_name, wav, "reference.wav", "audio/wav")
        .await
    {
        Ok(v) => {
            span.set_status(Status::Ok);
            HttpResponse::Ok().json(v)
        }
        Err(e) => {
            span.set_status(Status::error("create_voice failed"));
            log::error!("create_voice (from-library) failed: {:?}", e);
            HttpResponse::BadGateway().json(json!({ "error": format!("{e}") }))
        }
    }
 }
 /// Read a library file (audio or video) as a Chatterbox-ready reference: ffmpeg
 /// decodes/extracts its audio to mono 24 kHz WAV. Reading straight from the
 /// library path avoids slurping a (possibly large) video into memory.
 async fn prepare_reference_audio(abs: &Path) -> anyhow::Result<Vec<u8>> {
    run_ffmpeg_to_wav(&abs.to_string_lossy()).await
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    #[test]
    fn sanitize_voice_name_keeps_safe_chars() {
        assert_eq!(sanitize_voice_name("m").as_deref(), Some("m"));
        assert_eq!(
            sanitize_voice_name("  Cameron ").as_deref(),
            Some("Cameron")
        );
        assert_eq!(
            sanitize_voice_name("voice_01-a").as_deref(),
            Some("voice_01-a")
        );
    }
    #[test]
    fn sanitize_voice_name_strips_unsafe_chars() {
        // Path separators / dots / spaces become '-' and are trimmed at edges.
        assert_eq!(sanitize_voice_name("a b.c").as_deref(), Some("a-b-c"));
        assert_eq!(
            sanitize_voice_name("../etc/passwd").as_deref(),
            Some("etc-passwd")
        );
    }
    #[test]
    fn sanitize_voice_name_rejects_empty_or_all_unsafe() {
        assert_eq!(sanitize_voice_name(""), None);
        assert_eq!(sanitize_voice_name("   "), None);
        assert_eq!(sanitize_voice_name("../../"), None);
        assert_eq!(sanitize_voice_name("...."), None);
    }
    #[test]
    fn sanitize_voice_name_bounds_length() {
        let long = "a".repeat(200);
        assert_eq!(sanitize_voice_name(&long).unwrap().len(), 64);
    }
    #[test]
    fn clean_for_tts_strips_markdown() {
        assert_eq!(
            clean_for_tts("**Bold** and _italic_ and `code`"),
            "Bold and italic and code"
        );
        assert_eq!(clean_for_tts("# Title\n\nbody"), "Title\nbody");
        assert_eq!(
            clean_for_tts("See [docs](http://x.com) now"),
            "See docs now"
        );
        assert_eq!(clean_for_tts("- one\n- two"), "one\ntwo");
    }
    #[test]
    fn clean_for_tts_strips_emoji_and_urls() {
        assert_eq!(clean_for_tts("Hello 😀 world 🎉"), "Hello world");
        assert_eq!(
            clean_for_tts("visit https://example.com today"),
            "visit today"
        );
        // ZWJ-glued emoji sequence is fully removed.
        assert_eq!(clean_for_tts("family 👨‍👩‍👧 photo"), "family photo");
    }
    #[test]
    fn clean_for_tts_collapses_blank_lines_to_single_break() {
        // Chatterbox pauses (sometimes ~20s) per blank line, so paragraph
        // breaks must collapse to a single newline.
        assert_eq!(clean_for_tts("para one\n\npara two"), "para one\npara two");
        assert_eq!(clean_for_tts("a\n\n\n\nb"), "a\nb");
        // Whitespace-only "blank" lines collapse too.
        assert_eq!(clean_for_tts("a\n  \t \nb"), "a\nb");
        // A single newline is left alone.
        assert_eq!(clean_for_tts("a\nb"), "a\nb");
    }
    #[test]
    fn clean_for_tts_preserves_bracket_tags() {
        // Non-turbo Chatterbox ignores these; a future Turbo uses them as
        // paralinguistic cues — so we must not strip them.
        assert_eq!(clean_for_tts("hello [laugh] there"), "hello [laugh] there");
    }
 }
@@ -1,748 +0,0 @@
 use crate::ai::insight_chat::ChatStreamEvent;
 use std::collections::HashMap;
 use std::sync::Arc;
 use std::sync::Mutex as StdMutex;
 use std::sync::atomic::{AtomicU32, Ordering};
 use std::time::Instant;
 use tokio::sync::{Mutex, Notify};
 use tokio::task::AbortHandle;
 /// Maximum number of events buffered per turn. Agentic turns typically
 /// produce ~120 events; 500 provides 4× headroom. When exceeded, oldest
 /// events are evicted from the front.
 const MAX_BUFFERED_EVENTS: usize = 500;
 /// Turn status codes used by `TurnEntry::status`.
 #[derive(Debug, Clone, Copy, PartialEq)]
 pub enum TurnStatus {
    Running = 0,
    Done = 1,
    Error = 2,
    Cancelled = 3,
 }
 impl From<u32> for TurnStatus {
    fn from(v: u32) -> Self {
        match v {
            0 => TurnStatus::Running,
            1 => TurnStatus::Done,
            2 => TurnStatus::Error,
            3 => TurnStatus::Cancelled,
            _ => TurnStatus::Running,
        }
    }
 }
 impl TurnStatus {
    pub fn as_str(&self) -> &'static str {
        match self {
            TurnStatus::Running => "running",
            TurnStatus::Done => "done",
            TurnStatus::Error => "error",
            TurnStatus::Cancelled => "cancelled",
        }
    }
 }
 /// Shared metadata about a turn, read by the SSE replay handler to emit
 /// the initial `turn_info` event and to decide whether to wait for new
 /// events or close immediately.
 #[derive(Debug, Clone)]
 pub struct TurnInfo {
    pub turn_id: String,
    pub file_path: String,
    pub library_id: i32,
    pub status: TurnStatus,
    pub total_events_pushed: u32,
    pub buffered_count: u32,
 }
 /// Result of reading events at or after an absolute `skip_before` index.
 #[derive(Debug)]
 pub enum ReplayOutcome {
    /// New events are available. `next_skip` is the absolute index to pass
    /// on the next read (i.e. one past the last event returned).
    Events {
        events: Vec<ChatStreamEvent>,
        next_skip: u32,
    },
    /// The reader is caught up to the live edge — no events past `skip_before`
    /// yet. `next_skip` is the current high-water mark.
    CaughtUp { next_skip: u32 },
    /// `skip_before` points below the buffer's base index: the requested
    /// events were evicted. Maps to HTTP 410 Gone.
    Gone,
 }
 /// Per-turn state shared between the agentic loop (writer) and all SSE
 /// replay connections (readers).
 pub struct TurnEntry {
    pub turn_id: String,
    pub file_path: String,
    pub library_id: i32,
    /// Shared event buffer — multiple SSE connections can read independently.
    /// Each connection tracks its own `skip_before` offset.
    events: Mutex<Vec<ChatStreamEvent>>,
    /// Monotonic counter: total events pushed (may exceed events.len()
    /// due to eviction). Used for skip_before indexing.
    total_events_pushed: AtomicU32,
    /// The event index that this entry started with. Adjusts on eviction
    /// so that `skip_before` stays absolute across connections.
    base_index: AtomicU32,
    pub status: AtomicU32,
    /// Abort handle for the spawned agentic task, set once after spawn.
    /// Behind a std `Mutex` because the entry is shared via `Arc` and the
    /// handle is installed after the entry is already in the registry.
    abort_handle: StdMutex<Option<AbortHandle>>,
    pub created_at: Instant,
    notify: Arc<Notify>,
 }
 impl TurnEntry {
    pub fn new(turn_id: String, file_path: String, library_id: i32) -> Self {
        Self {
            turn_id,
            file_path,
            library_id,
            events: Mutex::new(Vec::new()),
            total_events_pushed: AtomicU32::new(0),
            base_index: AtomicU32::new(0),
            status: AtomicU32::new(TurnStatus::Running as u32),
            abort_handle: StdMutex::new(None),
            created_at: Instant::now(),
            notify: Arc::new(Notify::new()),
        }
    }
    /// Install the abort handle for the spawned agentic task. Called once,
    /// right after the task is spawned.
    pub fn set_abort_handle(&self, handle: AbortHandle) {
        *self.abort_handle.lock().expect("abort_handle poisoned") = Some(handle);
    }
    /// Abort the spawned agentic task, if a handle was installed. Returns
    /// `true` if a task was aborted.
    pub fn abort(&self) -> bool {
        if let Some(handle) = self
            .abort_handle
            .lock()
            .expect("abort_handle poisoned")
            .take()
        {
            handle.abort();
            true
        } else {
            false
        }
    }
    /// Push an event into the buffer. Evicts oldest events if the buffer
    /// exceeds `MAX_BUFFERED_EVENTS`. Notifies all waiting SSE connections.
    pub async fn push_event(&self, event: ChatStreamEvent) {
        {
            let mut events = self.events.lock().await;
            // Evict oldest events if we've hit the cap.
            if events.len() >= MAX_BUFFERED_EVENTS {
                // Drop the oldest event to make room and advance the base
                // index so skip_before stays absolute across connections.
                events.remove(0);
                self.base_index.fetch_add(1, Ordering::Relaxed);
            }
            events.push(event);
            // Increment while holding the buffer lock so the counter stays in
            // lock-step with the buffer even if multiple writers ever exist.
            self.total_events_pushed.fetch_add(1, Ordering::Relaxed);
        }
        self.notify.notify_waiters();
    }
    /// Get a snapshot of turn metadata for the `turn_info` SSE event.
    pub async fn info(&self) -> TurnInfo {
        let events = self.events.lock().await;
        let buffered = events.len() as u32;
        let total = self.total_events_pushed.load(Ordering::Relaxed);
        drop(events);
        TurnInfo {
            turn_id: self.turn_id.clone(),
            file_path: self.file_path.clone(),
            library_id: self.library_id,
            status: self.status.load(Ordering::Relaxed).into(),
            total_events_pushed: total,
            buffered_count: buffered,
        }
    }
    /// Set the terminal status and notify all waiters.
    pub fn set_terminal_status(&self, status: TurnStatus) {
        self.status.store(status as u32, Ordering::Relaxed);
        self.notify.notify_waiters();
    }
    /// Read buffered events at or after absolute index `skip_before` without
    /// waiting. Distinguishes "evicted" (Gone) from "caught up" (no new
    /// events yet) — the previous boolean/`Option` API conflated the two.
    pub async fn replay_from(&self, skip_before: u32) -> ReplayOutcome {
        let events = self.events.lock().await;
        let base = self.base_index.load(Ordering::Relaxed);
        // The buffer holds absolute indices [base, base + len). A request
        // below `base` asked for events that have been evicted.
        if skip_before < base {
            return ReplayOutcome::Gone;
        }
        let offset = (skip_before - base) as usize;
        let next_skip = base + events.len() as u32;
        if offset >= events.len() {
            // Caught up to (or past) the live edge — nothing new yet.
            return ReplayOutcome::CaughtUp { next_skip };
        }
        ReplayOutcome::Events {
            events: events[offset..].to_vec(),
            next_skip,
        }
    }
    /// Wait for the next batch of events past `skip_before`, the turn to
    /// finish, or eviction. Returns:
    /// - `Events` when new events are available (drained before any terminal
    ///   signal so the final `Done`/`Error` is never dropped),
    /// - `CaughtUp` only when the turn has reached a terminal status and the
    ///   reader is fully drained (the caller should close the stream),
    /// - `Gone` when `skip_before` points into evicted territory.
    pub async fn next_batch(&self, skip_before: u32) -> ReplayOutcome {
        loop {
            // Register interest BEFORE inspecting state so a push/terminal that
            // races between our read and our await can't be lost (Notify's
            // `notify_waiters` does not store a permit).
            let notified = self.notify.notified();
            tokio::pin!(notified);
            notified.as_mut().enable();
            match self.replay_from(skip_before).await {
                ReplayOutcome::CaughtUp { next_skip } => {
                    // No new events. If the turn is finished, every event
                    // (including the terminal one) has already been drained
                    // above on a prior call, so signal the caller to close.
                    if !self.is_running() {
                        return ReplayOutcome::CaughtUp { next_skip };
                    }
                    // Still running — wait for the next push or terminal.
                }
                other => return other, // Events or Gone
            }
            notified.await;
        }
    }
    /// Check if this turn is still running.
    pub fn is_running(&self) -> bool {
        self.status.load(Ordering::Relaxed) == TurnStatus::Running as u32
    }
 }
 /// In-memory registry of all active chat turns. Injected into `AppState`
 /// and shared across all handlers.
 pub struct TurnRegistry {
    entries: Mutex<HashMap<String, Arc<TurnEntry>>>,
    timeout_secs: u64,
 }
 impl TurnRegistry {
    pub fn new(timeout_secs: u64) -> Self {
        Self {
            entries: Mutex::new(HashMap::new()),
            timeout_secs,
        }
    }
    /// Returns the cleanup timeout in seconds.
    pub fn timeout_secs(&self) -> u64 {
        self.timeout_secs
    }
    /// Insert a new turn entry. Returns the turn_id.
    pub async fn insert(&self, entry: Arc<TurnEntry>) -> String {
        let turn_id = entry.turn_id.clone();
        let mut entries = self.entries.lock().await;
        entries.insert(turn_id.clone(), entry);
        turn_id
    }
    /// Look up a turn by id. Returns None if not found or expired.
    pub async fn get(&self, turn_id: &str) -> Option<Arc<TurnEntry>> {
        let entries = self.entries.lock().await;
        entries.get(turn_id).cloned()
    }
    /// Clean up stale entries older than the timeout. Returns the count of
    /// entries removed.
    pub async fn cleanup_stale(&self) -> usize {
        let mut entries = self.entries.lock().await;
        let _now = Instant::now();
        let stale: Vec<String> = entries
            .iter()
            .filter(|(_, entry)| entry.created_at.elapsed().as_secs() > self.timeout_secs)
            .map(|(id, _)| id.clone())
            .collect();
        for id in &stale {
            entries.remove(id);
        }
        if !stale.is_empty() {
            log::info!(
                "TurnRegistry: cleaned up {} stale entries (timeout={}s)",
                stale.len(),
                self.timeout_secs
            );
        }
        stale.len()
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use crate::ai::insight_chat::ChatStreamEvent;
    use std::time::Duration;
    /// Unwrap the events from a `ReplayOutcome::Events`, panicking otherwise.
    fn events_of(outcome: ReplayOutcome) -> Vec<ChatStreamEvent> {
        match outcome {
            ReplayOutcome::Events { events, .. } => events,
            other => panic!("expected Events, got {other:?}"),
        }
    }
    // ── TurnStatus ──────────────────────────────────────────────────
    #[test]
    fn turn_status_from_u32_valid_values() {
        assert_eq!(TurnStatus::from(0), TurnStatus::Running);
        assert_eq!(TurnStatus::from(1), TurnStatus::Done);
        assert_eq!(TurnStatus::from(2), TurnStatus::Error);
        assert_eq!(TurnStatus::from(3), TurnStatus::Cancelled);
    }
    #[test]
    fn turn_status_from_u32_unknown_defaults_to_running() {
        assert_eq!(TurnStatus::from(4), TurnStatus::Running);
        assert_eq!(TurnStatus::from(u32::MAX), TurnStatus::Running);
    }
    #[test]
    fn turn_status_as_str() {
        assert_eq!(TurnStatus::Running.as_str(), "running");
        assert_eq!(TurnStatus::Done.as_str(), "done");
        assert_eq!(TurnStatus::Error.as_str(), "error");
        assert_eq!(TurnStatus::Cancelled.as_str(), "cancelled");
    }
    // ── TurnEntry ───────────────────────────────────────────────────
    #[tokio::test]
    async fn turn_entry_push_and_replay() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        entry
            .push_event(ChatStreamEvent::TextDelta("hello".to_string()))
            .await;
        entry
            .push_event(ChatStreamEvent::TextDelta(" world".to_string()))
            .await;
        let events = events_of(entry.replay_from(0).await);
        assert_eq!(events.len(), 2);
    }
    #[tokio::test]
    async fn turn_entry_replay_with_skip() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        for i in 0..5 {
            entry
                .push_event(ChatStreamEvent::TextDelta(format!("e{i}")))
                .await;
        }
        // skip_before=0 → all 5 events
        let all = events_of(entry.replay_from(0).await);
        assert_eq!(all.len(), 5);
        // skip_before=2 → events 2,3,4 (3 events)
        let skipped = events_of(entry.replay_from(2).await);
        assert_eq!(skipped.len(), 3);
        // skip_before=5 → caught up to the live edge (not Gone).
        assert!(matches!(
            entry.replay_from(5).await,
            ReplayOutcome::CaughtUp { next_skip: 5 }
        ));
    }
    #[tokio::test]
    async fn turn_entry_replay_empty_by_default() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        // Empty buffer with skip_before=0 → caught up (nothing to replay yet).
        assert!(matches!(
            entry.replay_from(0).await,
            ReplayOutcome::CaughtUp { next_skip: 0 }
        ));
    }
    #[tokio::test]
    async fn turn_entry_is_running_initially() {
        let entry = TurnEntry::new("t1".to_string(), "/photo.jpg".to_string(), 1);
        assert!(entry.is_running());
    }
    #[tokio::test]
    async fn turn_entry_set_terminal_status() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        assert!(entry.is_running());
        entry.set_terminal_status(TurnStatus::Done);
        assert!(!entry.is_running());
    }
    #[tokio::test]
    async fn turn_entry_info() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            42,
        ));
        entry
            .push_event(ChatStreamEvent::TextDelta("x".to_string()))
            .await;
        entry.set_terminal_status(TurnStatus::Done);
        let info = entry.info().await;
        assert_eq!(info.turn_id, "t1");
        assert_eq!(info.file_path, "/photo.jpg");
        assert_eq!(info.library_id, 42);
        assert_eq!(info.status, TurnStatus::Done);
        assert_eq!(info.total_events_pushed, 1);
        assert_eq!(info.buffered_count, 1);
    }
    #[tokio::test]
    async fn turn_entry_eviction_caps_buffer() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        // Push MAX_BUFFERED_EVENTS + 10 events.
        for i in 0..(MAX_BUFFERED_EVENTS + 10) {
            entry
                .push_event(ChatStreamEvent::TextDelta(format!("e{i}")))
                .await;
        }
        // Asking from absolute 0 after eviction is Gone (0-9 were dropped).
        assert!(matches!(entry.replay_from(0).await, ReplayOutcome::Gone));
        // Reading from the new base (10) returns the full capped buffer.
        let events = events_of(entry.replay_from(10).await);
        assert_eq!(events.len(), MAX_BUFFERED_EVENTS);
        // First event should be at index 10 (0-9 were evicted).
        if let ChatStreamEvent::TextDelta(s) = &events[0] {
            assert_eq!(s, "e10");
        } else {
            panic!("expected TextDelta");
        }
        // Last event should be at index MAX_BUFFERED_EVENTS + 9.
        if let ChatStreamEvent::TextDelta(s) = &events[events.len() - 1] {
            assert_eq!(s, &format!("e{}", MAX_BUFFERED_EVENTS + 9));
        } else {
            panic!("expected TextDelta");
        }
    }
    #[tokio::test]
    async fn turn_entry_replay_evicted_index_is_gone() {
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        // Push one past the cap so exactly one event (index 0) is evicted.
        for i in 0..=MAX_BUFFERED_EVENTS {
            entry
                .push_event(ChatStreamEvent::TextDelta(format!("e{i}")))
                .await;
        }
        // Base is now 1; asking from absolute 0 is evicted territory → Gone.
        assert!(matches!(entry.replay_from(0).await, ReplayOutcome::Gone));
        // skip_before = MAX_BUFFERED_EVENTS → last event only (index valid).
        let last = events_of(entry.replay_from(MAX_BUFFERED_EVENTS as u32).await);
        assert_eq!(last.len(), 1);
        // skip_before = MAX_BUFFERED_EVENTS + 1 → caught up to the live edge.
        assert!(matches!(
            entry.replay_from((MAX_BUFFERED_EVENTS + 1) as u32).await,
            ReplayOutcome::CaughtUp { .. }
        ));
    }
    // ── TurnRegistry ────────────────────────────────────────────────
    #[tokio::test]
    async fn turn_registry_insert_and_get() {
        let registry = TurnRegistry::new(300);
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        let id = registry.insert(entry).await;
        assert_eq!(id, "t1");
        let retrieved = registry.get("t1").await;
        assert!(retrieved.is_some());
        assert_eq!(retrieved.unwrap().turn_id, "t1");
    }
    #[tokio::test]
    async fn turn_registry_get_nonexistent_returns_none() {
        let registry = TurnRegistry::new(300);
        assert!(registry.get("nonexistent").await.is_none());
    }
    #[tokio::test]
    async fn turn_registry_cleanup_stale_removes_old_entries() {
        let registry = TurnRegistry::new(0);
        let mut entry = TurnEntry::new("t1".to_string(), "/photo.jpg".to_string(), 1);
        entry.created_at = Instant::now() - Duration::from_secs(1);
        registry.insert(Arc::new(entry)).await;
        let cleaned = registry.cleanup_stale().await;
        assert_eq!(cleaned, 1);
        assert!(registry.get("t1").await.is_none());
    }
    #[tokio::test]
    async fn turn_registry_cleanup_stale_preserves_recent() {
        let registry = TurnRegistry::new(3600); // 1 hour
        let entry = Arc::new(TurnEntry::new(
            "t1".to_string(),
            "/photo.jpg".to_string(),
            1,
        ));
        registry.insert(entry).await;
        let cleaned = registry.cleanup_stale().await;
        assert_eq!(cleaned, 0);
        assert!(registry.get("t1").await.is_some());
    }
    #[tokio::test]
    async fn turn_registry_cleanup_stale_multiple() {
        let registry = TurnRegistry::new(0);
        for i in 0..5 {
            let mut entry = TurnEntry::new(format!("t{i}"), "/photo.jpg".to_string(), 1);
            entry.created_at = Instant::now() - Duration::from_secs(1);
            registry.insert(Arc::new(entry)).await;
        }
        let cleaned = registry.cleanup_stale().await;
        assert_eq!(cleaned, 5);
    }
    #[tokio::test]
    async fn turn_registry_timeout_secs() {
        let registry = TurnRegistry::new(600);
        assert_eq!(registry.timeout_secs(), 600);
    }
    // ── next_batch / live replay ────────────────────────────────────
    /// Drain a turn the way the SSE replay handler does: pull batches via
    /// `next_batch` until the turn is finished and fully drained.
    async fn drain_to_end(entry: Arc<TurnEntry>) -> Vec<ChatStreamEvent> {
        let mut out = Vec::new();
        let mut skip = 0u32;
        while let ReplayOutcome::Events { events, next_skip } = entry.next_batch(skip).await {
            out.extend(events);
            skip = next_skip;
        }
        out
    }
    fn is_terminal(ev: &ChatStreamEvent) -> bool {
        matches!(ev, ChatStreamEvent::Done { .. } | ChatStreamEvent::Error(_))
    }
    /// The core guarantee behind the replay rewrite: a reader waiting on
    /// `next_batch` always receives the terminal event, even though the
    /// writer flips status to terminal immediately after pushing it.
    #[tokio::test]
    async fn next_batch_always_delivers_terminal_event() {
        for _ in 0..50 {
            let entry = Arc::new(TurnEntry::new("t".into(), "/p.jpg".into(), 1));
            let writer = entry.clone();
            let w = tokio::spawn(async move {
                writer
                    .push_event(ChatStreamEvent::IterationStart { n: 1, max: 6 })
                    .await;
                writer
                    .push_event(ChatStreamEvent::TextDelta("hi".into()))
                    .await;
                // Push terminal then flip status with no await between — the
                // race that previously dropped the Done on the reader side.
                writer
                    .push_event(ChatStreamEvent::Done {
                        tool_calls_made: 0,
                        iterations_used: 1,
                        truncated: false,
                        prompt_tokens: None,
                        eval_tokens: None,
                        num_ctx: None,
                        amended_insight_id: None,
                        backend_used: "local".into(),
                        model_used: "m".into(),
                        cancelled: false,
                    })
                    .await;
                writer.set_terminal_status(TurnStatus::Done);
            });
            let events = drain_to_end(entry).await;
            w.await.unwrap();
            assert!(
                events.last().is_some_and(is_terminal),
                "terminal event missing; got {} events",
                events.len()
            );
            assert_eq!(events.len(), 3, "expected IterationStart, TextDelta, Done");
        }
    }
    /// A reader that connects before any event is pushed blocks in
    /// `next_batch` and then receives events as the writer produces them.
    #[tokio::test]
    async fn next_batch_waits_for_late_events() {
        let entry = Arc::new(TurnEntry::new("t".into(), "/p.jpg".into(), 1));
        let writer = entry.clone();
        tokio::spawn(async move {
            tokio::task::yield_now().await;
            writer
                .push_event(ChatStreamEvent::TextDelta("late".into()))
                .await;
            writer.set_terminal_status(TurnStatus::Done);
        });
        // First call blocks until the writer pushes, rather than returning
        // CaughtUp on the empty buffer of a running turn.
        match entry.next_batch(0).await {
            ReplayOutcome::Events { events, next_skip } => {
                assert_eq!(events.len(), 1);
                assert_eq!(next_skip, 1);
            }
            other => panic!("expected Events, got {other:?}"),
        }
    }
    #[tokio::test]
    async fn next_batch_closes_on_terminal_when_caught_up() {
        let entry = Arc::new(TurnEntry::new("t".into(), "/p.jpg".into(), 1));
        entry
            .push_event(ChatStreamEvent::TextDelta("x".into()))
            .await;
        entry.set_terminal_status(TurnStatus::Done);
        // Caught up (skip past the one buffered event) on a finished turn →
        // CaughtUp so the handler closes the stream rather than hanging.
        assert!(matches!(
            entry.next_batch(1).await,
            ReplayOutcome::CaughtUp { .. }
        ));
    }
    #[tokio::test]
    async fn next_batch_reports_gone_for_evicted_index() {
        let entry = Arc::new(TurnEntry::new("t".into(), "/p.jpg".into(), 1));
        for i in 0..=MAX_BUFFERED_EVENTS {
            entry
                .push_event(ChatStreamEvent::TextDelta(format!("e{i}")))
                .await;
        }
        // Index 0 was evicted (base advanced to 1).
        assert!(matches!(entry.next_batch(0).await, ReplayOutcome::Gone));
    }
    // ── abort handle (#1 cancellation) ──────────────────────────────
    #[tokio::test]
    async fn abort_handle_aborts_task_once() {
        let entry = Arc::new(TurnEntry::new("t".into(), "/p.jpg".into(), 1));
        // No handle installed yet → abort is a no-op.
        assert!(!entry.abort());
        let handle = tokio::spawn(async {
            // Long-lived task that only ends via abort.
            futures::future::pending::<()>().await;
        });
        entry.set_abort_handle(handle.abort_handle());
        assert!(entry.abort(), "first abort should fire");
        assert!(!entry.abort(), "handle is taken; second abort is a no-op");
        // The aborted task resolves to a cancellation JoinError.
        let join = handle.await;
        assert!(join.unwrap_err().is_cancelled());
    }
    #[tokio::test]
    async fn base_index_tracks_eviction() {
        let entry = Arc::new(TurnEntry::new("t".into(), "/p.jpg".into(), 1));
        for i in 0..(MAX_BUFFERED_EVENTS + 5) {
            entry
                .push_event(ChatStreamEvent::TextDelta(format!("e{i}")))
                .await;
        }
        let info = entry.info().await;
        // 5 events evicted; total keeps climbing, buffer stays capped.
        assert_eq!(info.total_events_pushed, (MAX_BUFFERED_EVENTS + 5) as u32);
        assert_eq!(info.buffered_count, MAX_BUFFERED_EVENTS as u32);
        // First live index is 5: reading from there yields the full buffer.
        let from_base = events_of(entry.replay_from(5).await);
        assert_eq!(from_base.len(), MAX_BUFFERED_EVENTS);
    }
 }
@@ -1,796 +0,0 @@
 //! Per-tick drains the watcher runs alongside ingest.
 //!
 //! These passes were previously inlined in `main.rs`; they exist because
 //! a quick scan only walks recently-modified files, so any backlog of
 //! rows missing a `content_hash` / `date_taken` / face detection
 //! wouldn't otherwise drain except during the once-an-hour full scan.
 //! Each function is bounded per call by a `*_PER_TICK` env-var cap.
 use std::collections::HashMap;
 use std::path::PathBuf;
 use std::sync::{Arc, Mutex};
 use log::{debug, info, warn};
 use crate::content_hash;
 use crate::database::ExifDao;
 use crate::date_resolver;
 use crate::face_watch;
 use crate::faces;
 use crate::file_types;
 use crate::libraries;
 use crate::tags;
 /// Compute and persist content_hash for image_exif rows where it's NULL.
 ///
 /// Bounded per call by `FACE_HASH_BACKFILL_MAX_PER_TICK` (default 2000)
 /// so a watcher tick on a large legacy library doesn't block for hours
 /// blake3-ing every photo at once. Subsequent scans pick up the rest.
 /// For 50k+ libraries the dedicated `cargo run --bin backfill_hashes`
 /// is still faster (it doesn't fight a watcher loop for the DAO mutex).
 ///
 /// Drains unhashed image_exif rows by querying them directly, independent
 /// of the filesystem walk. Quick scans only walk recently-modified files,
 /// so a backlog of pre-existing unhashed rows never enters
 /// `process_new_files`'s candidate set — left alone, it would only drain
 /// on full scans (default once an hour). Calling this every tick keeps
 /// the face-detection backlog moving regardless.
 ///
 /// Returns the number of rows successfully backfilled this pass.
 pub fn backfill_unhashed_backlog(
    context: &opentelemetry::Context,
    library: &libraries::Library,
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
 ) -> usize {
    let cap: i64 = dotenv::var("FACE_HASH_BACKFILL_MAX_PER_TICK")
        .ok()
        .and_then(|s| s.parse().ok())
        .filter(|n: &i64| *n > 0)
        .unwrap_or(2000);
    // Fetch up to cap+1 rows so we can tell "more remain" without a
    // separate count query. Across libraries — there's no per-library
    // filter on get_rows_missing_hash today — but we only ever update
    // rows whose library_id matches the caller's library, so other
    // libraries' rows just get skipped here and picked up on the next
    // library's tick. Negligible cost given the cap.
    let rows: Vec<(i32, String)> = {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        dao.get_rows_missing_hash(context, cap + 1)
            .unwrap_or_default()
    };
    if rows.is_empty() {
        return 0;
    }
    let more_than_cap = rows.len() as i64 > cap;
    let base_path = std::path::Path::new(&library.root_path);
    let mut backfilled = 0usize;
    let mut errors = 0usize;
    let mut skipped_other_lib = 0usize;
    for (lib_id, rel_path) in rows.iter().take(cap as usize) {
        if *lib_id != library.id {
            skipped_other_lib += 1;
            continue;
        }
        let abs = base_path.join(rel_path);
        if !abs.exists() {
            // File walked away — the watcher's reconciliation pass will
            // remove the orphan exif row eventually.
            continue;
        }
        match content_hash::compute(&abs) {
            Ok(id) => {
                let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
                if let Err(e) = dao.backfill_content_hash(
                    context,
                    library.id,
                    rel_path,
                    &id.content_hash,
                    id.size_bytes,
                ) {
                    warn!(
                        "face_watch: backfill_content_hash failed for {}: {:?}",
                        rel_path, e
                    );
                    errors += 1;
                } else {
                    backfilled += 1;
                }
            }
            Err(e) => {
                debug!(
                    "face_watch: hash compute failed for {} ({:?})",
                    abs.display(),
                    e
                );
                errors += 1;
            }
        }
    }
    if backfilled > 0 || errors > 0 || more_than_cap {
        info!(
            "face_watch: backfill pass for library '{}': hashed {} ({} error(s), {} skipped to other libraries; {} cap, more_remain={})",
            library.name, backfilled, errors, skipped_other_lib, cap, more_than_cap
        );
    }
    backfilled
 }
 /// Drain image_exif rows whose `date_taken` was never resolved or was
 /// resolved by the weakest fallback (`fs_time`). Runs the canonical-date
 /// waterfall — exiftool batch (one subprocess for the whole tick's
 /// rows) → filename regex → earliest_fs_time — and persists each
 /// resolution with its source tag. Capped per tick by
 /// `DATE_BACKFILL_MAX_PER_TICK` (default 500) so a 14k-row library
 /// drains over a few quick-scan ticks without blocking the watcher.
 ///
 /// kamadak-exif is intentionally skipped here: the row already has a
 /// NULL date_taken because the ingest path's kamadak-exif call returned
 /// nothing, and re-running it would just produce the same answer.
 /// exiftool is the meaningful new attempt — it handles videos and
 /// MakerNote-hosted dates kamadak can't reach.
 pub fn backfill_missing_date_taken(
    context: &opentelemetry::Context,
    library: &libraries::Library,
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
 ) -> usize {
    let cap: i64 = dotenv::var("DATE_BACKFILL_MAX_PER_TICK")
        .ok()
        .and_then(|s| s.parse().ok())
        .filter(|n: &i64| *n > 0)
        .unwrap_or(500);
    let rows: Vec<(i32, String)> = {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        dao.get_rows_needing_date_backfill(context, library.id, cap + 1)
            .unwrap_or_default()
    };
    if rows.is_empty() {
        return 0;
    }
    let more_than_cap = rows.len() as i64 > cap;
    let base_path = std::path::Path::new(&library.root_path);
    // Build absolute paths and drop rows whose files no longer exist —
    // the missing-file scan in library_maintenance retires deleted rows
    // separately. Without this filter, NULL-date rows for missing files
    // would loop through the drain forever (no source can resolve them).
    let mut existing: Vec<(String, PathBuf)> = Vec::with_capacity(rows.len());
    for (_, rel_path) in rows.iter().take(cap as usize) {
        let abs = base_path.join(rel_path);
        if abs.exists() {
            existing.push((rel_path.clone(), abs));
        }
    }
    if existing.is_empty() {
        return 0;
    }
    // One exiftool subprocess for the whole batch; the resolver falls
    // through to filename / fs_time per file when exiftool can't supply
    // a date (or isn't installed at all).
    let paths: Vec<PathBuf> = existing.iter().map(|(_, p)| p.clone()).collect();
    let resolved = date_resolver::resolve_dates_batch(&paths, &HashMap::new());
    let mut backfilled = 0usize;
    let mut unresolved = 0usize;
    let mut by_source: HashMap<&'static str, usize> = HashMap::new();
    {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        for (rel_path, abs) in &existing {
            let Some(rd) = resolved.get(abs).copied() else {
                unresolved += 1;
                continue;
            };
            match dao.backfill_date_taken(
                context,
                library.id,
                rel_path,
                rd.timestamp,
                rd.source.as_str(),
            ) {
                Ok(()) => {
                    backfilled += 1;
                    *by_source.entry(rd.source.as_str()).or_insert(0) += 1;
                }
                Err(e) => {
                    warn!(
                        "date_backfill: update failed for lib {} {}: {:?}",
                        library.id, rel_path, e
                    );
                }
            }
        }
    }
    if backfilled > 0 || unresolved > 0 || more_than_cap {
        info!(
            "date_backfill: library '{}': resolved {} ({:?}), {} unresolved, cap={}, more_remain={}",
            library.name, backfilled, by_source, unresolved, cap, more_than_cap
        );
    }
    backfilled
 }
 /// Per-tick face-detection drain. Pulls a capped batch of hashed-but-
 /// unscanned image_exif rows directly via the FaceDao anti-join and
 /// hands them to the existing detection pass. Runs on every tick (not
 /// just full scans) so the backlog moves at quick-scan cadence.
 /// Per-tick CLIP encoding drain. Mirrors `process_face_backlog`: pull
 /// up to `CLIP_BACKLOG_MAX_PER_TICK` candidates with a known
 /// `content_hash` but no `clip_embedding`, hand them to
 /// `clip_watch::run_clip_encoding_pass` for parallel fan-out, and let
 /// that module write the result back via `backfill_clip_embedding`.
 ///
 /// Idempotent — a row stays in the candidate set until its embedding
 /// lands, so a transient failure (Apollo unreachable, CUDA OOM) just
 /// defers to the next tick. Permanent failures (un-decodable bytes)
 /// retry every tick at this point; future Branch may add a status
 /// column like face_detections has.
 pub fn process_clip_backlog(
    context: &opentelemetry::Context,
    library: &libraries::Library,
    clip_client: &crate::ai::clip_client::ClipClient,
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
    excluded_dirs: &[String],
 ) {
    if !clip_client.is_enabled() {
        return;
    }
    let cap: i64 = dotenv::var("CLIP_BACKLOG_MAX_PER_TICK")
        .ok()
        .and_then(|s| s.parse().ok())
        .filter(|n: &i64| *n > 0)
        .unwrap_or(32);
    let rows: Vec<(String, String)> = {
        let mut dao = exif_dao.lock().expect("exif dao");
        match dao.list_clip_unencoded_candidates(context, library.id, cap) {
            Ok(r) => r,
            Err(e) => {
                warn!(
                    "clip_watch: list_clip_unencoded_candidates failed for library '{}': {:?}",
                    library.name, e
                );
                return;
            }
        }
    };
    if rows.is_empty() {
        return;
    }
    info!(
        "clip_watch: backlog drain — encoding {} candidate(s) for library '{}' (cap={})",
        rows.len(),
        library.name,
        cap
    );
    let candidates: Vec<crate::clip_watch::ClipCandidate> = rows
        .into_iter()
        .map(
            |(rel_path, content_hash)| crate::clip_watch::ClipCandidate {
                rel_path,
                content_hash,
            },
        )
        .collect();
    crate::clip_watch::run_clip_encoding_pass(
        library,
        excluded_dirs,
        clip_client,
        Arc::clone(exif_dao),
        candidates,
    );
 }
 pub fn process_face_backlog(
    context: &opentelemetry::Context,
    library: &libraries::Library,
    face_client: &crate::ai::face_client::FaceClient,
    face_dao: &Arc<Mutex<Box<dyn faces::FaceDao>>>,
    tag_dao: &Arc<Mutex<Box<dyn tags::TagDao>>>,
    excluded_dirs: &[String],
 ) {
    let cap: i64 = dotenv::var("FACE_BACKLOG_MAX_PER_TICK")
        .ok()
        .and_then(|s| s.parse().ok())
        .filter(|n: &i64| *n > 0)
        .unwrap_or(64);
    let rows: Vec<(String, String)> = {
        let mut dao = face_dao.lock().expect("face dao");
        match dao.list_unscanned_candidates(context, library.id, cap) {
            Ok(r) => r,
            Err(e) => {
                warn!(
                    "face_watch: list_unscanned_candidates failed for library '{}': {:?}",
                    library.name, e
                );
                return;
            }
        }
    };
    if rows.is_empty() {
        return;
    }
    info!(
        "face_watch: backlog drain — running detection on {} candidate(s) for library '{}' (cap={})",
        rows.len(),
        library.name,
        cap
    );
    let candidates: Vec<face_watch::FaceCandidate> = rows
        .into_iter()
        .map(|(rel_path, content_hash)| face_watch::FaceCandidate {
            rel_path,
            content_hash,
        })
        .collect();
    face_watch::run_face_detection_pass(
        library,
        excluded_dirs,
        face_client,
        Arc::clone(face_dao),
        Arc::clone(tag_dao),
        candidates,
    );
 }
 /// Compute content_hash for any image rows the walker just touched
 /// whose stored EXIF row is still hash-less. Called from
 /// `process_new_files` so freshly-ingested files don't have to wait for
 /// the next standalone `backfill_unhashed_backlog` tick before face
 /// detection can key on their bytes.
 ///
 /// Cap is on **successes only**. An earlier version counted errors too,
 /// so a pocket of chronically-unhashable files at the front of the
 /// table (vanished mid-scan, permission denied, etc.) burned the budget
 /// every tick and the rest of the backlog never advanced.
 pub fn backfill_missing_content_hashes(
    context: &opentelemetry::Context,
    files: &[(PathBuf, String)],
    library: &libraries::Library,
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
 ) {
    let image_paths: Vec<String> = files
        .iter()
        .filter(|(p, _)| !file_types::is_video_file(p))
        .map(|(_, rel)| rel.clone())
        .collect();
    if image_paths.is_empty() {
        return;
    }
    let exif_records = {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        dao.get_exif_batch(context, Some(library.id), &image_paths)
            .unwrap_or_default()
    };
    // Cheap lookup back from rel_path → absolute file_path so
    // content_hash::compute can read the bytes.
    let path_by_rel: HashMap<String, &PathBuf> =
        files.iter().map(|(p, rel)| (rel.clone(), p)).collect();
    let cap: usize = dotenv::var("FACE_HASH_BACKFILL_MAX_PER_TICK")
        .ok()
        .and_then(|s| s.parse().ok())
        .filter(|n: &usize| *n > 0)
        .unwrap_or(2000);
    // Count the unhashed backlog up front so we can surface "still needs
    // backfill: N" in the log — without it, a face-scan that's stuck at
    // 44% looks stalled when really it's chipping through hashes.
    let unhashed_total = exif_records
        .iter()
        .filter(|r| r.content_hash.is_none())
        .count();
    let mut backfilled = 0usize;
    let mut errors = 0usize;
    for record in &exif_records {
        if backfilled >= cap {
            break;
        }
        if record.content_hash.is_some() {
            continue;
        }
        let Some(file_path) = path_by_rel.get(&record.file_path) else {
            // Walked file went missing between the directory scan and now;
            // next tick will retry naturally.
            continue;
        };
        match content_hash::compute(file_path) {
            Ok(id) => {
                let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
                if let Err(e) = dao.backfill_content_hash(
                    context,
                    library.id,
                    &record.file_path,
                    &id.content_hash,
                    id.size_bytes,
                ) {
                    warn!(
                        "face_watch: backfill_content_hash failed for {}: {:?}",
                        record.file_path, e
                    );
                    errors += 1;
                } else {
                    backfilled += 1;
                }
            }
            Err(e) => {
                debug!(
                    "face_watch: hash compute failed for {} ({:?})",
                    file_path.display(),
                    e
                );
                errors += 1;
            }
        }
    }
    // Always log when there's an unhashed backlog so an operator
    // looking at "scan stuck at 44%" can see backfill is running and
    // how much remains. Quiet only when there's nothing to do.
    if unhashed_total > 0 || backfilled > 0 || errors > 0 {
        let remaining = unhashed_total.saturating_sub(backfilled);
        info!(
            "face_watch: backfilled {}/{} content_hash for library '{}' ({} error(s); {} still need backfill; cap={})",
            backfilled, unhashed_total, library.name, errors, remaining, cap
        );
    }
 }
 /// Build the face-detection candidate list for a scan tick.
 ///
 /// Returns `(rel_path, content_hash)` for every image file that has a
 /// content_hash recorded in image_exif but no row in face_detections
 /// yet. Re-querying image_exif here picks up rows the EXIF write loop
 /// just inserted alongside any pre-existing rows the watcher walked
 /// over — covers both new uploads and the initial backlog scan.
 pub fn build_face_candidates(
    context: &opentelemetry::Context,
    library: &libraries::Library,
    files: &[(PathBuf, String)],
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
    face_dao: &Arc<Mutex<Box<dyn faces::FaceDao>>>,
 ) -> Vec<face_watch::FaceCandidate> {
    // Restrict to image files; videos aren't face-scanned in v1 (kamadak
    // doesn't even register them in image_exif).
    let image_paths: Vec<String> = files
        .iter()
        .filter(|(p, _)| !file_types::is_video_file(p))
        .map(|(_, rel)| rel.clone())
        .collect();
    if image_paths.is_empty() {
        return Vec::new();
    }
    let exif_records = {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        dao.get_exif_batch(context, Some(library.id), &image_paths)
            .unwrap_or_default()
    };
    // rel_path → content_hash (only rows with a hash; without one we have
    // nothing to key face data against).
    let mut hash_by_path: HashMap<String, String> = HashMap::with_capacity(exif_records.len());
    for record in exif_records {
        if let Some(h) = record.content_hash {
            hash_by_path.insert(record.file_path, h);
        }
    }
    let mut candidates = Vec::new();
    let mut dao = face_dao.lock().expect("face dao");
    for rel_path in image_paths {
        let Some(hash) = hash_by_path.get(&rel_path) else {
            continue;
        };
        match dao.already_scanned(context, hash) {
            Ok(true) => continue,
            Ok(false) => candidates.push(face_watch::FaceCandidate {
                rel_path,
                content_hash: hash.clone(),
            }),
            Err(e) => {
                warn!("face_watch: already_scanned errored for {}: {:?}", hash, e);
            }
        }
    }
    candidates
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use std::fs;
    use std::sync::{Arc, Mutex};
    use diesel::prelude::*;
    use tempfile::TempDir;
    use crate::database::models::{InsertImageExif, InsertLibrary};
    use crate::database::test::in_memory_db_connection;
    use crate::database::{ExifDao, SqliteExifDao, schema};
    use crate::faces::{FaceDao, SqliteFaceDao};
    use crate::libraries::Library;
    fn ctx() -> opentelemetry::Context {
        opentelemetry::Context::new()
    }
    /// Everything `setup` hands back to a test: tempdir, library, shared
    /// connection, and the two DAOs. Aliased to keep clippy's
    /// type-complexity lint satisfied.
    type SetupFixture = (
        TempDir,
        Library,
        Arc<Mutex<diesel::SqliteConnection>>,
        Arc<Mutex<Box<dyn ExifDao>>>,
        Arc<Mutex<Box<dyn FaceDao>>>,
    );
    /// Build a tempdir-backed library + DAOs sharing a single in-memory
    /// SQLite connection (so cross-table joins like
    /// `list_unscanned_candidates` see consistent state).
    fn setup() -> SetupFixture {
        let tmp = TempDir::new().expect("tempdir");
        let mut conn = in_memory_db_connection();
        // Migration seeds library id=1 with a placeholder root; rewrite it
        // to point at the tempdir so `<root>/<rel_path>` resolves to real
        // files this test creates.
        diesel::update(schema::libraries::table.filter(schema::libraries::id.eq(1)))
            .set(schema::libraries::root_path.eq(tmp.path().to_string_lossy().to_string()))
            .execute(&mut conn)
            .expect("rewrite library 1 root");
        // Add a second library so cross-library skip cases have somewhere
        // to put their rows.
        diesel::insert_into(schema::libraries::table)
            .values(InsertLibrary {
                name: "other",
                root_path: "/tmp/other-test-lib",
                created_at: 0,
                enabled: true,
                excluded_dirs: None,
            })
            .execute(&mut conn)
            .expect("seed second library");
        let library = Library {
            id: 1,
            name: "main".to_string(),
            root_path: tmp.path().to_string_lossy().to_string(),
            enabled: true,
            excluded_dirs: Vec::new(),
        };
        let shared = Arc::new(Mutex::new(conn));
        let exif_dao: Arc<Mutex<Box<dyn ExifDao>>> = Arc::new(Mutex::new(Box::new(
            SqliteExifDao::from_shared(Arc::clone(&shared)),
        )));
        let face_dao: Arc<Mutex<Box<dyn FaceDao>>> = Arc::new(Mutex::new(Box::new(
            SqliteFaceDao::from_connection(Arc::clone(&shared)),
        )));
        (tmp, library, shared, exif_dao, face_dao)
    }
    fn insert_exif(
        exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
        lib_id: i32,
        rel: &str,
        content_hash: Option<&str>,
    ) {
        let mut dao = exif_dao.lock().unwrap();
        dao.store_exif(
            &ctx(),
            InsertImageExif {
                library_id: lib_id,
                file_path: rel.to_string(),
                camera_make: None,
                camera_model: None,
                lens_model: None,
                width: None,
                height: None,
                orientation: None,
                gps_latitude: None,
                gps_longitude: None,
                gps_altitude: None,
                focal_length: None,
                aperture: None,
                shutter_speed: None,
                iso: None,
                date_taken: None,
                created_time: 0,
                last_modified: 0,
                content_hash: content_hash.map(|s| s.to_string()),
                size_bytes: None,
                phash_64: None,
                dhash_64: None,
                date_taken_source: None,
            },
        )
        .expect("insert");
    }
    fn write_image(root: &std::path::Path, rel: &str, bytes: &[u8]) {
        let abs = root.join(rel);
        if let Some(parent) = abs.parent() {
            fs::create_dir_all(parent).expect("mkdir");
        }
        fs::write(abs, bytes).expect("write file");
    }
    #[test]
    fn backfill_unhashed_backlog_hashes_missing_rows_in_this_library() {
        let (tmp, library, _conn, exif_dao, _face_dao) = setup();
        write_image(tmp.path(), "a.jpg", b"alpha-bytes");
        write_image(tmp.path(), "b.jpg", b"bravo-bytes");
        insert_exif(&exif_dao, 1, "a.jpg", None);
        insert_exif(&exif_dao, 1, "b.jpg", None);
        let backfilled = backfill_unhashed_backlog(&ctx(), &library, &exif_dao);
        assert_eq!(backfilled, 2);
        let mut dao = exif_dao.lock().unwrap();
        let rows = dao
            .get_exif_batch(&ctx(), Some(1), &["a.jpg".to_string(), "b.jpg".to_string()])
            .unwrap();
        assert_eq!(rows.len(), 2);
        for r in rows {
            assert!(
                r.content_hash.is_some(),
                "row {} should have a hash",
                r.file_path
            );
        }
    }
    #[test]
    fn backfill_unhashed_backlog_skips_other_libraries_and_missing_files() {
        let (tmp, library, _conn, exif_dao, _face_dao) = setup();
        write_image(tmp.path(), "exists.jpg", b"hello");
        // Row for this library whose file is missing on disk:
        insert_exif(&exif_dao, 1, "ghost.jpg", None);
        insert_exif(&exif_dao, 1, "exists.jpg", None);
        // Row in the other library — must be skipped (different lib_id).
        insert_exif(&exif_dao, 2, "other.jpg", None);
        let backfilled = backfill_unhashed_backlog(&ctx(), &library, &exif_dao);
        assert_eq!(backfilled, 1, "only the existing in-library file hashes");
        let mut dao = exif_dao.lock().unwrap();
        let other = dao
            .get_exif_batch(&ctx(), Some(2), &["other.jpg".to_string()])
            .unwrap();
        assert_eq!(other.len(), 1);
        assert!(
            other[0].content_hash.is_none(),
            "other-library row must remain unhashed"
        );
        let ghost = dao
            .get_exif_batch(&ctx(), Some(1), &["ghost.jpg".to_string()])
            .unwrap();
        assert_eq!(ghost.len(), 1);
        assert!(
            ghost[0].content_hash.is_none(),
            "missing-on-disk row stays unhashed (reconciliation removes it later)"
        );
    }
    #[test]
    fn backfill_unhashed_backlog_respects_per_tick_cap() {
        // Env-var-driven cap; the function reads it on every call, so we
        // can set it just for this test and unset before returning.
        // Serial guard: tests in the same binary may share env, but each
        // backfill call re-reads — and we only care that the cap shape
        // (success count <= cap, more_remain logged) holds.
        unsafe {
            std::env::set_var("FACE_HASH_BACKFILL_MAX_PER_TICK", "2");
        }
        let (tmp, library, _conn, exif_dao, _face_dao) = setup();
        for i in 0..5 {
            let rel = format!("img_{}.jpg", i);
            write_image(tmp.path(), &rel, format!("bytes-{}", i).as_bytes());
            insert_exif(&exif_dao, 1, &rel, None);
        }
        let backfilled = backfill_unhashed_backlog(&ctx(), &library, &exif_dao);
        assert_eq!(backfilled, 2, "cap=2 must bound the per-tick successes");
        unsafe {
            std::env::remove_var("FACE_HASH_BACKFILL_MAX_PER_TICK");
        }
    }
    #[test]
    fn backfill_missing_content_hashes_skips_videos_and_hashed_rows() {
        let (tmp, library, _conn, exif_dao, _face_dao) = setup();
        // Two image rows (one already hashed, one not), one video.
        write_image(tmp.path(), "fresh.jpg", b"fresh-pixels");
        write_image(tmp.path(), "already.jpg", b"already-pixels");
        write_image(tmp.path(), "clip.mp4", b"video-bytes");
        insert_exif(&exif_dao, 1, "fresh.jpg", None);
        insert_exif(&exif_dao, 1, "already.jpg", Some("pre-existing-hash"));
        insert_exif(&exif_dao, 1, "clip.mp4", None);
        let files: Vec<(PathBuf, String)> = vec![
            (tmp.path().join("fresh.jpg"), "fresh.jpg".to_string()),
            (tmp.path().join("already.jpg"), "already.jpg".to_string()),
            (tmp.path().join("clip.mp4"), "clip.mp4".to_string()),
        ];
        backfill_missing_content_hashes(&ctx(), &files, &library, &exif_dao);
        let mut dao = exif_dao.lock().unwrap();
        let rows = dao
            .get_exif_batch(
                &ctx(),
                Some(1),
                &[
                    "fresh.jpg".to_string(),
                    "already.jpg".to_string(),
                    "clip.mp4".to_string(),
                ],
            )
            .unwrap();
        let by_path: HashMap<String, Option<String>> = rows
            .into_iter()
            .map(|r| (r.file_path, r.content_hash))
            .collect();
        assert!(
            by_path["fresh.jpg"].is_some(),
            "fresh image must get a hash"
        );
        assert_eq!(
            by_path["already.jpg"].as_deref(),
            Some("pre-existing-hash"),
            "already-hashed image left untouched"
        );
        assert!(
            by_path["clip.mp4"].is_none(),
            "video skipped (not face-scanned, no hash needed via this path)"
        );
    }
    #[test]
    fn build_face_candidates_filters_videos_unhashed_and_already_scanned() {
        let (tmp, library, _conn, exif_dao, face_dao) = setup();
        // Seed image_exif with: hashed unscanned, hashed scanned, unhashed,
        // and a video. Files don't need to exist on disk — the function
        // doesn't read them, only the DB rows.
        insert_exif(&exif_dao, 1, "fresh.jpg", Some("hash-fresh"));
        insert_exif(&exif_dao, 1, "scanned.jpg", Some("hash-scanned"));
        insert_exif(&exif_dao, 1, "unhashed.jpg", None);
        insert_exif(&exif_dao, 1, "clip.mp4", Some("hash-video"));
        // Mark `scanned.jpg`'s hash as already detected.
        {
            let mut dao = face_dao.lock().unwrap();
            dao.mark_status(&ctx(), 1, "hash-scanned", "scanned.jpg", "no_faces", "test")
                .expect("mark scanned");
        }
        let files: Vec<(PathBuf, String)> = vec![
            (tmp.path().join("fresh.jpg"), "fresh.jpg".to_string()),
            (tmp.path().join("scanned.jpg"), "scanned.jpg".to_string()),
            (tmp.path().join("unhashed.jpg"), "unhashed.jpg".to_string()),
            (tmp.path().join("clip.mp4"), "clip.mp4".to_string()),
        ];
        let candidates = build_face_candidates(&ctx(), &library, &files, &exif_dao, &face_dao);
        assert_eq!(
            candidates.len(),
            1,
            "exactly fresh.jpg should be a candidate"
        );
        assert_eq!(candidates[0].rel_path, "fresh.jpg");
        assert_eq!(candidates[0].content_hash, "hash-fresh");
    }
 }
@@ -14,7 +14,6 @@ use image_api::database::{
    SqliteInsightDao, SqliteKnowledgeDao, SqliteLocationHistoryDao, SqliteSearchHistoryDao,
    connect,
 };
 use image_api::faces::{FaceDao, SqliteFaceDao};
 use image_api::file_types::{IMAGE_EXTENSIONS, VIDEO_EXTENSIONS};
 use image_api::libraries::{self, Library};
 use image_api::tags::{SqliteTagDao, TagDao};
@@ -183,11 +182,6 @@ async fn main() -> anyhow::Result<()> {
        Arc::new(Mutex::new(Box::new(SqliteTagDao::default())));
    let knowledge_dao: Arc<Mutex<Box<dyn KnowledgeDao>>> =
        Arc::new(Mutex::new(Box::new(SqliteKnowledgeDao::new())));
    let face_dao: Arc<Mutex<Box<dyn FaceDao>>> =
        Arc::new(Mutex::new(Box::new(SqliteFaceDao::new())));
    let persona_dao: Arc<Mutex<Box<dyn image_api::database::PersonaDao>>> = Arc::new(Mutex::new(
        Box::new(image_api::database::SqlitePersonaDao::new()),
    ));
    // Pass the full library set so `resolve_full_path` probes every root,
    // even when --library restricts the walk. A rel_path shared across
@@ -195,7 +189,6 @@ async fn main() -> anyhow::Result<()> {
    let generator = InsightGenerator::new(
        ollama,
        None,
        None,
        sms_client,
        apollo_client,
        insight_dao.clone(),
@@ -205,9 +198,7 @@ async fn main() -> anyhow::Result<()> {
        location_dao,
        search_dao,
        tag_dao,
        face_dao,
        knowledge_dao,
        persona_dao,
        all_libs.clone(),
    );
@@ -340,8 +331,7 @@ async fn main() -> anyhow::Result<()> {
                None,
                Vec::new(),
                Vec::new(),
-                1, // operator user_id — populate_knowledge is single-user offline tool
+                false, // disable_writes — keep KB writes on for the population job
                "default".to_string(),
            )
            .await
        {
@@ -1,273 +0,0 @@
 //! Probe binary for CLIP semantic search.
 //!
 //! No DB writes. Walks a library's `image_exif` rows, encodes a sample
 //! via Apollo's `/encode_image`, encodes the user's --query via
 //! `/encode_text`, and prints the top-K most similar photos by cosine
 //! similarity so the operator can eyeball quality before committing to
 //! the persistence phase (column populated by backlog drain, search
 //! endpoint, UI).
 //!
 //! Usage:
 //!     cargo run --release --bin probe_clip_search -- \
 //!         --library 1 --limit 200 --query "a beach at sunset" --top 10
 //!
 //! Env: standard ImageApi `.env`. Requires either
 //! `APOLLO_CLIP_API_BASE_URL` or `APOLLO_API_BASE_URL` to be set.
 use std::path::{Path, PathBuf};
 use std::sync::{Arc, Mutex};
 use std::time::Instant;
 use clap::Parser;
 use log::{info, warn};
 use image_api::ai::clip_client::{ClipClient, ClipError, EncodeImageMeta};
 use image_api::database::{ExifDao, SqliteExifDao, connect};
 use image_api::exif;
 use image_api::file_types;
 use image_api::libraries::{self, Library};
 #[derive(Parser, Debug)]
 #[command(name = "probe_clip_search")]
 #[command(about = "Top-K CLIP semantic search over a sample of image_exif rows")]
 struct Args {
    /// Library id to sample from.
    #[arg(long)]
    library: i32,
    /// Max files to encode. CPU inference is slow (~1-3 s per photo at
    /// ViT-L/14); start small and grow once GPU is sorted.
    #[arg(long, default_value_t = 50)]
    limit: usize,
    /// Natural-language query. Empty triggers an error from Apollo.
    #[arg(long)]
    query: String,
    /// How many top results to print.
    #[arg(long, default_value_t = 10)]
    top: usize,
    /// Offset into the library's rel_path listing.
    #[arg(long, default_value_t = 0)]
    offset: i64,
    /// How many DB rows to scan before giving up on hitting the limit.
    #[arg(long, default_value_t = 5000)]
    max_scan: i64,
 }
 /// Same as `face_watch::read_image_bytes_for_detect` (which is pub(crate)).
 /// Inlined for the throwaway probe.
 fn read_image_bytes(path: &Path) -> std::io::Result<Vec<u8>> {
    if file_types::needs_ffmpeg_thumbnail(path)
        && let Some(preview) = exif::extract_embedded_jpeg_preview(path)
    {
        return Ok(preview);
    }
    std::fs::read(path)
 }
 /// Decode a base64'd LE float32 vector to a `Vec<f32>`.
 fn decode_f32_vec(b64: &str) -> anyhow::Result<Vec<f32>> {
    use base64::Engine;
    let bytes = base64::engine::general_purpose::STANDARD.decode(b64.as_bytes())?;
    if bytes.len() % 4 != 0 {
        anyhow::bail!("embedding byte length {} not divisible by 4", bytes.len());
    }
    let mut out = Vec::with_capacity(bytes.len() / 4);
    for chunk in bytes.chunks_exact(4) {
        out.push(f32::from_le_bytes([chunk[0], chunk[1], chunk[2], chunk[3]]));
    }
    Ok(out)
 }
 /// Plain dot product. Apollo L2-normalizes both sides, so this is cosine sim.
 fn dot(a: &[f32], b: &[f32]) -> f32 {
    a.iter().zip(b.iter()).map(|(x, y)| x * y).sum()
 }
 #[tokio::main]
 async fn main() -> anyhow::Result<()> {
    env_logger::init();
    dotenv::dotenv().ok();
    let args = Args::parse();
    if args.query.trim().is_empty() {
        anyhow::bail!("--query must not be empty");
    }
    let client = ClipClient::from_env();
    if !client.is_enabled() {
        anyhow::bail!(
            "ClipClient disabled: set APOLLO_CLIP_API_BASE_URL or APOLLO_API_BASE_URL in .env"
        );
    }
    match client.health().await {
        Ok(h) => info!(
            "clip engine: loaded={} device={} model={} dim={}",
            h.loaded, h.device, h.model_version, h.embedding_dim
        ),
        Err(e) => warn!("health probe failed (continuing): {e}"),
    }
    let mut seed_conn = connect();
    if let Some(base) = dotenv::var("BASE_PATH").ok().as_deref() {
        libraries::seed_or_patch_from_env(&mut seed_conn, base);
    }
    let libs = libraries::load_all(&mut seed_conn);
    drop(seed_conn);
    let lib: Library = libs
        .into_iter()
        .find(|l| l.id == args.library)
        .ok_or_else(|| anyhow::anyhow!("library id {} not found", args.library))?;
    info!(
        "probing library #{} ({}) at {}",
        lib.id, lib.name, lib.root_path
    );
    let dao: Arc<Mutex<Box<dyn ExifDao>>> = Arc::new(Mutex::new(Box::new(SqliteExifDao::new())));
    let ctx = opentelemetry::Context::new();
    // Encode the query up-front so the long image-encode loop doesn't
    // race a slow query encode. Fails fast on a misspelled query.
    let query_resp = client
        .encode_text(&args.query)
        .await
        .map_err(|e| anyhow::anyhow!("encode_text: {e}"))?;
    let query_vec = decode_f32_vec(&query_resp.embedding)?;
    info!(
        "query encoded ({}d, {}ms): {:?}",
        query_resp.embedding_dim, query_resp.duration_ms, args.query
    );
    // Page through (id, rel_path), filter to images on disk, encode up
    // to `limit`. Each encoded photo gets scored against the query and
    // kept in a top-K heap.
    const PAGE: i64 = 500;
    let mut offset = args.offset;
    let mut scanned: i64 = 0;
    let mut encoded = 0usize;
    let mut perm_fail = 0usize;
    let mut transient_fail = 0usize;
    let root = PathBuf::from(&lib.root_path);
    let started = Instant::now();
    // (similarity, rel_path) — we keep all scored results and sort at
    // the end. With limit≤few-hundred this is trivial.
    let mut scores: Vec<(f32, String)> = Vec::with_capacity(args.limit);
    'outer: loop {
        if scanned >= args.max_scan {
            warn!(
                "scan cap ({}) reached before hitting limit ({}); bump --max-scan to scan deeper",
                args.max_scan, args.limit
            );
            break;
        }
        let rows = {
            let mut guard = dao.lock().expect("dao lock");
            guard
                .list_rel_paths_for_library_page(&ctx, lib.id, PAGE, offset)
                .map_err(|e| anyhow::anyhow!("list rel_paths: {:?}", e))?
        };
        if rows.is_empty() {
            info!("no more rows after offset {}", offset);
            break;
        }
        offset += rows.len() as i64;
        scanned += rows.len() as i64;
        for (_id, rel_path) in rows {
            if encoded >= args.limit {
                break 'outer;
            }
            let abs = root.join(&rel_path);
            if !file_types::is_image_file(&abs) || !abs.exists() {
                continue;
            }
            let bytes = match read_image_bytes(&abs) {
                Ok(b) => b,
                Err(e) => {
                    warn!("read {rel_path}: {e}");
                    continue;
                }
            };
            let meta = EncodeImageMeta {
                content_hash: String::new(),
                library_id: lib.id,
                rel_path: rel_path.clone(),
            };
            let call_start = Instant::now();
            match client.encode_image(bytes, meta).await {
                Ok(resp) => {
                    encoded += 1;
                    let vec = match decode_f32_vec(&resp.embedding) {
                        Ok(v) => v,
                        Err(e) => {
                            warn!("decode {rel_path}: {e}");
                            continue;
                        }
                    };
                    if vec.len() != query_vec.len() {
                        warn!(
                            "dim mismatch for {rel_path}: image={} query={}",
                            vec.len(),
                            query_vec.len()
                        );
                        continue;
                    }
                    let sim = dot(&vec, &query_vec);
                    scores.push((sim, rel_path.clone()));
                    if encoded.is_multiple_of(10) {
                        info!(
                            "progress: {} encoded, {:.1}s elapsed",
                            encoded,
                            started.elapsed().as_secs_f32()
                        );
                    }
                    let _ = call_start;
                }
                Err(ClipError::Permanent(e)) => {
                    perm_fail += 1;
                    warn!("permanent encode failure for {rel_path}: {e}");
                }
                Err(ClipError::Transient(e)) => {
                    transient_fail += 1;
                    warn!("transient encode failure for {rel_path}: {e}");
                }
                Err(ClipError::Disabled) => {
                    anyhow::bail!("clip client became disabled mid-run; impossible");
                }
            }
        }
    }
    scores.sort_by(|a, b| b.0.partial_cmp(&a.0).unwrap_or(std::cmp::Ordering::Equal));
    let elapsed = started.elapsed();
    println!();
    println!(
        "── top {} for query: {:?} ──",
        args.top.min(scores.len()),
        args.query
    );
    for (i, (sim, path)) in scores.iter().take(args.top).enumerate() {
        println!("[{:>2}] sim={:.3}  {}", i + 1, sim, path);
    }
    println!();
    println!("── summary ─────────────────────────────────────");
    println!("query                : {:?}", args.query);
    println!("scanned rows         : {scanned}");
    println!("encoded photos       : {encoded}");
    println!("permanent failures   : {perm_fail}");
    println!("transient failures   : {transient_fail}");
    println!("elapsed              : {:.1}s", elapsed.as_secs_f32());
    if encoded > 0 {
        println!(
            "throughput           : {:.2} photos/s ({:.0}ms/photo avg)",
            encoded as f32 / elapsed.as_secs_f32().max(0.001),
            elapsed.as_millis() as f32 / encoded as f32
        );
    }
    Ok(())
 }
@@ -1,352 +0,0 @@
 //! `/photos/search?q=<text>` — CLIP semantic photo search.
 //!
 //! The route lives outside `files.rs` to keep that 1500+ line module
 //! focused on EXIF / tag listing. The flow is:
 //!
 //! 1. Parse query params (`q`, `limit`, `threshold`, optional `library`).
 //! 2. Call Apollo's `/api/internal/clip/encode_text` to get the query
 //!    vector (L2-normalized 768-d f32 for ViT-L/14).
 //! 3. Load every `(content_hash, clip_embedding)` for the scope from
 //!    `image_exif` via `ExifDao::list_clip_index`. ~28–43 MB for a 14k
 //!    library at ViT-L/14; loaded fresh per request — fast enough for
 //!    v1, optimize via an AppState cache later if needed.
 //! 4. Dot product (= cosine since both sides are L2-normalized), filter
 //!    above `threshold`, top-K by score.
 //! 5. Resolve each surviving hash back to a `(library_id, rel_path)` so
 //!    the frontend can render the photo / hand off to the carousel.
 //!
 //! Response shape is intentionally minimal — paths + score — so the
 //! frontend can reuse existing PhotoGrid rendering by joining against
 //! `/api/photos/match` (or calling `/image/metadata` lazily). Don't
 //! bake camera/EXIF metadata into this route; it would force a fan-out
 //! per result and balloon the response.
 use crate::AppState;
 use crate::ai::clip_client::ClipError;
 use crate::database::ExifDao;
 use actix_web::{HttpResponse, Result as ActixResult, web};
 use base64::Engine;
 use serde::{Deserialize, Serialize};
 use std::sync::Mutex;
 #[derive(Debug, Deserialize)]
 pub struct SearchQuery {
    /// Natural-language query. Required; empty triggers 400.
    pub q: String,
    /// Max results to return in this page. Capped to 200 server-side.
    /// Defaults to 20. Pair with `offset` for pagination.
    #[serde(default = "default_limit")]
    pub limit: usize,
    /// Zero-based offset into the sorted-and-filtered result set. The
    /// scoring loop still runs over the full embedding matrix on every
    /// page (cheap at personal-library scale — sub-100ms — and avoids
    /// stateful pagination cursors). Defaults to 0.
    #[serde(default)]
    pub offset: usize,
    /// Cosine-similarity floor below which results are dropped.
    /// 0.20 is the rough "this is plausibly relevant" line for OpenAI
    /// CLIP; tunable per call when sweeping. Defaults to 0.20.
    #[serde(default = "default_threshold")]
    pub threshold: f32,
    /// Optional single-library scope. Legacy param — new clients pass
    /// `library_ids` instead so multi-select scopes (Apollo's HUD library
    /// chips, FileViewer-React's library picker) actually filter. Kept
    /// for back-compat; `library_ids` wins when both are supplied.
    pub library: Option<i32>,
    /// Optional multi-library scope, comma-separated id list
    /// (`?library_ids=1,3`). Empty / omitted = every enabled library
    /// (the historical default). Apollo and FileViewer-React both send
    /// this when 2+ libraries are selected; the single-library case
    /// works through either param interchangeably.
    pub library_ids: Option<String>,
    /// Optional model-version filter. Defaults to the live engine's
    /// version (queried lazily). Forces a strict join so mid-flight
    /// model swaps can't mix geometries in a single response.
    #[serde(default)]
    pub model_version: Option<String>,
 }
 fn default_limit() -> usize {
    20
 }
 fn default_threshold() -> f32 {
    0.20
 }
 #[derive(Debug, Serialize)]
 pub struct SearchHit {
    pub library_id: i32,
    pub rel_path: String,
    pub content_hash: String,
    /// Cosine similarity in [-1, 1]. In practice OpenAI CLIP returns
    /// 0.10–0.40 for the typical photo library.
    pub score: f32,
 }
 #[derive(Debug, Serialize)]
 pub struct SearchResponse {
    pub query: String,
    pub model_version: String,
    pub threshold: f32,
    /// Total embeddings scored (= every photo in scope with a stored
    /// embedding). Same value across pages of the same query.
    pub considered: usize,
    /// Count of results above threshold, before pagination. Lets the
    /// client decide whether a "Load more" button is meaningful and
    /// stop fetching when ``offset + results.len() >= total_matching``.
    pub total_matching: usize,
    pub offset: usize,
    pub results: Vec<SearchHit>,
 }
 #[derive(Debug, Serialize)]
 struct SearchError {
    error: String,
 }
 /// Decode a stored `clip_embedding` BLOB back into a `Vec<f32>`. Returns
 /// `None` on malformed bytes — those rows get skipped rather than
 /// failing the whole query.
 fn decode_embedding(bytes: &[u8]) -> Option<Vec<f32>> {
    if bytes.is_empty() || !bytes.len().is_multiple_of(4) {
        return None;
    }
    let mut out = Vec::with_capacity(bytes.len() / 4);
    for chunk in bytes.chunks_exact(4) {
        out.push(f32::from_le_bytes([chunk[0], chunk[1], chunk[2], chunk[3]]));
    }
    Some(out)
 }
 #[inline]
 fn dot(a: &[f32], b: &[f32]) -> f32 {
    a.iter().zip(b.iter()).map(|(x, y)| x * y).sum()
 }
 pub async fn search_photos(
    state: web::Data<AppState>,
    exif_dao: web::Data<Mutex<Box<dyn ExifDao>>>,
    query: web::Query<SearchQuery>,
 ) -> ActixResult<HttpResponse> {
    let q_text = query.q.trim().to_string();
    if q_text.is_empty() {
        return Ok(HttpResponse::BadRequest().json(SearchError {
            error: "query parameter `q` is required".into(),
        }));
    }
    if !state.clip_client.is_enabled() {
        return Ok(HttpResponse::ServiceUnavailable().json(SearchError {
            error: "CLIP search is disabled (no Apollo CLIP endpoint configured)".into(),
        }));
    }
    let limit = query.limit.clamp(1, 200);
    let offset = query.offset;
    let threshold = query.threshold.clamp(-1.0, 1.0);
    // 1. Encode the query text. Fast — Apollo's text encoder is ~50ms
    // on CPU. Bail with a clear error message if Apollo's down so the
    // user sees "service unavailable" rather than empty results.
    let query_resp = match state.clip_client.encode_text(&q_text).await {
        Ok(r) => r,
        Err(ClipError::Permanent(e)) => {
            return Ok(HttpResponse::BadRequest().json(SearchError {
                error: format!("query rejected: {e}"),
            }));
        }
        Err(ClipError::Transient(e)) => {
            return Ok(HttpResponse::BadGateway().json(SearchError {
                error: format!("CLIP service unavailable: {e}"),
            }));
        }
        Err(ClipError::Disabled) => {
            return Ok(HttpResponse::ServiceUnavailable().json(SearchError {
                error: "CLIP service disabled".into(),
            }));
        }
    };
    // decode_embedding works on raw bytes; the wire format is b64.
    let query_bytes = base64::engine::general_purpose::STANDARD
        .decode(query_resp.embedding.as_bytes())
        .unwrap_or_default();
    let query_vec = match decode_embedding(&query_bytes) {
        Some(v) => v,
        None => {
            return Ok(HttpResponse::BadGateway().json(SearchError {
                error: "CLIP service returned a malformed query embedding".into(),
            }));
        }
    };
    // 2. Decide which library scope to search. `library_ids` (multi)
    // wins over the legacy `library` (single) when both are present;
    // either / both empty falls back to "every enabled library".
    let library_ids: Vec<i32> = if let Some(raw) = query.library_ids.as_deref() {
        let mut out: Vec<i32> = Vec::new();
        for piece in raw.split(',') {
            let trimmed = piece.trim();
            if trimmed.is_empty() {
                continue;
            }
            match trimmed.parse::<i32>() {
                Ok(id) => {
                    if !out.contains(&id) {
                        out.push(id);
                    }
                }
                Err(_) => {
                    return Ok(HttpResponse::BadRequest().json(SearchError {
                        error: format!("invalid library_ids entry: {trimmed:?}"),
                    }));
                }
            }
        }
        out
    } else if let Some(id) = query.library {
        vec![id]
    } else {
        Vec::new()
    };
    // 3. Pull the (hash, embedding) matrix. Lock contention here is
    // bounded — one big SELECT under a mutex Arc<Mutex<dyn ExifDao>>
    // and then we release before scoring. If this becomes a hotspot
    // we'll cache the decoded matrix in AppState with TTL.
    let ctx = opentelemetry::Context::current();
    let rows: Vec<(String, Vec<u8>)> = {
        let mut dao = exif_dao.lock().expect("exif dao");
        match dao.list_clip_index(
            &ctx,
            &library_ids,
            query
                .model_version
                .as_deref()
                .or(Some(&query_resp.model_version)),
        ) {
            Ok(r) => r,
            Err(e) => {
                log::warn!("clip_search: list_clip_index failed: {:?}", e);
                return Ok(HttpResponse::InternalServerError().json(SearchError {
                    error: "failed to load search index".into(),
                }));
            }
        }
    };
    let considered = rows.len();
    if considered == 0 {
        return Ok(HttpResponse::Ok().json(SearchResponse {
            query: q_text,
            model_version: query_resp.model_version,
            threshold,
            considered,
            total_matching: 0,
            offset,
            results: Vec::new(),
        }));
    }
    // 4. Score. Cap the loop's transient allocation; we keep all scores
    // and sort at the end. With ~14k entries the sort is microseconds.
    let mut scored: Vec<(f32, String)> = Vec::with_capacity(considered);
    for (hash, blob) in rows {
        let Some(emb) = decode_embedding(&blob) else {
            continue;
        };
        if emb.len() != query_vec.len() {
            continue;
        }
        let sim = dot(&emb, &query_vec);
        if sim < threshold {
            continue;
        }
        scored.push((sim, hash));
    }
    scored.sort_by(|a, b| b.0.partial_cmp(&a.0).unwrap_or(std::cmp::Ordering::Equal));
    let total_matching = scored.len();
    // Pagination — slice the sorted list at `[offset, offset+limit)`.
    // Offsets past the end produce empty pages rather than an error so
    // the client can stop fetching naturally on "load more" past the end.
    let scored: Vec<(f32, String)> = if offset >= total_matching {
        Vec::new()
    } else {
        let end = (offset + limit).min(total_matching);
        scored[offset..end].to_vec()
    };
    if scored.is_empty() {
        return Ok(HttpResponse::Ok().json(SearchResponse {
            query: q_text,
            model_version: query_resp.model_version,
            threshold,
            considered,
            total_matching,
            offset,
            results: Vec::new(),
        }));
    }
    // 5. Resolve each surviving hash back to a `(library_id, rel_path)`.
    // `get_rel_paths_by_hash` returns every rel_path; we pick the first
    // one for the result. Apollo / the UI can fetch alternatives via
    // /image/metadata when needed.
    let hashes: Vec<String> = scored.iter().map(|(_, h)| h.clone()).collect();
    let path_map = {
        let mut dao = exif_dao.lock().expect("exif dao");
        match dao.get_rel_paths_for_hashes(&ctx, &hashes) {
            Ok(m) => m,
            Err(e) => {
                log::warn!("clip_search: get_rel_paths_for_hashes failed: {:?}", e);
                return Ok(HttpResponse::InternalServerError().json(SearchError {
                    error: "failed to resolve photo paths".into(),
                }));
            }
        }
    };
    // We need (library_id, rel_path) — get_rel_paths_for_hashes only
    // returns rel_paths. Cross-reference via find_by_content_hash to
    // pick the library too. Single call per surviving hash; cheap at
    // top-20.
    let mut results = Vec::with_capacity(scored.len());
    {
        let mut dao = exif_dao.lock().expect("exif dao");
        for (score, hash) in scored {
            let row = match dao.find_by_content_hash(&ctx, &hash) {
                Ok(Some(r)) => r,
                Ok(None) => continue,
                Err(e) => {
                    log::warn!(
                        "clip_search: find_by_content_hash failed for {}: {:?}",
                        hash,
                        e
                    );
                    continue;
                }
            };
            // Prefer get_rel_paths_for_hashes's first entry if it
            // exists (it shares semantics with `image_exif`'s natural
            // order), falling back to the ImageExif row.
            let rel_path = path_map
                .get(&hash)
                .and_then(|paths| paths.first().cloned())
                .unwrap_or(row.file_path);
            results.push(SearchHit {
                library_id: row.library_id,
                rel_path,
                content_hash: hash,
                score,
            });
        }
    }
    Ok(HttpResponse::Ok().json(SearchResponse {
        query: q_text,
        model_version: query_resp.model_version,
        threshold,
        considered,
        total_matching,
        offset,
        results,
    }))
 }
@@ -1,246 +0,0 @@
 //! CLIP-encoding pass for the file watcher.
 //!
 //! `process_clip_backlog` in `backfill.rs` calls [`run_clip_encoding_pass`]
 //! with the page of candidates returned by
 //! `ExifDao::list_clip_unencoded_candidates`. We walk those, fan out K
 //! parallel encode calls to Apollo, and persist the resulting embeddings
 //! into `image_exif.clip_embedding` / `clip_model_version`.
 //!
 //! Unlike the face pipeline, CLIP has no marker rows — a permanent
 //! failure (un-decodable bytes) leaves the row's `clip_embedding` NULL
 //! and the drain will retry on the next tick. For personal-library
 //! scale this is fine; the per-tick cap bounds the wasted work, and
 //! `file_types::is_image_file` filters out videos / non-media client-
 //! side so most permanent failures are decoded-but-corrupt files (rare).
 //!
 //! The watcher thread isn't in any pre-existing async context, so we
 //! build a short-lived tokio runtime per pass and `block_on` the join
 //! of K encode futures. Concurrency knob: `CLIP_ENCODE_CONCURRENCY`
 //! (default 4 — lower than faces because Apollo's CLIP path doesn't
 //! release the GIL between preprocess and forward as cleanly).
 use crate::ai::clip_client::{ClipClient, ClipError, EncodeImageMeta};
 use crate::database::ExifDao;
 use crate::exif;
 use crate::file_types;
 use crate::libraries::Library;
 use crate::memories::PathExcluder;
 use log::{debug, info, warn};
 use std::path::Path;
 use std::sync::{Arc, Mutex};
 use tokio::sync::Semaphore;
 /// One file the watcher would like to CLIP-encode. Built from the DAO
 /// `list_clip_unencoded_candidates` result — needs the `content_hash`
 /// for traceability in Apollo's log lines, even though the embedding
 /// itself is keyed on `(library_id, rel_path)` for the back-write.
 #[derive(Debug, Clone)]
 pub struct ClipCandidate {
    pub rel_path: String,
    pub content_hash: String,
 }
 /// Synchronous entry point. Returns once every candidate has been
 /// processed (or definitively skipped). No-op when the client is
 /// disabled so the caller can call unconditionally.
 pub fn run_clip_encoding_pass(
    library: &Library,
    excluded_dirs: &[String],
    clip_client: &ClipClient,
    exif_dao: Arc<Mutex<Box<dyn ExifDao>>>,
    candidates: Vec<ClipCandidate>,
 ) {
    if !clip_client.is_enabled() {
        return;
    }
    if candidates.is_empty() {
        return;
    }
    let base = Path::new(&library.root_path);
    let filtered = filter_excluded(base, excluded_dirs, candidates, Some(&library.name));
    if filtered.is_empty() {
        return;
    }
    let concurrency: usize = std::env::var("CLIP_ENCODE_CONCURRENCY")
        .ok()
        .and_then(|s| s.parse().ok())
        .filter(|n: &usize| *n > 0)
        .unwrap_or(4);
    info!(
        "clip_watch: encoding {} candidate(s) for library '{}' (concurrency {})",
        filtered.len(),
        library.name,
        concurrency
    );
    let rt = match tokio::runtime::Builder::new_multi_thread()
        .worker_threads(2)
        .enable_all()
        .build()
    {
        Ok(rt) => rt,
        Err(e) => {
            warn!("clip_watch: failed to build tokio runtime: {e}");
            return;
        }
    };
    let library_id = library.id;
    let library_root = library.root_path.clone();
    rt.block_on(async move {
        let sem = Arc::new(Semaphore::new(concurrency));
        let mut handles = Vec::with_capacity(filtered.len());
        for cand in filtered {
            let permit_sem = sem.clone();
            let clip_client = clip_client.clone();
            let exif_dao = exif_dao.clone();
            let library_root = library_root.clone();
            handles.push(tokio::spawn(async move {
                let _permit = permit_sem.acquire().await.expect("clip semaphore");
                process_one(library_id, &library_root, cand, &clip_client, exif_dao).await;
            }));
        }
        for h in handles {
            let _ = h.await;
        }
    });
 }
 async fn process_one(
    library_id: i32,
    library_root: &str,
    cand: ClipCandidate,
    clip_client: &ClipClient,
    exif_dao: Arc<Mutex<Box<dyn ExifDao>>>,
 ) {
    let abs = Path::new(library_root).join(&cand.rel_path);
    let bytes = match read_image_bytes_for_encode(&abs) {
        Ok(b) => b,
        Err(e) => {
            // Same rationale as face_watch: don't mark — the file may
            // have been moved/renamed mid-scan; let the next pass retry.
            warn!(
                "clip_watch: read failed for {} (lib {}): {}",
                cand.rel_path, library_id, e
            );
            return;
        }
    };
    let meta = EncodeImageMeta {
        content_hash: cand.content_hash.clone(),
        library_id,
        rel_path: cand.rel_path.clone(),
    };
    let ctx = opentelemetry::Context::current();
    match clip_client.encode_image(bytes, meta).await {
        Ok(resp) => {
            let emb_bytes = match resp.decode_embedding() {
                Ok(b) => b,
                Err(e) => {
                    warn!("clip_watch: bad embedding for {}: {:?}", cand.rel_path, e);
                    return;
                }
            };
            let mut dao = exif_dao.lock().expect("exif dao");
            if let Err(e) = dao.backfill_clip_embedding(
                &ctx,
                library_id,
                &cand.rel_path,
                &emb_bytes,
                &resp.model_version,
            ) {
                warn!(
                    "clip_watch: backfill_clip_embedding failed for {}: {:?}",
                    cand.rel_path, e
                );
                return;
            }
            debug!(
                "clip_watch: {} → dim={} ({}ms, {})",
                cand.rel_path, resp.embedding_dim, resp.duration_ms, resp.model_version
            );
        }
        Err(ClipError::Permanent(e)) => {
            // No marker — the row sits with NULL embedding and the drain
            // retries next pass. For personal-library scale the cost of
            // re-attempting permanently-broken files is bounded by the
            // per-tick cap. If this becomes a recurring noise source,
            // add a `clip_status` column with `failed` semantics like
            // face_detections has.
            warn!(
                "clip_watch: permanent failure on {} (will retry next pass): {}",
                cand.rel_path, e
            );
        }
        Err(ClipError::Transient(e)) => {
            debug!(
                "clip_watch: transient on {}: {} (will retry next pass)",
                cand.rel_path, e
            );
        }
        Err(ClipError::Disabled) => {
            // Defensive — the entry-point already checked is_enabled().
        }
    }
 }
 /// Drop candidates whose paths land in an excluded dir or whose
 /// extension isn't an image. Mirrors `face_watch::filter_excluded` so
 /// the two backlogs stay shape-consistent. Library name is passed
 /// purely for the log line that surfaces an exclusion hit.
 pub fn filter_excluded(
    base: &Path,
    excluded_dirs: &[String],
    candidates: Vec<ClipCandidate>,
    library_name: Option<&str>,
 ) -> Vec<ClipCandidate> {
    let excluder = if excluded_dirs.is_empty() {
        None
    } else {
        Some(PathExcluder::new(base, excluded_dirs))
    };
    candidates
        .into_iter()
        .filter(|c| {
            let abs = base.join(&c.rel_path);
            if !file_types::is_image_file(&abs) {
                debug!(
                    "clip_watch: skipping non-image '{}' (lib {})",
                    c.rel_path,
                    library_name.unwrap_or("<unknown>")
                );
                return false;
            }
            if let Some(ex) = excluder.as_ref()
                && ex.is_excluded(&abs)
            {
                debug!(
                    "clip_watch: skipping excluded '{}' (lib {})",
                    c.rel_path,
                    library_name.unwrap_or("<unknown>")
                );
                return false;
            }
            true
        })
        .collect()
 }
 /// Read image bytes for CLIP encoding. Same logic as
 /// `face_watch::read_image_bytes_for_detect` — RAW / HEIC files don't
 /// decode in Apollo's PIL pipeline, so we pull the embedded JPEG
 /// preview the thumbnail pipeline already extracts. Plain JPEG / PNG /
 /// WebP go through a direct read.
 pub fn read_image_bytes_for_encode(path: &Path) -> std::io::Result<Vec<u8>> {
    if file_types::needs_ffmpeg_thumbnail(path)
        && let Some(preview) = exif::extract_embedded_jpeg_preview(path)
    {
        return Ok(preview);
    }
    std::fs::read(path)
 }
@@ -50,32 +50,14 @@ pub fn thumbnail_path(thumbs_dir: &Path, hash: &str) -> PathBuf {
    thumbs_dir.join(shard).join(format!("{}.jpg", hash))
 }
 /// Hash-keyed large-preview path: `<thumbs_dir>/_large/<hash[..2]>/<hash>.jpg`.
 /// Kept under the same root as 200px thumbs so deployments don't need a
 /// second env var, but namespaced under `_large/` so the existing 200px
 /// shards don't collide with the larger derivative.
 pub fn large_preview_path(thumbs_dir: &Path, hash: &str) -> PathBuf {
    let shard = shard_prefix(hash);
    thumbs_dir
        .join("_large")
        .join(shard)
        .join(format!("{}.jpg", hash))
 }
 /// Hash-keyed xlarge-preview path: `<thumbs_dir>/_xlarge/<hash[..2]>/<hash>.jpg`.
 pub fn xlarge_preview_path(thumbs_dir: &Path, hash: &str) -> PathBuf {
    let shard = shard_prefix(hash);
    thumbs_dir
        .join("_xlarge")
        .join(shard)
        .join(format!("{}.jpg", hash))
 }
 /// Hash-keyed HLS output directory: `<video_dir>/<hash[..2]>/<hash>/`.
 /// The playlist lives at `playlist.m3u8` inside this directory and its
-/// segments are co-located so HLS relative references Just Work. See
+/// segments are co-located so HLS relative references Just Work.
-/// [`crate::video::hls_paths`] for the filename constants and the
+///
-/// per-file helpers built on this dir.
+/// Allow-dead until Branch B/C rewires the HLS pipeline to use it; the
 /// helper lives here today so Branch A's path layout decisions stay
 /// adjacent to thumbnail/legacy ones.
 #[allow(dead_code)]
 pub fn hls_dir(video_dir: &Path, hash: &str) -> PathBuf {
    let shard = shard_prefix(hash);
    video_dir.join(shard).join(hash)
@@ -141,9 +123,6 @@ mod tests {
        let p = thumbnail_path(thumbs, "abcdef0123");
        assert_eq!(p, PathBuf::from("/tmp/thumbs/ab/abcdef0123.jpg"));
        let l = large_preview_path(thumbs, "abcdef0123");
        assert_eq!(l, PathBuf::from("/tmp/thumbs/_large/ab/abcdef0123.jpg"));
        let video = Path::new("/tmp/video");
        let d = hls_dir(video, "1234deadbeef");
        assert_eq!(d, PathBuf::from("/tmp/video/12/1234deadbeef"));
@@ -194,8 +194,6 @@ pub enum MediaType {
 #[serde(rename_all = "lowercase")]
 pub enum PhotoSize {
    Full,
    XLarge,
    Large,
    Thumb,
 }
@@ -274,7 +274,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
                source_file: event.source_file,
            })
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn store_events_batch(
@@ -348,7 +348,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
            Ok(inserted)
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn find_events_in_range(
@@ -373,7 +373,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
            .map(|rows| rows.into_iter().map(|r| r.to_calendar_event()).collect())
            .map_err(|e| anyhow::anyhow!("Query error: {:?}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_similar_events(
@@ -429,7 +429,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
            Ok(scored_events.into_iter().take(limit).map(|(_, event)| event).collect())
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_relevant_events_hybrid(
@@ -500,7 +500,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
                Ok(events_in_range.into_iter().take(limit).map(|r| r.to_calendar_event()).collect())
            }
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn event_exists(
@@ -528,7 +528,7 @@ impl CalendarEventDao for SqliteCalendarEventDao {
            Ok(result.count > 0)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_event_count(&mut self, context: &opentelemetry::Context) -> Result<i64, DbError> {
@@ -551,6 +551,6 @@ impl CalendarEventDao for SqliteCalendarEventDao {
            Ok(result.count)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
 }
@@ -75,11 +75,6 @@ pub trait DailySummaryDao: Sync + Send {
        context: &opentelemetry::Context,
        contact: &str,
    ) -> Result<i64, DbError>;
    /// Cheap presence check — returns true iff at least one daily summary row
    /// exists. Used by gating logic that only needs "is the table empty?",
    /// avoiding a `COUNT(*)` full scan on large corpora.
    fn has_any_summaries(&mut self, context: &opentelemetry::Context) -> Result<bool, DbError>;
 }
 pub struct SqliteDailySummaryDao {
@@ -190,7 +185,7 @@ impl DailySummaryDao for SqliteDailySummaryDao {
                model_version: summary.model_version,
            })
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn find_similar_summaries(
@@ -286,7 +281,7 @@ impl DailySummaryDao for SqliteDailySummaryDao {
            Ok(top_results)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_similar_summaries_with_time_weight(
@@ -408,7 +403,7 @@ impl DailySummaryDao for SqliteDailySummaryDao {
            Ok(top_results)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn summary_exists(
@@ -435,7 +430,7 @@ impl DailySummaryDao for SqliteDailySummaryDao {
            Ok(count > 0)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_summary_count(
@@ -457,31 +452,7 @@ impl DailySummaryDao for SqliteDailySummaryDao {
            .map(|r| r.count)
            .map_err(|e| anyhow::anyhow!("Count query error: {:?}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn has_any_summaries(&mut self, context: &opentelemetry::Context) -> Result<bool, DbError> {
        trace_db_call(context, "query", "has_any_summaries", |_span| {
            let mut conn = self
                .connection
                .lock()
                .expect("Unable to get DailySummaryDao");
            #[derive(QueryableByName)]
            struct ProbeResult {
                #[diesel(sql_type = diesel::sql_types::Integer)]
                #[allow(dead_code)]
                one: i32,
            }
            let rows: Vec<ProbeResult> =
                diesel::sql_query("SELECT 1 as one FROM daily_conversation_summaries LIMIT 1")
                    .load(conn.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Failed to probe daily summaries: {}", e))?;
            Ok(!rows.is_empty())
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
 }
@@ -1,681 +0,0 @@
 use diesel::prelude::*;
 use diesel::sqlite::SqliteConnection;
 use std::ops::DerefMut;
 use std::sync::{Arc, Mutex};
 use crate::database::models::{
    InsertInsightGenerationJob, InsightGenerationJob, InsightGenerationType, InsightJobStatus,
 };
 use crate::database::schema;
 use crate::database::{DbError, DbErrorKind, connect};
 use crate::otel::trace_db_call;
 /// Tracks async insight generation jobs. Each call to `create_job` inserts
 /// a new row; the application layer prevents concurrent running jobs by
 /// cancelling the old one before creating a new one.
 pub trait InsightGenerationJobDao: Sync + Send {
    /// Insert a new running job. Always creates a new row (no upsert).
    /// Cleans up terminal-state rows for the same key first.
    fn create_job(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
        generation_type: InsightGenerationType,
    ) -> Result<i32, DbError>;
    /// Mark a job as completed with the resulting insight id. Only updates
    /// if the job is still in "running" status (prevents overwriting a
    /// cancelled job with a late-completing task).
    fn complete_job(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
        insight_id: i32,
    ) -> Result<(), DbError>;
    /// Mark a job as failed with an error message. Only updates if the job
    /// is still in "running" status.
    fn fail_job(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
        error_message: &str,
    ) -> Result<(), DbError>;
    /// Cancel a specific job by id. Only updates if the job is still
    /// in "running" status. Returns true if a row was updated.
    fn cancel_job(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
    ) -> Result<bool, DbError>;
    /// Cancel all running jobs for a given file. Returns the number of
    /// jobs cancelled.
    fn cancel_active_jobs(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
    ) -> Result<usize, DbError>;
    /// Find the latest running job for a given file. Returns None if no
    /// running job exists.
    fn get_active_job(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
    ) -> Result<Option<InsightGenerationJob>, DbError>;
    /// Find any job by id regardless of status.
    fn get_job_by_id(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
    ) -> Result<Option<InsightGenerationJob>, DbError>;
    /// Mark all jobs still in "running" status as "failed" with a recovery
    /// error message. Returns the number of jobs recovered.
    fn recover_orphaned_jobs(&mut self, context: &opentelemetry::Context)
    -> Result<usize, DbError>;
 }
 pub struct SqliteInsightGenerationJobDao {
    connection: Arc<Mutex<SqliteConnection>>,
 }
 impl Default for SqliteInsightGenerationJobDao {
    fn default() -> Self {
        Self::new()
    }
 }
 impl SqliteInsightGenerationJobDao {
    pub fn new() -> Self {
        Self {
            connection: Arc::new(Mutex::new(connect())),
        }
    }
    #[cfg(test)]
    pub fn from_connection(conn: Arc<Mutex<SqliteConnection>>) -> Self {
        Self { connection: conn }
    }
 }
 impl InsightGenerationJobDao for SqliteInsightGenerationJobDao {
    fn create_job(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
        generation_type: InsightGenerationType,
    ) -> Result<i32, DbError> {
        trace_db_call(context, "insert", "create_job", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            let now = std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .expect("Time went backwards")
                .as_secs() as i64;
            let new_job = InsertInsightGenerationJob {
                library_id,
                path: file_path.to_string(),
                gen_type: generation_type.to_string(),
                status: InsightJobStatus::Running.to_string(),
                started_at: now,
            };
            diesel::insert_into(dsl::insight_generation_jobs)
                .values(&new_job)
                .execute(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Failed to insert job: {}", e))?;
            dsl::insight_generation_jobs
                .filter(
                    dsl::library_id
                        .eq(library_id)
                        .and(dsl::file_path.eq(file_path))
                        .and(dsl::generation_type.eq(generation_type.as_str()))
                        .and(dsl::status.eq(InsightJobStatus::Running.as_str())),
                )
                .select(dsl::id)
                .order(dsl::id.desc())
                .first::<i32>(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Failed to get job id: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn complete_job(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
        insight_id: i32,
    ) -> Result<(), DbError> {
        trace_db_call(context, "update", "complete_job", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            let now = std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .expect("Time went backwards")
                .as_secs() as i64;
            // Only update if still running — prevents cancelled job from
            // being overwritten by a late-completing task.
            diesel::update(
                dsl::insight_generation_jobs.filter(
                    dsl::id
                        .eq(job_id)
                        .and(dsl::status.eq(InsightJobStatus::Running.as_str())),
                ),
            )
            .set((
                dsl::status.eq(InsightJobStatus::Completed.as_str()),
                dsl::completed_at.eq(Some(now)),
                dsl::result_insight_id.eq(Some(insight_id)),
            ))
            .execute(connection.deref_mut())
            .map(|_| ())
            .map_err(|e| anyhow::anyhow!("Failed to complete job: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn fail_job(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
        error_message: &str,
    ) -> Result<(), DbError> {
        trace_db_call(context, "update", "fail_job", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            let now = std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .expect("Time went backwards")
                .as_secs() as i64;
            // Only update if still running.
            diesel::update(
                dsl::insight_generation_jobs.filter(
                    dsl::id
                        .eq(job_id)
                        .and(dsl::status.eq(InsightJobStatus::Running.as_str())),
                ),
            )
            .set((
                dsl::status.eq(InsightJobStatus::Failed.as_str()),
                dsl::completed_at.eq(Some(now)),
                dsl::error_message.eq(Some(error_message.to_string())),
            ))
            .execute(connection.deref_mut())
            .map(|_| ())
            .map_err(|e| anyhow::anyhow!("Failed to fail job: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn cancel_job(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
    ) -> Result<bool, DbError> {
        trace_db_call(context, "update", "cancel_job", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            let now = std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .expect("Time went backwards")
                .as_secs() as i64;
            let rows = diesel::update(
                dsl::insight_generation_jobs.filter(
                    dsl::id
                        .eq(job_id)
                        .and(dsl::status.eq(InsightJobStatus::Running.as_str())),
                ),
            )
            .set((
                dsl::status.eq(InsightJobStatus::Cancelled.as_str()),
                dsl::completed_at.eq(Some(now)),
                dsl::error_message.eq(Some("cancelled by user".to_string())),
            ))
            .execute(connection.deref_mut())
            .map_err(|e| anyhow::anyhow!("Failed to cancel job: {}", e))?;
            Ok(rows > 0)
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn cancel_active_jobs(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
    ) -> Result<usize, DbError> {
        trace_db_call(context, "update", "cancel_active_jobs", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            let now = std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .expect("Time went backwards")
                .as_secs() as i64;
            let rows = diesel::update(
                dsl::insight_generation_jobs.filter(
                    dsl::library_id
                        .eq(library_id)
                        .and(dsl::file_path.eq(file_path))
                        .and(dsl::status.eq(InsightJobStatus::Running.as_str())),
                ),
            )
            .set((
                dsl::status.eq(InsightJobStatus::Cancelled.as_str()),
                dsl::completed_at.eq(Some(now)),
                dsl::error_message.eq(Some("cancelled by newer request".to_string())),
            ))
            .execute(connection.deref_mut())
            .map_err(|e| anyhow::anyhow!("Failed to cancel active jobs: {}", e))?;
            Ok(rows)
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn get_active_job(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
    ) -> Result<Option<InsightGenerationJob>, DbError> {
        trace_db_call(context, "query", "get_active_job", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            dsl::insight_generation_jobs
                .filter(
                    dsl::library_id
                        .eq(library_id)
                        .and(dsl::file_path.eq(file_path))
                        .and(dsl::status.eq(InsightJobStatus::Running.as_str())),
                )
                .order(dsl::id.desc())
                .first::<InsightGenerationJob>(connection.deref_mut())
                .optional()
                .map_err(|e| anyhow::anyhow!("Failed to get active job: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn get_job_by_id(
        &mut self,
        context: &opentelemetry::Context,
        job_id: i32,
    ) -> Result<Option<InsightGenerationJob>, DbError> {
        trace_db_call(context, "query", "get_job_by_id", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            dsl::insight_generation_jobs
                .filter(dsl::id.eq(job_id))
                .first::<InsightGenerationJob>(connection.deref_mut())
                .optional()
                .map_err(|e| anyhow::anyhow!("Failed to get job: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn recover_orphaned_jobs(
        &mut self,
        context: &opentelemetry::Context,
    ) -> Result<usize, DbError> {
        trace_db_call(context, "update", "recover_orphaned_jobs", |_span| {
            use schema::insight_generation_jobs::dsl;
            let mut connection = self
                .connection
                .lock()
                .expect("Unable to lock InsightGenerationJobDao");
            let now = std::time::SystemTime::now()
                .duration_since(std::time::UNIX_EPOCH)
                .expect("Time went backwards")
                .as_secs() as i64;
            let rows = diesel::update(
                dsl::insight_generation_jobs
                    .filter(dsl::status.eq(InsightJobStatus::Running.as_str())),
            )
            .set((
                dsl::status.eq(InsightJobStatus::Failed.as_str()),
                dsl::completed_at.eq(Some(now)),
                dsl::error_message.eq(Some("server crashed while running".to_string())),
            ))
            .execute(connection.deref_mut())
            .map_err(|e| anyhow::anyhow!("Failed to recover orphaned jobs: {}", e))?;
            Ok(rows)
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use diesel::Connection;
    use diesel_migrations::{EmbeddedMigrations, MigrationHarness, embed_migrations};
    const DB_MIGRATIONS: EmbeddedMigrations = embed_migrations!();
    fn setup_dao() -> SqliteInsightGenerationJobDao {
        let mut conn = SqliteConnection::establish(":memory:")
            .expect("Unable to create in-memory db connection");
        conn.run_pending_migrations(DB_MIGRATIONS)
            .expect("Failure running DB migrations");
        SqliteInsightGenerationJobDao::from_connection(Arc::new(Mutex::new(conn)))
    }
    fn ctx() -> opentelemetry::Context {
        opentelemetry::Context::new()
    }
    #[test]
    fn create_job_inserts_new_row() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id_1 = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        let job_id_2 = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        assert_ne!(job_id_1, job_id_2, "each create_job call inserts a new row");
    }
    #[test]
    fn complete_job_sets_result() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        dao.complete_job(&ctx, job_id, 42).unwrap();
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(job.status, InsightJobStatus::Completed.as_str());
        assert_eq!(job.result_insight_id, Some(42));
        assert!(job.completed_at.is_some());
    }
    #[test]
    fn fail_job_sets_error() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Agentic)
            .unwrap();
        dao.fail_job(&ctx, job_id, "model timeout").unwrap();
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(job.status, InsightJobStatus::Failed.as_str());
        assert_eq!(job.error_message.as_deref(), Some("model timeout"));
        assert!(job.completed_at.is_some());
    }
    #[test]
    fn get_active_job_returns_none_when_completed() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        // Job is running
        let active = dao.get_active_job(&ctx, 1, "photos/test.jpg").unwrap();
        assert!(active.is_some());
        assert_eq!(active.unwrap().id, job_id);
        // Complete it
        dao.complete_job(&ctx, job_id, 1).unwrap();
        // No longer active
        let active = dao.get_active_job(&ctx, 1, "photos/test.jpg").unwrap();
        assert!(active.is_none());
    }
    #[test]
    fn cancel_active_jobs() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        let cancelled = dao.cancel_active_jobs(&ctx, 1, "photos/test.jpg").unwrap();
        assert_eq!(cancelled, 1, "should cancel 1 running job");
        // Job is no longer active
        let active = dao.get_active_job(&ctx, 1, "photos/test.jpg").unwrap();
        assert!(active.is_none());
        // Job exists with cancelled status
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(job.status, InsightJobStatus::Cancelled.as_str());
        // Cancelling again returns 0 (nothing to cancel)
        let cancelled2 = dao.cancel_active_jobs(&ctx, 1, "photos/test.jpg").unwrap();
        assert_eq!(cancelled2, 0, "should return 0 when no running job");
    }
    #[test]
    fn get_active_job_scoped_by_library() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id_1 = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        let job_id_2 = dao
            .create_job(&ctx, 2, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        assert_ne!(
            job_id_1, job_id_2,
            "different libraries should have separate jobs"
        );
        // Complete lib1's job
        dao.complete_job(&ctx, job_id_1, 1).unwrap();
        // lib1 has no active job
        let active1 = dao.get_active_job(&ctx, 1, "photos/test.jpg").unwrap();
        assert!(active1.is_none());
        // lib2 still has active job
        let active2 = dao.get_active_job(&ctx, 2, "photos/test.jpg").unwrap();
        assert!(active2.is_some());
        assert_eq!(active2.unwrap().id, job_id_2);
    }
    #[test]
    fn get_job_by_id_finds_any_status() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        // Find while running
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(job.status, InsightJobStatus::Running.as_str());
        // Complete it
        dao.complete_job(&ctx, job_id, 99).unwrap();
        // Still findable
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(job.status, InsightJobStatus::Completed.as_str());
        assert_eq!(job.result_insight_id, Some(99));
    }
    #[test]
    fn recover_orphaned_jobs() {
        let mut dao = setup_dao();
        let ctx = ctx();
        // Create two running jobs
        let job_id_1 = dao
            .create_job(&ctx, 1, "photos/a.jpg", InsightGenerationType::Standard)
            .unwrap();
        let job_id_2 = dao
            .create_job(&ctx, 1, "photos/b.jpg", InsightGenerationType::Agentic)
            .unwrap();
        // Complete one
        dao.complete_job(&ctx, job_id_1, 1).unwrap();
        // Recover should only affect the running job
        let recovered = dao.recover_orphaned_jobs(&ctx).unwrap();
        assert_eq!(recovered, 1, "should recover exactly 1 running job");
        // job_id_1 is still completed
        let job1 = dao.get_job_by_id(&ctx, job_id_1).unwrap().unwrap();
        assert_eq!(job1.status, InsightJobStatus::Completed.as_str());
        // job_id_2 is now failed with recovery message
        let job2 = dao.get_job_by_id(&ctx, job_id_2).unwrap().unwrap();
        assert_eq!(job2.status, InsightJobStatus::Failed.as_str());
        assert_eq!(
            job2.error_message.as_deref(),
            Some("server crashed while running")
        );
        // Second recovery is a no-op
        let recovered2 = dao.recover_orphaned_jobs(&ctx).unwrap();
        assert_eq!(recovered2, 0, "no running jobs remain");
    }
    #[test]
    fn complete_job_noop_when_cancelled() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        dao.cancel_job(&ctx, job_id).unwrap();
        // Late-completing task tries to mark as completed — should be a no-op
        dao.complete_job(&ctx, job_id, 42).unwrap();
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(
            job.status,
            InsightJobStatus::Cancelled.as_str(),
            "cancelled status must not be overwritten by late complete"
        );
        assert_eq!(
            job.result_insight_id, None,
            "insight_id must stay None when complete is a no-op"
        );
    }
    #[test]
    fn fail_job_noop_when_cancelled() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Agentic)
            .unwrap();
        dao.cancel_job(&ctx, job_id).unwrap();
        // Late-failing task tries to mark as failed — should be a no-op
        dao.fail_job(&ctx, job_id, "timeout after 120s").unwrap();
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(
            job.status,
            InsightJobStatus::Cancelled.as_str(),
            "cancelled status must not be overwritten by late fail"
        );
        assert_eq!(
            job.error_message.as_deref(),
            Some("cancelled by user"),
            "error_message must reflect the cancel, not the late fail"
        );
    }
    #[test]
    fn cancel_job_by_id() {
        let mut dao = setup_dao();
        let ctx = ctx();
        let job_id = dao
            .create_job(&ctx, 1, "photos/test.jpg", InsightGenerationType::Standard)
            .unwrap();
        let cancelled = dao.cancel_job(&ctx, job_id).unwrap();
        assert!(cancelled, "should cancel running job");
        let job = dao.get_job_by_id(&ctx, job_id).unwrap().unwrap();
        assert_eq!(job.status, InsightJobStatus::Cancelled.as_str());
        assert!(job.completed_at.is_some());
        // Cancelling again is a no-op
        let cancelled2 = dao.cancel_job(&ctx, job_id).unwrap();
        assert!(!cancelled2, "already cancelled job should return false");
    }
 }
@@ -21,22 +21,6 @@ pub trait InsightDao: Sync + Send {
        file_path: &str,
    ) -> Result<Option<PhotoInsight>, DbError>;
    /// Library-scoped variant of `get_insight`. The default `get_insight`
    /// finds any `is_current=true` row matching `file_path` across
    /// libraries — fine for the photo-grid metadata fetch (cross-library
    /// merge), wrong for the chat path: a regenerate on lib1 flips lib1's
    /// row to `is_current=false` and inserts a new lib1 row, but
    /// lib2's untouched `is_current=true` row for the same rel_path
    /// would still satisfy the path-only query and shadow the regen on
    /// the next history fetch. Always pass a library_id when you have
    /// one (chat / insight write paths always do).
    fn get_current_insight_for_library(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
    ) -> Result<Option<PhotoInsight>, DbError>;
    /// Return the most recent current insight whose rel_path is one of
    /// `paths`. Used for content-hash sharing: the caller expands a
    /// single file into all rel_paths with the same content_hash, then
@@ -47,6 +31,7 @@ pub trait InsightDao: Sync + Send {
        paths: &[String],
    ) -> Result<Option<PhotoInsight>, DbError>;
    #[allow(dead_code)]
    fn get_insight_history(
        &mut self,
        context: &opentelemetry::Context,
@@ -81,17 +66,6 @@ pub trait InsightDao: Sync + Send {
        approved: bool,
    ) -> Result<(), DbError>;
    /// Rate a specific insight version by primary key, regardless of
    /// `is_current`. Used by the per-file history view to approve/reject
    /// previously generated (superseded) versions, which the path-based
    /// `rate_insight` (current row only) cannot reach.
    fn rate_insight_by_id(
        &mut self,
        context: &opentelemetry::Context,
        insight_id: i32,
        approved: bool,
    ) -> Result<(), DbError>;
    fn get_approved_insights(
        &mut self,
        context: &opentelemetry::Context,
@@ -100,15 +74,13 @@ pub trait InsightDao: Sync + Send {
    /// Replace the `training_messages` JSON blob on the current row for
    /// `(library_id, rel_path)`. Used by chat-turn append mode to persist
    /// the extended conversation without inserting a new insight version.
    /// Returns the number of rows affected (0 if no current row matched,
    /// indicating a concurrent regenerate/reconcile flipped `is_current`).
    fn update_training_messages(
        &mut self,
        context: &opentelemetry::Context,
        library_id: i32,
        file_path: &str,
        training_messages_json: &str,
-    ) -> Result<usize, DbError>;
+    ) -> Result<(), DbError>;
 }
 pub struct SqliteInsightDao {
@@ -171,13 +143,13 @@ impl InsightDao for SqliteInsightDao {
            )
            .set(is_current.eq(false))
            .execute(connection.deref_mut())
-            .map_err(|e| anyhow::anyhow!("Failed to flip is_current: {}", e))?;
+            .map_err(|_| anyhow::anyhow!("Update is_current error"))?;
            // Insert the new insight as current
            diesel::insert_into(photo_insights)
                .values(&insight)
                .execute(connection.deref_mut())
-                .map_err(|e| anyhow::anyhow!("Failed to insert insight: {}", e))?;
+                .map_err(|_| anyhow::anyhow!("Insert error"))?;
            // Retrieve the inserted record (is_current = true)
            photo_insights
@@ -185,12 +157,9 @@ impl InsightDao for SqliteInsightDao {
                .filter(rel_path.eq(&insight.file_path))
                .filter(is_current.eq(true))
                .first::<PhotoInsight>(connection.deref_mut())
-                .map_err(|e| anyhow::anyhow!("Failed to retrieve inserted insight: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
        .map_err(|e| {
            log::error!("store_insight failed: {}", e);
            DbError::new(DbErrorKind::InsertError)
        })
        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn get_insight(
@@ -208,36 +177,9 @@ impl InsightDao for SqliteInsightDao {
                .filter(is_current.eq(true))
                .first::<PhotoInsight>(connection.deref_mut())
                .optional()
-                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_current_insight_for_library(
        &mut self,
        context: &opentelemetry::Context,
        lib_id: i32,
        path: &str,
    ) -> Result<Option<PhotoInsight>, DbError> {
        trace_db_call(
            context,
            "query",
            "get_current_insight_for_library",
            |_span| {
                use schema::photo_insights::dsl::*;
                let mut connection = self.connection.lock().expect("Unable to get InsightDao");
                photo_insights
                    .filter(library_id.eq(lib_id))
                    .filter(rel_path.eq(path))
                    .filter(is_current.eq(true))
                    .first::<PhotoInsight>(connection.deref_mut())
                    .optional()
                    .map_err(|e| anyhow::anyhow!("Query error: {}", e))
            },
        )
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn get_insight_for_paths(
@@ -259,9 +201,9 @@ impl InsightDao for SqliteInsightDao {
                .order(generated_at.desc())
                .first::<PhotoInsight>(connection.deref_mut())
                .optional()
-                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_insight_history(
@@ -278,9 +220,9 @@ impl InsightDao for SqliteInsightDao {
                .filter(rel_path.eq(path))
                .order(generated_at.desc())
                .load::<PhotoInsight>(connection.deref_mut())
-                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_insight_by_id(
@@ -297,9 +239,9 @@ impl InsightDao for SqliteInsightDao {
                .find(insight_id)
                .first::<PhotoInsight>(connection.deref_mut())
                .optional()
-                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn delete_insight(
@@ -315,9 +257,9 @@ impl InsightDao for SqliteInsightDao {
            diesel::delete(photo_insights.filter(rel_path.eq(path)))
                .execute(connection.deref_mut())
                .map(|_| ())
-                .map_err(|e| anyhow::anyhow!("Delete error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Delete error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_all_insights(
@@ -333,9 +275,9 @@ impl InsightDao for SqliteInsightDao {
                .filter(is_current.eq(true))
                .order(generated_at.desc())
                .load::<PhotoInsight>(connection.deref_mut())
-                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn rate_insight(
@@ -357,29 +299,9 @@ impl InsightDao for SqliteInsightDao {
            .set(approved.eq(Some(is_approved)))
            .execute(connection.deref_mut())
            .map(|_| ())
-            .map_err(|e| anyhow::anyhow!("Update error: {}", e))
+            .map_err(|_| anyhow::anyhow!("Update error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
+        .map_err(|_| DbError::new(DbErrorKind::UpdateError))
    }
    fn rate_insight_by_id(
        &mut self,
        context: &opentelemetry::Context,
        target_id: i32,
        is_approved: bool,
    ) -> Result<(), DbError> {
        trace_db_call(context, "update", "rate_insight_by_id", |_span| {
            use schema::photo_insights::dsl::*;
            let mut connection = self.connection.lock().expect("Unable to get InsightDao");
            diesel::update(photo_insights.find(target_id))
                .set(approved.eq(Some(is_approved)))
                .execute(connection.deref_mut())
                .map(|_| ())
                .map_err(|e| anyhow::anyhow!("Update error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn get_approved_insights(
@@ -396,9 +318,9 @@ impl InsightDao for SqliteInsightDao {
                .filter(training_messages.is_not_null())
                .order(generated_at.desc())
                .load::<PhotoInsight>(connection.deref_mut())
-                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
+                .map_err(|_| anyhow::anyhow!("Query error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn update_training_messages(
@@ -407,7 +329,7 @@ impl InsightDao for SqliteInsightDao {
        lib_id: i32,
        path: &str,
        training_messages_json: &str,
-    ) -> Result<usize, DbError> {
+    ) -> Result<(), DbError> {
        trace_db_call(context, "update", "update_training_messages", |_span| {
            use schema::photo_insights::dsl::*;
@@ -421,95 +343,9 @@ impl InsightDao for SqliteInsightDao {
            )
            .set(training_messages.eq(Some(training_messages_json.to_string())))
            .execute(connection.deref_mut())
-            .map_err(|e| anyhow::anyhow!("Update error: {}", e))
+            .map(|_| ())
            .map_err(|_| anyhow::anyhow!("Update error"))
        })
-        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
+        .map_err(|_| DbError::new(DbErrorKind::UpdateError))
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use crate::database::test::in_memory_db_connection;
    fn dao() -> SqliteInsightDao {
        let conn = Arc::new(Mutex::new(in_memory_db_connection()));
        SqliteInsightDao::from_connection(conn)
    }
    /// Build an insight insert with sensible defaults; tests override the
    /// fields they care about (path, generated_at, model).
    fn insert(path: &str, generated_at: i64, model: &str) -> InsertPhotoInsight {
        InsertPhotoInsight {
            library_id: 1,
            file_path: path.to_string(),
            title: format!("title for {model}"),
            summary: "summary".to_string(),
            generated_at,
            model_version: model.to_string(),
            is_current: true,
            training_messages: None,
            backend: "local".to_string(),
            fewshot_source_ids: None,
            content_hash: None,
            num_ctx: None,
            temperature: None,
            top_p: None,
            top_k: None,
            min_p: None,
            system_prompt: None,
            persona_id: None,
            prompt_eval_count: None,
            eval_count: None,
        }
    }
    #[test]
    fn get_insight_history_returns_all_versions_newest_first() {
        let cx = opentelemetry::Context::new();
        let mut dao = dao();
        // store_insight flips prior rows to is_current=false, so three
        // generations for the same path leave a 3-row history.
        dao.store_insight(&cx, insert("a.jpg", 100, "m1")).unwrap();
        dao.store_insight(&cx, insert("a.jpg", 200, "m2")).unwrap();
        dao.store_insight(&cx, insert("a.jpg", 300, "m3")).unwrap();
        // A different path must not leak into the history.
        dao.store_insight(&cx, insert("b.jpg", 250, "other"))
            .unwrap();
        let history = dao.get_insight_history(&cx, "a.jpg").unwrap();
        assert_eq!(history.len(), 3);
        assert_eq!(
            history.iter().map(|i| i.generated_at).collect::<Vec<_>>(),
            vec![300, 200, 100],
            "history should be newest-first"
        );
        // Exactly one version is current (the latest generation).
        let current: Vec<_> = history.iter().filter(|i| i.is_current).collect();
        assert_eq!(current.len(), 1);
        assert_eq!(current[0].generated_at, 300);
    }
    #[test]
    fn rate_insight_by_id_rates_only_the_targeted_version() {
        let cx = opentelemetry::Context::new();
        let mut dao = dao();
        dao.store_insight(&cx, insert("a.jpg", 100, "m1")).unwrap();
        dao.store_insight(&cx, insert("a.jpg", 200, "m2")).unwrap();
        // History is newest-first: [200 (current), 100 (superseded)].
        let history = dao.get_insight_history(&cx, "a.jpg").unwrap();
        let old_version = history.iter().find(|i| i.generated_at == 100).unwrap();
        assert!(!old_version.is_current);
        dao.rate_insight_by_id(&cx, old_version.id, true).unwrap();
        let history = dao.get_insight_history(&cx, "a.jpg").unwrap();
        let old = history.iter().find(|i| i.generated_at == 100).unwrap();
        let current = history.iter().find(|i| i.generated_at == 200).unwrap();
        assert_eq!(old.approved, Some(true), "targeted version is rated");
        assert_eq!(current.approved, None, "current version is untouched");
    }
 }
@@ -273,7 +273,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
                source_file: location.source_file,
            })
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn store_locations_batch(
@@ -350,7 +350,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            Ok(inserted)
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn find_nearest_location(
@@ -385,7 +385,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            Ok(results.into_iter().next().map(|r| r.to_location_record()))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_locations_in_range(
@@ -413,7 +413,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            .map(|rows| rows.into_iter().map(|r| r.to_location_record()).collect())
            .map_err(|e| anyhow::anyhow!("Query error: {:?}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_locations_near_point(
@@ -468,7 +468,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            Ok(filtered)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn location_exists(
@@ -502,7 +502,7 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            Ok(result.count > 0)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_location_count(&mut self, context: &opentelemetry::Context) -> Result<i64, DbError> {
@@ -525,6 +525,6 @@ impl LocationHistoryDao for SqliteLocationHistoryDao {
            Ok(result.count)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
 }
@@ -1,75 +1,9 @@
 use crate::database::schema::{
-    entities, entity_facts, entity_photo_links, favorites, image_exif, insight_generation_jobs,
+    entities, entity_facts, entity_photo_links, favorites, image_exif, libraries, photo_insights,
-    libraries, personas, photo_insights, users, video_preview_clips,
+    users, video_preview_clips,
 };
 use serde::Serialize;
 /// Possible statuses for an insight generation job.
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, FromSqlRow)]
 #[serde(rename_all = "snake_case")]
 pub enum InsightJobStatus {
    Running,
    Completed,
    Failed,
    Cancelled,
 }
 impl InsightJobStatus {
    pub fn as_str(&self) -> &'static str {
        match self {
            Self::Running => "running",
            Self::Completed => "completed",
            Self::Failed => "failed",
            Self::Cancelled => "cancelled",
        }
    }
    pub fn parse(s: &str) -> Self {
        match s {
            "running" => Self::Running,
            "completed" => Self::Completed,
            "failed" => Self::Failed,
            "cancelled" => Self::Cancelled,
            other => {
                log::warn!(
                    "Unknown InsightJobStatus value: {:?}, treating as failed",
                    other
                );
                Self::Failed
            }
        }
    }
 }
 impl std::fmt::Display for InsightJobStatus {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.write_str(self.as_str())
    }
 }
 /// Type of insight generation (standard vs agentic).
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
 #[serde(rename_all = "snake_case")]
 pub enum InsightGenerationType {
    Standard,
    Agentic,
 }
 impl InsightGenerationType {
    pub fn as_str(&self) -> &'static str {
        match self {
            Self::Standard => "standard",
            Self::Agentic => "agentic",
        }
    }
 }
 impl std::fmt::Display for InsightGenerationType {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
        f.write_str(self.as_str())
    }
 }
 #[derive(Insertable)]
 #[diesel(table_name = users)]
 pub struct InsertUser<'a> {
@@ -180,15 +114,6 @@ pub struct ImageExif {
    /// Snapshot of the prior `date_taken_source` taken on first manual
    /// override. NULL when no override is active.
    pub original_date_taken_source: Option<String>,
    /// L2-normalized CLIP image embedding (raw little-endian float32 bytes;
    /// length depends on the model — 768×4 for ViT-L/14, 512×4 for ViT-B/32).
    /// NULL until Apollo's CLIP service has encoded this photo via the
    /// backfill drain. Used by `/photos/search` for semantic queries.
    pub clip_embedding: Option<Vec<u8>>,
    /// Which CLIP model produced `clip_embedding` (e.g. `"ViT-L/14"`). A
    /// swap of `APOLLO_CLIP_MODEL` re-eligibilizes rows whose stored
    /// version differs so the drain rebuilds them.
    pub clip_model_version: Option<String>,
 }
 #[derive(Insertable)]
@@ -218,15 +143,6 @@ pub struct InsertPhotoInsight {
    /// inserted before the hash is available stay null and the
    /// reconciliation pass backfills them.
    pub content_hash: Option<String>,
    pub num_ctx: Option<i32>,
    pub temperature: Option<f32>,
    pub top_p: Option<f32>,
    pub top_k: Option<i32>,
    pub min_p: Option<f32>,
    pub system_prompt: Option<String>,
    pub persona_id: Option<String>,
    pub prompt_eval_count: Option<i32>,
    pub eval_count: Option<i32>,
 }
 #[derive(Serialize, Queryable, Clone, Debug)]
@@ -246,15 +162,6 @@ pub struct PhotoInsight {
    pub backend: String,
    pub fewshot_source_ids: Option<String>,
    pub content_hash: Option<String>,
    pub num_ctx: Option<i32>,
    pub temperature: Option<f32>,
    pub top_p: Option<f32>,
    pub top_k: Option<i32>,
    pub min_p: Option<f32>,
    pub system_prompt: Option<String>,
    pub persona_id: Option<String>,
    pub prompt_eval_count: Option<i32>,
    pub eval_count: Option<i32>,
 }
 // --- Libraries ---
@@ -331,44 +238,6 @@ pub struct InsertEntityFact {
    pub confidence: f32,
    pub status: String,
    pub created_at: i64,
    /// Which persona authored this fact. Shared entities, persona-tagged
    /// facts: each persona accumulates its own voice over the same
    /// real-world referents. Defaults to `'default'` for legacy rows
    /// (see migration 2026-05-09-000000).
    pub persona_id: String,
    /// Author's user_id. Required for the composite FK to
    /// `personas(user_id, persona_id)` (migration 2026-05-10-000000) and
    /// for cross-user fact isolation: two users with the same 'default'
    /// persona must not see each other's facts. Always paired with
    /// `persona_id` — they're a unit.
    pub user_id: i32,
    /// Real-world period the fact is/was true (unix seconds). NULL on
    /// either side = unbounded — `valid_from IS NULL` reads as
    /// "always-true-back-to-the-beginning", `valid_until IS NULL` as
    /// "still-true-now-or-unknown". Distinguishes valid time from
    /// transaction time (`created_at` is when we recorded the fact,
    /// not when it was true in the world). See migration
    /// 2026-05-10-000100.
    pub valid_from: Option<i64>,
    pub valid_until: Option<i64>,
    /// Points at the entity_facts.id that replaced this one. Set by
    /// the supersede endpoint; status flips to 'superseded' in the
    /// same transaction. See migration 2026-05-10-000200.
    pub superseded_by: Option<i32>,
    /// Provenance for model audit — see migration 2026-05-10-000300.
    /// `created_by_model` is the LLM identifier (e.g. "qwen2.5:7b",
    /// "anthropic/claude-sonnet-4") or NULL for legacy / manual rows.
    /// `created_by_backend` is "local" / "hybrid" / "manual" / NULL.
    pub created_by_model: Option<String>,
    pub created_by_backend: Option<String>,
    /// Audit trail for mutations after creation — see migration
    /// 2026-05-10-000500. `last_modified_*` stamp on any update
    /// (status flip, valid-time edit, supersede, manual PATCH);
    /// `last_modified_at` is unix seconds. NULL on rows that have
    /// never been touched since creation.
    pub last_modified_by_model: Option<String>,
    pub last_modified_by_backend: Option<String>,
    pub last_modified_at: Option<i64>,
 }
 #[derive(Serialize, Queryable, Clone, Debug)]
@@ -383,16 +252,6 @@ pub struct EntityFact {
    pub confidence: f32,
    pub status: String,
    pub created_at: i64,
    pub persona_id: String,
    pub user_id: i32,
    pub valid_from: Option<i64>,
    pub valid_until: Option<i64>,
    pub superseded_by: Option<i32>,
    pub created_by_model: Option<String>,
    pub created_by_backend: Option<String>,
    pub last_modified_by_model: Option<String>,
    pub last_modified_by_backend: Option<String>,
    pub last_modified_at: Option<i64>,
 }
 #[derive(Insertable)]
@@ -415,45 +274,6 @@ pub struct EntityPhotoLink {
    pub role: String,
 }
 // --- Personas ---
 #[derive(Insertable)]
 #[diesel(table_name = personas)]
 pub struct InsertPersona<'a> {
    pub user_id: i32,
    pub persona_id: &'a str,
    pub name: &'a str,
    pub system_prompt: &'a str,
    pub is_built_in: bool,
    pub include_all_memories: bool,
    pub created_at: i64,
    pub updated_at: i64,
    /// "Strict mode" — agent reads only see facts with status =
    /// 'reviewed' (human-verified). Default false. See migration
    /// 2026-05-10-000400.
    pub reviewed_only_facts: bool,
    /// Gate for the agent's update_fact / supersede_fact tools.
    /// Default false — fresh personas let the agent create but not
    /// alter or replace. Operator opts in once a model has earned
    /// trust. See migration 2026-05-10-000500.
    pub allow_agent_corrections: bool,
 }
 #[derive(Serialize, Queryable, Clone, Debug)]
 pub struct Persona {
    pub id: i32,
    pub user_id: i32,
    pub persona_id: String,
    pub name: String,
    pub system_prompt: String,
    pub is_built_in: bool,
    pub include_all_memories: bool,
    pub created_at: i64,
    pub updated_at: i64,
    pub reviewed_only_facts: bool,
    pub allow_agent_corrections: bool,
 }
 #[derive(Insertable)]
 #[diesel(table_name = video_preview_clips)]
 pub struct InsertVideoPreviewClip {
@@ -478,30 +298,3 @@ pub struct VideoPreviewClip {
    pub created_at: String,
    pub updated_at: String,
 }
 #[derive(Insertable)]
 #[diesel(table_name = insight_generation_jobs)]
 pub struct InsertInsightGenerationJob {
    pub library_id: i32,
    #[diesel(column_name = file_path)]
    pub path: String,
    #[diesel(column_name = generation_type)]
    pub gen_type: String,
    pub status: String,
    pub started_at: i64,
 }
 #[derive(Queryable, Serialize, Clone, Debug)]
 pub struct InsightGenerationJob {
    pub id: i32,
    pub library_id: i32,
    #[diesel(column_name = file_path)]
    pub path: String,
    #[diesel(column_name = generation_type)]
    pub gen_type: String,
    pub status: String,
    pub started_at: i64,
    pub completed_at: Option<i64>,
    pub result_insight_id: Option<i32>,
    pub error_message: Option<String>,
 }
@@ -1,447 +0,0 @@
 #![allow(dead_code)]
 use diesel::prelude::*;
 use diesel::sqlite::SqliteConnection;
 use std::ops::DerefMut;
 use std::sync::{Arc, Mutex};
 use crate::database::models::{InsertPersona, Persona};
 use crate::database::schema;
 use crate::database::{DbError, DbErrorKind, connect};
 use crate::otel::trace_db_call;
 /// Patch shape for update_persona. None = leave field alone. Built-ins are
 /// allowed to flip `include_all_memories` but should reject name/prompt
 /// edits at the handler layer (built-in copy lives in the migration).
 pub struct PersonaPatch {
    pub name: Option<String>,
    pub system_prompt: Option<String>,
    pub include_all_memories: Option<bool>,
    pub reviewed_only_facts: Option<bool>,
    pub allow_agent_corrections: Option<bool>,
 }
 /// One row of a bulk migration upload. Fields named to match the JSON
 /// shape the mobile client uploads (`POST /personas/migrate`).
 pub struct ImportPersona {
    pub persona_id: String,
    pub name: String,
    pub system_prompt: String,
    pub is_built_in: bool,
    pub created_at: i64,
 }
 pub trait PersonaDao: Sync + Send {
    fn list_personas(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
    ) -> Result<Vec<Persona>, DbError>;
    fn get_persona(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        persona_id: &str,
    ) -> Result<Option<Persona>, DbError>;
    fn create_persona(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        persona_id: &str,
        name: &str,
        system_prompt: &str,
        is_built_in: bool,
        include_all_memories: bool,
    ) -> Result<Persona, DbError>;
    fn update_persona(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        persona_id: &str,
        patch: PersonaPatch,
    ) -> Result<Option<Persona>, DbError>;
    fn delete_persona(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        persona_id: &str,
    ) -> Result<bool, DbError>;
    /// Idempotent bulk import. INSERT OR IGNORE on (user_id, persona_id)
    /// — re-uploading the same set is a no-op. Returns the number of rows
    /// actually inserted (skipped duplicates don't count).
    fn bulk_import(
        &mut self,
        cx: &opentelemetry::Context,
        user_id: i32,
        personas: &[ImportPersona],
    ) -> Result<usize, DbError>;
 }
 pub struct SqlitePersonaDao {
    connection: Arc<Mutex<SqliteConnection>>,
 }
 impl Default for SqlitePersonaDao {
    fn default() -> Self {
        Self::new()
    }
 }
 impl SqlitePersonaDao {
    pub fn new() -> Self {
        Self {
            connection: Arc::new(Mutex::new(connect())),
        }
    }
    pub fn from_connection(conn: Arc<Mutex<SqliteConnection>>) -> Self {
        Self { connection: conn }
    }
 }
 impl PersonaDao for SqlitePersonaDao {
    fn list_personas(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
    ) -> Result<Vec<Persona>, DbError> {
        trace_db_call(cx, "query", "list_personas", |_span| {
            use schema::personas::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            personas
                .filter(user_id.eq(uid))
                .order(created_at.asc())
                .load::<Persona>(conn.deref_mut())
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn get_persona(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        pid: &str,
    ) -> Result<Option<Persona>, DbError> {
        trace_db_call(cx, "query", "get_persona", |_span| {
            use schema::personas::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            personas
                .filter(user_id.eq(uid))
                .filter(persona_id.eq(pid))
                .first::<Persona>(conn.deref_mut())
                .optional()
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn create_persona(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        pid: &str,
        nm: &str,
        prompt: &str,
        builtin: bool,
        include_all: bool,
    ) -> Result<Persona, DbError> {
        trace_db_call(cx, "insert", "create_persona", |_span| {
            use schema::personas::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            let now = chrono::Utc::now().timestamp_millis();
            diesel::insert_into(personas)
                .values(InsertPersona {
                    user_id: uid,
                    persona_id: pid,
                    name: nm,
                    system_prompt: prompt,
                    is_built_in: builtin,
                    include_all_memories: include_all,
                    created_at: now,
                    updated_at: now,
                    reviewed_only_facts: false,
                    allow_agent_corrections: false,
                })
                .execute(conn.deref_mut())
                .map_err(|e| anyhow::anyhow!("Insert error: {}", e))?;
            personas
                .filter(user_id.eq(uid))
                .filter(persona_id.eq(pid))
                .first::<Persona>(conn.deref_mut())
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
    }
    fn update_persona(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        pid: &str,
        patch: PersonaPatch,
    ) -> Result<Option<Persona>, DbError> {
        trace_db_call(cx, "update", "update_persona", |_span| {
            use schema::personas::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            let now = chrono::Utc::now().timestamp_millis();
            // Apply each field as its own UPDATE — keeps types simple
            // (Diesel's tuple updates don't compose cleanly across optional
            // columns) and matches the pattern already in use for entities
            // (knowledge_dao.rs::update_entity).
            if let Some(ref new_name) = patch.name {
                diesel::update(personas.filter(user_id.eq(uid)).filter(persona_id.eq(pid)))
                    .set((name.eq(new_name), updated_at.eq(now)))
                    .execute(conn.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Update name error: {}", e))?;
            }
            if let Some(ref new_prompt) = patch.system_prompt {
                diesel::update(personas.filter(user_id.eq(uid)).filter(persona_id.eq(pid)))
                    .set((system_prompt.eq(new_prompt), updated_at.eq(now)))
                    .execute(conn.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Update prompt error: {}", e))?;
            }
            if let Some(new_include_all) = patch.include_all_memories {
                diesel::update(personas.filter(user_id.eq(uid)).filter(persona_id.eq(pid)))
                    .set((include_all_memories.eq(new_include_all), updated_at.eq(now)))
                    .execute(conn.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Update include_all error: {}", e))?;
            }
            if let Some(new_reviewed_only) = patch.reviewed_only_facts {
                diesel::update(personas.filter(user_id.eq(uid)).filter(persona_id.eq(pid)))
                    .set((
                        reviewed_only_facts.eq(new_reviewed_only),
                        updated_at.eq(now),
                    ))
                    .execute(conn.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Update reviewed_only_facts error: {}", e))?;
            }
            if let Some(new_allow_corrections) = patch.allow_agent_corrections {
                diesel::update(personas.filter(user_id.eq(uid)).filter(persona_id.eq(pid)))
                    .set((
                        allow_agent_corrections.eq(new_allow_corrections),
                        updated_at.eq(now),
                    ))
                    .execute(conn.deref_mut())
                    .map_err(|e| anyhow::anyhow!("Update allow_agent_corrections error: {}", e))?;
            }
            personas
                .filter(user_id.eq(uid))
                .filter(persona_id.eq(pid))
                .first::<Persona>(conn.deref_mut())
                .optional()
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
    }
    fn delete_persona(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        pid: &str,
    ) -> Result<bool, DbError> {
        trace_db_call(cx, "delete", "delete_persona", |_span| {
            use schema::personas::dsl::*;
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            let n = diesel::delete(personas.filter(user_id.eq(uid)).filter(persona_id.eq(pid)))
                .execute(conn.deref_mut())
                .map_err(|e| anyhow::anyhow!("Delete error: {}", e))?;
            Ok(n > 0)
        })
        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
    }
    fn bulk_import(
        &mut self,
        cx: &opentelemetry::Context,
        uid: i32,
        rows: &[ImportPersona],
    ) -> Result<usize, DbError> {
        trace_db_call(cx, "insert", "bulk_import_personas", |_span| {
            let mut conn = self.connection.lock().expect("PersonaDao lock");
            let now = chrono::Utc::now().timestamp_millis();
            let mut inserted = 0usize;
            // INSERT OR IGNORE on the (user_id, persona_id) UNIQUE so
            // re-running migrate is a no-op for personas already on the
            // server.
            for p in rows {
                let n = diesel::sql_query(
                    "INSERT OR IGNORE INTO personas (user_id, persona_id, name, system_prompt, \
                     is_built_in, include_all_memories, created_at, updated_at) \
                     VALUES (?, ?, ?, ?, ?, 0, ?, ?)",
                )
                .bind::<diesel::sql_types::Integer, _>(uid)
                .bind::<diesel::sql_types::Text, _>(&p.persona_id)
                .bind::<diesel::sql_types::Text, _>(&p.name)
                .bind::<diesel::sql_types::Text, _>(&p.system_prompt)
                .bind::<diesel::sql_types::Bool, _>(p.is_built_in)
                .bind::<diesel::sql_types::BigInt, _>(p.created_at)
                .bind::<diesel::sql_types::BigInt, _>(now)
                .execute(conn.deref_mut())
                .map_err(|e| anyhow::anyhow!("Insert error: {}", e))?;
                inserted += n;
            }
            Ok(inserted)
        })
        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use crate::database::test::in_memory_db_connection;
    fn dao_with_user(username: &str) -> (SqlitePersonaDao, i32) {
        use crate::database::schema::users::dsl as u;
        let conn = Arc::new(Mutex::new(in_memory_db_connection()));
        diesel::insert_into(u::users)
            .values((u::username.eq(username), u::password.eq("x")))
            .execute(conn.lock().unwrap().deref_mut())
            .unwrap();
        let user_id: i32 = u::users
            .filter(u::username.eq(username))
            .select(u::id)
            .first(conn.lock().unwrap().deref_mut())
            .unwrap();
        (SqlitePersonaDao::from_connection(conn), user_id)
    }
    #[test]
    fn create_and_list_round_trip() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("alice");
        // The migration seeds 3 built-ins for any existing user; alice
        // was created post-migration so she starts empty.
        let p = dao
            .create_persona(&cx, uid, "custom-1", "Custom A", "prompt A", false, false)
            .unwrap();
        assert_eq!(p.persona_id, "custom-1");
        assert_eq!(p.user_id, uid);
        assert!(!p.is_built_in);
        let list = dao.list_personas(&cx, uid).unwrap();
        assert_eq!(list.len(), 1);
        assert_eq!(list[0].persona_id, "custom-1");
    }
    #[test]
    fn unique_constraint_blocks_duplicate_persona_id() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("bob");
        dao.create_persona(&cx, uid, "x", "X", "p", false, false)
            .unwrap();
        let err = dao.create_persona(&cx, uid, "x", "X2", "p2", false, false);
        assert!(
            err.is_err(),
            "second insert with same persona_id should fail"
        );
    }
    #[test]
    fn bulk_import_is_idempotent() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("carol");
        let rows = vec![
            ImportPersona {
                persona_id: "custom-a".into(),
                name: "A".into(),
                system_prompt: "p1".into(),
                is_built_in: false,
                created_at: 1,
            },
            ImportPersona {
                persona_id: "custom-b".into(),
                name: "B".into(),
                system_prompt: "p2".into(),
                is_built_in: false,
                created_at: 2,
            },
        ];
        let first = dao.bulk_import(&cx, uid, &rows).unwrap();
        assert_eq!(first, 2);
        let second = dao.bulk_import(&cx, uid, &rows).unwrap();
        assert_eq!(second, 0, "re-import should insert nothing");
        assert_eq!(dao.list_personas(&cx, uid).unwrap().len(), 2);
    }
    #[test]
    fn dao_update_does_not_block_built_ins() {
        // Documenting contract: the DAO is intentionally permissive —
        // `update_persona` will apply name/system_prompt edits to ANY
        // row, including built-ins. The guard against editing built-in
        // identity (name + systemPrompt) lives in the HTTP handler
        // (src/personas.rs::update_persona). If you find yourself
        // wanting to add the guard here too, prefer that — defence in
        // depth — but keep this test passing so anyone who removes
        // the handler guard gets a failing call site, not silent data
        // corruption.
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("eve");
        dao.create_persona(&cx, uid, "default", "Default", "old", true, false)
            .unwrap();
        let updated = dao
            .update_persona(
                &cx,
                uid,
                "default",
                PersonaPatch {
                    name: Some("Renamed".into()),
                    system_prompt: Some("new prompt".into()),
                    include_all_memories: None,
                    reviewed_only_facts: None,
                    allow_agent_corrections: None,
                },
            )
            .unwrap()
            .unwrap();
        assert_eq!(updated.name, "Renamed");
        assert_eq!(updated.system_prompt, "new prompt");
        assert!(
            updated.is_built_in,
            "is_built_in flag should be unchanged by patch"
        );
    }
    #[test]
    fn update_toggles_include_all_memories() {
        let cx = opentelemetry::Context::new();
        let (mut dao, uid) = dao_with_user("dan");
        dao.create_persona(&cx, uid, "j", "Journal", "p", true, false)
            .unwrap();
        let updated = dao
            .update_persona(
                &cx,
                uid,
                "j",
                PersonaPatch {
                    name: None,
                    system_prompt: None,
                    include_all_memories: Some(true),
                    reviewed_only_facts: None,
                    allow_agent_corrections: None,
                },
            )
            .unwrap()
            .unwrap();
        assert!(updated.include_all_memories);
    }
 }
@@ -96,7 +96,7 @@ impl PreviewDao for SqlitePreviewDao {
                .map(|_| ())
                .map_err(|e| anyhow::anyhow!("Insert error: {}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn update_status(
@@ -126,7 +126,7 @@ impl PreviewDao for SqlitePreviewDao {
                .map(|_| ())
                .map_err(|e| anyhow::anyhow!("Update error: {}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::UpdateError, e))
+        .map_err(|_| DbError::new(DbErrorKind::UpdateError))
    }
    fn get_preview(
@@ -148,7 +148,7 @@ impl PreviewDao for SqlitePreviewDao {
                Err(e) => Err(anyhow::anyhow!("Query error: {}", e)),
            }
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_previews_batch(
@@ -170,7 +170,7 @@ impl PreviewDao for SqlitePreviewDao {
                .load::<VideoPreviewClip>(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_by_status(
@@ -188,7 +188,7 @@ impl PreviewDao for SqlitePreviewDao {
                .load::<VideoPreviewClip>(connection.deref_mut())
                .map_err(|e| anyhow::anyhow!("Query error: {}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
 }
@@ -57,28 +57,30 @@ impl ReconcileStats {
 /// watcher tick. Errors are logged but never propagated; reconciliation
 /// is best-effort and a transient DB hiccup must not stall the watcher.
 pub fn run(conn: &mut SqliteConnection) -> ReconcileStats {
-    let stats = ReconcileStats {
+    let mut stats = ReconcileStats::default();
-        tagged_photo_hashes_filled: match backfill_tagged_photo_hashes(conn) {
+
-            Ok(n) => n,
+    stats.tagged_photo_hashes_filled = match backfill_tagged_photo_hashes(conn) {
-            Err(e) => {
+        Ok(n) => n,
-                warn!("reconcile: tagged_photo hash backfill failed: {:?}", e);
+        Err(e) => {
-                0
+            warn!("reconcile: tagged_photo hash backfill failed: {:?}", e);
-            }
+            0
-        },
+        }
-        photo_insights_hashes_filled: match backfill_photo_insights_hashes(conn) {
+    };
-            Ok(n) => n,
+
-            Err(e) => {
+    stats.photo_insights_hashes_filled = match backfill_photo_insights_hashes(conn) {
-                warn!("reconcile: photo_insights hash backfill failed: {:?}", e);
+        Ok(n) => n,
-                0
+        Err(e) => {
-            }
+            warn!("reconcile: photo_insights hash backfill failed: {:?}", e);
-        },
+            0
-        photo_insights_demoted: match collapse_insight_currents(conn) {
+        }
-            Ok(n) => n,
+    };
-            Err(e) => {
+
-                warn!("reconcile: photo_insights scalar merge failed: {:?}", e);
+    stats.photo_insights_demoted = match collapse_insight_currents(conn) {
-                0
+        Ok(n) => n,
-            }
+        Err(e) => {
-        },
+            warn!("reconcile: photo_insights scalar merge failed: {:?}", e);
            0
        }
    };
    if stats.changed() {
@@ -57,16 +57,6 @@ diesel::table! {
        confidence -> Float,
        status -> Text,
        created_at -> BigInt,
        persona_id -> Text,
        user_id -> Integer,
        valid_from -> Nullable<BigInt>,
        valid_until -> Nullable<BigInt>,
        superseded_by -> Nullable<Integer>,
        created_by_model -> Nullable<Text>,
        created_by_backend -> Nullable<Text>,
        last_modified_by_model -> Nullable<Text>,
        last_modified_by_backend -> Nullable<Text>,
        last_modified_at -> Nullable<BigInt>,
    }
 }
@@ -138,8 +128,6 @@ diesel::table! {
        date_taken_source -> Nullable<Text>,
        original_date_taken -> Nullable<BigInt>,
        original_date_taken_source -> Nullable<Text>,
        clip_embedding -> Nullable<Binary>,
        clip_model_version -> Nullable<Text>,
    }
 }
@@ -171,22 +159,6 @@ diesel::table! {
    }
 }
 diesel::table! {
    personas (id) {
        id -> Integer,
        user_id -> Integer,
        persona_id -> Text,
        name -> Text,
        system_prompt -> Text,
        is_built_in -> Bool,
        include_all_memories -> Bool,
        created_at -> BigInt,
        updated_at -> BigInt,
        reviewed_only_facts -> Bool,
        allow_agent_corrections -> Bool,
    }
 }
 diesel::table! {
    persons (id) {
        id -> Integer,
@@ -216,15 +188,6 @@ diesel::table! {
        backend -> Text,
        fewshot_source_ids -> Nullable<Text>,
        content_hash -> Nullable<Text>,
        num_ctx -> Nullable<Integer>,
        temperature -> Nullable<Float>,
        top_p -> Nullable<Float>,
        top_k -> Nullable<Integer>,
        min_p -> Nullable<Float>,
        system_prompt -> Nullable<Text>,
        persona_id -> Nullable<Text>,
        prompt_eval_count -> Nullable<Integer>,
        eval_count -> Nullable<Integer>,
    }
 }
@@ -280,28 +243,12 @@ diesel::table! {
    }
 }
 diesel::table! {
    insight_generation_jobs (id) {
        id -> Integer,
        library_id -> Integer,
        file_path -> Text,
        generation_type -> Text,
        status -> Text,
        started_at -> BigInt,
        completed_at -> Nullable<BigInt>,
        result_insight_id -> Nullable<Integer>,
        error_message -> Nullable<Text>,
    }
 }
 diesel::joinable!(entity_facts -> photo_insights (source_insight_id));
 diesel::joinable!(entity_photo_links -> entities (entity_id));
 diesel::joinable!(entity_photo_links -> libraries (library_id));
 diesel::joinable!(face_detections -> libraries (library_id));
 diesel::joinable!(face_detections -> persons (person_id));
 diesel::joinable!(image_exif -> libraries (library_id));
 diesel::joinable!(insight_generation_jobs -> libraries (library_id));
 diesel::joinable!(personas -> users (user_id));
 diesel::joinable!(persons -> entities (entity_id));
 diesel::joinable!(photo_insights -> libraries (library_id));
 diesel::joinable!(tagged_photo -> tags (tag_id));
@@ -316,10 +263,8 @@ diesel::allow_tables_to_appear_in_same_query!(
    face_detections,
    favorites,
    image_exif,
    insight_generation_jobs,
    libraries,
    location_history,
    personas,
    persons,
    photo_insights,
    search_history,
@@ -227,7 +227,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
                source_file: search.source_file,
            })
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn store_searches_batch(
@@ -283,7 +283,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
            Ok(inserted)
        })
-        .map_err(|e| DbError::log(DbErrorKind::InsertError, e))
+        .map_err(|_| DbError::new(DbErrorKind::InsertError))
    }
    fn find_searches_in_range(
@@ -310,7 +310,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
            .map(|rows| rows.into_iter().map(|r| r.to_search_record()).collect())
            .map_err(|e| anyhow::anyhow!("Query error: {:?}", e))
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_similar_searches(
@@ -372,7 +372,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
                .map(|(_, search)| search)
                .collect())
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn find_relevant_searches_hybrid(
@@ -459,7 +459,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
                    .collect())
            }
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn search_exists(
@@ -490,7 +490,7 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
            Ok(result.count > 0)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
    fn get_search_count(&mut self, context: &opentelemetry::Context) -> Result<i64, DbError> {
@@ -513,6 +513,6 @@ impl SearchHistoryDao for SqliteSearchHistoryDao {
            Ok(result.count)
        })
-        .map_err(|e| DbError::log(DbErrorKind::QueryError, e))
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
    }
 }
@@ -230,21 +230,12 @@ fn exiftool_available() -> bool {
 /// One-file exiftool invocation. Used by the upload + GPS-write paths,
 /// which deal with one file at a time. The batch path uses
 /// `exiftool_dates_batch` so we don't pay subprocess startup per row.
 ///
 /// Notably absent: `-fast` / `-fast2`. For QuickTime/MP4 files whose
 /// `moov` atom sits at the end (non-faststart, common for Snapchat
 /// exports and any MP4 muxed without `-movflags +faststart`), `-fast2`
 /// causes exiftool to skip the trailer and return no `CreateDate` /
 /// `MediaCreateDate`, dropping us to the `fs_time` fallback for files
 /// that actually have a real capture date. We pre-filter to files that
 /// kamadak-exif couldn't read, so the JPEG fast-path is already covered
 /// — paying full-scan cost on the residual is the right trade.
 fn exiftool_date_single(path: &Path) -> Option<i64> {
    if !exiftool_available() {
        return None;
    }
    let mut cmd = Command::new("exiftool");
-    cmd.arg("-j").arg("-q").arg("-d").arg("%s");
+    cmd.arg("-j").arg("-q").arg("-d").arg("%s").arg("-fast2");
    for tag in EXIFTOOL_DATE_TAGS {
        cmd.arg(format!("-{}", tag));
    }
@@ -270,10 +261,7 @@ fn exiftool_dates_batch(paths: &[&Path]) -> HashMap<PathBuf, i64> {
    }
    let mut cmd = Command::new("exiftool");
-    // No `-fast2` — see exiftool_date_single for the rationale (QuickTime
+    cmd.arg("-j").arg("-q").arg("-d").arg("%s").arg("-fast2");
    // moov-at-end files miss CreateDate / MediaCreateDate when the trailer
    // is skipped).
    cmd.arg("-j").arg("-q").arg("-d").arg("%s");
    for tag in EXIFTOOL_DATE_TAGS {
        cmd.arg(format!("-{}", tag));
    }
@@ -47,7 +47,7 @@ use std::sync::{Arc, Mutex};
 /// Visual identity. The optional `entity_id` bridges this person to an
 /// LLM-extracted knowledge-graph entity (textual side). Persons are NOT
 /// auto-bridged at creation — only when the user explicitly links them in
-/// the management UI.
+/// the management UI, or when bootstrap finds an exact-name match.
 #[derive(Serialize, Queryable, Clone, Debug)]
 pub struct Person {
    pub id: i32,
@@ -366,10 +366,6 @@ pub struct EmbeddingsQuery {
    pub limit: i64,
    #[serde(default)]
    pub offset: i64,
    /// Restrict to one person's faces. Used by the similar-unassigned
    /// suggester to fetch a centroid pool. When set, takes precedence
    /// over `unassigned` (the more specific filter wins).
    pub person_id: Option<i32>,
 }
 fn default_unassigned() -> bool {
@@ -433,7 +429,6 @@ pub trait FaceDao: Send + Sync {
        ctx: &opentelemetry::Context,
        library_id: Option<i32>,
        unassigned: bool,
        person_id: Option<i32>,
        limit: i64,
        offset: i64,
    ) -> anyhow::Result<Vec<(FaceDetectionRow, String)>>;
@@ -508,10 +503,6 @@ pub trait FaceDao: Send + Sync {
        into: i32,
    ) -> anyhow::Result<Person>;
    /// Cheap presence probe — returns true iff at least one face has been
    /// detected (excluding marker rows). Used by chat-tool gating.
    fn has_any_faces(&mut self, ctx: &opentelemetry::Context) -> anyhow::Result<bool>;
    /// Resolve `(library_id, rel_path)` → `content_hash` via image_exif.
    /// Returns None when the photo hasn't been EXIF-indexed yet (no row
    /// in image_exif) or when the row exists but content_hash is NULL.
@@ -868,7 +859,6 @@ impl FaceDao for SqliteFaceDao {
        ctx: &opentelemetry::Context,
        library_id: Option<i32>,
        unassigned: bool,
        person_id: Option<i32>,
        limit: i64,
        offset: i64,
    ) -> anyhow::Result<Vec<(FaceDetectionRow, String)>> {
@@ -882,13 +872,7 @@ impl FaceDao for SqliteFaceDao {
            if let Some(lib) = library_id {
                query = query.filter(face_detections::library_id.eq(lib));
            }
-            // person_id is the more specific filter — when both it and
+            if unassigned {
            // `unassigned` are supplied, prefer the explicit person id and
            // ignore the IS NULL constraint (which would always return
            // empty for an assigned person).
            if let Some(pid) = person_id {
                query = query.filter(face_detections::person_id.eq(pid));
            } else if unassigned {
                query = query.filter(face_detections::person_id.is_null());
            }
            let rows = query
@@ -1024,14 +1008,9 @@ impl FaceDao for SqliteFaceDao {
                if let Some(lib) = library_id {
                    q = q.filter(face_detections::library_id.eq(lib));
                }
-                q.select(
+                q.select(diesel::dsl::count_distinct(face_detections::content_hash))
-                    #[allow(deprecated)]
+                    .first(conn.deref_mut())
-                    {
+                    .with_context(|| "stats: scanned")?
                        diesel::dsl::count_distinct(face_detections::content_hash)
                    },
                )
                .first(conn.deref_mut())
                .with_context(|| "stats: scanned")?
            };
            let with_faces: i64 = {
                let mut q = face_detections::table
@@ -1040,14 +1019,9 @@ impl FaceDao for SqliteFaceDao {
                if let Some(lib) = library_id {
                    q = q.filter(face_detections::library_id.eq(lib));
                }
-                q.select(
+                q.select(diesel::dsl::count_distinct(face_detections::content_hash))
-                    #[allow(deprecated)]
+                    .first(conn.deref_mut())
-                    {
+                    .with_context(|| "stats: with_faces")?
                        diesel::dsl::count_distinct(face_detections::content_hash)
                    },
                )
                .first(conn.deref_mut())
                .with_context(|| "stats: with_faces")?
            };
            let no_faces: i64 = {
                let mut q = face_detections::table
@@ -1056,14 +1030,9 @@ impl FaceDao for SqliteFaceDao {
                if let Some(lib) = library_id {
                    q = q.filter(face_detections::library_id.eq(lib));
                }
-                q.select(
+                q.select(diesel::dsl::count_distinct(face_detections::content_hash))
-                    #[allow(deprecated)]
+                    .first(conn.deref_mut())
-                    {
+                    .with_context(|| "stats: no_faces")?
                        diesel::dsl::count_distinct(face_detections::content_hash)
                    },
                )
                .first(conn.deref_mut())
                .with_context(|| "stats: no_faces")?
            };
            let failed: i64 = {
                let mut q = face_detections::table
@@ -1072,14 +1041,9 @@ impl FaceDao for SqliteFaceDao {
                if let Some(lib) = library_id {
                    q = q.filter(face_detections::library_id.eq(lib));
                }
-                q.select(
+                q.select(diesel::dsl::count_distinct(face_detections::content_hash))
-                    #[allow(deprecated)]
+                    .first(conn.deref_mut())
-                    {
+                    .with_context(|| "stats: failed")?
                        diesel::dsl::count_distinct(face_detections::content_hash)
                    },
                )
                .first(conn.deref_mut())
                .with_context(|| "stats: failed")?
            };
            // Image-extension filter mirrors `list_unscanned_candidates` so
            // SCANNED can actually reach 100%: videos sit in `image_exif` but
@@ -1468,19 +1432,6 @@ impl FaceDao for SqliteFaceDao {
        })
    }
    fn has_any_faces(&mut self, ctx: &opentelemetry::Context) -> anyhow::Result<bool> {
        let mut conn = self.connection.lock().expect("face dao lock");
        trace_db_call(ctx, "query", "has_any_faces", |_span| {
            face_detections::table
                .filter(face_detections::status.eq("detected"))
                .select(face_detections::id)
                .first::<i32>(conn.deref_mut())
                .optional()
                .map(|x| x.is_some())
                .with_context(|| "has_any_faces query")
        })
    }
    fn resolve_content_hash(
        &mut self,
        ctx: &opentelemetry::Context,
@@ -1708,10 +1659,18 @@ where
                .route(web::get().to(list_persons_handler::<D>))
                .route(web::post().to(create_person_handler::<D>)),
        )
        .service(
            web::resource("/persons/bootstrap")
                .route(web::post().to(bootstrap_persons_handler::<D>)),
        )
        .service(
            web::resource("/persons/ignore-bucket")
                .route(web::post().to(ignore_bucket_handler::<D>)),
        )
        .service(
            web::resource("/tags/people-bootstrap-candidates")
                .route(web::get().to(bootstrap_candidates_handler::<D>)),
        )
        .service(
            web::resource("/persons/{id}")
                .route(web::get().to(get_person_handler::<D>))
@@ -1726,6 +1685,340 @@ where
        )
 }
 // ── Bootstrap (Phase 4) ─────────────────────────────────────────────────────
 #[derive(Serialize, Debug, Clone)]
 pub struct BootstrapCandidate {
    /// Display name — most-frequent capitalization across the case-insensitive
    /// group, or simply the first one seen if it's a tie.
    pub name: String,
    /// Lowercased name; the stable key for grouping and the auto-bind path.
    pub normalized_name: String,
    /// Sum of `tagged_photo` counts across all capitalizations of this name.
    pub usage_count: i64,
    /// Heuristic suggestion; the UI defaults this to checked but the user
    /// confirms before [`bootstrap_persons_handler`] actually creates rows.
    pub looks_like_person: bool,
    /// True when a `persons` row already exists for this name (any case).
    /// The UI hides these — re-running bootstrap is idempotent so it's fine
    /// either way, but the noise isn't worth showing.
    pub already_exists: bool,
 }
 #[derive(Serialize, Debug)]
 pub struct BootstrapCandidatesResponse {
    pub candidates: Vec<BootstrapCandidate>,
 }
 #[derive(Deserialize, Debug)]
 pub struct BootstrapPersonsReq {
    pub names: Vec<String>,
 }
 #[derive(Serialize, Debug)]
 pub struct BootstrapPersonsResponse {
    pub created: Vec<Person>,
    pub skipped: Vec<BootstrapSkipped>,
 }
 #[derive(Serialize, Debug)]
 pub struct BootstrapSkipped {
    pub name: String,
    pub reason: String,
 }
 /// Hard filter for the bootstrap candidate list. Returns true if the tag
 /// could plausibly be a person name; returns false to drop it from the
 /// candidates entirely (not just leave looks_like_person=false).
 ///
 /// Rules — all required:
 /// - At least 3 characters after trimming. Two-letter tags ("AB", "OK")
 ///   are almost always abbreviations or markers, not names.
 /// - No emoji or symbol-class characters. SQL-side string sort already
 ///   surfaces those at the top of the tag list; filtering them keeps
 ///   the candidate UI focused on names rather than chart-junk.
 /// - No control characters or null bytes.
 pub(crate) fn is_plausible_name_token(raw: &str) -> bool {
    let trimmed = raw.trim();
    if trimmed.chars().count() < 3 {
        return false;
    }
    for c in trimmed.chars() {
        // Letter / mark / decimal-digit / connector-punctuation /
        // dash / apostrophe / period / whitespace are all plausible in a
        // name. Anything else (emoji, symbols, math operators, arrows,
        // box drawing, control codes) disqualifies the whole tag.
        if c.is_alphabetic()
            || c.is_whitespace()
            || matches!(c, '\'' | '-' | '.' | '_' | '\u{2019}')
        {
            continue;
        }
        if c.is_ascii_digit() {
            // Digits don't disqualify here — `looks_like_person` rejects
            // them later, but `is_plausible_name_token` is just about
            // "could this be in the candidate list at all?". A tag like
            // "Sarah2" stays as a candidate (display-flagged not-a-person
            // by looks_like_person) so the operator can still spot and
            // confirm it manually if it's an alias.
            continue;
        }
        return false;
    }
    true
 }
 /// Conservative "this tag *might* be a person name" heuristic. False
 /// negatives are fine — the operator confirms in the UI before any row
 /// is created. False positives are also fine for the same reason; the
 /// goal is just to default sensible candidates to checked.
 ///
 /// Rules:
 /// - 1–2 whitespace-separated words
 /// - Each word starts with an uppercase character
 /// - No digits anywhere (rejects "Trip 2018", "2024", etc.)
 /// - Single-word names not on a small denylist of common non-person
 ///   tags (cat, christmas, beach, ...). Two-word names skip the
 ///   denylist because a real two-word person name is the dominant
 ///   case ("Sarah Smith") and false-blocking it is worse than false-
 ///   accepting "Sunset Walk".
 pub(crate) fn looks_like_person(raw: &str) -> bool {
    let trimmed = raw.trim();
    if trimmed.is_empty() {
        return false;
    }
    let words: Vec<&str> = trimmed.split_whitespace().collect();
    if !(1..=2).contains(&words.len()) {
        return false;
    }
    for w in &words {
        let Some(first) = w.chars().next() else {
            return false;
        };
        if !first.is_uppercase() {
            return false;
        }
        if w.chars().any(|c| c.is_ascii_digit()) {
            return false;
        }
    }
    if words.len() == 1 {
        const DENY: &[&str] = &[
            // Pets / animals
            "cat",
            "dog",
            "kitten",
            "puppy",
            "bird",
            "fish",
            "pet",
            "pets",
            // Events / occasions
            "birthday",
            "christmas",
            "halloween",
            "easter",
            "thanksgiving",
            "wedding",
            "anniversary",
            "vacation",
            "holiday",
            "party",
            "trip",
            "graduation",
            "concert",
            // Places (generic)
            "home",
            "work",
            "beach",
            "park",
            "hotel",
            "restaurant",
            "office",
            "house",
            "garden",
            // Subjects / styles
            "food",
            "sunset",
            "sunrise",
            "landscape",
            "portrait",
            "selfie",
            "nature",
            "flowers",
            "flower",
            "snow",
            "rain",
            "sky",
            // Buckets
            "untagged",
            "favorites",
            "favourites",
            "misc",
            "other",
            "random",
        ];
        let lower = trimmed.to_lowercase();
        if DENY.iter().any(|w| *w == lower) {
            return false;
        }
    }
    true
 }
 async fn bootstrap_candidates_handler<D: FaceDao>(
    _: Claims,
    request: HttpRequest,
    face_dao: web::Data<Mutex<D>>,
    tag_dao: web::Data<Mutex<crate::tags::SqliteTagDao>>,
 ) -> impl Responder {
    use std::collections::HashMap;
    let context = extract_context_from_request(&request);
    let span = global_tracer().start_with_context("faces.bootstrap_candidates", &context);
    let span_context = opentelemetry::Context::current_with_span(span);
    // All tags + their counts. Path filter unused — bootstrap is library-wide.
    let tags_with_counts = {
        let mut td = tag_dao.lock().expect("tag dao lock");
        match crate::tags::TagDao::get_all_tags(&mut *td, &span_context, None) {
            Ok(t) => t,
            Err(e) => return HttpResponse::InternalServerError().body(format!("{:#}", e)),
        }
    };
    // Group by lowercase name. Pick the most-frequent capitalization
    // for the display name (ties broken by first-seen). Filter out
    // short tags and tags carrying non-name characters (emojis, symbols)
    // before grouping — they're noise no operator would tick, so showing
    // them just makes the candidate list harder to scan.
    struct Group {
        display: String,
        display_freq: i64,
        total_count: i64,
    }
    let mut groups: HashMap<String, Group> = HashMap::new();
    for (count, tag) in tags_with_counts {
        if !is_plausible_name_token(&tag.name) {
            continue;
        }
        let lower = tag.name.to_lowercase();
        let g = groups.entry(lower).or_insert_with(|| Group {
            display: tag.name.clone(),
            display_freq: 0,
            total_count: 0,
        });
        g.total_count += count;
        if count > g.display_freq {
            g.display = tag.name.clone();
            g.display_freq = count;
        }
    }
    // Cross-reference against existing persons (bulk one-query lookup).
    let lower_names: Vec<String> = groups.keys().cloned().collect();
    let existing = {
        let mut fd = face_dao.lock().expect("face dao lock");
        match fd.find_persons_by_names_ci(&span_context, &lower_names) {
            Ok(m) => m,
            Err(e) => return HttpResponse::InternalServerError().body(format!("{:#}", e)),
        }
    };
    let mut candidates: Vec<BootstrapCandidate> = groups
        .into_iter()
        .map(|(lower, g)| BootstrapCandidate {
            looks_like_person: looks_like_person(&g.display),
            already_exists: existing.contains_key(&lower),
            name: g.display,
            normalized_name: lower,
            usage_count: g.total_count,
        })
        .collect();
    // Sort: persons-first heuristic by descending count, then alphabetical.
    // Persons-likely candidates surface near the top so the user doesn't
    // scroll past dozens of "vacation"-style tags to find them.
    candidates.sort_by(|a, b| {
        b.looks_like_person
            .cmp(&a.looks_like_person)
            .then(b.usage_count.cmp(&a.usage_count))
            .then(a.normalized_name.cmp(&b.normalized_name))
    });
    HttpResponse::Ok().json(BootstrapCandidatesResponse { candidates })
 }
 async fn bootstrap_persons_handler<D: FaceDao>(
    _: Claims,
    request: HttpRequest,
    body: web::Json<BootstrapPersonsReq>,
    face_dao: web::Data<Mutex<D>>,
 ) -> impl Responder {
    let context = extract_context_from_request(&request);
    let span = global_tracer().start_with_context("faces.bootstrap_persons", &context);
    let span_context = opentelemetry::Context::current_with_span(span);
    let mut created: Vec<Person> = Vec::new();
    let mut skipped: Vec<BootstrapSkipped> = Vec::new();
    let mut dao = face_dao.lock().expect("face dao lock");
    // Pre-fetch the existing-name set so a duplicate request reports
    // "already exists" (skipped) rather than firing N inserts that all
    // 409 against the UNIQUE COLLATE NOCASE constraint.
    let lower_names: Vec<String> = body.names.iter().map(|n| n.to_lowercase()).collect();
    let existing = match dao.find_persons_by_names_ci(&span_context, &lower_names) {
        Ok(m) => m,
        Err(e) => return HttpResponse::InternalServerError().body(format!("{:#}", e)),
    };
    for name in &body.names {
        let trimmed = name.trim();
        if trimmed.is_empty() {
            skipped.push(BootstrapSkipped {
                name: name.clone(),
                reason: "empty name".into(),
            });
            continue;
        }
        let lower = trimmed.to_lowercase();
        if existing.contains_key(&lower) {
            skipped.push(BootstrapSkipped {
                name: trimmed.to_string(),
                reason: "person already exists".into(),
            });
            continue;
        }
        match dao.create_person(
            &span_context,
            &CreatePersonReq {
                name: trimmed.to_string(),
                notes: None,
                entity_id: None,
                is_ignored: false,
            },
            /*from_tag*/ true,
        ) {
            Ok(p) => created.push(p),
            Err(e) => {
                if is_unique_violation(&e) {
                    // Race with a concurrent create; treat as skipped.
                    skipped.push(BootstrapSkipped {
                        name: trimmed.to_string(),
                        reason: "person already exists".into(),
                    });
                } else {
                    skipped.push(BootstrapSkipped {
                        name: trimmed.to_string(),
                        reason: format!("{:#}", e),
                    });
                }
            }
        }
    }
    HttpResponse::Ok().json(BootstrapPersonsResponse { created, skipped })
 }
 // ── Stats / list ────────────────────────────────────────────────────────────
 #[derive(Deserialize)]
@@ -1822,7 +2115,6 @@ async fn embeddings_handler<D: FaceDao>(
        &span_context,
        query.library,
        query.unassigned,
        query.person_id,
        limit,
        offset,
    )
@@ -2038,19 +2330,12 @@ async fn update_face_handler<D: FaceDao>(
            match dao.get_face(&span_context, id) {
                Ok(Some(r)) => r,
                Ok(None) => return HttpResponse::NotFound().finish(),
-                Err(e) => {
+                Err(e) => return HttpResponse::InternalServerError().body(e.to_string()),
                    warn!("PATCH /image/faces/{}: 500 — get_face failed: {:#}", id, e);
                    return HttpResponse::InternalServerError().body(e.to_string());
                }
            }
        };
        let library = match app_state.library_by_id(current.library_id) {
            Some(l) => l.clone(),
            None => {
                warn!(
                    "PATCH /image/faces/{}: 500 — face row references unknown library_id {}",
                    id, current.library_id
                );
                return HttpResponse::InternalServerError().body(format!(
                    "face row references unknown library_id {}",
                    current.library_id
@@ -2133,17 +2418,7 @@ async fn update_face_handler<D: FaceDao>(
    let mut dao = face_dao.lock().expect("face dao lock");
    let row = match dao.update_face(&span_context, id, person_patch, bbox_patch, new_embedding) {
        Ok(r) => r,
-        Err(e) => {
+        Err(e) => return HttpResponse::InternalServerError().body(e.to_string()),
            // The full anyhow chain (`{:#}`) shows the diesel cause behind
            // the short context string we surface in the response body —
            // SQLITE_BUSY here usually means another DAO's writer held the
            // lock past `busy_timeout` (5s), which is invisible in `{}`.
            warn!(
                "PATCH /image/faces/{}: 500 — update_face failed: {:#}",
                id, e
            );
            return HttpResponse::InternalServerError().body(e.to_string());
        }
    };
    // Hydrate person_name so the response shape matches GET /image/faces
    // — the carousel overlay does an optimistic replace on this row, and
@@ -2151,13 +2426,7 @@ async fn update_face_handler<D: FaceDao>(
    // VFD label off the bbox even though the assignment didn't change.
    match hydrate_face_with_person(&mut *dao, &span_context, row) {
        Ok(joined) => HttpResponse::Ok().json(joined),
-        Err(e) => {
+        Err(e) => HttpResponse::InternalServerError().body(e.to_string()),
            warn!(
                "PATCH /image/faces/{}: 500 — hydrate_face_with_person failed: {:#}",
                id, e
            );
            HttpResponse::InternalServerError().body(e.to_string())
        }
    }
 }
@@ -2510,7 +2779,77 @@ mod tests {
        );
    }
-    // ── Phase 4: cosine + DAO support ───────────────────────────────────
+    // ── Phase 4: bootstrap heuristic + cosine + DAO support ─────────────
    #[test]
    fn is_plausible_name_token_filters_short_and_emoji() {
        // Hard filter applied before grouping — emojis and tags shorter
        // than 3 chars never make it into the candidate list, regardless
        // of looks_like_person's later assessment.
        assert!(is_plausible_name_token("Cameron"));
        assert!(is_plausible_name_token("Sarah Smith"));
        assert!(is_plausible_name_token("O'Brien"));
        assert!(is_plausible_name_token("Jean-Luc"));
        assert!(is_plausible_name_token("St. James"));
        assert!(is_plausible_name_token("Renée"));
        assert!(is_plausible_name_token("José"));
        // Asian script names — the alphabetic/letter check covers any
        // script, not just Latin.
        assert!(is_plausible_name_token("田中太郎"));
        // Below the 3-character floor.
        assert!(!is_plausible_name_token(""));
        assert!(!is_plausible_name_token(" "));
        assert!(!is_plausible_name_token("Bo"));
        assert!(!is_plausible_name_token("AB"));
        // Trim before counting — surrounding whitespace doesn't count.
        assert!(!is_plausible_name_token("  AB  "));
        // Emoji / symbol classes get the whole tag dropped.
        assert!(!is_plausible_name_token("🐱cat"));
        assert!(!is_plausible_name_token("Heart ❤"));
        assert!(!is_plausible_name_token("📸Photo"));
        assert!(!is_plausible_name_token("→ Trip"));
        assert!(!is_plausible_name_token("★Vacation"));
        // Digits are kept (handled by looks_like_person, not here).
        assert!(is_plausible_name_token("Trip 2018"));
        assert!(is_plausible_name_token("2024"));
    }
    #[test]
    fn looks_like_person_accepts_typical_names() {
        assert!(looks_like_person("Cameron"));
        assert!(looks_like_person("Sarah Smith"));
        assert!(looks_like_person("Mary Jane"));
        // Non-ASCII title-cased single word still counts.
        assert!(looks_like_person("Renée"));
    }
    #[test]
    fn looks_like_person_rejects_obvious_non_people() {
        // Digits, lowercase, three-or-more words, denylist hits.
        assert!(!looks_like_person("2018"));
        assert!(!looks_like_person("Trip 2018"));
        assert!(!looks_like_person("trip"));
        assert!(!looks_like_person("Birthday Party Cake"));
        assert!(!looks_like_person("cat"));
        assert!(!looks_like_person("Cat")); // denied even when title-cased
        assert!(!looks_like_person("Christmas"));
        assert!(!looks_like_person("home"));
        assert!(!looks_like_person(""));
        assert!(!looks_like_person("   "));
    }
    #[test]
    fn looks_like_person_two_words_skips_denylist() {
        // Two-word names get a pass on the single-word denylist —
        // "Sunset Walk" is much more likely a real album than a person,
        // but false-accepting is fine because the operator confirms.
        // What matters is we don't false-reject "Sarah Smith".
        assert!(looks_like_person("Sunset Walk"));
        assert!(looks_like_person("Sarah Smith"));
    }
    #[test]
    fn cosine_similarity_known_vectors() {
@@ -2983,87 +3322,6 @@ mod tests {
        assert_eq!(faces[0].person_id, Some(alice.id));
    }
    #[test]
    fn list_embeddings_filters_by_person_id() {
        // Apollo's similar-unassigned suggester relies on this filter to
        // pull a single person's embeddings without paging the whole
        // detected set client-side. When person_id is set it must win
        // over `unassigned=true` (otherwise the IS NULL constraint would
        // always return an empty set for an assigned person).
        let mut dao = fresh_dao();
        diesel::sql_query(
            "INSERT OR IGNORE INTO libraries (id, name, root_path, created_at) \
             VALUES (1, 'main', '/tmp', 0)",
        )
        .execute(dao.connection.lock().unwrap().deref_mut())
        .expect("seed libraries");
        let alice = dao
            .create_person(
                &ctx(),
                &CreatePersonReq {
                    name: "Alice".into(),
                    notes: None,
                    entity_id: None,
                    is_ignored: false,
                },
                false,
            )
            .unwrap();
        let bob = dao
            .create_person(
                &ctx(),
                &CreatePersonReq {
                    name: "Bob".into(),
                    notes: None,
                    entity_id: None,
                    is_ignored: false,
                },
                false,
            )
            .unwrap();
        let mk_row = |hash: &str, person: Option<i32>| InsertFaceDetectionInput {
            library_id: 1,
            content_hash: hash.into(),
            rel_path: format!("{hash}.jpg"),
            bbox: Some((0.1, 0.1, 0.2, 0.2)),
            embedding: Some(vec![0u8; 2048]),
            confidence: Some(0.9),
            source: "auto".into(),
            person_id: person,
            status: "detected".into(),
            model_version: "buffalo_l".into(),
        };
        dao.store_detection(&ctx(), mk_row("a1", Some(alice.id)))
            .unwrap();
        dao.store_detection(&ctx(), mk_row("a2", Some(alice.id)))
            .unwrap();
        dao.store_detection(&ctx(), mk_row("b1", Some(bob.id)))
            .unwrap();
        dao.store_detection(&ctx(), mk_row("u1", None)).unwrap();
        // person_id=alice returns only alice's two faces — ignoring the
        // (default-true) `unassigned` filter, which would have selected
        // u1 only.
        let alice_rows = dao
            .list_embeddings(&ctx(), None, true, Some(alice.id), 100, 0)
            .unwrap();
        assert_eq!(alice_rows.len(), 2);
        assert!(
            alice_rows
                .iter()
                .all(|(r, _)| r.person_id == Some(alice.id))
        );
        // unassigned=true with no person_id behaves as before.
        let unassigned_rows = dao
            .list_embeddings(&ctx(), None, true, None, 100, 0)
            .unwrap();
        assert_eq!(unassigned_rows.len(), 1);
        assert_eq!(unassigned_rows[0].0.content_hash, "u1");
    }
    // ── crop_image_to_bbox ──────────────────────────────────────────────
    // Pure helper used by the manual face-create handler. Generate a tiny
    // image in memory, write it to a temp file, then exercise the bbox
@@ -53,7 +53,6 @@ pub fn walk_library_files(base_path: &Path, excluded_dirs: &[String]) -> Vec<Dir
 /// used by the watcher's quick-scan tick to skip the long tail. Files
 /// whose metadata can't be read are kept; the caller's batch EXIF lookup
 /// dedups against existing rows.
 #[allow(dead_code)]
 pub fn enumerate_indexable_files(
    base_path: &Path,
    excluded_dirs: &[String],
@@ -22,42 +22,8 @@ pub fn needs_ffmpeg_thumbnail(path: &Path) -> bool {
 /// Supported video file extensions
 pub const VIDEO_EXTENSIONS: &[&str] = &["mp4", "mov", "avi", "mkv"];
 /// Audio file extensions accepted as voice-clone references (TTS). Mirrors
 /// the formats Chatterbox can decode (wav/mp3/flac/m4a/aac/ogg).
 pub const AUDIO_EXTENSIONS: &[&str] = &["wav", "mp3", "flac", "m4a", "aac", "ogg", "oga", "opus"];
 /// Filenames that are filesystem metadata, not real media — exact
 /// basename match. Extend if a new platform sidecar appears (Windows
 /// Thumbs.db / desktop.ini live here too if those libraries land).
 const METADATA_FILENAMES: &[&str] = &[".DS_Store"];
 /// True if the basename is a filesystem metadata sidecar that should be
 /// invisible to every media predicate.
 ///
 /// macOS writes `._<name>` AppleDouble companions when copying to
 /// non-HFS volumes — each holds the extended attributes of `<name>`,
 /// NOT a copy of the bytes. Same extension as the real file, so a
 /// pure-extension match treats `._photo.jpg` as a JPEG, ships it to
 /// the decoder, and accumulates failed rows: face_detections
 /// `status='failed'`, clip_embedding `status='failed'`, plus a
 /// pointless `image_exif` row whose `content_hash` will be the hash
 /// of the metadata blob. The downstream noise (failed-row counts that
 /// never go to zero, 422 bursts to Apollo, evictor timer reset by
 /// those 422s) is the visible damage. `.DS_Store` is the per-directory
 /// version (Finder view state) — no extension, but cheap to guard
 /// here too in case some future predicate matches by content type.
 pub fn is_filesystem_metadata(path: &Path) -> bool {
    let Some(name) = path.file_name().and_then(|n| n.to_str()) else {
        return false;
    };
    name.starts_with("._") || METADATA_FILENAMES.contains(&name)
 }
 /// Check if a path has an image extension
 pub fn is_image_file(path: &Path) -> bool {
    if is_filesystem_metadata(path) {
        return false;
    }
    if let Some(ext) = path.extension().and_then(|e| e.to_str()) {
        let ext_lower = ext.to_lowercase();
        IMAGE_EXTENSIONS.contains(&ext_lower.as_str())
@@ -68,9 +34,6 @@ pub fn is_image_file(path: &Path) -> bool {
 /// Check if a path has a video extension
 pub fn is_video_file(path: &Path) -> bool {
    if is_filesystem_metadata(path) {
        return false;
    }
    if let Some(ext) = path.extension().and_then(|e| e.to_str()) {
        let ext_lower = ext.to_lowercase();
        VIDEO_EXTENSIONS.contains(&ext_lower.as_str())
@@ -79,19 +42,6 @@ pub fn is_video_file(path: &Path) -> bool {
    }
 }
 /// Check if a path has an audio extension (voice-clone references)
 pub fn is_audio_file(path: &Path) -> bool {
    if is_filesystem_metadata(path) {
        return false;
    }
    if let Some(ext) = path.extension().and_then(|e| e.to_str()) {
        let ext_lower = ext.to_lowercase();
        AUDIO_EXTENSIONS.contains(&ext_lower.as_str())
    } else {
        false
    }
 }
 /// Check if a path has a supported media extension (image or video)
 pub fn is_media_file(path: &Path) -> bool {
    is_image_file(path) || is_video_file(path)
@@ -148,46 +98,4 @@ mod tests {
        assert!(!is_media_file(Path::new("document.txt")));
        assert!(!is_media_file(Path::new("no_extension")));
    }
    #[test]
    fn test_apple_double_excluded_from_media() {
        // The bug-of-record: ImageApi was shipping macOS AppleDouble
        // sidecars to Apollo's CLIP/face decoders, accumulating failed
        // rows and pinning Apollo's eviction timer with the 422 burst.
        // Predicate-level guard means every downstream walker
        // (face_watch, backfill, clip_watch, watcher) inherits the fix
        // without touching their filters.
        assert!(!is_image_file(Path::new("._photo.jpg")));
        assert!(!is_image_file(Path::new("dir/._photo.JPG")));
        assert!(!is_image_file(Path::new("a/b/._DSC_2182-S.jpg")));
        assert!(!is_video_file(Path::new("._video.mp4")));
        assert!(!is_media_file(Path::new("._photo.png")));
        // A real file that merely starts with "_" (no leading dot) is
        // not AppleDouble — must NOT be filtered.
        assert!(is_image_file(Path::new("_photo.jpg")));
    }
    #[test]
    fn test_ds_store_excluded() {
        // Finder per-directory metadata. No image extension so
        // is_image_file would already say false; the guard makes the
        // predicate's *reason* explicit and covers a hypothetical
        // future caller matching by basename.
        assert!(!is_image_file(Path::new(".DS_Store")));
        assert!(!is_video_file(Path::new(".DS_Store")));
        assert!(!is_media_file(Path::new("some/dir/.DS_Store")));
        assert!(is_filesystem_metadata(Path::new(".DS_Store")));
        assert!(is_filesystem_metadata(Path::new("dir/.DS_Store")));
    }
    #[test]
    fn test_dotfiles_other_than_apple_double_are_unaffected() {
        // We deliberately scope to `._*` + the exact .DS_Store name —
        // not all dotfiles — because a user could plausibly name a
        // cover image `.cover.jpg` and we shouldn't silently drop it.
        // If that turns out to be wrong, broaden here; for now,
        // narrow + explicit > broad + surprising.
        assert!(is_image_file(Path::new(".cover.jpg")));
        assert!(!is_filesystem_metadata(Path::new(".cover.jpg")));
    }
 }
@@ -10,7 +10,6 @@ use std::path::{Path, PathBuf};
 use std::sync::Mutex;
 use std::time::SystemTime;
 use crate::AppState;
 use crate::data::{
    Claims, ExifBatchRequest, ExifBatchResponse, ExifSummary, FilesRequest, FilterMode, MediaType,
    PhotosResponse, SortType,
@@ -19,8 +18,8 @@ use crate::database::ExifDao;
 use crate::file_types;
 use crate::geo::{gps_bounding_box, haversine_distance};
 use crate::memories::extract_date_from_filename;
 use crate::thumbnails::create_thumbnails;
 use crate::utils::earliest_fs_time;
 use crate::{AppState, create_thumbnails};
 use actix_web::web::Data;
 use actix_web::{
    HttpRequest, HttpResponse,
@@ -1511,8 +1510,6 @@ mod tests {
            date_taken_source,
            original_date_taken: None,
            original_date_taken_source: None,
            clip_embedding: None,
            clip_model_version: None,
        }
    }
@@ -1552,8 +1549,6 @@ mod tests {
                date_taken_source: data.date_taken_source.clone(),
                original_date_taken: None,
                original_date_taken_source: None,
                clip_embedding: None,
                clip_model_version: None,
            })
        }
@@ -1600,8 +1595,6 @@ mod tests {
                date_taken_source: data.date_taken_source.clone(),
                original_date_taken: None,
                original_date_taken_source: None,
                clip_embedding: None,
                clip_model_version: None,
            })
        }
@@ -1695,21 +1688,6 @@ mod tests {
            Ok(())
        }
        fn list_distinct_content_hashes(
            &mut self,
            _context: &opentelemetry::Context,
        ) -> Result<Vec<String>, DbError> {
            Ok(Vec::new())
        }
        fn list_paths_and_hashes_for_library(
            &mut self,
            _context: &opentelemetry::Context,
            _library_id: i32,
        ) -> Result<Vec<(String, Option<String>)>, DbError> {
            Ok(Vec::new())
        }
        fn get_rows_needing_date_backfill(
            &mut self,
            _context: &opentelemetry::Context,
@@ -1938,35 +1916,6 @@ mod tests {
        ) -> Result<(), DbError> {
            Ok(())
        }
        fn list_clip_unencoded_candidates(
            &mut self,
            _context: &opentelemetry::Context,
            _library_id: i32,
            _limit: i64,
        ) -> Result<Vec<(String, String)>, DbError> {
            Ok(Vec::new())
        }
        fn backfill_clip_embedding(
            &mut self,
            _context: &opentelemetry::Context,
            _library_id: i32,
            _rel_path: &str,
            _embedding: &[u8],
            _model_version: &str,
        ) -> Result<(), DbError> {
            Ok(())
        }
        fn list_clip_index(
            &mut self,
            _context: &opentelemetry::Context,
            _library_ids: &[i32],
            _model_version: Option<&str>,
        ) -> Result<Vec<(String, Vec<u8>)>, DbError> {
            Ok(Vec::new())
        }
    }
    mod api {
@@ -1,128 +0,0 @@
 //! User-favorites endpoints. Favorites are keyed on `(user_id, rel_path)`
 //! and shared across libraries — a favorite created in lib1 is visible
 //! under lib2 if the same rel_path resolves there too.
 use std::sync::Mutex;
 use actix_web::{
    HttpRequest, HttpResponse, Responder, delete, get, put,
    web::{self, Data},
 };
 use log::{error, info, warn};
 use opentelemetry::trace::{Span, Status, Tracer};
 use crate::data::{AddFavoriteRequest, Claims, PhotosResponse};
 use crate::database::{DbError, DbErrorKind, FavoriteDao};
 use crate::otel::{extract_context_from_request, global_tracer};
 #[get("image/favorites")]
 pub async fn favorites(
    claims: Claims,
    request: HttpRequest,
    favorites_dao: Data<Mutex<Box<dyn FavoriteDao>>>,
 ) -> impl Responder {
    let tracer = global_tracer();
    let context = extract_context_from_request(&request);
    let mut span = tracer.start_with_context("get favorites", &context);
    match web::block(move || {
        favorites_dao
            .lock()
            .expect("Unable to get FavoritesDao")
            .get_favorites(claims.sub.parse::<i32>().unwrap())
    })
    .await
    {
        Ok(Ok(favorites)) => {
            let favorites = favorites
                .into_iter()
                .map(|favorite| favorite.path)
                .collect::<Vec<String>>();
            span.set_status(Status::Ok);
            // Favorites are library-agnostic (shared by rel_path), so we
            // intentionally leave photo_libraries empty to signal "no badge".
            HttpResponse::Ok().json(PhotosResponse {
                photos: favorites,
                dirs: Vec::new(),
                photo_libraries: Vec::new(),
                total_count: None,
                has_more: None,
                next_offset: None,
            })
        }
        Ok(Err(e)) => {
            span.set_status(Status::error(format!("Error getting favorites: {:?}", e)));
            error!("Error getting favorites: {:?}", e);
            HttpResponse::InternalServerError().finish()
        }
        Err(_) => HttpResponse::InternalServerError().finish(),
    }
 }
 #[put("image/favorites")]
 pub async fn put_add_favorite(
    claims: Claims,
    body: web::Json<AddFavoriteRequest>,
    favorites_dao: Data<Mutex<Box<dyn FavoriteDao>>>,
 ) -> impl Responder {
    if let Ok(user_id) = claims.sub.parse::<i32>() {
        let path = body.path.clone();
        match web::block::<_, Result<usize, DbError>>(move || {
            favorites_dao
                .lock()
                .expect("Unable to get FavoritesDao")
                .add_favorite(user_id, &path)
        })
        .await
        {
            Ok(Err(e)) if e.kind == DbErrorKind::AlreadyExists => {
                warn!("Favorite: {} exists for user: {}", &body.path, user_id);
                HttpResponse::Ok()
            }
            Ok(Err(e)) => {
                error!("{:?} {}. for user: {}", e, body.path, user_id);
                HttpResponse::BadRequest()
            }
            Ok(Ok(_)) => {
                info!("Adding favorite \"{}\" for userid: {}", body.path, user_id);
                HttpResponse::Created()
            }
            Err(e) => {
                error!("Blocking error while inserting favorite: {:?}", e);
                HttpResponse::InternalServerError()
            }
        }
    } else {
        error!("Unable to parse sub as i32: {}", claims.sub);
        HttpResponse::BadRequest()
    }
 }
 #[delete("image/favorites")]
 pub async fn delete_favorite(
    claims: Claims,
    body: web::Query<AddFavoriteRequest>,
    favorites_dao: Data<Mutex<Box<dyn FavoriteDao>>>,
 ) -> impl Responder {
    if let Ok(user_id) = claims.sub.parse::<i32>() {
        let path = body.path.clone();
        web::block(move || {
            favorites_dao
                .lock()
                .expect("Unable to get favorites dao")
                .remove_favorite(user_id, path);
        })
        .await
        .unwrap();
        info!(
            "Removing favorite \"{}\" for userid: {}",
            body.path, user_id
        );
        HttpResponse::Ok()
    } else {
        error!("Unable to parse sub as i32: {}", claims.sub);
        HttpResponse::BadRequest()
    }
 }
@@ -1,9 +0,0 @@
 //! HTTP route handlers, grouped by domain.
 //!
 //! These were previously inlined in `main.rs`; moving them out keeps
 //! `main()` focused on startup wiring and makes each domain
 //! independently testable with `actix_web::test::init_service`.
 pub mod favorites;
 pub mod image;
 pub mod video;
@@ -1,879 +0,0 @@
 //! Video-related endpoints: HLS playlist generation, segment streaming,
 //! and the short-clip preview pipeline.
 use std::collections::HashMap;
 use std::path::PathBuf;
 use std::sync::Mutex;
 use actix_files::NamedFile;
 use actix_web::{
    HttpRequest, HttpResponse, Responder, get, post,
    web::{self, Data},
 };
 use log::{debug, error, info, warn};
 use opentelemetry::KeyValue;
 use opentelemetry::trace::{Span, Status, Tracer};
 use serde::Serialize;
 use crate::content_hash;
 use crate::data::{
    Claims, PreviewClipRequest, PreviewStatusItem, PreviewStatusRequest, PreviewStatusResponse,
    ThumbnailRequest,
 };
 use crate::database::{ExifDao, PreviewDao};
 use crate::files::is_valid_full_path;
 use crate::libraries;
 use crate::otel::{extract_context_from_request, global_tracer};
 use crate::state::AppState;
 use crate::video::actors::{
    GeneratePreviewClipMessage, QueueVideosMessage, VideoToQueue, probe_video_stream_meta,
 };
 use crate::video::hls_paths;
 /// Response body for `POST /video/generate`. Clients consume
 /// `playlist_url` (hash-keyed, stable across libraries and renames)
 /// and poll for readiness via the URL itself.
 #[derive(Serialize, Debug)]
 struct GenerateVideoResponse {
    /// Hash-keyed URL to the HLS playlist. Resolves to
    /// `$VIDEO_PATH/<shard>/<hash>/playlist.m3u8` server-side. Relative
    /// segment refs inside the playlist resolve correctly because the
    /// browser appends to this URL's path.
    playlist_url: String,
    /// blake3 content hash of the source video. Stable per byte content,
    /// so duplicate uploads / archive ingests share one set of HLS
    /// output.
    content_hash: String,
    /// `true` iff the playlist file is already on disk. `false` means a
    /// transcode was queued; clients should retry the URL after a short
    /// delay (or rely on HLS.js's own retry policy).
    ready: bool,
    /// Source-video frame rate in Hz, probed via ffprobe. `None` when the
    /// probe failed or ffprobe couldn't parse either rate field — clients
    /// fall back to their own default (typically 30) for frame stepping.
    #[serde(skip_serializing_if = "Option::is_none")]
    frame_rate: Option<f32>,
 }
 #[post("/video/generate")]
 pub async fn generate_video(
    _claims: Claims,
    request: HttpRequest,
    app_state: Data<AppState>,
    exif_dao: Data<std::sync::Mutex<Box<dyn ExifDao>>>,
    body: web::Json<ThumbnailRequest>,
 ) -> impl Responder {
    let tracer = global_tracer();
    let context = extract_context_from_request(&request);
    let mut span = tracer.start_with_context("generate_video", &context);
    let preferred_library = libraries::resolve_library_param(&app_state, body.library.as_deref())
        .ok()
        .flatten()
        .unwrap_or_else(|| app_state.primary_library());
    // Try the resolved library first, then fall back to any other library
    // that actually contains the file — handles union-mode requests where
    // the mobile client passes no library but the file lives in a
    // non-primary library. Track which library won so the DB lookup is
    // scoped correctly.
    let resolved = is_valid_full_path(&preferred_library.root_path, &body.path, false)
        .filter(|p| p.exists())
        .map(|p| (preferred_library.id, preferred_library.root_path.clone(), p))
        .or_else(|| {
            app_state.libraries.iter().find_map(|lib| {
                if lib.id == preferred_library.id {
                    return None;
                }
                is_valid_full_path(&lib.root_path, &body.path, false)
                    .filter(|p| p.exists())
                    .map(|p| (lib.id, lib.root_path.clone(), p))
            })
        });
    let Some((resolved_library_id, resolved_root, full_path)) = resolved else {
        span.set_status(Status::error(format!("invalid path {:?}", &body.path)));
        return HttpResponse::BadRequest().finish();
    };
    // Build the rel_path used to look up the row. Forward-slash normalized
    // so the lookup matches DB rows on Windows — see `rel_path_for_lookup`.
    let full_path_str = full_path.to_string_lossy().to_string();
    let rel_path = rel_path_for_lookup(&full_path_str, &resolved_root);
    // DB lookup first. Cheap and avoids re-reading the file off disk for
    // already-ingested videos.
    let hash_from_db: Option<String> = {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        match dao.get_exif_batch(
            &context,
            Some(resolved_library_id),
            std::slice::from_ref(&rel_path),
        ) {
            Ok(rows) => rows.into_iter().next().and_then(|r| r.content_hash),
            Err(e) => {
                warn!(
                    "exif_dao.get_exif_batch failed for {} (lib {}): {:?}",
                    rel_path, resolved_library_id, e
                );
                None
            }
        }
    };
    // Best-effort fallback: compute on-the-fly when the DB row hasn't
    // been written or is mid-backfill. Read-only — no library mutation.
    let content_hash_str = match hash_from_db {
        Some(h) => h,
        None => match content_hash::compute(&full_path) {
            Ok(id) => id.content_hash,
            Err(e) => {
                error!(
                    "Failed to compute content_hash for {}: {}",
                    full_path.display(),
                    e
                );
                span.set_status(Status::error(format!("hash compute failed: {}", e)));
                return HttpResponse::InternalServerError().finish();
            }
        },
    };
    let video_dir = std::path::Path::new(&app_state.video_path);
    let playlist_path = hls_paths::playlist_for_hash(video_dir, &content_hash_str);
    let sentinel_path = hls_paths::sentinel_for_hash(video_dir, &content_hash_str);
    let ready = playlist_path.exists();
    if !ready && !sentinel_path.exists() {
        // Kick off generation via the existing actor pipeline. Fire-and-
        // forget — the playlist appears at `playlist_path` once ffmpeg
        // + rename complete. The client polls the URL.
        info!(
            "/video/generate: queueing playlist for {} (hash={})",
            full_path.display(),
            &content_hash_str[..content_hash_str.len().min(16)]
        );
        app_state.playlist_manager.do_send(QueueVideosMessage {
            videos: vec![VideoToQueue {
                video_path: full_path.clone(),
                content_hash: content_hash_str.clone(),
            }],
        });
        span.add_event(
            "playlist_queued",
            vec![KeyValue::new("content_hash", content_hash_str.clone())],
        );
    } else if ready {
        span.add_event(
            "playlist_already_present",
            vec![KeyValue::new("content_hash", content_hash_str.clone())],
        );
    } else {
        // Sentinel present — past transcode attempt failed. Return the
        // URL anyway (it'll 404 / 5xx at fetch time) so the client gets
        // a deterministic answer. Operator must delete the sentinel to
        // force a retry.
        warn!(
            "/video/generate: unsupported sentinel present for {} (hash={}); not re-queueing",
            full_path.display(),
            &content_hash_str[..content_hash_str.len().min(16)]
        );
    }
    let playlist_url = format!(
        "/video/hls/{}/{}",
        content_hash_str,
        hls_paths::PLAYLIST_FILENAME
    );
    // Probe the source for frame rate so the mobile scrubber can step at
    // the right interval. Cheap (~tens of ms) and only runs once per video
    // open. Probe failures degrade silently — clients have a fallback.
    let frame_rate = probe_video_stream_meta(&full_path.to_string_lossy())
        .await
        .frame_rate;
    span.set_status(Status::Ok);
    HttpResponse::Ok().json(GenerateVideoResponse {
        playlist_url,
        content_hash: content_hash_str,
        ready,
        frame_rate,
    })
 }
 /// Serve HLS playlist or segment files under the hash-keyed layout
 /// `$VIDEO_PATH/<shard>/<hash>/<file>`. The matched `{file}` must be
 /// either `playlist.m3u8` or a `segment_NNN.ts` style segment; any other
 /// shape is 400'd to defend against operators stashing other content in
 /// the hash dir.
 #[get("/video/hls/{hash}/{file}")]
 pub async fn stream_hls_file(
    request: HttpRequest,
    _: Claims,
    path: web::Path<(String, String)>,
    app_state: Data<AppState>,
 ) -> impl Responder {
    let tracer = global_tracer();
    let context = extract_context_from_request(&request);
    let mut span = tracer.start_with_context("stream_hls_file", &context);
    let (hash, file) = path.into_inner();
    if !is_valid_hash(&hash) {
        span.set_status(Status::error("invalid hash"));
        return HttpResponse::BadRequest().body("invalid hash");
    }
    if !is_allowed_hls_filename(&file) {
        span.set_status(Status::error("invalid file"));
        return HttpResponse::BadRequest().body("invalid file");
    }
    let shard = &hash[..2];
    let file_path = PathBuf::from(&app_state.video_path)
        .join(shard)
        .join(&hash)
        .join(&file);
    // Path-traversal guard: canonicalize both sides and require the file
    // to live under `app_state.video_path`. `is_valid_hash` /
    // `is_allowed_hls_filename` already block dangerous strings, but
    // belt-and-suspenders here is cheap.
    let canonical_base = match std::fs::canonicalize(&app_state.video_path) {
        Ok(p) => p,
        Err(e) => {
            error!("Failed to canonicalize VIDEO_PATH: {:?}", e);
            span.set_status(Status::error("VIDEO_PATH not canonicalisable"));
            return HttpResponse::InternalServerError().finish();
        }
    };
    let canonical_file = match std::fs::canonicalize(&file_path) {
        Ok(p) => p,
        Err(_) => {
            debug!("HLS file not found: {}", file_path.display());
            span.set_status(Status::error("not found"));
            return HttpResponse::NotFound().finish();
        }
    };
    if !canonical_file.starts_with(&canonical_base) {
        warn!(
            "Path traversal attempt: {} resolved outside VIDEO_PATH",
            file_path.display()
        );
        span.set_status(Status::error("traversal"));
        return HttpResponse::Forbidden().finish();
    }
    match NamedFile::open(&canonical_file) {
        Ok(f) => {
            span.set_status(Status::Ok);
            f.into_response(&request)
        }
        Err(_) => {
            span.set_status(Status::error("not found"));
            HttpResponse::NotFound().finish()
        }
    }
 }
 /// 64 lowercase-or-upper hex chars. Strict so we don't accept arbitrary
 /// strings that might canonicalize into trouble.
 fn is_valid_hash(s: &str) -> bool {
    s.len() == 64 && s.bytes().all(|b| b.is_ascii_hexdigit())
 }
 /// Compute the forward-slash `rel_path` used to look up a video's
 /// `image_exif` row, from its absolute path string and the library root.
 ///
 /// Normalizing to forward slashes is essential on Windows: `file_scan`
 /// stores rel_paths forward-slash regardless of OS, but a raw strip of a
 /// backslash Windows path (`Z:\...\pic\Melissa\clip.mp4`) yields
 /// `Melissa\clip.mp4`. `get_exif_batch` does an exact match with no
 /// normalization, so the backslash form misses and the handler falls back
 /// to re-hashing the entire file on every request.
 fn rel_path_for_lookup(full_path_str: &str, resolved_root: &str) -> String {
    full_path_str
        .strip_prefix(resolved_root)
        .unwrap_or(full_path_str)
        .trim_start_matches(['/', '\\'])
        .replace('\\', "/")
 }
 /// Allowed file names inside a hash dir. `playlist.m3u8` plus segment
 /// files matching the `segment_NNN.ts` template that `PlaylistGenerator`
 /// writes via `hls_paths::SEGMENT_TEMPLATE`. Anything else (including
 /// `.tmp`, `.unsupported`, dotfiles) returns 400 — these are internal
 /// artifacts the client should never request.
 fn is_allowed_hls_filename(name: &str) -> bool {
    if name == hls_paths::PLAYLIST_FILENAME {
        return true;
    }
    if let Some(rest) = name.strip_prefix("segment_")
        && let Some(num) = rest.strip_suffix(".ts")
        && !num.is_empty()
        && num.bytes().all(|b| b.is_ascii_digit())
    {
        return true;
    }
    false
 }
 #[get("/video/preview")]
 pub async fn get_video_preview(
    _claims: Claims,
    request: HttpRequest,
    req: web::Query<PreviewClipRequest>,
    app_state: Data<AppState>,
    preview_dao: Data<Mutex<Box<dyn PreviewDao>>>,
 ) -> impl Responder {
    let tracer = global_tracer();
    let context = extract_context_from_request(&request);
    let mut span = tracer.start_with_context("get_video_preview", &context);
    // Validate path
    let full_path = match is_valid_full_path(&app_state.base_path, &req.path, true) {
        Some(path) => path,
        None => {
            span.set_status(Status::error("Invalid path"));
            return HttpResponse::BadRequest().json(serde_json::json!({"error": "Invalid path"}));
        }
    };
    let full_path_str = full_path.to_string_lossy().to_string();
    // Use relative path (from BASE_PATH) for DB storage, consistent with EXIF convention
    let relative_path = full_path_str
        .strip_prefix(&app_state.base_path)
        .unwrap_or(&full_path_str)
        .trim_start_matches(['/', '\\'])
        .to_string();
    // Check preview status in DB
    let preview = {
        let mut dao = preview_dao.lock().expect("Unable to lock PreviewDao");
        dao.get_preview(&context, &relative_path)
    };
    match preview {
        Ok(Some(clip)) => match clip.status.as_str() {
            "complete" => {
                let preview_path = PathBuf::from(&app_state.preview_clips_path)
                    .join(&relative_path)
                    .with_extension("mp4");
                match NamedFile::open(&preview_path) {
                    Ok(file) => {
                        span.set_status(Status::Ok);
                        file.into_response(&request)
                    }
                    Err(_) => {
                        // File missing on disk but DB says complete - reset and regenerate
                        let mut dao = preview_dao.lock().expect("Unable to lock PreviewDao");
                        let _ = dao.update_status(
                            &context,
                            &relative_path,
                            "pending",
                            None,
                            None,
                            None,
                        );
                        app_state
                            .preview_clip_generator
                            .do_send(GeneratePreviewClipMessage {
                                video_path: full_path_str,
                            });
                        span.set_status(Status::Ok);
                        HttpResponse::Accepted().json(serde_json::json!({
                            "status": "processing",
                            "path": req.path
                        }))
                    }
                }
            }
            "processing" => {
                span.set_status(Status::Ok);
                HttpResponse::Accepted().json(serde_json::json!({
                    "status": "processing",
                    "path": req.path
                }))
            }
            "failed" => {
                let error_msg = clip
                    .error_message
                    .unwrap_or_else(|| "Unknown error".to_string());
                span.set_status(Status::error(format!("Generation failed: {}", error_msg)));
                HttpResponse::InternalServerError().json(serde_json::json!({
                    "error": format!("Generation failed: {}", error_msg)
                }))
            }
            _ => {
                // pending or unknown status - trigger generation
                app_state
                    .preview_clip_generator
                    .do_send(GeneratePreviewClipMessage {
                        video_path: full_path_str,
                    });
                span.set_status(Status::Ok);
                HttpResponse::Accepted().json(serde_json::json!({
                    "status": "processing",
                    "path": req.path
                }))
            }
        },
        Ok(None) => {
            // No record exists - insert as pending and trigger generation
            {
                let mut dao = preview_dao.lock().expect("Unable to lock PreviewDao");
                let _ = dao.insert_preview(&context, &relative_path, "pending");
            }
            app_state
                .preview_clip_generator
                .do_send(GeneratePreviewClipMessage {
                    video_path: full_path_str,
                });
            span.set_status(Status::Ok);
            HttpResponse::Accepted().json(serde_json::json!({
                "status": "processing",
                "path": req.path
            }))
        }
        Err(_) => {
            span.set_status(Status::error("Database error"));
            HttpResponse::InternalServerError().json(serde_json::json!({"error": "Database error"}))
        }
    }
 }
 #[post("/video/preview/status")]
 pub async fn get_preview_status(
    _claims: Claims,
    request: HttpRequest,
    body: web::Json<PreviewStatusRequest>,
    app_state: Data<AppState>,
    preview_dao: Data<Mutex<Box<dyn PreviewDao>>>,
 ) -> impl Responder {
    let tracer = global_tracer();
    let context = extract_context_from_request(&request);
    let mut span = tracer.start_with_context("get_preview_status", &context);
    // Limit to 200 paths per request
    if body.paths.len() > 200 {
        span.set_status(Status::error("Too many paths"));
        return HttpResponse::BadRequest()
            .json(serde_json::json!({"error": "Maximum 200 paths per request"}));
    }
    let previews = {
        let mut dao = preview_dao.lock().expect("Unable to lock PreviewDao");
        dao.get_previews_batch(&context, &body.paths)
    };
    match previews {
        Ok(clips) => {
            // Build a map of file_path -> VideoPreviewClip for quick lookup
            let clip_map: HashMap<String, _> = clips
                .into_iter()
                .map(|clip| (clip.file_path.clone(), clip))
                .collect();
            let mut items: Vec<PreviewStatusItem> = Vec::with_capacity(body.paths.len());
            for path in &body.paths {
                if let Some(clip) = clip_map.get(path) {
                    // Re-queue generation for stale pending/failed records
                    if clip.status == "pending" || clip.status == "failed" {
                        let full_path = format!(
                            "{}/{}",
                            app_state.base_path.trim_end_matches(['/', '\\']),
                            path.trim_start_matches(['/', '\\'])
                        );
                        app_state
                            .preview_clip_generator
                            .do_send(GeneratePreviewClipMessage {
                                video_path: full_path,
                            });
                    }
                    items.push(PreviewStatusItem {
                        path: path.clone(),
                        status: clip.status.clone(),
                        preview_url: if clip.status == "complete" {
                            Some(format!("/video/preview?path={}", urlencoding::encode(path)))
                        } else {
                            None
                        },
                    });
                } else {
                    // No record exists — insert as pending and trigger generation
                    {
                        let mut dao = preview_dao.lock().expect("Unable to lock PreviewDao");
                        let _ = dao.insert_preview(&context, path, "pending");
                    }
                    // Build full path for ffmpeg (actor needs the absolute path for input)
                    let full_path = format!(
                        "{}/{}",
                        app_state.base_path.trim_end_matches(['/', '\\']),
                        path.trim_start_matches(['/', '\\'])
                    );
                    info!("Triggering preview generation for '{}'", path);
                    app_state
                        .preview_clip_generator
                        .do_send(GeneratePreviewClipMessage {
                            video_path: full_path,
                        });
                    items.push(PreviewStatusItem {
                        path: path.clone(),
                        status: "pending".to_string(),
                        preview_url: None,
                    });
                }
            }
            span.set_status(Status::Ok);
            HttpResponse::Ok().json(PreviewStatusResponse { previews: items })
        }
        Err(_) => {
            span.set_status(Status::error("Database error"));
            HttpResponse::InternalServerError().json(serde_json::json!({"error": "Database error"}))
        }
    }
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use crate::data::Claims;
    use crate::database::PreviewDao;
    use crate::testhelpers::TestPreviewDao;
    use actix_web::App;
    #[test]
    fn is_valid_hash_requires_64_ascii_hex() {
        assert!(is_valid_hash(&"a".repeat(64)));
        assert!(is_valid_hash(&"F".repeat(64)));
        assert!(is_valid_hash(&format!("ab{}", "0".repeat(62))));
        assert!(!is_valid_hash(&"a".repeat(63)));
        assert!(!is_valid_hash(&"a".repeat(65)));
        // Anything outside the hex alphabet — including '/', '.', '..' —
        // is rejected up front so the path-traversal canonicalisation
        // never has to defend the boundary alone.
        assert!(!is_valid_hash(&format!("/{}", "a".repeat(63))));
        assert!(!is_valid_hash(&format!("..{}", "a".repeat(62))));
        assert!(!is_valid_hash(&"g".repeat(64)));
    }
    #[test]
    fn is_allowed_hls_filename_accepts_only_playlist_and_segments() {
        assert!(is_allowed_hls_filename("playlist.m3u8"));
        assert!(is_allowed_hls_filename("segment_000.ts"));
        assert!(is_allowed_hls_filename("segment_999.ts"));
        assert!(is_allowed_hls_filename("segment_0.ts"));
        // Internal artifacts the client should never request.
        assert!(!is_allowed_hls_filename("playlist.m3u8.tmp"));
        assert!(!is_allowed_hls_filename("playlist.unsupported"));
        // Traversal / path components — defence in depth alongside
        // the actix path matcher itself.
        assert!(!is_allowed_hls_filename(".."));
        assert!(!is_allowed_hls_filename("../etc/passwd"));
        assert!(!is_allowed_hls_filename("segment_abc.ts"));
        assert!(!is_allowed_hls_filename("segment_.ts"));
        assert!(!is_allowed_hls_filename(""));
    }
    #[test]
    fn rel_path_for_lookup_normalizes_windows_separators() {
        // Windows: backslash root + backslash full path. The stored row is
        // forward-slash (`Melissa/clip.mp4`), so without normalization the
        // lookup misses and the handler re-hashes the whole file.
        assert_eq!(
            rel_path_for_lookup(r"Z:\Media\pic\Melissa\clip.mp4", r"Z:\Media\pic"),
            "Melissa/clip.mp4"
        );
    }
    #[test]
    fn rel_path_for_lookup_handles_unix_separators() {
        assert_eq!(
            rel_path_for_lookup("/media/pic/Melissa/clip.mp4", "/media/pic"),
            "Melissa/clip.mp4"
        );
    }
    #[test]
    fn rel_path_for_lookup_file_at_root_has_no_separator() {
        // A file directly in the library root has no internal separator, so
        // the bug never manifested here — guard against a regression anyway.
        assert_eq!(
            rel_path_for_lookup(r"Z:\Media\pic\clip.mp4", r"Z:\Media\pic"),
            "clip.mp4"
        );
        assert_eq!(
            rel_path_for_lookup("/media/pic/clip.mp4", "/media/pic"),
            "clip.mp4"
        );
    }
    #[test]
    fn rel_path_for_lookup_strips_leading_separators() {
        // Both separator styles are trimmed from the front after the root
        // is stripped, regardless of which form the join produced.
        assert_eq!(
            rel_path_for_lookup(r"Z:\Media\pic\sub\a.mp4", r"Z:\Media\pic"),
            "sub/a.mp4"
        );
        assert_eq!(
            rel_path_for_lookup("/media/pic//sub/a.mp4", "/media/pic"),
            "sub/a.mp4"
        );
    }
    #[test]
    fn rel_path_for_lookup_falls_back_when_root_does_not_match() {
        // If the root doesn't prefix the path (e.g. a stale mount), we keep
        // the whole path but still normalize separators rather than panic.
        assert_eq!(
            rel_path_for_lookup(r"D:\other\Melissa\clip.mp4", r"Z:\Media\pic"),
            "D:/other/Melissa/clip.mp4"
        );
    }
    fn make_token() -> String {
        let claims = Claims::valid_user("1".to_string());
        jsonwebtoken::encode(
            &jsonwebtoken::Header::default(),
            &claims,
            &jsonwebtoken::EncodingKey::from_secret(b"test_key"),
        )
        .unwrap()
    }
    fn make_preview_dao(dao: TestPreviewDao) -> Data<Mutex<Box<dyn PreviewDao>>> {
        Data::new(Mutex::new(Box::new(dao) as Box<dyn PreviewDao>))
    }
    #[actix_rt::test]
    async fn test_get_preview_status_returns_pending_for_unknown() {
        let dao = TestPreviewDao::new();
        let preview_dao = make_preview_dao(dao);
        let app_state = Data::new(AppState::test_state());
        let token = make_token();
        let app = actix_web::test::init_service(
            App::new()
                .service(get_preview_status)
                .app_data(app_state)
                .app_data(preview_dao.clone()),
        )
        .await;
        let req = actix_web::test::TestRequest::post()
            .uri("/video/preview/status")
            .insert_header(("Authorization", format!("Bearer {}", token)))
            .set_json(serde_json::json!({"paths": ["photos/new_video.mp4"]}))
            .to_request();
        let resp = actix_web::test::call_service(&app, req).await;
        assert_eq!(resp.status(), 200);
        let body: serde_json::Value = actix_web::test::read_body_json(resp).await;
        let previews = body["previews"].as_array().unwrap();
        assert_eq!(previews.len(), 1);
        assert_eq!(previews[0]["status"], "pending");
        // Verify the DAO now has a pending record
        let mut dao_lock = preview_dao.lock().unwrap();
        let ctx = opentelemetry::Context::new();
        let clip = dao_lock.get_preview(&ctx, "photos/new_video.mp4").unwrap();
        assert!(clip.is_some());
        assert_eq!(clip.unwrap().status, "pending");
    }
    #[actix_rt::test]
    async fn test_get_preview_status_returns_complete_with_url() {
        let mut dao = TestPreviewDao::new();
        let ctx = opentelemetry::Context::new();
        dao.insert_preview(&ctx, "photos/done.mp4", "pending")
            .unwrap();
        dao.update_status(
            &ctx,
            "photos/done.mp4",
            "complete",
            Some(9.5),
            Some(500000),
            None,
        )
        .unwrap();
        let preview_dao = make_preview_dao(dao);
        let app_state = Data::new(AppState::test_state());
        let token = make_token();
        let app = actix_web::test::init_service(
            App::new()
                .service(get_preview_status)
                .app_data(app_state)
                .app_data(preview_dao),
        )
        .await;
        let req = actix_web::test::TestRequest::post()
            .uri("/video/preview/status")
            .insert_header(("Authorization", format!("Bearer {}", token)))
            .set_json(serde_json::json!({"paths": ["photos/done.mp4"]}))
            .to_request();
        let resp = actix_web::test::call_service(&app, req).await;
        assert_eq!(resp.status(), 200);
        let body: serde_json::Value = actix_web::test::read_body_json(resp).await;
        let previews = body["previews"].as_array().unwrap();
        assert_eq!(previews.len(), 1);
        assert_eq!(previews[0]["status"], "complete");
        assert!(
            previews[0]["preview_url"]
                .as_str()
                .unwrap()
                .contains("photos%2Fdone.mp4")
        );
    }
    #[actix_rt::test]
    async fn test_get_preview_status_rejects_over_200_paths() {
        let dao = TestPreviewDao::new();
        let preview_dao = make_preview_dao(dao);
        let app_state = Data::new(AppState::test_state());
        let token = make_token();
        let app = actix_web::test::init_service(
            App::new()
                .service(get_preview_status)
                .app_data(app_state)
                .app_data(preview_dao),
        )
        .await;
        let paths: Vec<String> = (0..201).map(|i| format!("video_{}.mp4", i)).collect();
        let req = actix_web::test::TestRequest::post()
            .uri("/video/preview/status")
            .insert_header(("Authorization", format!("Bearer {}", token)))
            .set_json(serde_json::json!({"paths": paths}))
            .to_request();
        let resp = actix_web::test::call_service(&app, req).await;
        assert_eq!(resp.status(), 400);
    }
    #[actix_rt::test]
    async fn test_get_preview_status_mixed_statuses() {
        let mut dao = TestPreviewDao::new();
        let ctx = opentelemetry::Context::new();
        dao.insert_preview(&ctx, "a.mp4", "pending").unwrap();
        dao.insert_preview(&ctx, "b.mp4", "pending").unwrap();
        dao.update_status(&ctx, "b.mp4", "complete", Some(10.0), Some(100000), None)
            .unwrap();
        let preview_dao = make_preview_dao(dao);
        let app_state = Data::new(AppState::test_state());
        let token = make_token();
        let app = actix_web::test::init_service(
            App::new()
                .service(get_preview_status)
                .app_data(app_state)
                .app_data(preview_dao),
        )
        .await;
        let req = actix_web::test::TestRequest::post()
            .uri("/video/preview/status")
            .insert_header(("Authorization", format!("Bearer {}", token)))
            .set_json(serde_json::json!({"paths": ["a.mp4", "b.mp4", "c.mp4"]}))
            .to_request();
        let resp = actix_web::test::call_service(&app, req).await;
        assert_eq!(resp.status(), 200);
        let body: serde_json::Value = actix_web::test::read_body_json(resp).await;
        let previews = body["previews"].as_array().unwrap();
        assert_eq!(previews.len(), 3);
        // a.mp4 is pending
        assert_eq!(previews[0]["path"], "a.mp4");
        assert_eq!(previews[0]["status"], "pending");
        // b.mp4 is complete with URL
        assert_eq!(previews[1]["path"], "b.mp4");
        assert_eq!(previews[1]["status"], "complete");
        assert!(previews[1]["preview_url"].is_string());
        // c.mp4 was not found — handler inserts pending
        assert_eq!(previews[2]["path"], "c.mp4");
        assert_eq!(previews[2]["status"], "pending");
    }
    /// Verifies that the status endpoint re-queues generation for stale
    /// "pending" and "failed" records (e.g., after a server restart or
    /// when clip files were deleted). The do_send to the actor exercises
    /// the re-queue code path; the actor runs against temp dirs so it
    /// won't panic.
    #[actix_rt::test]
    async fn test_get_preview_status_requeues_pending_and_failed() {
        let mut dao = TestPreviewDao::new();
        let ctx = opentelemetry::Context::new();
        // Simulate stale records left from a previous server run
        dao.insert_preview(&ctx, "stale/pending.mp4", "pending")
            .unwrap();
        dao.insert_preview(&ctx, "stale/failed.mp4", "pending")
            .unwrap();
        dao.update_status(
            &ctx,
            "stale/failed.mp4",
            "failed",
            None,
            None,
            Some("ffmpeg error"),
        )
        .unwrap();
        let preview_dao = make_preview_dao(dao);
        let app_state = Data::new(AppState::test_state());
        let token = make_token();
        let app = actix_web::test::init_service(
            App::new()
                .service(get_preview_status)
                .app_data(app_state)
                .app_data(preview_dao),
        )
        .await;
        let req = actix_web::test::TestRequest::post()
            .uri("/video/preview/status")
            .insert_header(("Authorization", format!("Bearer {}", token)))
            .set_json(serde_json::json!({
                "paths": ["stale/pending.mp4", "stale/failed.mp4"]
            }))
            .to_request();
        let resp = actix_web::test::call_service(&app, req).await;
        assert_eq!(resp.status(), 200);
        let body: serde_json::Value = actix_web::test::read_body_json(resp).await;
        let previews = body["previews"].as_array().unwrap();
        assert_eq!(previews.len(), 2);
        // Both records are returned with their current status
        assert_eq!(previews[0]["path"], "stale/pending.mp4");
        assert_eq!(previews[0]["status"], "pending");
        assert!(previews[0].get("preview_url").is_none());
        assert_eq!(previews[1]["path"], "stale/failed.mp4");
        assert_eq!(previews[1]["status"], "failed");
        assert!(previews[1].get("preview_url").is_none());
    }
 }
@@ -1,409 +0,0 @@
 //! Per-library HLS readiness: Prometheus gauges + `/hls/stats` endpoint.
 //!
 //! The new hash-keyed pipeline transcodes lazily — most of a freshly
 //! mounted library is "pending" for the first hour, and operators want
 //! a live read on "how much work is left, am I CPU-bound, do I need to
 //! bump `HLS_CONCURRENCY`." This module supplies both surfaces against
 //! the same compute path:
 //!
 //! - **Prometheus gauges** `imageserver_hls_videos_total{library}`,
 //!   `..._with_playlist{library}`, `..._pending{library}`,
 //!   `..._unsupported{library}`. Updated every watcher full-scan tick
 //!   and on every `/hls/stats` request, so the freshness matches
 //!   whichever surface the operator is watching.
 //!
 //! - **`GET /hls/stats`** returns a JSON snapshot of the same counts
 //!   plus a top-level cross-library aggregate. Claims-protected
 //!   (matches every other authenticated read in this crate).
 //!
 //! Cost is O(distinct video hashes per library), each row needing a
 //! single `stat()` on the playlist file. On a 100k-video library that's
 //! noticeable; on a typical home library (few thousand) it's noise.
 //! We call from explicit triggers only — never per-request from
 //! middleware — so the cost is bounded.
 use std::collections::HashSet;
 use std::path::Path;
 use std::sync::{Arc, Mutex};
 use actix_web::{HttpResponse, Responder, get, web};
 use lazy_static::lazy_static;
 use log::{info, warn};
 use prometheus::IntGaugeVec;
 use serde::Serialize;
 use crate::data::Claims;
 use crate::database::ExifDao;
 use crate::file_types;
 use crate::libraries::Library;
 use crate::state::AppState;
 use crate::video::hls_paths;
 lazy_static! {
    pub static ref HLS_VIDEOS_TOTAL: IntGaugeVec = IntGaugeVec::new(
        prometheus::Opts::new(
            "imageserver_hls_videos_total",
            "Distinct video content hashes per library known to image_exif",
        ),
        &["library"],
    )
    .expect("HLS_VIDEOS_TOTAL");
    pub static ref HLS_VIDEOS_WITH_PLAYLIST: IntGaugeVec = IntGaugeVec::new(
        prometheus::Opts::new(
            "imageserver_hls_videos_with_playlist",
            "Videos whose hash-keyed HLS playlist is already on disk",
        ),
        &["library"],
    )
    .expect("HLS_VIDEOS_WITH_PLAYLIST");
    pub static ref HLS_VIDEOS_PENDING: IntGaugeVec = IntGaugeVec::new(
        prometheus::Opts::new(
            "imageserver_hls_videos_pending",
            "Videos whose hash-keyed HLS playlist is not yet on disk",
        ),
        &["library"],
    )
    .expect("HLS_VIDEOS_PENDING");
    pub static ref HLS_VIDEOS_UNSUPPORTED: IntGaugeVec = IntGaugeVec::new(
        prometheus::Opts::new(
            "imageserver_hls_videos_unsupported",
            "Videos with an `.unsupported` sentinel — ffmpeg refused; \
             operator must delete to retry",
        ),
        &["library"],
    )
    .expect("HLS_VIDEOS_UNSUPPORTED");
 }
 /// Per-library HLS readiness snapshot.
 #[derive(Serialize, Debug, Clone, PartialEq, Eq)]
 pub struct HlsLibraryStats {
    pub library_id: i32,
    pub library: String,
    /// Distinct video content hashes (dedupes intra-library bytes-at-N-paths).
    pub total: usize,
    /// Of `total`, hashes whose `playlist.m3u8` is on disk.
    pub with_playlist: usize,
    /// Of `total`, hashes whose ffmpeg attempt left a `.unsupported`
    /// sentinel. Counted separately because they won't progress without
    /// operator intervention (delete the sentinel to retry).
    pub unsupported: usize,
    /// `total - (with_playlist + unsupported)` — videos awaiting transcode.
    pub pending: usize,
    /// Distinct rel_paths under this library that are video files but
    /// whose `image_exif.content_hash` is still NULL (mid-backfill).
    /// These don't yet count toward `total` because they're invisible
    /// to the hash-keyed pipeline; surfaced so the operator can see
    /// "hash backfill, then transcode" pipeline depth.
    pub hashless_videos: usize,
 }
 /// JSON response body for `GET /hls/stats`.
 #[derive(Serialize, Debug)]
 pub struct HlsStatsResponse {
    pub libraries: Vec<HlsLibraryStats>,
    pub total: usize,
    pub with_playlist: usize,
    pub pending: usize,
    pub unsupported: usize,
    pub hashless_videos: usize,
 }
 /// Compute current readiness per library and publish to Prometheus.
 /// Returns the same data so callers can serialise it. The publish step
 /// is idempotent on the gauge — old values get overwritten.
 pub fn compute_and_publish(
    libraries: &[Library],
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
    video_dir: &Path,
 ) -> Vec<HlsLibraryStats> {
    let ctx = opentelemetry::Context::new();
    let mut out = Vec::with_capacity(libraries.len());
    for lib in libraries {
        let stats = compute_for_library(&ctx, lib, exif_dao, video_dir);
        publish_gauges(&stats);
        out.push(stats);
    }
    out
 }
 fn publish_gauges(s: &HlsLibraryStats) {
    HLS_VIDEOS_TOTAL
        .with_label_values(&[s.library.as_str()])
        .set(s.total as i64);
    HLS_VIDEOS_WITH_PLAYLIST
        .with_label_values(&[s.library.as_str()])
        .set(s.with_playlist as i64);
    HLS_VIDEOS_PENDING
        .with_label_values(&[s.library.as_str()])
        .set(s.pending as i64);
    HLS_VIDEOS_UNSUPPORTED
        .with_label_values(&[s.library.as_str()])
        .set(s.unsupported as i64);
 }
 fn compute_for_library(
    ctx: &opentelemetry::Context,
    lib: &Library,
    exif_dao: &Arc<Mutex<Box<dyn ExifDao>>>,
    video_dir: &Path,
 ) -> HlsLibraryStats {
    let rows = {
        let mut dao = exif_dao.lock().expect("Unable to lock ExifDao");
        match dao.list_paths_and_hashes_for_library(ctx, lib.id) {
            Ok(r) => r,
            Err(e) => {
                warn!(
                    "hls_stats: list_paths_and_hashes_for_library failed for lib {}: {:?}",
                    lib.id, e
                );
                Vec::new()
            }
        }
    };
    stats_from_rows(lib, &rows, video_dir)
 }
 /// Pure function — same compute as [`compute_for_library`] but works
 /// on caller-supplied rows. Split out so tests don't need a full
 /// `ExifDao` mock; the integration path is exercised through
 /// `compute_and_publish` against the real SQLite DAO at runtime.
 fn stats_from_rows(
    lib: &Library,
    rows: &[(String, Option<String>)],
    video_dir: &Path,
 ) -> HlsLibraryStats {
    let mut hashes: HashSet<String> = HashSet::new();
    let mut hashless_videos = 0usize;
    for (rel_path, hash_opt) in rows {
        if !file_types::is_video_file(Path::new(rel_path)) {
            continue;
        }
        match hash_opt {
            Some(h) => {
                hashes.insert(h.clone());
            }
            None => {
                hashless_videos += 1;
            }
        }
    }
    let mut with_playlist = 0usize;
    let mut unsupported = 0usize;
    for h in &hashes {
        if hls_paths::playlist_for_hash(video_dir, h).exists() {
            with_playlist += 1;
        } else if hls_paths::sentinel_for_hash(video_dir, h).exists() {
            unsupported += 1;
        }
    }
    let total = hashes.len();
    let pending = total.saturating_sub(with_playlist + unsupported);
    HlsLibraryStats {
        library_id: lib.id,
        library: lib.name.clone(),
        total,
        with_playlist,
        unsupported,
        pending,
        hashless_videos,
    }
 }
 /// Log a single info line summarising readiness across all libraries.
 /// Called by the watcher at the end of a full-scan tick so operators
 /// who tail the log see the headline number without scraping
 /// Prometheus.
 pub fn log_summary(stats: &[HlsLibraryStats]) {
    let total: usize = stats.iter().map(|s| s.total).sum();
    let with_playlist: usize = stats.iter().map(|s| s.with_playlist).sum();
    let pending: usize = stats.iter().map(|s| s.pending).sum();
    let unsupported: usize = stats.iter().map(|s| s.unsupported).sum();
    let hashless: usize = stats.iter().map(|s| s.hashless_videos).sum();
    let per_lib: Vec<String> = stats
        .iter()
        .map(|s| {
            format!(
                "{}={}/{} pending={} unsupported={} hashless={}",
                s.library, s.with_playlist, s.total, s.pending, s.unsupported, s.hashless_videos,
            )
        })
        .collect();
    info!(
        "HLS readiness: {}/{} playlists on disk, {} pending, {} unsupported, {} hashless videos | per-library: [{}]",
        with_playlist,
        total,
        pending,
        unsupported,
        hashless,
        per_lib.join(", "),
    );
 }
 #[get("/hls/stats")]
 pub async fn hls_stats_handler(
    _claims: Claims,
    app_state: web::Data<AppState>,
    exif_dao: web::Data<Mutex<Box<dyn ExifDao>>>,
 ) -> impl Responder {
    let libraries = app_state.libraries.clone();
    let video_dir = std::path::PathBuf::from(&app_state.video_path);
    let exif_dao = exif_dao.into_inner();
    // Synchronous file IO + DB query — run on a blocking pool so the
    // actix worker thread stays free for other requests.
    let stats =
        match web::block(move || compute_and_publish(&libraries, &exif_dao, &video_dir)).await {
            Ok(s) => s,
            Err(e) => {
                warn!("/hls/stats: blocking task failed: {:?}", e);
                Vec::new()
            }
        };
    let total: usize = stats.iter().map(|s| s.total).sum();
    let with_playlist: usize = stats.iter().map(|s| s.with_playlist).sum();
    let pending: usize = stats.iter().map(|s| s.pending).sum();
    let unsupported: usize = stats.iter().map(|s| s.unsupported).sum();
    let hashless_videos: usize = stats.iter().map(|s| s.hashless_videos).sum();
    HttpResponse::Ok().json(HlsStatsResponse {
        libraries: stats,
        total,
        with_playlist,
        pending,
        unsupported,
        hashless_videos,
    })
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use tempfile::tempdir;
    fn lib(id: i32, name: &str) -> Library {
        Library {
            id,
            name: name.into(),
            root_path: String::new(),
            enabled: true,
            excluded_dirs: Vec::new(),
        }
    }
    fn rows(vs: Vec<(&str, Option<&str>)>) -> Vec<(String, Option<String>)> {
        vs.into_iter()
            .map(|(p, h)| (p.to_string(), h.map(|s| s.to_string())))
            .collect()
    }
    fn touch(dir: &Path, rel: &str) {
        let p = dir.join(rel);
        std::fs::create_dir_all(p.parent().unwrap()).unwrap();
        std::fs::write(p, b"").unwrap();
    }
    #[test]
    fn videos_only_count_in_total() {
        let tmp = tempdir().unwrap();
        let r = rows(vec![
            ("photos/IMG.jpg", Some(&"a".repeat(64))), // image: ignored
            ("clip.mp4", Some(&"b".repeat(64))),
            ("vid.mov", Some(&"c".repeat(64))),
        ]);
        let stats = stats_from_rows(&lib(1, "main"), &r, tmp.path());
        assert_eq!(stats.total, 2);
        assert_eq!(stats.with_playlist, 0);
        assert_eq!(stats.pending, 2);
        assert_eq!(stats.unsupported, 0);
        assert_eq!(stats.hashless_videos, 0);
    }
    #[test]
    fn hash_dedup_collapses_duplicate_rel_paths() {
        let tmp = tempdir().unwrap();
        let r = rows(vec![
            ("a/clip.mp4", Some(&"a".repeat(64))),
            ("b/clip.mp4", Some(&"a".repeat(64))), // same bytes, dup
            ("other.mp4", Some(&"b".repeat(64))),
        ]);
        let stats = stats_from_rows(&lib(1, "main"), &r, tmp.path());
        assert_eq!(stats.total, 2, "duplicate hashes collapse");
    }
    #[test]
    fn playlist_existence_promotes_to_with_playlist() {
        let tmp = tempdir().unwrap();
        let hash = "a".repeat(64);
        touch(tmp.path(), &format!("aa/{}/playlist.m3u8", hash));
        let r = rows(vec![("clip.mp4", Some(&hash))]);
        let stats = stats_from_rows(&lib(1, "main"), &r, tmp.path());
        assert_eq!(stats.total, 1);
        assert_eq!(stats.with_playlist, 1);
        assert_eq!(stats.pending, 0);
    }
    #[test]
    fn sentinel_existence_promotes_to_unsupported() {
        let tmp = tempdir().unwrap();
        let hash = "b".repeat(64);
        touch(tmp.path(), &format!("bb/{}/playlist.unsupported", hash));
        let r = rows(vec![("clip.mov", Some(&hash))]);
        let stats = stats_from_rows(&lib(1, "main"), &r, tmp.path());
        assert_eq!(stats.total, 1);
        assert_eq!(stats.unsupported, 1);
        assert_eq!(stats.with_playlist, 0);
        assert_eq!(stats.pending, 0);
    }
    #[test]
    fn null_hash_videos_are_hashless_not_total() {
        let tmp = tempdir().unwrap();
        let r = rows(vec![
            ("clip.mp4", None),
            ("other.mp4", Some(&"a".repeat(64))),
        ]);
        let stats = stats_from_rows(&lib(1, "main"), &r, tmp.path());
        assert_eq!(stats.total, 1, "hashless row excluded from total");
        assert_eq!(stats.hashless_videos, 1);
    }
    #[test]
    fn publish_gauges_sets_per_library_value() {
        let s = HlsLibraryStats {
            library_id: 7,
            library: "test_publish_a".into(),
            total: 5,
            with_playlist: 2,
            pending: 3,
            unsupported: 0,
            hashless_videos: 0,
        };
        publish_gauges(&s);
        assert_eq!(
            HLS_VIDEOS_TOTAL
                .with_label_values(&["test_publish_a"])
                .get(),
            5
        );
        assert_eq!(
            HLS_VIDEOS_PENDING
                .with_label_values(&["test_publish_a"])
                .get(),
            3
        );
        assert_eq!(
            HLS_VIDEOS_WITH_PLAYLIST
                .with_label_values(&["test_publish_a"])
                .get(),
            2
        );
    }
 }
@@ -7,8 +7,6 @@ pub mod ai;
 pub mod auth;
 pub mod bin_progress;
 pub mod cleanup;
 pub mod clip_search;
 pub mod clip_watch;
 pub mod content_hash;
 pub mod data;
 pub mod database;
@@ -28,13 +26,11 @@ pub mod memories;
 pub mod otel;
 pub mod parsers;
 pub mod perceptual_hash;
 pub mod personas;
 pub mod service;
 pub mod state;
 pub mod tags;
 #[cfg(test)]
 pub mod testhelpers;
 pub mod thumbnails;
 pub mod utils;
 pub mod video;
@@ -42,3 +38,20 @@ pub mod video;
 pub use data::{Claims, ThumbnailRequest};
 pub use database::{connect, schema};
 pub use state::AppState;
 // Stub functions for modules that reference main.rs
 // These are not used by cleanup_files binary
 use std::path::Path;
 use walkdir::DirEntry;
 pub fn create_thumbnails(_libs: &[libraries::Library], _excluded_dirs: &[String]) {
    // Stub - implemented in main.rs
 }
 pub fn update_media_counts(_media_dir: &Path, _excluded_dirs: &[String]) {
    // Stub - implemented in main.rs
 }
 pub fn is_video(entry: &DirEntry) -> bool {
    file_types::direntry_is_video(entry)
 }
@@ -1,9 +1,8 @@
-use actix_web::{HttpResponse, Responder, get, patch, web, web::Data};
+use actix_web::{HttpResponse, Responder, get, web::Data};
 use chrono::Utc;
 use diesel::prelude::*;
 use diesel::sqlite::SqliteConnection;
 use log::{info, warn};
 use serde::Deserialize;
 use std::collections::HashMap;
 use std::path::{Path, PathBuf};
 use std::sync::{Arc, RwLock};
@@ -80,21 +79,14 @@ impl Library {
    }
 }
-/// Parse an excluded_dirs string into a Vec, dropping empty entries.
+/// Parse a comma-separated excluded_dirs column into a Vec, dropping
-/// NULL → empty Vec. Duplicates are preserved — `PathExcluder` accepts
+/// empty entries (mirrors `AppState::parse_excluded_dirs` for the env
-/// repeats, and the storage-side normaliser is where dedup happens.
+/// var). NULL → empty Vec.
 ///
 /// Accepts both `,` and newline (`\n` / `\r\n`) as separators so the
 /// UI's textarea can submit one-entry-per-line input without forcing
 /// the operator to remember commas. The DB stores the canonical
 /// comma-joined form (see `normalize_excluded_dirs_input`); the
 /// newline path matters mostly for the frontend submit, but mirroring
 /// it here keeps the parse direction round-trip safe.
 pub fn parse_excluded_dirs_column(raw: Option<&str>) -> Vec<String> {
    match raw {
        None => Vec::new(),
        Some(s) => s
-            .split([',', '\n', '\r'])
+            .split(',')
            .map(str::trim)
            .filter(|s| !s.is_empty())
            .map(String::from)
@@ -102,118 +94,6 @@ pub fn parse_excluded_dirs_column(raw: Option<&str>) -> Vec<String> {
    }
 }
 /// Validate a single excluded_dirs entry, normalising trivial cosmetic
 /// differences and rejecting forms that `PathExcluder` would silently
 /// drop. Returns the entry to store, or an error message describing
 /// what's wrong with it.
 ///
 /// Rules:
 /// - Backslashes are rejected — PathExcluder strips only a leading `/`;
 ///   a Windows-typed `\photos` or `photos\2024` lands in the
 ///   component-pattern bucket and never matches anything. Suggest the
 ///   forward-slash form.
 /// - A Windows drive letter prefix (`Z:` etc.) is rejected — excluded
 ///   entries are *relative to the library root*, not absolute system
 ///   paths.
 /// - A no-leading-slash entry containing `/` is rejected — the
 ///   component-pattern path matches a single segment only; the user
 ///   almost certainly meant the leading-slash form.
 /// - A `..` segment in a path entry is rejected — `base.join("../x")`
 ///   doesn't canonicalise, so the resulting prefix never matches and
 ///   the exclude silently fails.
 /// - Trailing slashes on path entries are stripped silently
 ///   (`/photos/` → `/photos`) — purely cosmetic.
 pub fn validate_excluded_dirs_entry(entry: &str) -> Result<String, String> {
    let trimmed = entry.trim();
    if trimmed.is_empty() {
        return Err("empty entry".to_string());
    }
    if trimmed.contains('\\') {
        return Err(format!(
            "'{}': use forward slashes — backslash paths never match on the watcher's component-by-component compare",
            trimmed
        ));
    }
    // Windows drive letter prefix like `Z:` or `Z:/something`. A
    // length-2 ASCII-alpha + colon is the canonical form; we don't
    // bother with longer multi-letter Windows drive-equivalents
    // (`\\?\Volume{…}`) since the backslash check already catches them.
    let bytes = trimmed.as_bytes();
    if bytes.len() >= 2 && bytes[0].is_ascii_alphabetic() && bytes[1] == b':' {
        return Err(format!(
            "'{}': excluded entries are relative to the library root, not absolute system paths — drop the drive letter",
            trimmed
        ));
    }
    if let Some(rel) = trimmed.strip_prefix('/') {
        // Path form. Reject `..` traversal — `base.join(\"../x\")` doesn't
        // canonicalise, so `path.starts_with(...)` never matches.
        if rel.split('/').any(|seg| seg == "..") {
            return Err(format!(
                "'{}': '..' segments don't normalise — the prefix-match never fires",
                trimmed
            ));
        }
        // Strip a trailing slash if any (`/photos/` → `/photos`). Purely
        // cosmetic; PathBuf::starts_with treats both forms identically.
        let stripped = if rel.ends_with('/') {
            format!("/{}", rel.trim_end_matches('/'))
        } else {
            trimmed.to_string()
        };
        // After stripping, an empty rel ("/" alone) excludes the root —
        // certainly a typo.
        if stripped == "/" {
            return Err("'/': excluding the library root is almost certainly a typo".to_string());
        }
        Ok(stripped)
    } else {
        // Component-pattern form: must be a single segment. A `/`
        // anywhere here is the common "I forgot the leading slash" typo
        // — reject so the user fixes it instead of staring at an
        // exclude that does nothing.
        if trimmed.contains('/') {
            return Err(format!(
                "'{}': multi-segment names only match with a leading slash — try '/{}'",
                trimmed, trimmed
            ));
        }
        Ok(trimmed.to_string())
    }
 }
 /// Canonicalise an excluded_dirs string for storage: validate each
 /// entry, then parse → trim → dedupe (preserving insertion order) →
 /// comma-join with no inner whitespace. Empty / whitespace-only input
 /// → `Ok(None)` (writes NULL). Any entry that fails validation aborts
 /// the whole patch with a descriptive error so the operator can fix
 /// the typo before retrying.
 ///
 /// Used by `PATCH /libraries/{id}` so two users typing the same entries
 /// in different orders / casings / whitespace land on the same stored
 /// form, and a typo'd duplicate (`@eaDir, @eaDir`) collapses on save.
 /// Round-trip stable: writing the output back through this function
 /// yields the same string.
 pub fn normalize_excluded_dirs_input(raw: &str) -> Result<Option<String>, String> {
    let parsed = parse_excluded_dirs_column(Some(raw));
    if parsed.is_empty() {
        return Ok(None);
    }
    let mut seen = std::collections::HashSet::new();
    let mut deduped: Vec<String> = Vec::with_capacity(parsed.len());
    for entry in parsed {
        let validated = validate_excluded_dirs_entry(&entry)?;
        if seen.insert(validated.clone()) {
            deduped.push(validated);
        }
    }
    if deduped.is_empty() {
        Ok(None)
    } else {
        Ok(Some(deduped.join(",")))
    }
 }
 impl From<LibraryRow> for Library {
    fn from(row: LibraryRow) -> Self {
        Library {
@@ -454,30 +334,16 @@ pub struct LibraryStatus {
 #[derive(serde::Serialize)]
 pub struct LibrariesResponse {
    pub libraries: Vec<LibraryStatus>,
    /// Globally-excluded paths/patterns from the `EXCLUDED_DIRS` env var.
    /// Applied **in union** with each library's own `excluded_dirs`. Surfaced
    /// here so an admin UI can show the operator "you already skip these
    /// everywhere" before they add per-library entries that would duplicate
    /// the global list. Read-only — globals live in `.env` and aren't
    /// mutable via the API today.
    pub global_excluded_dirs: Vec<String>,
 }
 #[get("/libraries")]
 pub async fn list_libraries(_claims: Claims, app_state: Data<AppState>) -> impl Responder {
    // Read from the live view so a recent PATCH /libraries/{id} that
    // flipped `enabled` or rewrote `excluded_dirs` surfaces immediately
    // — the immutable `app_state.libraries` snapshot is stale once the
    // first mutation lands.
    let live_guard = app_state
        .live_libraries
        .read()
        .unwrap_or_else(|e| e.into_inner());
    let health_guard = app_state
        .library_health
        .read()
        .unwrap_or_else(|e| e.into_inner());
-    let libraries = live_guard
+    let libraries = app_state
        .libraries
        .iter()
        .map(|lib| LibraryStatus {
            library: lib.clone(),
@@ -487,123 +353,7 @@ pub async fn list_libraries(_claims: Claims, app_state: Data<AppState>) -> impl
                .unwrap_or(LibraryHealth::Online),
        })
        .collect();
-    HttpResponse::Ok().json(LibrariesResponse {
+    HttpResponse::Ok().json(LibrariesResponse { libraries })
        libraries,
        global_excluded_dirs: app_state.excluded_dirs.clone(),
    })
 }
 /// Body for PATCH /libraries/{id}. Both fields are optional — omitting
 /// one leaves it untouched. `excluded_dirs` is the same comma-separated
 /// shape as the DB column; an empty string clears (writes NULL).
 #[derive(Deserialize, Debug)]
 pub struct PatchLibraryBody {
    pub enabled: Option<bool>,
    pub excluded_dirs: Option<String>,
 }
 /// Mutate one library row. The watcher reads `app_state.live_libraries`
 /// at the top of each tick, so a successful PATCH is picked up within
 /// one WATCH_QUICK_INTERVAL_SECONDS without restart — no separate
 /// `apply_now` signal. Returns the updated `Library` so the caller can
 /// render the new state without a follow-up GET.
 ///
 /// Despite CLAUDE.md noting "Toggle via SQL; there is intentionally no
 /// HTTP endpoint for library mutation", we now expose this for Apollo's
 /// Settings panel. The single-user trust model hasn't changed; the
 /// endpoint just removes the SSH-and-sqlite3 step.
 #[patch("/libraries/{id}")]
 pub async fn patch_library(
    _claims: Claims,
    path: web::Path<i32>,
    body: web::Json<PatchLibraryBody>,
    app_state: Data<AppState>,
 ) -> impl Responder {
    let lib_id = path.into_inner();
    let body = body.into_inner();
    if body.enabled.is_none() && body.excluded_dirs.is_none() {
        return HttpResponse::UnprocessableEntity().body("empty patch body");
    }
    let mut conn = crate::database::connect();
    // Build the SET clause. Diesel's set() takes a tuple of assignments;
    // we apply each field independently so an absent field doesn't get
    // forced to NULL / its default.
    let mut affected = 0usize;
    if let Some(enabled) = body.enabled {
        match diesel::update(libraries::table.filter(libraries::id.eq(lib_id)))
            .set(libraries::enabled.eq(enabled))
            .execute(&mut conn)
        {
            Ok(n) => affected = affected.max(n),
            Err(e) => {
                warn!(
                    "PATCH /libraries/{}: enabled update failed: {:?}",
                    lib_id, e
                );
                return HttpResponse::InternalServerError().body(format!("{}", e));
            }
        }
    }
    if let Some(raw) = body.excluded_dirs.as_deref() {
        // Canonicalise on write — trim, dedupe, validate, drop empties —
        // so the DB stores a round-trip-stable form regardless of how
        // messy the user typed it. Empty / whitespace-only → NULL
        // (matches a never-set library). Validation failures (Windows
        // backslash paths, drive letters, `..` traversal, etc.) bounce
        // back as 422 so the operator can fix the typo.
        let normalised = match normalize_excluded_dirs_input(raw) {
            Ok(v) => v,
            Err(msg) => return HttpResponse::UnprocessableEntity().body(msg),
        };
        let stored: Option<&str> = normalised.as_deref();
        match diesel::update(libraries::table.filter(libraries::id.eq(lib_id)))
            .set(libraries::excluded_dirs.eq(stored))
            .execute(&mut conn)
        {
            Ok(n) => affected = affected.max(n),
            Err(e) => {
                warn!(
                    "PATCH /libraries/{}: excluded_dirs update failed: {:?}",
                    lib_id, e
                );
                return HttpResponse::InternalServerError().body(format!("{}", e));
            }
        }
    }
    if affected == 0 {
        return HttpResponse::NotFound().body(format!("library id {} not found", lib_id));
    }
    // Refresh the live view from the canonical DB state. Reloading the
    // whole table (rather than mutating one entry in place) is cheap
    // (handful of rows) and keeps the in-memory and DB views trivially
    // consistent.
    let fresh = load_all(&mut conn);
    let updated = fresh.iter().find(|l| l.id == lib_id).cloned();
    {
        let mut live = app_state
            .live_libraries
            .write()
            .unwrap_or_else(|e| e.into_inner());
        *live = fresh;
    }
    match updated {
        Some(lib) => {
            info!(
                "PATCH /libraries/{}: enabled={:?} excluded_dirs={:?} → applied",
                lib_id, body.enabled, body.excluded_dirs
            );
            HttpResponse::Ok().json(lib)
        }
        None => {
            HttpResponse::NotFound().body(format!("library id {} not found after update", lib_id))
        }
    }
 }
 #[cfg(test)]
@@ -746,40 +496,6 @@ mod tests {
        );
    }
    #[test]
    fn parse_excluded_dirs_column_splits_on_newlines_too() {
        // Newline-separated input from a textarea submit. One-per-line
        // is the recommended UX because "I forgot the comma" was a
        // recurring footgun (.thumbnails .thumbnails2 silently
        // becomes a single never-matching pattern).
        assert_eq!(
            parse_excluded_dirs_column(Some("@eaDir\n.thumbnails\n/private")),
            vec![
                "@eaDir".to_string(),
                ".thumbnails".to_string(),
                "/private".to_string()
            ]
        );
        // Windows line endings (CRLF) — the carriage return is its own
        // separator so the trailing empty token between \r and \n gets
        // trimmed + dropped.
        assert_eq!(
            parse_excluded_dirs_column(Some("a\r\nb\r\nc")),
            vec!["a".to_string(), "b".to_string(), "c".to_string()]
        );
        // Mixed comma + newline — the user pastes from one source,
        // adds a few entries inline. Both work, in any combination.
        assert_eq!(
            parse_excluded_dirs_column(Some("a, b\nc,d")),
            vec![
                "a".to_string(),
                "b".to_string(),
                "c".to_string(),
                "d".to_string()
            ]
        );
    }
    #[test]
    fn effective_excluded_dirs_unions_global_and_per_library() {
        let lib_no_extras = Library {
@@ -807,175 +523,6 @@ mod tests {
        assert_eq!(combined.len(), 3);
    }
    #[test]
    fn effective_excluded_dirs_keeps_overlap_between_global_and_per_library() {
        // Two sources both excluding `@eaDir` is legal — `PathExcluder`
        // accepts repeats, and there's no behavioral reason to dedupe
        // here. Documents the design choice so a future refactor that
        // tightens this is forced to update both code and tests.
        let globals = vec!["@eaDir".to_string()];
        let lib = Library {
            id: 1,
            name: "main".into(),
            root_path: "/x".into(),
            enabled: true,
            excluded_dirs: vec!["@eaDir".to_string(), "/private".to_string()],
        };
        let combined = lib.effective_excluded_dirs(&globals);
        // 2 occurrences of @eaDir + /private = 3 entries total.
        assert_eq!(combined, vec!["@eaDir", "@eaDir", "/private"]);
    }
    #[test]
    fn normalize_excluded_dirs_input_handles_empty_and_whitespace() {
        assert_eq!(normalize_excluded_dirs_input(""), Ok(None));
        assert_eq!(normalize_excluded_dirs_input("   "), Ok(None));
        assert_eq!(normalize_excluded_dirs_input(",,,"), Ok(None));
        assert_eq!(normalize_excluded_dirs_input(" , , "), Ok(None));
    }
    #[test]
    fn normalize_excluded_dirs_input_trims_per_entry() {
        // Inner whitespace stripped on each item, comma-joined without
        // spaces. Mirrors how parse_excluded_dirs_column reads it back.
        assert_eq!(
            normalize_excluded_dirs_input(" @eaDir , /private , .thumbnails "),
            Ok(Some("@eaDir,/private,.thumbnails".to_string()))
        );
    }
    #[test]
    fn normalize_excluded_dirs_input_dedupes_preserving_first_occurrence() {
        // Exact-string duplicates collapse; the first occurrence wins
        // (preserves the operator's typed order so they recognise their
        // intent on round-trip).
        assert_eq!(
            normalize_excluded_dirs_input("@eaDir, /private, @eaDir, /private"),
            Ok(Some("@eaDir,/private".to_string()))
        );
        // Whitespace-distinct entries collapse to the same canonical
        // form. Case is preserved — `Foo` and `foo` are different keys
        // (filesystem case-sensitivity is platform-dependent; we don't
        // make that call here).
        assert_eq!(
            normalize_excluded_dirs_input(" Foo,foo, Foo "),
            Ok(Some("Foo,foo".to_string()))
        );
    }
    #[test]
    fn normalize_excluded_dirs_input_is_round_trip_stable() {
        // Writing the normaliser's output back through it yields the
        // same string. PATCH-clearing edits round-trip cleanly through
        // parse_excluded_dirs_column too.
        let raw = " /a/b ,, /a/b , c ";
        let once = normalize_excluded_dirs_input(raw)
            .expect("validation passes")
            .expect("not empty");
        let twice = normalize_excluded_dirs_input(&once)
            .expect("validation passes")
            .expect("not empty");
        assert_eq!(once, twice);
        // Parsing the stored form back gives the deduped Vec.
        assert_eq!(
            parse_excluded_dirs_column(Some(&once)),
            vec!["/a/b".to_string(), "c".to_string()]
        );
    }
    #[test]
    fn validate_rejects_backslash_paths() {
        // Windows-typed entries land in the component-pattern bucket
        // and never match — reject so the user gets feedback instead
        // of a silent no-op.
        assert!(validate_excluded_dirs_entry(r"\photos").is_err());
        assert!(validate_excluded_dirs_entry(r"photos\2024").is_err());
        assert!(validate_excluded_dirs_entry(r"\\server\share").is_err());
        // The error message names the entry and points at the fix.
        let err = validate_excluded_dirs_entry(r"\photos").unwrap_err();
        assert!(err.contains("forward slashes"), "{}", err);
    }
    #[test]
    fn validate_rejects_windows_drive_letters() {
        assert!(validate_excluded_dirs_entry("Z:/photos").is_err());
        assert!(validate_excluded_dirs_entry("z:photos").is_err());
        // Single-letter alpha + colon is the canonical drive prefix;
        // the message should steer toward the relative form.
        let err = validate_excluded_dirs_entry("Z:/foo").unwrap_err();
        assert!(err.contains("relative to the library root"), "{}", err);
    }
    #[test]
    fn validate_rejects_multi_segment_name_without_leading_slash() {
        // The common "I forgot the slash" typo. Today this would store
        // a never-matching component pattern; we catch it.
        let err = validate_excluded_dirs_entry("photos/2024").unwrap_err();
        assert!(err.contains("multi-segment"), "{}", err);
        // And the suggestion shows the corrected form.
        assert!(err.contains("/photos/2024"), "{}", err);
    }
    #[test]
    fn validate_rejects_parent_dir_traversal_in_path_entries() {
        // base.join("../sensitive") doesn't canonicalise, so the
        // resulting prefix never starts_with anything the walker sees.
        assert!(validate_excluded_dirs_entry("/../secret").is_err());
        assert!(validate_excluded_dirs_entry("/photos/../keys").is_err());
        // Same string as a non-leading-slash component is fine — it
        // just never matches (you'd literally need a directory named
        // `..` which is impossible on every filesystem we care about),
        // but the validator accepts it because the failure mode isn't
        // a silent footgun in that direction.
        assert!(validate_excluded_dirs_entry("..").is_ok());
    }
    #[test]
    fn validate_strips_trailing_slash_on_path_entries() {
        assert_eq!(validate_excluded_dirs_entry("/photos/").unwrap(), "/photos");
        assert_eq!(
            validate_excluded_dirs_entry("/photos//").unwrap(),
            "/photos"
        );
        // Bare "/" is rejected — almost certainly a typo for the
        // library root.
        assert!(validate_excluded_dirs_entry("/").is_err());
        assert!(validate_excluded_dirs_entry("///").is_err());
    }
    #[test]
    fn validate_passes_valid_entries() {
        for entry in &[
            "/photos",
            "/photos/2024",
            "/media/raw/private",
            "@eaDir",
            ".thumbnails",
            ".DS_Store",
            "node_modules",
        ] {
            assert!(
                validate_excluded_dirs_entry(entry).is_ok(),
                "expected {} to pass",
                entry
            );
        }
    }
    #[test]
    fn normalize_aborts_on_invalid_entry() {
        // One bad entry kills the whole patch — better to surface the
        // problem than to silently apply N-1 of N changes.
        let err = normalize_excluded_dirs_input("/photos, photos/2024").unwrap_err();
        assert!(err.contains("photos/2024"), "{}", err);
        // A valid mix succeeds — the bad-entry test isn't accidentally
        // matching the good prefix.
        assert_eq!(
            normalize_excluded_dirs_input("/photos, @eaDir, /private/"),
            Ok(Some("/photos,@eaDir,/private".to_string()))
        );
    }
    fn probe_lib(id: i32, root: String) -> Library {
        Library {
            id,
@@ -1052,7 +599,7 @@ mod tests {
            enabled: true,
            excluded_dirs: Vec::new(),
        };
-        let map = new_health_map(std::slice::from_ref(&lib));
+        let map = new_health_map(&[lib.clone()]);
        // First probe: empty dir, no prior data — Online.
        let s1 = refresh_health(&map, &lib, false);
@@ -296,7 +296,6 @@ impl GcStats {
            || self.revived > 0
    }
    #[allow(dead_code)]
    pub fn total_deleted(&self) -> usize {
        self.deleted_face_detections + self.deleted_tagged_photo + self.deleted_photo_insights
    }
@@ -206,37 +206,12 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
        let timestamp_str = captures.get(1)?.as_str();
        let len = timestamp_str.len();
        // Snapchat used real unix-second filenames in its early era
        // (e.g. `Snapchat-1383929602.jpg` = 2013-11-08), then switched to
        // monotonic sequential IDs whose digits overlap plausible epoch
        // ranges (`Snapchat-1021849065.mp4` truncates to 2002, actually
        // saved 2021; `Snapchat-1751031586660373917.jpg` is 19 digits,
        // truncates to 2002, actually 2016). Discriminate by:
        //   - exactly 10 captured digits AND post-2011-09-23 (launch) → real epoch
        //   - anything else under this prefix → sequential ID, fall through
        // The Snapchat-launch floor catches the 10-digit-2002 case; the
        // length=10 gate catches the multi-digit sequential IDs (which
        // get truncated to 16 by the regex above).
        let lower = filename.to_ascii_lowercase();
        let is_snapchat = lower.starts_with("snapchat-");
        if is_snapchat && len != 10 {
            return None;
        }
        // Skip autogenerated filenames that start with "10000" (e.g., 1000004178.jpg)
        // These are not timestamps but auto-generated file IDs
        if timestamp_str.starts_with("10000") {
            return None;
        }
        // A leading zero rules out a real unix timestamp at any sane
        // resolution (seconds since 2001-09-09, ms since 1970-01-01 are
        // both 10+ digits with no leading zero). Filenames like
        // `000227580005.jpg` are sequential scan IDs, not timestamps.
        if timestamp_str.starts_with('0') {
            return None;
        }
        // Try milliseconds first (13 digits exactly)
        if len == 13
            && let Some(date_time) = timestamp_str
@@ -244,7 +219,6 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
                .ok()
                .and_then(DateTime::from_timestamp_millis)
                .map(|naive_dt| naive_dt.fixed_offset())
                .and_then(plausible_filename_date)
        {
            return Some(date_time);
        }
@@ -257,7 +231,6 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
                .ok()
                .and_then(|timestamp_secs| DateTime::from_timestamp(timestamp_secs, 0))
                .map(|naive_dt| naive_dt.fixed_offset())
                .and_then(plausible_filename_date)
        {
            return Some(date_time);
        }
@@ -269,15 +242,7 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
                .ok()
                .and_then(|timestamp_secs| DateTime::from_timestamp(timestamp_secs, 0))
                .map(|naive_dt| naive_dt.fixed_offset())
                .and_then(plausible_filename_date)
        {
            // Snapchat launched 2011-09-23. A 10-digit Snapchat filename
            // dated before that is a sequential ID (e.g.
            // `Snapchat-1021849065.mp4` parses to 2002), not a real epoch.
            const SNAPCHAT_LAUNCH_TS: i64 = 1_316_736_000;
            if is_snapchat && date_time.timestamp() < SNAPCHAT_LAUNCH_TS {
                return None;
            }
            return Some(date_time);
        }
@@ -288,7 +253,6 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
                .ok()
                .and_then(DateTime::from_timestamp_millis)
                .map(|naive_dt| naive_dt.fixed_offset())
                .and_then(plausible_filename_date)
        {
            return Some(date_time);
        }
@@ -297,27 +261,6 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
    None
 }
 /// Sanity gate for filename-derived timestamps. Real photo capture dates
 /// live in a narrow window; values outside it are almost always sequential
 /// scan IDs (`000227580005.jpg` → 1970) or arbitrary numeric suffixes
 /// (`IMG_21323906751390.jpeg` → 2037) that the regex caught by accident.
 /// Rejecting them lets the date_resolver waterfall fall through to
 /// `fs_time`, which is a much better proxy for content age than a fake
 /// epoch date.
 fn plausible_filename_date(dt: DateTime<FixedOffset>) -> Option<DateTime<FixedOffset>> {
    use chrono::Datelike;
    let year = dt.year();
    // 1995 predates digital photography for most users; allowing one year
    // past `now` covers clock-skew on freshly-taken shots without letting
    // 2037 timestamps through.
    let max_year = Utc::now().year() + 1;
    if (1995..=max_year).contains(&year) {
        Some(dt)
    } else {
        None
    }
 }
 /// Convert a `date_taken` Unix-seconds value to a `NaiveDate` in the
 /// client's local time. Falls back to server-local when the client didn't
 /// send a tz hint.
@@ -647,49 +590,6 @@ mod tests {
        );
    }
    #[test]
    fn test_extract_date_from_filename_leading_zero_scan_id_should_not_match() {
        // Sequential film-scan IDs like 000227580005.jpg parsed as a 12-digit
        // ms timestamp resolve to 1970-01-03; the leading zero rules out a
        // real epoch value at any sane resolution. Resolver should fall
        // through to fs_time instead of pinning the photo to 1970.
        assert!(extract_date_from_filename("000227580005.jpg").is_none());
    }
    #[test]
    fn test_extract_date_from_filename_far_future_should_not_match() {
        // IMG_21323906751390.jpeg → first 10 digits = 2132390675 → 2037.
        // Plausibility gate rejects it so the resolver falls through to
        // fs_time (which carries the real ingest date).
        assert!(extract_date_from_filename("IMG_21323906751390.jpeg").is_none());
    }
    #[test]
    fn test_extract_date_from_filename_snapchat_sequential_ids_rejected() {
        // Modern Snapchat-prefixed filenames carry sequential app-assigned
        // IDs whose digits happen to fall inside plausible epoch ranges
        // when truncated. Reported cases (real save dates per FileModifyDate):
        //   Snapchat-1021849065.mp4         → 10 digits → 2002-05-19  (saved 2021)
        //   Snapchat-1751031586660373917.jpg → 19 digits → 2002-09-09  (saved 2016)
        // We discriminate by length + Snapchat-launch floor: only exactly
        // 10 digits AND post-2011-09-23 (Snapchat launch) is treated as
        // a real unix epoch. Anything else falls through to fs_time.
        assert!(extract_date_from_filename("Snapchat-1021849065.mp4").is_none());
        assert!(extract_date_from_filename("Snapchat-1751031586660373917.jpg").is_none());
        // Case-insensitive match — lowercase variant should also reject.
        assert!(extract_date_from_filename("snapchat-1021849065.mp4").is_none());
    }
    #[test]
    fn test_extract_date_from_filename_snapchat_early_era_unix_epoch() {
        // Early Snapchat (2013-2014ish) wrote real unix-second filenames.
        // Snapchat-1383929602.jpg → 1383929602 = 2013-11-08 16:53:22 UTC.
        // The blanket-prefix denial introduced for sequential IDs broke
        // these — restore via a length=10 + post-launch sanity gate.
        let date_time = extract_date_from_filename("Snapchat-1383929602.jpg").unwrap();
        assert_eq!(date_time.timestamp(), 1383929602);
    }
    // The obsolete `test_memory_date_priority_*` tests covered the old
    // request-time waterfall in `get_memory_date_with_priority`. Their
    // replacement lives in `crate::date_resolver::tests` (resolver
@@ -1,355 +0,0 @@
 //! HTTP handlers for the server-side persona store.
 //!
 //! Personas previously lived only in mobile AsyncStorage; this module
 //! elevates them so they can sync across devices and so the
 //! `entity_facts.persona_id` column has something to reference.
 //!
 //! Built-in personas (default / journal / factual) are seeded by the
 //! migration. Customs are created here and may be migrated up from a
 //! device's local store via `POST /personas/migrate`.
 use actix_web::dev::{ServiceFactory, ServiceRequest};
 use actix_web::{App, HttpResponse, Responder, web};
 use serde::{Deserialize, Serialize};
 use std::sync::Mutex;
 use crate::data::Claims;
 use crate::database::models::Persona;
 use crate::database::{ImportPersona, PersonaDao, PersonaPatch};
 // ---------------------------------------------------------------------------
 // Wire shapes — camelCase out the door, snake_case from the DB.
 // ---------------------------------------------------------------------------
 #[derive(Serialize)]
 pub struct PersonaView {
    pub id: String,
    pub name: String,
    #[serde(rename = "systemPrompt")]
    pub system_prompt: String,
    #[serde(rename = "isBuiltIn")]
    pub is_built_in: bool,
    #[serde(rename = "includeAllMemories")]
    pub include_all_memories: bool,
    #[serde(rename = "createdAt")]
    pub created_at: i64,
    #[serde(rename = "updatedAt")]
    pub updated_at: i64,
    /// "Strict mode" — when true, the agent's recall_* tools return
    /// only facts whose status is 'reviewed'. See migration
    /// 2026-05-10-000400.
    #[serde(rename = "reviewedOnlyFacts")]
    pub reviewed_only_facts: bool,
    /// Gate for the agent's update_fact / supersede_fact tools.
    /// Default false — fresh personas let the agent create but not
    /// alter. See migration 2026-05-10-000500.
    #[serde(rename = "allowAgentCorrections")]
    pub allow_agent_corrections: bool,
 }
 impl From<Persona> for PersonaView {
    fn from(p: Persona) -> Self {
        Self {
            id: p.persona_id,
            name: p.name,
            system_prompt: p.system_prompt,
            is_built_in: p.is_built_in,
            include_all_memories: p.include_all_memories,
            created_at: p.created_at,
            updated_at: p.updated_at,
            reviewed_only_facts: p.reviewed_only_facts,
            allow_agent_corrections: p.allow_agent_corrections,
        }
    }
 }
 #[derive(Deserialize)]
 pub struct CreatePersonaRequest {
    pub name: String,
    #[serde(rename = "systemPrompt")]
    pub system_prompt: String,
    /// Optional caller-provided id. When present (e.g. a client that
    /// already minted `"custom-1735124234"` locally and is upgrading from
    /// the AsyncStorage-only era), the server uses it; collisions return
    /// 409. When absent the server mints `"custom-<ms>"`.
    #[serde(default, rename = "personaId")]
    pub persona_id: Option<String>,
 }
 #[derive(Deserialize)]
 pub struct UpdatePersonaRequest {
    #[serde(default)]
    pub name: Option<String>,
    #[serde(default, rename = "systemPrompt")]
    pub system_prompt: Option<String>,
    #[serde(default, rename = "includeAllMemories")]
    pub include_all_memories: Option<bool>,
    #[serde(default, rename = "reviewedOnlyFacts")]
    pub reviewed_only_facts: Option<bool>,
    #[serde(default, rename = "allowAgentCorrections")]
    pub allow_agent_corrections: Option<bool>,
 }
 #[derive(Deserialize)]
 pub struct MigrateRequest {
    pub personas: Vec<MigratePersona>,
 }
 #[derive(Deserialize)]
 pub struct MigratePersona {
    pub id: String,
    pub name: String,
    #[serde(rename = "systemPrompt")]
    pub system_prompt: String,
    #[serde(default, rename = "isBuiltIn")]
    pub is_built_in: bool,
    #[serde(default, rename = "createdAt")]
    pub created_at: Option<i64>,
 }
 #[derive(Serialize)]
 pub struct MigrateResponse {
    pub inserted: usize,
 }
 // ---------------------------------------------------------------------------
 // Service registration
 // ---------------------------------------------------------------------------
 pub type PersonaDaoData = web::Data<Mutex<Box<dyn PersonaDao>>>;
 pub fn add_persona_services<T>(app: App<T>) -> App<T>
 where
    T: ServiceFactory<ServiceRequest, Config = (), Error = actix_web::Error, InitError = ()>,
 {
    app.service(
        web::scope("/personas")
            .service(web::resource("/migrate").route(web::post().to(migrate_personas)))
            .service(
                web::resource("")
                    .route(web::get().to(list_personas))
                    .route(web::post().to(create_persona)),
            )
            .service(
                web::resource("/{persona_id}")
                    .route(web::put().to(update_persona))
                    .route(web::delete().to(delete_persona)),
            ),
    )
 }
 // ---------------------------------------------------------------------------
 // Handlers
 // ---------------------------------------------------------------------------
 fn user_id_from_claims(claims: &Claims) -> Option<i32> {
    claims.sub.parse::<i32>().ok()
 }
 async fn list_personas(claims: Claims, dao: PersonaDaoData) -> impl Responder {
    let Some(uid) = user_id_from_claims(&claims) else {
        return HttpResponse::Unauthorized().json(serde_json::json!({"error": "Invalid claims"}));
    };
    let cx = opentelemetry::Context::current();
    let mut dao = dao.lock().expect("Unable to lock PersonaDao");
    match dao.list_personas(&cx, uid) {
        Ok(rows) => {
            let views: Vec<PersonaView> = rows.into_iter().map(PersonaView::from).collect();
            HttpResponse::Ok().json(views)
        }
        Err(e) => {
            log::error!("list_personas error: {:?}", e);
            HttpResponse::InternalServerError().json(serde_json::json!({"error": "Database error"}))
        }
    }
 }
 async fn create_persona(
    claims: Claims,
    body: web::Json<CreatePersonaRequest>,
    dao: PersonaDaoData,
 ) -> impl Responder {
    let Some(uid) = user_id_from_claims(&claims) else {
        return HttpResponse::Unauthorized().json(serde_json::json!({"error": "Invalid claims"}));
    };
    if body.name.trim().is_empty() {
        return HttpResponse::BadRequest().json(serde_json::json!({"error": "name is required"}));
    }
    if body.system_prompt.trim().is_empty() {
        return HttpResponse::BadRequest()
            .json(serde_json::json!({"error": "systemPrompt is required"}));
    }
    let cx = opentelemetry::Context::current();
    let mut dao = dao.lock().expect("Unable to lock PersonaDao");
    let pid = match body.persona_id.as_deref() {
        Some(s) if !s.trim().is_empty() => s.to_string(),
        _ => format!("custom-{}", chrono::Utc::now().timestamp_millis()),
    };
    if matches!(pid.as_str(), "default" | "journal" | "factual") {
        return HttpResponse::Conflict()
            .json(serde_json::json!({"error": "persona id collides with a built-in"}));
    }
    // Pre-check existence so we can return 409 cleanly. The DB UNIQUE
    // would also catch it, but parsing Diesel's "constraint violation"
    // out of a generic DbError is uglier than a quick lookup.
    if let Ok(Some(_)) = dao.get_persona(&cx, uid, &pid) {
        return HttpResponse::Conflict()
            .json(serde_json::json!({"error": "persona already exists"}));
    }
    match dao.create_persona(
        &cx,
        uid,
        &pid,
        &body.name,
        &body.system_prompt,
        false,
        false,
    ) {
        Ok(p) => HttpResponse::Created().json(PersonaView::from(p)),
        Err(e) => {
            log::error!("create_persona error: {:?}", e);
            HttpResponse::InternalServerError().json(serde_json::json!({"error": "Database error"}))
        }
    }
 }
 async fn update_persona(
    claims: Claims,
    path: web::Path<String>,
    body: web::Json<UpdatePersonaRequest>,
    dao: PersonaDaoData,
 ) -> impl Responder {
    let Some(uid) = user_id_from_claims(&claims) else {
        return HttpResponse::Unauthorized().json(serde_json::json!({"error": "Invalid claims"}));
    };
    let pid = path.into_inner();
    let cx = opentelemetry::Context::current();
    let mut dao = dao.lock().expect("Unable to lock PersonaDao");
    // Built-in personas are owned by the migration; the canonical voice
    // text lives in source. A client renaming or rewriting the prompt
    // here would diverge from what new users get seeded with and hide
    // the operator's actual customization (their own custom persona)
    // from the picker. `include_all_memories` stays editable on
    // built-ins — that's a per-user preference, not the persona's
    // identity. Mirrors the same guard delete_persona enforces below.
    match dao.get_persona(&cx, uid, &pid) {
        Ok(Some(p)) if p.is_built_in => {
            let editing_identity = body.name.is_some() || body.system_prompt.is_some();
            if editing_identity {
                return HttpResponse::Conflict().json(serde_json::json!({
                    "error": "Cannot edit name or systemPrompt of a built-in persona"
                }));
            }
        }
        Ok(None) => {
            return HttpResponse::NotFound()
                .json(serde_json::json!({"error": "Persona not found"}));
        }
        Err(e) => {
            log::error!("update_persona lookup error: {:?}", e);
            return HttpResponse::InternalServerError()
                .json(serde_json::json!({"error": "Database error"}));
        }
        Ok(Some(_)) => {}
    }
    let patch = PersonaPatch {
        name: body.name.clone(),
        system_prompt: body.system_prompt.clone(),
        include_all_memories: body.include_all_memories,
        reviewed_only_facts: body.reviewed_only_facts,
        allow_agent_corrections: body.allow_agent_corrections,
    };
    match dao.update_persona(&cx, uid, &pid, patch) {
        Ok(Some(p)) => HttpResponse::Ok().json(PersonaView::from(p)),
        Ok(None) => {
            HttpResponse::NotFound().json(serde_json::json!({"error": "Persona not found"}))
        }
        Err(e) => {
            log::error!("update_persona error: {:?}", e);
            HttpResponse::InternalServerError().json(serde_json::json!({"error": "Database error"}))
        }
    }
 }
 async fn delete_persona(
    claims: Claims,
    path: web::Path<String>,
    dao: PersonaDaoData,
 ) -> impl Responder {
    let Some(uid) = user_id_from_claims(&claims) else {
        return HttpResponse::Unauthorized().json(serde_json::json!({"error": "Invalid claims"}));
    };
    let pid = path.into_inner();
    let cx = opentelemetry::Context::current();
    let mut dao = dao.lock().expect("Unable to lock PersonaDao");
    match dao.get_persona(&cx, uid, &pid) {
        Ok(Some(p)) if p.is_built_in => {
            return HttpResponse::Conflict()
                .json(serde_json::json!({"error": "Cannot delete built-in persona"}));
        }
        Ok(None) => {
            return HttpResponse::NotFound()
                .json(serde_json::json!({"error": "Persona not found"}));
        }
        Err(e) => {
            log::error!("delete_persona lookup error: {:?}", e);
            return HttpResponse::InternalServerError()
                .json(serde_json::json!({"error": "Database error"}));
        }
        Ok(Some(_)) => {}
    }
    match dao.delete_persona(&cx, uid, &pid) {
        Ok(_) => HttpResponse::NoContent().finish(),
        Err(e) => {
            log::error!("delete_persona error: {:?}", e);
            HttpResponse::InternalServerError().json(serde_json::json!({"error": "Database error"}))
        }
    }
 }
 async fn migrate_personas(
    claims: Claims,
    body: web::Json<MigrateRequest>,
    dao: PersonaDaoData,
 ) -> impl Responder {
    let Some(uid) = user_id_from_claims(&claims) else {
        return HttpResponse::Unauthorized().json(serde_json::json!({"error": "Invalid claims"}));
    };
    let cx = opentelemetry::Context::current();
    let mut dao = dao.lock().expect("Unable to lock PersonaDao");
    // Filter out built-in ids — those are already seeded by the
    // migration and re-importing them would be a no-op anyway thanks to
    // INSERT OR IGNORE, but skipping early avoids the UNIQUE round-trip.
    let now = chrono::Utc::now().timestamp_millis();
    let rows: Vec<ImportPersona> = body
        .personas
        .iter()
        .filter(|p| !matches!(p.id.as_str(), "default" | "journal" | "factual"))
        .map(|p| ImportPersona {
            persona_id: p.id.clone(),
            name: p.name.clone(),
            system_prompt: p.system_prompt.clone(),
            is_built_in: p.is_built_in,
            created_at: p.created_at.unwrap_or(now),
        })
        .collect();
    match dao.bulk_import(&cx, uid, &rows) {
        Ok(inserted) => HttpResponse::Ok().json(MigrateResponse { inserted }),
        Err(e) => {
            log::error!("migrate_personas error: {:?}", e);
            HttpResponse::InternalServerError().json(serde_json::json!({"error": "Database error"}))
        }
    }
 }
@@ -1,46 +1,31 @@
 use crate::ai::apollo_client::ApolloClient;
 use crate::ai::clip_client::ClipClient;
 use crate::ai::face_client::FaceClient;
 use crate::ai::insight_chat::{ChatLockMap, InsightChatService};
 use crate::ai::llamacpp::LlamaCppClient;
 use crate::ai::openrouter::OpenRouterClient;
 use crate::ai::turn_registry::TurnRegistry;
 use crate::ai::{InsightGenerator, OllamaClient, SmsApiClient};
 use crate::database::{
-    CalendarEventDao, DailySummaryDao, ExifDao, InsightDao, InsightGenerationJobDao, KnowledgeDao,
+    CalendarEventDao, DailySummaryDao, ExifDao, InsightDao, KnowledgeDao, LocationHistoryDao,
-    LocationHistoryDao, SearchHistoryDao, SqliteCalendarEventDao, SqliteDailySummaryDao,
+    SearchHistoryDao, SqliteCalendarEventDao, SqliteDailySummaryDao, SqliteExifDao,
-    SqliteExifDao, SqliteInsightDao, SqliteInsightGenerationJobDao, SqliteKnowledgeDao,
+    SqliteInsightDao, SqliteKnowledgeDao, SqliteLocationHistoryDao, SqliteSearchHistoryDao,
-    SqliteLocationHistoryDao, SqliteSearchHistoryDao, connect,
+    connect,
 };
 use crate::database::{PreviewDao, SqlitePreviewDao};
 use crate::faces;
 use crate::libraries::{self, Library, LibraryHealthMap};
 use crate::tags::{SqliteTagDao, TagDao};
 use crate::video::actors::{
    PlaylistGenerator, PreviewClipGenerator, StreamActor, VideoPlaylistManager,
 };
 use actix::{Actor, Addr};
 use std::collections::HashMap;
 use std::env;
-use std::sync::{Arc, Mutex, RwLock};
+use std::sync::{Arc, Mutex};
 pub struct AppState {
    pub stream_manager: Arc<Addr<StreamActor>>,
    pub playlist_manager: Arc<Addr<VideoPlaylistManager>>,
    pub preview_clip_generator: Arc<Addr<PreviewClipGenerator>>,
    /// All configured media libraries. Ordered by `id` ascending; the first
-    /// entry is the primary library. Frozen at startup — handlers that
+    /// entry is the primary library.
    /// only need stable lookup (id → name / root_path) read this. Mutable
    /// flags (`enabled`, `excluded_dirs`) reflect their startup values;
    /// for live state see [`AppState::live_libraries`].
    pub libraries: Vec<Library>,
    /// Live view of the libraries table, shared mutably between the
    /// watcher (which reads it at the top of each tick to honour the
    /// latest `enabled` / `excluded_dirs`) and the PATCH /libraries/{id}
    /// handler (which writes it on a successful mutation). The split
    /// from [`AppState::libraries`] is deliberate: handlers that only
    /// look up by id don't need to take a lock per request.
    pub live_libraries: Arc<RwLock<Vec<Library>>>,
    /// Per-library availability snapshot. Updated by the file watcher at
    /// the top of each tick via `libraries::refresh_health`. HTTP handlers
    /// read it (e.g. `/libraries` surfacing). See "Library availability
@@ -65,25 +50,15 @@ pub struct AppState {
    /// Curated list of OpenRouter model ids exposed to clients. Sourced from
    /// `OPENROUTER_ALLOWED_MODELS` (comma-separated). Empty when unset.
    pub openrouter_allowed_models: Vec<String>,
    /// `None` when `LLAMA_SWAP_URL` is not configured. Consulted only when a
    /// request explicitly opts into `backend=llamacpp`. Same shape as the
    /// `openrouter` slot — present here so handlers can route to it without
    /// threading through the generator.
    #[allow(dead_code)]
    pub llamacpp: Option<Arc<LlamaCppClient>>,
    /// Curated list of llama-swap model ids exposed to clients. Sourced from
    /// `LLAMA_SWAP_ALLOWED_MODELS` (comma-separated). Empty when unset; the
    /// server then falls back to `LLAMA_SWAP_PRIMARY_MODEL`.
    pub llamacpp_allowed_models: Vec<String>,
    pub sms_client: SmsApiClient,
    pub insight_generator: InsightGenerator,
    /// Chat continuation service. Hold an Arc so handlers can clone cheaply.
    pub insight_chat: Arc<InsightChatService>,
-    pub turn_registry: Arc<TurnRegistry>,
+    /// Face inference client (calls Apollo's `/api/internal/faces/*`).
    /// Disabled (`is_enabled() == false`) when neither `APOLLO_FACE_API_BASE_URL`
    /// nor `APOLLO_API_BASE_URL` is set; the file-watch hook (Phase 3) and
    /// manual-face-create handler short-circuit in that case.
    pub face_client: FaceClient,
    pub clip_client: ClipClient,
    pub insight_job_dao: Arc<Mutex<Box<dyn InsightGenerationJobDao>>>,
    pub insight_job_handles: Arc<Mutex<HashMap<i32, tokio::task::AbortHandle>>>,
 }
 impl AppState {
@@ -114,24 +89,18 @@ impl AppState {
        ollama: OllamaClient,
        openrouter: Option<Arc<OpenRouterClient>>,
        openrouter_allowed_models: Vec<String>,
        llamacpp: Option<Arc<LlamaCppClient>>,
        llamacpp_allowed_models: Vec<String>,
        sms_client: SmsApiClient,
        insight_generator: InsightGenerator,
        insight_chat: Arc<InsightChatService>,
        turn_registry: Arc<TurnRegistry>,
        preview_dao: Arc<Mutex<Box<dyn PreviewDao>>>,
        face_client: FaceClient,
        clip_client: ClipClient,
        insight_job_dao: Arc<Mutex<Box<dyn InsightGenerationJobDao>>>,
        insight_job_handles: Arc<Mutex<HashMap<i32, tokio::task::AbortHandle>>>,
    ) -> Self {
        assert!(
            !libraries_vec.is_empty(),
            "AppState::new requires at least one library"
        );
        let base_path = libraries_vec[0].root_path.clone();
-        let playlist_generator = PlaylistGenerator::new(video_path.clone());
+        let playlist_generator = PlaylistGenerator::new();
        let video_playlist_manager =
            VideoPlaylistManager::new(video_path.clone(), playlist_generator.start());
@@ -142,13 +111,11 @@ impl AppState {
        );
        let library_health = libraries::new_health_map(&libraries_vec);
        let live_libraries = Arc::new(RwLock::new(libraries_vec.clone()));
        Self {
            stream_manager,
            playlist_manager: Arc::new(video_playlist_manager.start()),
            preview_clip_generator: Arc::new(preview_clip_generator.start()),
            libraries: libraries_vec,
            live_libraries,
            library_health,
            base_path,
            thumbnail_path,
@@ -159,16 +126,10 @@ impl AppState {
            ollama,
            openrouter,
            openrouter_allowed_models,
            llamacpp,
            llamacpp_allowed_models,
            sms_client,
            insight_generator,
            insight_chat,
            turn_registry,
            face_client,
            clip_client,
            insight_job_dao,
            insight_job_handles,
        }
    }
@@ -205,9 +166,6 @@ impl Default for AppState {
        let openrouter = build_openrouter_from_env();
        let openrouter_allowed_models = parse_openrouter_allowed_models();
        let llamacpp = build_llamacpp_from_env();
        let llamacpp_allowed_models = parse_llamacpp_allowed_models();
        let sms_api_url =
            env::var("SMS_API_URL").unwrap_or_else(|_| "http://localhost:8000".to_string());
        let sms_api_token = env::var("SMS_API_TOKEN").ok();
@@ -227,9 +185,6 @@ impl Default for AppState {
            .or_else(|| env::var("APOLLO_API_BASE_URL").ok());
        let face_client = FaceClient::new(face_client_url);
        // CLIP inference client. Same env var fallback as face_client.
        let clip_client = ClipClient::from_env();
        // Initialize DAOs
        let insight_dao: Arc<Mutex<Box<dyn InsightDao>>> =
            Arc::new(Mutex::new(Box::new(SqliteInsightDao::new())));
@@ -251,17 +206,6 @@ impl Default for AppState {
            Arc::new(Mutex::new(Box::new(SqliteTagDao::default())));
        let knowledge_dao: Arc<Mutex<Box<dyn KnowledgeDao>>> =
            Arc::new(Mutex::new(Box::new(SqliteKnowledgeDao::new())));
        let persona_dao: Arc<Mutex<Box<dyn crate::database::PersonaDao>>> = Arc::new(Mutex::new(
            Box::new(crate::database::SqlitePersonaDao::new()),
        ));
        let face_dao: Arc<Mutex<Box<dyn faces::FaceDao>>> =
            Arc::new(Mutex::new(Box::new(faces::SqliteFaceDao::new())));
        // Initialize insight generation job DAO (async generation tracking)
        let insight_job_dao: Arc<Mutex<Box<dyn InsightGenerationJobDao>>> =
            Arc::new(Mutex::new(Box::new(SqliteInsightGenerationJobDao::new())));
        let insight_job_handles: Arc<Mutex<HashMap<i32, tokio::task::AbortHandle>>> =
            Arc::new(Mutex::new(HashMap::new()));
        // Load base path and ensure the primary library row reflects it.
        let base_path = env::var("BASE_PATH").expect("BASE_PATH was not set in the env");
@@ -278,7 +222,6 @@ impl Default for AppState {
        let insight_generator = InsightGenerator::new(
            ollama.clone(),
            openrouter.clone(),
            llamacpp.clone(),
            sms_client.clone(),
            apollo_client.clone(),
            insight_dao.clone(),
@@ -288,9 +231,7 @@ impl Default for AppState {
            location_dao.clone(),
            search_dao.clone(),
            tag_dao.clone(),
            face_dao.clone(),
            knowledge_dao,
            persona_dao,
            libraries_vec.clone(),
        );
@@ -300,18 +241,12 @@ impl Default for AppState {
            Arc::new(tokio::sync::Mutex::new(std::collections::HashMap::new()));
        let insight_chat = Arc::new(InsightChatService::new(
            Arc::new(insight_generator.clone()),
            ollama.clone(),
            openrouter.clone(),
            insight_dao.clone(),
            chat_locks,
        ));
        // Turn registry for reconnectable chat turns. 5-minute timeout for
        // stale turns (background cleaner drops entries older than this).
        let timeout_secs: u64 = env::var("INSIGHT_CHAT_TURN_TIMEOUT_SECS")
            .ok()
            .and_then(|v| v.parse().ok())
            .unwrap_or(300);
        let turn_registry = Arc::new(TurnRegistry::new(timeout_secs));
        // Ensure preview clips directory exists
        let preview_clips_path =
            env::var("PREVIEW_CLIPS_DIRECTORY").unwrap_or_else(|_| "preview_clips".to_string());
@@ -329,17 +264,11 @@ impl Default for AppState {
            ollama,
            openrouter,
            openrouter_allowed_models,
            llamacpp,
            llamacpp_allowed_models,
            sms_client,
            insight_generator,
            insight_chat,
            turn_registry,
            preview_dao,
            face_client,
            clip_client,
            insight_job_dao,
            insight_job_handles,
        )
    }
 }
@@ -375,45 +304,10 @@ fn parse_openrouter_allowed_models() -> Vec<String> {
        .collect()
 }
 /// Build a `LlamaCppClient` from environment variables. Returns `None` when
 /// `LLAMA_SWAP_URL` is unset. The client is constructed unconditionally
 /// when the URL is set (so it's available even under `LLM_BACKEND=ollama`
 /// for ad-hoc tooling), but the agentic / chat paths only route through it
 /// when `LLM_BACKEND=llamacpp`. Slot ids default to the names the bundled
 /// `llama-swap/config.yaml` uses — `chat` / `vision` / `embed`.
 fn build_llamacpp_from_env() -> Option<Arc<LlamaCppClient>> {
    let base_url = env::var("LLAMA_SWAP_URL").ok()?;
    let primary_model = env::var("LLAMA_SWAP_PRIMARY_MODEL").ok();
    let mut client = LlamaCppClient::new(Some(base_url), primary_model);
    if let Ok(model) = env::var("LLAMA_SWAP_EMBEDDING_MODEL") {
        client.set_embedding_model(model);
    }
    if let Ok(model) = env::var("LLAMA_SWAP_VISION_MODEL") {
        client.set_vision_model(model);
    }
    if let Ok(model) = env::var("LLAMA_SWAP_TTS_MODEL") {
        client.set_tts_model(model);
    }
    Some(Arc::new(client))
 }
 /// Parse `LLAMA_SWAP_ALLOWED_MODELS` (comma-separated) into a vec. Used to
 /// populate the model picker when `LLM_BACKEND=llamacpp` — `/insights/models`
 /// surfaces these slots with capabilities. Empty when unset.
 fn parse_llamacpp_allowed_models() -> Vec<String> {
    env::var("LLAMA_SWAP_ALLOWED_MODELS")
        .unwrap_or_default()
        .split(',')
        .map(|s| s.trim().to_string())
        .filter(|s| !s.is_empty())
        .collect()
 }
 #[cfg(test)]
 impl AppState {
    /// Creates an AppState instance for testing with temporary directories
    pub fn test_state() -> Self {
        use crate::database::insight_generation_job_dao::SqliteInsightGenerationJobDao;
        use actix::Actor;
        // Create a base temporary directory
        let temp_dir = tempfile::tempdir().expect("Failed to create temp directory");
@@ -454,11 +348,6 @@ impl AppState {
            Arc::new(Mutex::new(Box::new(SqliteTagDao::default())));
        let knowledge_dao: Arc<Mutex<Box<dyn KnowledgeDao>>> =
            Arc::new(Mutex::new(Box::new(SqliteKnowledgeDao::new())));
        let persona_dao: Arc<Mutex<Box<dyn crate::database::PersonaDao>>> = Arc::new(Mutex::new(
            Box::new(crate::database::SqlitePersonaDao::new()),
        ));
        let face_dao: Arc<Mutex<Box<dyn faces::FaceDao>>> =
            Arc::new(Mutex::new(Box::new(faces::SqliteFaceDao::new())));
        // Initialize test InsightGenerator with all data sources
        let base_path_str = base_path.to_string_lossy().to_string();
@@ -472,7 +361,6 @@ impl AppState {
        let insight_generator = InsightGenerator::new(
            ollama.clone(),
            None,
            None,
            sms_client.clone(),
            apollo_client.clone(),
            insight_dao.clone(),
@@ -482,9 +370,7 @@ impl AppState {
            location_dao.clone(),
            search_dao.clone(),
            tag_dao.clone(),
            face_dao.clone(),
            knowledge_dao,
            persona_dao,
            vec![test_lib],
        );
@@ -492,13 +378,12 @@ impl AppState {
            Arc::new(tokio::sync::Mutex::new(std::collections::HashMap::new()));
        let insight_chat = Arc::new(InsightChatService::new(
            Arc::new(insight_generator.clone()),
            ollama.clone(),
            None,
            insight_dao.clone(),
            chat_locks,
        ));
        // Turn registry for test state.
        let turn_registry = Arc::new(TurnRegistry::new(300));
        // Initialize test preview DAO
        let preview_dao: Arc<Mutex<Box<dyn PreviewDao>>> =
            Arc::new(Mutex::new(Box::new(SqlitePreviewDao::new())));
@@ -522,17 +407,11 @@ impl AppState {
            ollama,
            None,
            Vec::new(),
            None,
            Vec::new(),
            sms_client,
            insight_generator,
            insight_chat,
            turn_registry,
            preview_dao,
            FaceClient::new(None), // disabled in test
            ClipClient::new(None), // disabled in test
            Arc::new(Mutex::new(Box::new(SqliteInsightGenerationJobDao::new()))), // placeholder for test
            Arc::new(Mutex::new(HashMap::new())), // placeholder for test
        )
    }
 }
@@ -144,7 +144,6 @@ impl PreviewDao for TestPreviewDao {
        } else {
            Err(DbError {
                kind: DbErrorKind::UpdateError,
                source: None,
            })
        }
    }
@@ -1,478 +0,0 @@
 //! Thumbnail generation + the media-count Prometheus gauges.
 //!
 //! Startup and per-tick scans walk each library and produce a 200×200
 //! thumbnail under `THUMBNAILS/<library_id>/<rel_path>`, falling through
 //! a fast path (`image` crate), a RAW-preview path (`exif::extract_embedded_jpeg_preview`),
 //! and ffmpeg for video / HEIF / NEF / ARW. Files that fail every
 //! decoder get a sibling `.unsupported` sentinel so subsequent scans
 //! skip them silently.
 use std::path::{Path, PathBuf};
 use std::process::Command;
 use image::GenericImageView;
 use image::codecs::jpeg::JpegEncoder;
 use lazy_static::lazy_static;
 use log::{debug, error, info, warn};
 use opentelemetry::{
    KeyValue,
    trace::{Span, TraceContextExt, Tracer},
 };
 use prometheus::IntGauge;
 use rayon::prelude::*;
 use walkdir::DirEntry;
 use crate::content_hash;
 use crate::exif;
 use crate::file_types;
 use crate::libraries;
 use crate::otel::global_tracer;
 use crate::video::actors::{generate_image_thumbnail_ffmpeg, generate_video_thumbnail};
 /// Maximum long-edge size (px) for the large preview tier. Tuned to look
 /// crisp full-screen on a 3× phone (≈1290×2796 native) and to hold up
 /// through a few stops of pinch-zoom before the original streams in.
 /// Bigger doesn't help: callers that need true full resolution request
 /// `size=full` and the handler streams the original bytes.
 pub const LARGE_PREVIEW_MAX_DIM: u32 = 2048;
 /// JPEG quality for the large and xlarge preview tiers. 85 is the
 /// conventional "indistinguishable from source at viewing size" point —
 /// well above the `image` crate's default ~75, but well below quality-90+
 /// territory where file size doubles for no perceptible win.
 const LARGE_PREVIEW_JPEG_QUALITY: u8 = 85;
 /// Maximum long-edge size (px) for the xlarge preview tier. Bridges the
 /// gap between `large` (2048px, ~16MB decoded) and the original bytes
 /// (potentially 48+ MP / ~192MB decoded). At 4096px the decoded bitmap is
 /// ~64MB — enough for 2-3× pinch-zoom on any phone before the viewer
 /// needs to stream the true original.
 pub const XLARGE_PREVIEW_MAX_DIM: u32 = 4096;
 lazy_static! {
    pub static ref IMAGE_GAUGE: IntGauge = IntGauge::new(
        "imageserver_image_total",
        "Count of the images on the server"
    )
    .unwrap();
    pub static ref VIDEO_GAUGE: IntGauge = IntGauge::new(
        "imageserver_video_total",
        "Count of the videos on the server"
    )
    .unwrap();
 }
 /// Sentinel path written next to a would-be thumbnail when a file cannot be
 /// decoded by either the `image` crate or ffmpeg. Its presence causes future
 /// scans to skip the file instead of re-logging the failure.
 pub fn unsupported_thumbnail_sentinel(thumb_path: &Path) -> PathBuf {
    let mut s = thumb_path.as_os_str().to_owned();
    s.push(".unsupported");
    PathBuf::from(s)
 }
 pub fn generate_image_thumbnail(src: &Path, thumb_path: &Path) -> std::io::Result<()> {
    // The `image` crate doesn't auto-apply EXIF Orientation on load, and
    // saving back out as JPEG drops EXIF entirely — so without baking the
    // rotation into the pixels here, browsers see the raw landscape buffer
    // of a portrait phone shot and render it sideways. Read once up front
    // and apply to whichever decode branch we end up taking.
    let orientation = exif::read_orientation(src).unwrap_or(1);
    // RAW formats (ARW/NEF/CR2/etc): try the file's embedded JPEG preview
    // first. Avoids ffmpeg choking on proprietary RAW compression (Sony ARW
    // in particular), and is faster than decoding RAW pixels anyway.
    if let Some(preview) = exif::extract_embedded_jpeg_preview(src) {
        let img = image::load_from_memory(&preview).map_err(|e| {
            std::io::Error::new(
                std::io::ErrorKind::InvalidData,
                format!("decode embedded preview {:?}: {}", src, e),
            )
        })?;
        let img = exif::apply_orientation(img, orientation);
        let scaled = img.thumbnail(200, u32::MAX);
        scaled
            .save_with_format(thumb_path, image::ImageFormat::Jpeg)
            .map_err(|e| std::io::Error::other(format!("save {:?}: {}", thumb_path, e)))?;
        return Ok(());
    }
    if file_types::needs_ffmpeg_thumbnail(src) {
        return generate_image_thumbnail_ffmpeg(src, thumb_path);
    }
    let img = image::open(src).map_err(|e| {
        std::io::Error::new(std::io::ErrorKind::InvalidData, format!("{:?}: {}", src, e))
    })?;
    let img = exif::apply_orientation(img, orientation);
    let scaled = img.thumbnail(200, u32::MAX);
    scaled
        .save(thumb_path)
        .map_err(|e| std::io::Error::other(format!("save {:?}: {}", thumb_path, e)))?;
    Ok(())
 }
 /// Generate the on-demand large-preview tier (≈2048 long edge JPEG).
 ///
 /// Mirrors [`generate_image_thumbnail`]'s decode waterfall — embedded RAW
 /// preview, then ffmpeg for HEIC/HEIF, then the `image` crate — but
 /// resizes to [`LARGE_PREVIEW_MAX_DIM`] instead of 200 and encodes at
 /// quality 85 rather than the crate default. Caller is expected to have
 /// already created the destination's parent dir.
 ///
 /// Does not upscale: if the source's long edge is already below the cap,
 /// the file is encoded at its native size (still re-saved as JPEG so the
 /// served bytes match for callers that key off `Content-Length`).
 pub fn generate_large_preview(src: &Path, dest: &Path) -> std::io::Result<()> {
    let orientation = exif::read_orientation(src).unwrap_or(1);
    // RAW: prefer the in-file embedded JPEG preview over raw-sensor decode.
    // The preview is typically already 1–2 MP and avoids RAW codec quirks.
    if let Some(preview) = exif::extract_embedded_jpeg_preview(src) {
        let img = image::load_from_memory(&preview).map_err(|e| {
            std::io::Error::new(
                std::io::ErrorKind::InvalidData,
                format!("decode embedded preview {:?}: {}", src, e),
            )
        })?;
        let img = exif::apply_orientation(img, orientation);
        return encode_large_jpeg(img, dest);
    }
    if file_types::needs_ffmpeg_thumbnail(src) {
        return generate_large_preview_ffmpeg(src, dest);
    }
    let img = image::open(src).map_err(|e| {
        std::io::Error::new(std::io::ErrorKind::InvalidData, format!("{:?}: {}", src, e))
    })?;
    let img = exif::apply_orientation(img, orientation);
    encode_large_jpeg(img, dest)
 }
 /// Resize-if-needed + JPEG-encode at q85. Used by both the embedded-preview
 /// and image-crate-decode branches of `generate_large_preview`.
 fn encode_large_jpeg(img: image::DynamicImage, dest: &Path) -> std::io::Result<()> {
    let (w, h) = img.dimensions();
    let max_dim = w.max(h);
    // Avoid upscaling tiny sources — pointless work and adds nothing for
    // the viewer. `thumbnail` would scale up freely; explicit guard.
    let scaled = if max_dim > LARGE_PREVIEW_MAX_DIM {
        img.thumbnail(LARGE_PREVIEW_MAX_DIM, LARGE_PREVIEW_MAX_DIM)
    } else {
        img
    };
    let file = std::fs::File::create(dest)
        .map_err(|e| std::io::Error::other(format!("create {:?}: {}", dest, e)))?;
    let mut writer = std::io::BufWriter::new(file);
    let mut encoder = JpegEncoder::new_with_quality(&mut writer, LARGE_PREVIEW_JPEG_QUALITY);
    encoder
        .encode_image(&scaled)
        .map_err(|e| std::io::Error::other(format!("encode {:?}: {}", dest, e)))?;
    Ok(())
 }
 /// ffmpeg path for HEIC/HEIF (image crate can't decode these). Mirrors
 /// [`crate::video::actors::generate_image_thumbnail_ffmpeg`] but scales
 /// to the large-preview cap instead of 200.
 fn generate_large_preview_ffmpeg(src: &Path, dest: &Path) -> std::io::Result<()> {
    // scale=W:-1 with force_original_aspect_ratio=decrease + the min(iw,W)
    // trick caps the long edge regardless of orientation, mirroring what
    // image::thumbnail does for the non-ffmpeg branch.
    let vf = format!(
        "scale='if(gt(iw,ih),min(iw,{cap}),-1)':'if(gt(iw,ih),-1,min(ih,{cap}))'",
        cap = LARGE_PREVIEW_MAX_DIM
    );
    let output = Command::new("ffmpeg")
        .arg("-y")
        .arg("-i")
        .arg(src)
        .arg("-vframes")
        .arg("1")
        .arg("-vf")
        .arg(&vf)
        .arg("-q:v")
        // ffmpeg's mjpeg qscale: 2 ≈ ~q95, 5 ≈ ~q85, 10 ≈ ~q70. We pick
        // 5 to match the non-ffmpeg branch's q85 target.
        .arg("5")
        .arg("-f")
        .arg("image2")
        .arg("-c:v")
        .arg("mjpeg")
        .arg(dest)
        .output()?;
    if !output.status.success() {
        return Err(std::io::Error::other(format!(
            "ffmpeg failed ({}): {}",
            output.status,
            String::from_utf8_lossy(&output.stderr).trim()
        )));
    }
    Ok(())
 }
 /// Generate the on-demand xlarge-preview tier (≈4096 long edge JPEG).
 ///
 /// Same waterfall as [`generate_large_preview`] but targeting
 /// [`XLARGE_PREVIEW_MAX_DIM`]. Sources whose long edge is already below
 /// the cap are encoded at native size (no upscale).
 pub fn generate_xlarge_preview(src: &Path, dest: &Path) -> std::io::Result<()> {
    let orientation = exif::read_orientation(src).unwrap_or(1);
    if let Some(preview) = exif::extract_embedded_jpeg_preview(src) {
        let img = image::load_from_memory(&preview).map_err(|e| {
            std::io::Error::new(
                std::io::ErrorKind::InvalidData,
                format!("decode embedded preview {:?}: {}", src, e),
            )
        })?;
        let img = exif::apply_orientation(img, orientation);
        return encode_xlarge_jpeg(img, dest);
    }
    if file_types::needs_ffmpeg_thumbnail(src) {
        return generate_xlarge_preview_ffmpeg(src, dest);
    }
    let img = image::open(src).map_err(|e| {
        std::io::Error::new(std::io::ErrorKind::InvalidData, format!("{:?}: {}", src, e))
    })?;
    let img = exif::apply_orientation(img, orientation);
    encode_xlarge_jpeg(img, dest)
 }
 fn encode_xlarge_jpeg(img: image::DynamicImage, dest: &Path) -> std::io::Result<()> {
    let (w, h) = img.dimensions();
    let max_dim = w.max(h);
    let scaled = if max_dim > XLARGE_PREVIEW_MAX_DIM {
        img.thumbnail(XLARGE_PREVIEW_MAX_DIM, XLARGE_PREVIEW_MAX_DIM)
    } else {
        img
    };
    let file = std::fs::File::create(dest)
        .map_err(|e| std::io::Error::other(format!("create {:?}: {}", dest, e)))?;
    let mut writer = std::io::BufWriter::new(file);
    let mut encoder = JpegEncoder::new_with_quality(&mut writer, LARGE_PREVIEW_JPEG_QUALITY);
    encoder
        .encode_image(&scaled)
        .map_err(|e| std::io::Error::other(format!("encode {:?}: {}", dest, e)))?;
    Ok(())
 }
 fn generate_xlarge_preview_ffmpeg(src: &Path, dest: &Path) -> std::io::Result<()> {
    let vf = format!(
        "scale='if(gt(iw,ih),min(iw,{cap}),-1)':'if(gt(iw,ih),-1,min(ih,{cap}))'",
        cap = XLARGE_PREVIEW_MAX_DIM
    );
    let output = Command::new("ffmpeg")
        .arg("-y")
        .arg("-i")
        .arg(src)
        .arg("-vframes")
        .arg("1")
        .arg("-vf")
        .arg(&vf)
        .arg("-q:v")
        .arg("5")
        .arg("-f")
        .arg("image2")
        .arg("-c:v")
        .arg("mjpeg")
        .arg(dest)
        .output()?;
    if !output.status.success() {
        return Err(std::io::Error::other(format!(
            "ffmpeg failed ({}): {}",
            output.status,
            String::from_utf8_lossy(&output.stderr).trim()
        )));
    }
    Ok(())
 }
 pub fn create_thumbnails(libs: &[libraries::Library], excluded_dirs: &[String]) {
    let tracer = global_tracer();
    let span = tracer.start("creating thumbnails");
    let thumbs = &dotenv::var("THUMBNAILS").expect("THUMBNAILS not defined");
    let thumbnail_directory: &Path = Path::new(thumbs);
    for lib in libs {
        info!(
            "Scanning thumbnails for library '{}' at {}",
            lib.name, lib.root_path
        );
        let images = PathBuf::from(&lib.root_path);
        // Effective excludes = global env-var excludes ∪ library row's
        // excluded_dirs. Lets a parent-library mount skip the subtree
        // already covered by a child library.
        let effective_excludes = lib.effective_excluded_dirs(excluded_dirs);
        // Prune EXCLUDED_DIRS so we don't generate thumbnails-of-thumbnails
        // for Synology @eaDir trees. file_scan handles filter_entry pruning.
        crate::file_scan::walk_library_files(&images, &effective_excludes)
            .into_par_iter()
            .for_each(|entry| {
                let src = entry.path();
                let Ok(relative_path) = src.strip_prefix(&images) else {
                    return;
                };
                // Library-scoped legacy path: prevents two libraries with
                // the same rel_path from clobbering each other's thumbs.
                // Hash-keyed promotion happens lazily on first hash-aware
                // request — keeping this loop ExifDao-free preserves the
                // current "cargo build && go" startup story.
                let thumb_path = content_hash::library_scoped_legacy_path(
                    thumbnail_directory,
                    lib.id,
                    relative_path,
                );
                let bare_legacy = thumbnail_directory.join(relative_path);
                // Backwards-compat check: if a single-library install has a
                // bare-legacy thumb here already, accept it as present.
                // Same for the sentinel. Means we don't redo work after
                // upgrade and we don't leave stale duplicates around.
                if thumb_path.exists()
                    || bare_legacy.exists()
                    || unsupported_thumbnail_sentinel(&thumb_path).exists()
                    || unsupported_thumbnail_sentinel(&bare_legacy).exists()
                {
                    return;
                }
                let Some(parent) = thumb_path.parent() else {
                    return;
                };
                if let Err(e) = std::fs::create_dir_all(parent) {
                    error!("Failed to create thumbnail dir {:?}: {}", parent, e);
                    return;
                }
                if is_video(&entry) {
                    let mut video_span = tracer.start_with_context(
                        "generate_video_thumbnail",
                        &opentelemetry::Context::new()
                            .with_remote_span_context(span.span_context().clone()),
                    );
                    video_span.set_attributes(vec![
                        KeyValue::new("type", "video"),
                        KeyValue::new("file-name", thumb_path.display().to_string()),
                        KeyValue::new("library", lib.name.clone()),
                    ]);
                    debug!("Generating video thumbnail: {:?}", thumb_path);
                    if let Err(e) = generate_video_thumbnail(src, &thumb_path) {
                        let sentinel = unsupported_thumbnail_sentinel(&thumb_path);
                        error!(
                            "Unable to thumbnail video {:?}: {}. Writing sentinel {:?}",
                            src, e, sentinel
                        );
                        if let Err(se) = std::fs::write(&sentinel, b"") {
                            warn!("Failed to write sentinel {:?}: {}", sentinel, se);
                        }
                    }
                    video_span.end();
                } else if is_image(&entry) {
                    match generate_image_thumbnail(src, &thumb_path) {
                        Ok(_) => info!("Saved thumbnail: {:?}", thumb_path),
                        Err(e) => {
                            let sentinel = unsupported_thumbnail_sentinel(&thumb_path);
                            error!(
                                "Unable to thumbnail {:?}: {}. Writing sentinel {:?}",
                                src, e, sentinel
                            );
                            if let Err(se) = std::fs::write(&sentinel, b"") {
                                warn!("Failed to write sentinel {:?}: {}", sentinel, se);
                            }
                        }
                    }
                }
            });
    }
    debug!("Finished making thumbnails");
    for lib in libs {
        let effective_excludes = lib.effective_excluded_dirs(excluded_dirs);
        update_media_counts(Path::new(&lib.root_path), &effective_excludes);
    }
 }
 pub fn update_media_counts(media_dir: &Path, excluded_dirs: &[String]) {
    let mut image_count = 0;
    let mut video_count = 0;
    for entry in crate::file_scan::walk_library_files(media_dir, excluded_dirs) {
        if is_image(&entry) {
            image_count += 1;
        } else if is_video(&entry) {
            video_count += 1;
        }
    }
    IMAGE_GAUGE.set(image_count);
    VIDEO_GAUGE.set(video_count);
 }
 pub fn is_image(entry: &DirEntry) -> bool {
    file_types::direntry_is_image(entry)
 }
 pub fn is_video(entry: &DirEntry) -> bool {
    file_types::direntry_is_video(entry)
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use std::fs;
    use tempfile::TempDir;
    #[test]
    fn unsupported_thumbnail_sentinel_appends_suffix() {
        let p = Path::new("/thumbs/lib1/photo.jpg");
        let s = unsupported_thumbnail_sentinel(p);
        assert_eq!(s, PathBuf::from("/thumbs/lib1/photo.jpg.unsupported"));
    }
    #[test]
    fn unsupported_thumbnail_sentinel_preserves_extension_so_existing_thumb_is_distinct() {
        // A future scan checks both `thumb.exists()` and
        // `sentinel.exists()` — they must be distinct paths.
        let p = Path::new("foo.jpeg");
        let s = unsupported_thumbnail_sentinel(p);
        assert_ne!(s, PathBuf::from("foo.jpeg"));
        assert!(s.to_string_lossy().ends_with(".unsupported"));
    }
    #[test]
    fn unsupported_thumbnail_sentinel_handles_paths_without_extension() {
        let p = Path::new("/thumbs/notes");
        let s = unsupported_thumbnail_sentinel(p);
        assert_eq!(s, PathBuf::from("/thumbs/notes.unsupported"));
    }
    /// Smoke-test update_media_counts: build a tempdir with two images
    /// and one video, run the walker, and assert the gauges line up.
    /// Exercises the is_image / is_video classifier on real DirEntry
    /// values without needing a Prometheus registry.
    #[test]
    fn update_media_counts_counts_images_and_videos_in_tempdir() {
        let tmp = TempDir::new().expect("tempdir");
        fs::write(tmp.path().join("a.jpg"), b"").unwrap();
        fs::write(tmp.path().join("b.png"), b"").unwrap();
        fs::write(tmp.path().join("c.mp4"), b"").unwrap();
        fs::write(tmp.path().join("notes.txt"), b"").unwrap();
        // Reset gauges first in case another test mutated them — the
        // gauges are process-global statics.
        IMAGE_GAUGE.set(0);
        VIDEO_GAUGE.set(0);
        update_media_counts(tmp.path(), &[]);
        assert_eq!(IMAGE_GAUGE.get(), 2, "jpg + png");
        assert_eq!(VIDEO_GAUGE.get(), 1, "mp4");
    }
 }
@@ -1,18 +1,18 @@
 use crate::content_hash;
 use crate::database::PreviewDao;
 use crate::is_video;
 use crate::libraries::Library;
 use crate::otel::global_tracer;
-use crate::video::ffmpeg::{generate_preview_clip, get_duration_seconds_blocking};
+use crate::video::ffmpeg::generate_preview_clip;
 use crate::video::hls_paths;
 use actix::prelude::*;
-use log::{debug, error, info, warn};
+use log::{debug, error, info, trace, warn};
 use opentelemetry::KeyValue;
 use opentelemetry::trace::{Span, Status, Tracer};
 use std::io::Result;
 use std::path::{Path, PathBuf};
-use std::process::{Command, Stdio};
+use std::process::{Child, Command, ExitStatus, Stdio};
 use std::sync::{Arc, Mutex};
 use tokio::sync::Semaphore;
 use walkdir::{DirEntry, WalkDir};
 // ffmpeg -i test.mp4 -c:v h264 -flags +cgop -g 30 -hls_time 3 out.m3u8
 // ffmpeg -i "filename.mp4" -preset veryfast -c:v libx264 -f hls -hls_list_size 100 -hls_time 2 -crf 24 -vf scale=1080:-2,setsar=1:1 attempt/vid_out.m3u8
@@ -22,72 +22,104 @@ impl Actor for StreamActor {
    type Context = Context<Self>;
 }
-/// A video paired with its content hash, ready to be queued for HLS
+pub struct ProcessMessage(pub String, pub Child);
-/// playlist generation. Hash is required because all output paths are
+
-/// keyed on it; callers that lack a hash (rows mid-backfill) must skip
+impl Message for ProcessMessage {
-/// the video rather than fabricate one.
+    type Result = Result<ExitStatus>;
 #[derive(Debug, Clone)]
 pub struct VideoToQueue {
    pub video_path: PathBuf,
    pub content_hash: String,
 }
-pub fn generate_video_thumbnail(path: &Path, destination: &Path) -> std::io::Result<()> {
+impl Handler<ProcessMessage> for StreamActor {
-    // Probe duration up front and seek to ~50% — gives a more
+    type Result = Result<ExitStatus>;
    // representative frame than a fixed offset (skipping title cards on
    // long videos, landing inside the clip on 1–2s Snapchat MP4s) and
    // sidesteps the seek-past-EOF class of bug entirely. When duration
    // probing fails (LRV files, fragmented MP4s, ffprobe missing) fall
    // back to the first frame: ugly but reliable.
    //
    // -vf scale + -c:v mjpeg mirrors `generate_image_thumbnail_ffmpeg`. The
    // filter chain matters as much as the scale does: without it, ffmpeg
    // hands the decoded frame straight to the mjpeg encoder, which rejects
    // any non-yuvj420p source ("Non full-range YUV is non-standard"). The
    // filter chain lets ffmpeg auto-insert the pix_fmt converter the
    // encoder needs, which is how the image-thumbnail path already handles
    // the same class of source.
    let seek = get_duration_seconds_blocking(path).map(|d| format!("{:.3}", d / 2.0));
-    let mut cmd = Command::new("ffmpeg");
+    fn handle(&mut self, msg: ProcessMessage, _ctx: &mut Self::Context) -> Self::Result {
-    cmd.arg("-y");
+        trace!("Message received");
-    if let Some(s) = &seek {
+        let mut process = msg.1;
-        cmd.arg("-ss").arg(s);
+        let result = process.wait();
        debug!(
            "Finished waiting for: {:?}. Code: {:?}",
            msg.0,
            result
                .as_ref()
                .map_or(-1, |status| status.code().unwrap_or(-1))
        );
        result
    }
-    let output = cmd
+}
 pub fn playlist_file_for(playlist_dir: &str, video_path: &Path) -> PathBuf {
    let filename = video_path
        .file_name()
        .and_then(|n| n.to_str())
        .unwrap_or("unknown");
    PathBuf::from(format!("{}/{}.m3u8", playlist_dir, filename))
 }
 /// Sentinel path written next to a would-be playlist when ffmpeg cannot
 /// transcode the source (e.g. truncated mp4 with no moov atom). Its presence
 /// causes future scans to skip the file instead of re-running ffmpeg every
 /// pass. Delete the `.unsupported` file to force a retry.
 pub fn playlist_unsupported_sentinel(playlist_file: &Path) -> PathBuf {
    let mut s = playlist_file.as_os_str().to_owned();
    s.push(".unsupported");
    PathBuf::from(s)
 }
 pub async fn create_playlist(video_path: &str, playlist_file: &str) -> Result<Child> {
    if Path::new(playlist_file).exists() {
        debug!("Playlist already exists: {}", playlist_file);
        return Err(std::io::Error::from(std::io::ErrorKind::AlreadyExists));
    }
    let result = Command::new("ffmpeg")
        .arg("-i")
-        .arg(path)
+        .arg(video_path)
        .arg("-c:v")
        .arg("h264")
        .arg("-crf")
        .arg("21")
        .arg("-preset")
        .arg("veryfast")
        .arg("-hls_time")
        .arg("3")
        .arg("-hls_list_size")
        .arg("0")
        .arg("-hls_playlist_type")
        .arg("vod")
        .arg("-vf")
        .arg("scale='min(1080,iw)':-2,setsar=1:1")
        .arg(playlist_file)
        .stdout(Stdio::null())
        .stderr(Stdio::null())
        .spawn();
    let start_time = std::time::Instant::now();
    loop {
        actix::clock::sleep(std::time::Duration::from_secs(1)).await;
        if Path::new(playlist_file).exists()
            || std::time::Instant::now() - start_time > std::time::Duration::from_secs(5)
        {
            break;
        }
    }
    result
 }
 pub fn generate_video_thumbnail(path: &Path, destination: &Path) {
    Command::new("ffmpeg")
        .arg("-ss")
        .arg("3")
        .arg("-i")
        .arg(path.to_str().unwrap())
        .arg("-vframes")
        .arg("1")
        .arg("-vf")
        .arg("scale=200:-1")
        .arg("-f")
        .arg("image2")
        .arg("-c:v")
        .arg("mjpeg")
        .arg(destination)
-        .output()?;
+        .output()
-
+        .expect("Failure to create video frame");
    if !output.status.success() {
        return Err(std::io::Error::other(format!(
            "ffmpeg failed ({}): {}",
            output.status,
            String::from_utf8_lossy(&output.stderr).trim()
        )));
    }
    // ffmpeg can exit 0 without writing a frame for malformed files where
    // the probe duration lies. Confirm a non-empty file actually landed —
    // returning Err makes the caller write the `.unsupported` sentinel so
    // we stop re-detecting on every scan.
    let wrote = std::fs::metadata(destination)
        .map(|m| m.len() > 0)
        .unwrap_or(false);
    if !wrote {
        return Err(std::io::Error::other(
            "ffmpeg exited successfully but produced no thumbnail output",
        ));
    }
    Ok(())
 }
 /// Use ffmpeg to extract a 200px-wide thumbnail from formats the `image` crate
@@ -122,36 +154,16 @@ pub fn generate_image_thumbnail_ffmpeg(path: &Path, destination: &Path) -> std::
 /// Video stream metadata needed to pick HLS encode settings. Populated by
 /// a single ffprobe call to avoid spawning multiple subprocesses per video.
 #[derive(Debug, Default)]
-pub struct VideoStreamMeta {
+struct VideoStreamMeta {
-    pub is_h264: bool,
+    is_h264: bool,
    /// Rotation in degrees (0/90/180/270). Checks both the legacy `rotate`
    /// stream tag and the modern display-matrix side data.
-    pub rotation: i32,
+    rotation: i32,
    /// Frames per second. Prefers `avg_frame_rate` (handles VFR better than
    /// `r_frame_rate`, which lies on variable-framerate sources). `None`
    /// when ffprobe couldn't parse either field — caller picks a fallback.
    pub frame_rate: Option<f32>,
 }
 /// Parse ffprobe's rational frame-rate strings (`"30000/1001"`,
 /// `"60/1"`, `"0/0"`). Rejects 0/0 (ffprobe's "unknown" sentinel),
 /// non-positive results, and anything wildly out of range so a malformed
 /// probe can't poison the scrubber's step size.
 fn parse_ffprobe_rational(s: &str) -> Option<f32> {
    let (num, den) = s.split_once('/')?;
    let num: f32 = num.parse().ok()?;
    let den: f32 = den.parse().ok()?;
    if den.abs() < f32::EPSILON {
        return None;
    }
    let v = num / den;
    (v.is_finite() && v > 0.0 && v < 1000.0).then_some(v)
 }
 /// Probe video stream metadata in one ffprobe call. Returns default (codec
-/// unknown, rotation 0, fps None) on any failure — callers fall back to
+/// unknown, rotation 0) on any failure — callers fall back to transcoding.
-/// transcoding / a default framerate.
+async fn probe_video_stream_meta(video_path: &str) -> VideoStreamMeta {
 pub async fn probe_video_stream_meta(video_path: &str) -> VideoStreamMeta {
    let output = tokio::process::Command::new("ffprobe")
        .arg("-v")
        .arg("error")
@@ -159,16 +171,8 @@ pub async fn probe_video_stream_meta(video_path: &str) -> VideoStreamMeta {
        .arg("v:0")
        .arg("-print_format")
        .arg("json")
        // NOTE: request `stream_side_data_list` (stream-level side data, read
        // from the moov atom), NOT the bare `side_data_list` section. On modern
        // ffprobe the latter is the *frame* side-data section, which forces
        // ffprobe to enumerate every frame — reading the entire mdat over the
        // network. For non-faststart phone clips on an SMB mount that turned a
        // metadata probe into a full-file read (tens of seconds per open). The
        // Display Matrix rotation we need is present at stream level, so this
        // keeps codec/fps/rotation while reading only the header.
        .arg("-show_entries")
-        .arg("stream=codec_name,r_frame_rate,avg_frame_rate:stream_tags=rotate:stream_side_data_list")
+        .arg("stream=codec_name:stream_tags=rotate:side_data_list")
        .arg(video_path)
        .output()
        .await;
@@ -219,29 +223,12 @@ pub async fn probe_video_stream_meta(video_path: &str) -> VideoStreamMeta {
        })
        .unwrap_or(0);
    // ffprobe reports frame rates as rational strings like "30000/1001".
    // Prefer avg_frame_rate (handles VFR) and fall back to r_frame_rate.
    let frame_rate = stream
        .get("avg_frame_rate")
        .and_then(|v| v.as_str())
        .and_then(parse_ffprobe_rational)
        .or_else(|| {
            stream
                .get("r_frame_rate")
                .and_then(|v| v.as_str())
                .and_then(parse_ffprobe_rational)
        });
    debug!(
-        "Probed {}: codec_h264={}, rotation={}°, fps={:?}",
+        "Probed {}: codec_h264={}, rotation={}°",
-        video_path, is_h264, rotation, frame_rate
+        video_path, is_h264, rotation
    );
-    VideoStreamMeta {
+    VideoStreamMeta { is_h264, rotation }
        is_h264,
        rotation,
        frame_rate,
    }
 }
 /// Probe the max keyframe interval (GOP) in the first ~30s of a video.
@@ -301,17 +288,17 @@ async fn get_max_gop_seconds(video_path: &str) -> Option<f64> {
 }
 pub struct VideoPlaylistManager {
-    video_dir: PathBuf,
+    playlist_dir: PathBuf,
    playlist_generator: Addr<PlaylistGenerator>,
 }
 impl VideoPlaylistManager {
    pub fn new<P: Into<PathBuf>>(
-        video_dir: P,
+        playlist_dir: P,
        playlist_generator: Addr<PlaylistGenerator>,
    ) -> Self {
        Self {
-            video_dir: video_dir.into(),
+            playlist_dir: playlist_dir.into(),
            playlist_generator,
        }
    }
@@ -321,68 +308,144 @@ impl Actor for VideoPlaylistManager {
    type Context = Context<Self>;
 }
 impl Handler<ScanDirectoryMessage> for VideoPlaylistManager {
    type Result = ResponseFuture<()>;
    fn handle(&mut self, msg: ScanDirectoryMessage, _ctx: &mut Self::Context) -> Self::Result {
        let tracer = global_tracer();
        let mut span = tracer.start("videoplaylistmanager.scan_directory");
        let start = std::time::Instant::now();
        info!(
            "Starting scan directory for video playlist generation: {}",
            msg.directory
        );
        let playlist_output_dir = self.playlist_dir.clone();
        let playlist_dir_str = playlist_output_dir.to_str().unwrap().to_string();
        let video_files = WalkDir::new(&msg.directory)
            .into_iter()
            .filter_map(|e| e.ok())
            .filter(|e| e.file_type().is_file())
            .filter(is_video)
            .filter(|e| {
                let playlist = playlist_file_for(&playlist_dir_str, e.path());
                !playlist.exists() && !playlist_unsupported_sentinel(&playlist).exists()
            })
            .collect::<Vec<DirEntry>>();
        let scan_dir_name = msg.directory.clone();
        let playlist_generator = self.playlist_generator.clone();
        Box::pin(async move {
            for e in video_files {
                let path = e.path();
                let path_as_str = path.to_str().unwrap();
                debug!(
                    "Sending generate playlist message for path: {}",
                    path_as_str
                );
                match playlist_generator
                    .send(GeneratePlaylistMessage {
                        playlist_path: playlist_output_dir.to_str().unwrap().to_string(),
                        video_path: PathBuf::from(path),
                    })
                    .await
                    .expect("Failed to send generate playlist message")
                {
                    Ok(_) => {
                        span.add_event(
                            "Playlist generated",
                            vec![KeyValue::new("video_path", path_as_str.to_string())],
                        );
                        debug!(
                            "Successfully generated playlist for file: '{}'",
                            path_as_str
                        );
                    }
                    Err(e) if e.kind() == std::io::ErrorKind::AlreadyExists => {
                        debug!("Playlist already exists for '{:?}', skipping", path);
                    }
                    Err(e) => {
                        warn!("Failed to generate playlist for path '{:?}'. {:?}", path, e);
                    }
                }
            }
            span.add_event(
                "Finished directory scan",
                vec![KeyValue::new("directory", scan_dir_name.to_string())],
            );
            info!(
                "Finished directory scan of '{}' in {:?}",
                scan_dir_name,
                start.elapsed()
            );
        })
    }
 }
 impl Handler<QueueVideosMessage> for VideoPlaylistManager {
    type Result = ();
    fn handle(&mut self, msg: QueueVideosMessage, _ctx: &mut Self::Context) -> Self::Result {
-        if msg.videos.is_empty() {
+        if msg.video_paths.is_empty() {
            return;
        }
-        let video_dir = self.video_dir.clone();
+        info!(
            "Queueing {} videos for HLS playlist generation",
            msg.video_paths.len()
        );
        let playlist_output_dir = self.playlist_dir.clone();
        let playlist_dir_str = playlist_output_dir.to_str().unwrap().to_string();
        let playlist_generator = self.playlist_generator.clone();
-        let mut queued = 0usize;
+        for video_path in msg.video_paths {
-        let mut already_present = 0usize;
+            let playlist = playlist_file_for(&playlist_dir_str, &video_path);
-        for VideoToQueue {
+            if playlist.exists() || playlist_unsupported_sentinel(&playlist).exists() {
            video_path,
            content_hash,
        } in msg.videos
        {
            let playlist = hls_paths::playlist_for_hash(&video_dir, &content_hash);
            let sentinel = hls_paths::sentinel_for_hash(&video_dir, &content_hash);
            if playlist.exists() || sentinel.exists() {
                already_present += 1;
                continue;
            }
-            debug!(
+            let path_str = video_path.to_string_lossy().to_string();
-                "Queueing playlist generation for {} (hash={})",
+            debug!("Queueing playlist generation for: {}", path_str);
-                video_path.display(),
+
                short_hash(&content_hash)
            );
            playlist_generator.do_send(GeneratePlaylistMessage {
                playlist_path: playlist_dir_str.clone(),
                video_path,
                content_hash,
            });
            queued += 1;
        }
        info!(
            "Queue tick: {} queued, {} skipped (playlist or sentinel already on disk)",
            queued, already_present
        );
    }
 }
 #[derive(Message)]
 #[rtype(result = "()")]
 pub struct ScanDirectoryMessage {
    pub(crate) directory: String,
 }
 #[derive(Message)]
 #[rtype(result = "()")]
 pub struct QueueVideosMessage {
-    pub videos: Vec<VideoToQueue>,
+    pub video_paths: Vec<PathBuf>,
 }
 #[derive(Message)]
 #[rtype(result = "Result<()>")]
 pub struct GeneratePlaylistMessage {
    pub video_path: PathBuf,
-    pub content_hash: String,
+    pub playlist_path: String,
 }
 pub struct PlaylistGenerator {
    semaphore: Arc<Semaphore>,
    video_dir: PathBuf,
 }
 impl PlaylistGenerator {
-    pub(crate) fn new<P: Into<PathBuf>>(video_dir: P) -> Self {
+    pub(crate) fn new() -> Self {
        // Concurrency is tunable via HLS_CONCURRENCY so operators can dial
        // it to their hardware: 1 on weak Synology boxes to avoid thermal
        // throttling, higher on desktops with spare cores.
@@ -394,7 +457,6 @@ impl PlaylistGenerator {
        info!("PlaylistGenerator: concurrency={}", concurrency);
        PlaylistGenerator {
            semaphore: Arc::new(Semaphore::new(concurrency)),
            video_dir: video_dir.into(),
        }
    }
 }
@@ -408,23 +470,20 @@ impl Handler<GeneratePlaylistMessage> for PlaylistGenerator {
    fn handle(&mut self, msg: GeneratePlaylistMessage, _ctx: &mut Self::Context) -> Self::Result {
        let video_file = msg.video_path.to_str().unwrap().to_owned();
-        let content_hash_str = msg.content_hash.clone();
+        let playlist_path = msg.playlist_path.as_str().to_owned();
        let semaphore = self.semaphore.clone();
        let video_dir = self.video_dir.clone();
-        let hash_dir = content_hash::hls_dir(&video_dir, &content_hash_str);
+        let playlist_file = format!(
-        let playlist_path = hls_paths::playlist_for_hash(&video_dir, &content_hash_str);
+            "{}/{}.m3u8",
-        let sentinel_path = hls_paths::sentinel_for_hash(&video_dir, &content_hash_str);
+            playlist_path,
-        let segment_template = hls_paths::segment_template_for_hash(&video_dir, &content_hash_str);
+            msg.video_path.file_name().unwrap().to_str().unwrap()
-        let playlist_file = playlist_path.to_string_lossy().to_string();
+        );
        let segment_pattern = segment_template.to_string_lossy().to_string();
        let tracer = global_tracer();
        let mut span = tracer
            .span_builder("playlistgenerator.generate_playlist")
            .with_attributes(vec![
                KeyValue::new("video_file", video_file.clone()),
                KeyValue::new("content_hash", content_hash_str.clone()),
                KeyValue::new("playlist_file", playlist_file.clone()),
            ])
            .start(&tracer);
@@ -448,7 +507,7 @@ impl Handler<GeneratePlaylistMessage> for PlaylistGenerator {
                )],
            );
-            if playlist_path.exists() {
+            if Path::new(&playlist_file).exists() {
                debug!("Playlist already exists: {}", playlist_file);
                span.set_status(Status::error(format!(
                    "Playlist already exists: {}",
@@ -457,19 +516,6 @@ impl Handler<GeneratePlaylistMessage> for PlaylistGenerator {
                return Err(std::io::Error::from(std::io::ErrorKind::AlreadyExists));
            }
            // Ensure the shard + hash directory exist. Idempotent — the
            // dir may already be present from a prior attempt that wrote
            // a sentinel before being cleared for retry.
            if let Err(e) = tokio::fs::create_dir_all(&hash_dir).await {
                error!(
                    "Failed to create HLS hash dir {}: {}",
                    hash_dir.display(),
                    e
                );
                span.set_status(Status::error(format!("mkdir failed: {}", e)));
                return Err(e);
            }
            // One ffprobe call for codec + rotation metadata.
            let stream_meta = probe_video_stream_meta(&video_file).await;
            let is_h264 = stream_meta.is_h264;
@@ -530,11 +576,16 @@ impl Handler<GeneratePlaylistMessage> for PlaylistGenerator {
                span.add_event("Transcoding to h264", vec![]);
            }
-            // Encode to a .tmp playlist alongside the final inside the
+            // Encode to a .tmp playlist and explicit segment names so a failed
-            // hash dir, so a concurrent scan never sees a half-written
+            // encode leaves predictable artifacts we can clean up — and so a
-            // .m3u8 as "done". Segments use the hash-keyed template;
+            // concurrent scan doesn't see a half-written .m3u8 as "done".
            // ffmpeg writes them next to the playlist (relative refs).
            let playlist_tmp = format!("{}.tmp", playlist_file);
            let video_stem = msg
                .video_path
                .file_name()
                .and_then(|n| n.to_str())
                .unwrap_or("video");
            let segment_pattern = format!("{}/{}_%03d.ts", playlist_path, video_stem);
            let mut cmd = tokio::process::Command::new("ffmpeg");
            cmd.arg("-y").arg("-i").arg(&video_file);
@@ -623,12 +674,12 @@ impl Handler<GeneratePlaylistMessage> for PlaylistGenerator {
            let success = matches!(&ffmpeg_result, Ok(out) if out.status.success());
            if success {
-                if let Err(e) = tokio::fs::rename(&playlist_tmp, &playlist_path).await {
+                if let Err(e) = tokio::fs::rename(&playlist_tmp, &playlist_file).await {
                    error!(
                        "ffmpeg succeeded but rename {} -> {} failed: {}",
                        playlist_tmp, playlist_file, e
                    );
-                    cleanup_partial_hls(&hash_dir).await;
+                    cleanup_partial_hls(&playlist_tmp, playlist_path.as_str(), video_stem).await;
                    span.set_status(Status::error(format!("rename failed: {}", e)));
                    return Err(e);
                }
@@ -645,17 +696,18 @@ impl Handler<GeneratePlaylistMessage> for PlaylistGenerator {
                    Err(e) => format!("ffmpeg failed: {}", e),
                };
                error!("ffmpeg failed for {}: {}", video_file, detail);
-                cleanup_partial_hls(&hash_dir).await;
+                cleanup_partial_hls(&playlist_tmp, playlist_path.as_str(), video_stem).await;
-                if let Err(se) = tokio::fs::write(&sentinel_path, b"").await {
+                let sentinel = playlist_unsupported_sentinel(Path::new(&playlist_file));
                if let Err(se) = tokio::fs::write(&sentinel, b"").await {
                    warn!(
                        "Failed to write playlist sentinel {}: {}",
-                        sentinel_path.display(),
+                        sentinel.display(),
                        se
                    );
                } else {
                    info!(
                        "Wrote playlist sentinel {} so future scans skip {}",
-                        sentinel_path.display(),
+                        sentinel.display(),
                        video_file
                    );
                }
@@ -666,47 +718,29 @@ impl Handler<GeneratePlaylistMessage> for PlaylistGenerator {
    }
 }
-/// Delete the partial playlist (.tmp) and any segment files left behind by
+/// Delete the temp playlist and any segment files that ffmpeg may have written
-/// a failed ffmpeg run. Wipes every non-sentinel file in the hash dir;
+/// before failing. Called both on ffmpeg error and on rename failure so a
-/// retains the sentinel if one has already been written by an earlier
+/// retry on the next scan starts from a clean slate.
-/// caller in the same path (today there is none, but kept defensively so
+async fn cleanup_partial_hls(playlist_tmp: &str, playlist_dir: &str, video_stem: &str) {
-/// the function is safe to call after sentinel write too).
+    let _ = tokio::fs::remove_file(playlist_tmp).await;
-async fn cleanup_partial_hls(hash_dir: &Path) {
+
-    let Ok(mut entries) = tokio::fs::read_dir(hash_dir).await else {
+    let segment_prefix = format!("{}_", video_stem);
    let Ok(mut entries) = tokio::fs::read_dir(playlist_dir).await else {
        return;
    };
    while let Ok(Some(entry)) = entries.next_entry().await {
-        let path = entry.path();
+        let Some(name) = entry.file_name().to_str().map(str::to_owned) else {
        let is_sentinel = path
            .file_name()
            .and_then(|n| n.to_str())
            .map(|n| n == hls_paths::UNSUPPORTED_SENTINEL_FILENAME)
            .unwrap_or(false);
        if is_sentinel {
            continue;
-        }
+        };
-        if let Err(e) = tokio::fs::remove_file(&path).await {
+        if name.starts_with(&segment_prefix)
-            warn!(
+            && name.ends_with(".ts")
-                "Failed to remove partial HLS file {}: {}",
+            && let Err(e) = tokio::fs::remove_file(entry.path()).await
-                path.display(),
+        {
-                e
+            warn!("Failed to remove partial segment {}: {}", name, e);
            );
        }
    }
 }
 /// First 16 chars of a content hash for log lines. Short enough to keep
 /// log volume sane, long enough that distinct hashes don't collide in
 /// practice.
 fn short_hash(hash: &str) -> &str {
    let end = hash
        .char_indices()
        .nth(16)
        .map(|(i, _)| i)
        .unwrap_or(hash.len());
    &hash[..end]
 }
 #[derive(Message)]
 #[rtype(result = "()")]
 pub struct GeneratePreviewClipMessage {
@@ -831,50 +865,3 @@ impl Handler<GeneratePreviewClipMessage> for PreviewClipGenerator {
        })
    }
 }
 #[cfg(test)]
 mod tests {
    use super::parse_ffprobe_rational;
    #[test]
    fn parses_common_rational_framerates() {
        // NTSC 29.97 fps
        assert!((parse_ffprobe_rational("30000/1001").unwrap() - 29.970_03).abs() < 1e-3);
        // Plain integer fps
        assert!((parse_ffprobe_rational("30/1").unwrap() - 30.0).abs() < 1e-6);
        assert!((parse_ffprobe_rational("60/1").unwrap() - 60.0).abs() < 1e-6);
        // iPhone slow-mo
        assert!((parse_ffprobe_rational("240/1").unwrap() - 240.0).abs() < 1e-6);
    }
    #[test]
    fn rejects_ffprobe_unknown_sentinel() {
        // 0/0 is ffprobe's way of saying "I don't know" — must not be
        // interpreted as 0 fps.
        assert_eq!(parse_ffprobe_rational("0/0"), None);
    }
    #[test]
    fn rejects_malformed_input() {
        assert_eq!(parse_ffprobe_rational(""), None);
        assert_eq!(parse_ffprobe_rational("30"), None);
        assert_eq!(parse_ffprobe_rational("/1"), None);
        assert_eq!(parse_ffprobe_rational("30/"), None);
        assert_eq!(parse_ffprobe_rational("abc/def"), None);
    }
    #[test]
    fn rejects_non_positive_results() {
        // Negative numerator -> negative fps; meaningless.
        assert_eq!(parse_ffprobe_rational("-30/1"), None);
        // Zero numerator -> zero fps; also meaningless for frame stepping.
        assert_eq!(parse_ffprobe_rational("0/1"), None);
    }
    #[test]
    fn rejects_out_of_range() {
        // Anything > 1000 fps is almost certainly garbage probe output,
        // not a real source. (Real high-speed capture maxes near 1 kHz.)
        assert_eq!(parse_ffprobe_rational("999999/1"), None);
    }
 }
@@ -223,83 +223,20 @@ impl Ffmpeg {
 }
 /// Get video duration in seconds as f64 for precise interval calculation.
-///
+async fn get_duration_seconds(input_file: &str) -> Result<f64> {
-/// Returns `Ok(None)` when ffprobe runs successfully but the container has no
+    Command::new("ffprobe")
 /// readable duration (notably GoPro `LRV` low-res preview files, some
 /// fragmented MP4s, and short Snapchat clips with stripped headers). Callers
 /// can fall back to a duration-agnostic encode rather than treating this as
 /// a hard failure — previously the `parse::<f64>` on empty stdout produced
 /// "cannot parse float from empty string" and poisoned the preview-clip row
 /// with status=failed, which the watcher would re-queue every full scan.
 async fn get_duration_seconds(input_file: &str) -> Result<Option<f64>> {
    if let Some(d) = probe_duration(input_file, "format=duration").await? {
        return Ok(Some(d));
    }
    // Fall back to the per-stream duration — populated for some MP4s where
    // the format-level duration tag is missing.
    probe_duration(input_file, "stream=duration").await
 }
 /// Synchronous cousin of `get_duration_seconds`, for callers running on
 /// blocking thread pools (Rayon). Same fallback strategy: tries
 /// `format=duration`, then `stream=duration`. Returns `None` for any
 /// failure — ffprobe missing, container without a duration tag, parse
 /// error — so callers can pick a duration-agnostic default.
 pub fn get_duration_seconds_blocking(input_file: &std::path::Path) -> Option<f64> {
    if let Some(d) = probe_duration_blocking(input_file, "format=duration") {
        return Some(d);
    }
    probe_duration_blocking(input_file, "stream=duration")
 }
 fn probe_duration_blocking(input_file: &std::path::Path, show_entries: &str) -> Option<f64> {
    let out = std::process::Command::new("ffprobe")
        .args(["-v", "quiet"])
        .args(["-show_entries", show_entries])
        .args(["-of", "csv=p=0"])
        .arg("-i")
        .arg(input_file)
        .output()
        .ok()?;
    let raw = String::from_utf8_lossy(&out.stdout);
    parse_ffprobe_duration(&raw)
 }
 async fn probe_duration(input_file: &str, show_entries: &str) -> Result<Option<f64>> {
    let out = Command::new("ffprobe")
        .args(["-v", "quiet"])
        .args(["-show_entries", show_entries])
        .args(["-of", "csv=p=0"])
        .args(["-i", input_file])
        .args(["-show_entries", "format=duration"])
        .args(["-v", "quiet"])
        .args(["-of", "csv=p=0"])
        .output()
-        .await?;
+        .await
-    let raw = String::from_utf8_lossy(&out.stdout);
+        .map(|out| String::from_utf8_lossy(&out.stdout).trim().to_string())
-    Ok(parse_ffprobe_duration(&raw))
+        .and_then(|duration_str| {
-}
+            duration_str
-
+                .parse::<f64>()
-/// Parse ffprobe's `csv=p=0` duration output. Returns the first valid
+                .map_err(|e| std::io::Error::other(e.to_string()))
-/// positive finite duration, or `None` when there isn't one.
+        })
 ///
 /// Stream-level queries (`-show_entries stream=duration`) emit one value per
 /// stream, one per line; format-level queries emit a single line. The shape
 /// also varies — `N/A` for streams without a known duration, empty string
 /// for containers without the tag at all, and (rarely) `0`/`-1` for
 /// fragmented MP4s. All of those have to map to `None` so the caller can
 /// fall back to a duration-agnostic encode.
 fn parse_ffprobe_duration(stdout: &str) -> Option<f64> {
    for line in stdout.lines() {
        let trimmed = line.trim();
        if trimmed.is_empty() || trimmed == "N/A" {
            continue;
        }
        if let Ok(d) = trimmed.parse::<f64>()
            && d.is_finite()
            && d > 0.0
        {
            return Some(d);
        }
    }
    None
 }
 /// Generate a preview clip from a video file.
@@ -331,39 +268,28 @@ pub async fn generate_preview_clip(input_file: &str, output_file: &str) -> Resul
    cmd.arg("-i").arg(input_file);
-    // Branch on duration. `None` means ffprobe couldn't tell us — we treat
+    if duration < 1.0 {
-    // it like the <1s case and just transcode the whole file. The selected
+        // Very short video (<1s): transcode the whole thing to 480p MP4
-    // clip-duration we report back is computed alongside, so callers don't
+        // format=yuv420p ensures 10-bit sources are converted to 8-bit for h264_nvenc
-    // need to re-probe.
+        cmd.args(["-vf", "scale=-2:480,format=yuv420p"]);
-    let clip_duration = match duration {
+    } else {
-        None => {
+        let segment_count = if duration < 10.0 {
-            warn!(
+            duration.floor() as u32
-                "Unknown duration for '{}', transcoding whole file as preview",
+        } else {
-                input_file
+            10
-            );
+        };
-            cmd.args(["-vf", "scale=-2:480,format=yuv420p"]);
+        let interval = duration / segment_count as f64;
-            // Cap the encode at 10s so a long video with stripped duration
+
-            // metadata doesn't spend forever generating a "preview".
+        // format=yuv420p ensures 10-bit sources are converted to 8-bit for h264_nvenc
-            cmd.args(["-t", "10"]);
+        let vf = format!(
-            10.0
+            "select='lt(mod(t,{:.4}),1)',setpts=N/FRAME_RATE/TB,fps=30,scale=-2:480,format=yuv420p",
-        }
+            interval
-        Some(d) if d < 1.0 => {
+        );
-            cmd.args(["-vf", "scale=-2:480,format=yuv420p"]);
+        let af = format!("aselect='lt(mod(t,{:.4}),1)',asetpts=N/SR/TB", interval);
-            d
+
-        }
+        cmd.args(["-vf", &vf]);
-        Some(d) => {
+        cmd.args(["-af", &af]);
-            let segment_count = if d < 10.0 { d.floor() as u32 } else { 10 };
+    }
            let interval = d / segment_count as f64;
            let vf = format!(
                "select='lt(mod(t,{:.4}),1)',setpts=N/FRAME_RATE/TB,fps=30,scale=-2:480,format=yuv420p",
                interval
            );
            let af = format!("aselect='lt(mod(t,{:.4}),1)',asetpts=N/SR/TB", interval);
            cmd.args(["-vf", &vf]);
            cmd.args(["-af", &af]);
            if d < 10.0 { d.floor() } else { 10.0 }
        }
    };
    // Force 30fps output so high-framerate sources (60fps) don't play back
    // at double speed due to select/setpts timestamp mismatches.
@@ -394,6 +320,14 @@ pub async fn generate_preview_clip(input_file: &str, output_file: &str) -> Resul
    let metadata = std::fs::metadata(output_file)?;
    let file_size = metadata.len();
    let clip_duration = if duration < 1.0 {
        duration
    } else if duration < 10.0 {
        duration.floor()
    } else {
        10.0
    };
    info!(
        "Generated preview clip '{}' ({:.1}s, {} bytes) in {:?}",
        output_file,
@@ -404,69 +338,3 @@ pub async fn generate_preview_clip(input_file: &str, output_file: &str) -> Resul
    Ok((clip_duration, file_size))
 }
 #[cfg(test)]
 mod tests {
    use super::parse_ffprobe_duration;
    #[test]
    fn empty_output_returns_none() {
        // The original bug: ffprobe -show_entries format=duration returned
        // "" for some GoPro LRV files, and `parse::<f64>` panicked with
        // "cannot parse float from empty string".
        assert_eq!(parse_ffprobe_duration(""), None);
        assert_eq!(parse_ffprobe_duration("\n"), None);
        assert_eq!(parse_ffprobe_duration("   \n  \n"), None);
    }
    #[test]
    fn na_returns_none() {
        // ffprobe emits "N/A" for streams without a known duration.
        assert_eq!(parse_ffprobe_duration("N/A"), None);
        assert_eq!(parse_ffprobe_duration("N/A\nN/A\n"), None);
    }
    #[test]
    fn parses_simple_duration() {
        assert_eq!(parse_ffprobe_duration("12.345"), Some(12.345));
        assert_eq!(parse_ffprobe_duration("12.345\n"), Some(12.345));
        assert_eq!(parse_ffprobe_duration("0.5"), Some(0.5));
    }
    #[test]
    fn rejects_non_positive_durations() {
        // Fragmented MP4s and broken containers occasionally report 0 or a
        // negative duration. Treat as "unknown" so the caller falls back to
        // whole-file transcoding rather than dividing by zero downstream.
        assert_eq!(parse_ffprobe_duration("0"), None);
        assert_eq!(parse_ffprobe_duration("0.0"), None);
        assert_eq!(parse_ffprobe_duration("-1.5"), None);
    }
    #[test]
    fn rejects_non_finite_durations() {
        assert_eq!(parse_ffprobe_duration("inf"), None);
        assert_eq!(parse_ffprobe_duration("nan"), None);
    }
    #[test]
    fn first_valid_line_wins_for_stream_query() {
        // `-show_entries stream=duration` emits one value per stream. For a
        // video file the video stream is first; we accept it and ignore
        // any audio-stream values that follow.
        assert_eq!(parse_ffprobe_duration("12.5\n8.3\n"), Some(12.5));
    }
    #[test]
    fn skips_leading_na_and_blank_lines() {
        // Stream queries can put N/A first (e.g. data stream before the
        // video stream); the parser should keep scanning.
        assert_eq!(parse_ffprobe_duration("N/A\n\n7.25\n"), Some(7.25));
    }
    #[test]
    fn rejects_garbage() {
        assert_eq!(parse_ffprobe_duration("not a number"), None);
        assert_eq!(parse_ffprobe_duration("12.5abc"), None);
    }
 }
@@ -1,84 +0,0 @@
 //! Path layout for hash-keyed HLS output.
 //!
 //! Source-of-truth is [`crate::content_hash::hls_dir`], which gives
 //! `<video_dir>/<hash[..2]>/<hash>/`. The playlist, the per-segment files,
 //! and the "ffmpeg refused" sentinel all live inside that directory so a
 //! `.m3u8` written with relative segment references resolves correctly
 //! at serve time without any URL rewriting.
 use std::path::{Path, PathBuf};
 use crate::content_hash;
 /// Standard filename for the HLS playlist inside a hash dir. Fixed so
 /// the URL contract is `playlist.m3u8` regardless of the source video's
 /// original basename.
 pub const PLAYLIST_FILENAME: &str = "playlist.m3u8";
 /// Sentinel filename written when ffmpeg refused to transcode the
 /// source. Presence in the hash dir tells future scans to skip the file
 /// instead of re-running ffmpeg every tick. Delete to force a retry.
 pub const UNSUPPORTED_SENTINEL_FILENAME: &str = "playlist.unsupported";
 /// Segment-name template passed to ffmpeg via `-hls_segment_filename`.
 /// Segments live inside the hash dir; the playlist's relative refs
 /// resolve to siblings automatically.
 pub const SEGMENT_TEMPLATE: &str = "segment_%03d.ts";
 /// Path to the HLS playlist for a video identified by content hash.
 pub fn playlist_for_hash(video_dir: &Path, hash: &str) -> PathBuf {
    content_hash::hls_dir(video_dir, hash).join(PLAYLIST_FILENAME)
 }
 /// Path to the unsupported-source sentinel for a hash.
 pub fn sentinel_for_hash(video_dir: &Path, hash: &str) -> PathBuf {
    content_hash::hls_dir(video_dir, hash).join(UNSUPPORTED_SENTINEL_FILENAME)
 }
 /// Absolute path used as ffmpeg's `-hls_segment_filename` value.
 pub fn segment_template_for_hash(video_dir: &Path, hash: &str) -> PathBuf {
    content_hash::hls_dir(video_dir, hash).join(SEGMENT_TEMPLATE)
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    #[test]
    fn playlist_path_lives_inside_sharded_hash_dir() {
        let video = Path::new("/var/video");
        let p = playlist_for_hash(video, "abcdef0123456789");
        assert_eq!(
            p,
            PathBuf::from("/var/video/ab/abcdef0123456789/playlist.m3u8")
        );
    }
    #[test]
    fn sentinel_path_lives_alongside_playlist() {
        let video = Path::new("/var/video");
        let s = sentinel_for_hash(video, "abcdef0123456789");
        assert_eq!(
            s,
            PathBuf::from("/var/video/ab/abcdef0123456789/playlist.unsupported")
        );
    }
    #[test]
    fn segment_template_lives_alongside_playlist() {
        let video = Path::new("/var/video");
        let t = segment_template_for_hash(video, "abcdef0123456789");
        assert_eq!(
            t,
            PathBuf::from("/var/video/ab/abcdef0123456789/segment_%03d.ts")
        );
    }
    #[test]
    fn distinct_hashes_yield_distinct_dirs() {
        let video = Path::new("/var/video");
        let a = playlist_for_hash(video, "1111aaaa");
        let b = playlist_for_hash(video, "2222bbbb");
        assert_ne!(a.parent(), b.parent());
    }
 }
@@ -1,243 +0,0 @@
 //! One-shot retirement of the pre-content-hash HLS output layout.
 //!
 //! Before the hash-keyed layout landed, the actor pipeline wrote every
 //! playlist as `$VIDEO_PATH/<source-basename>.m3u8` with sibling
 //! `<source-basename>_NNN.ts` segments and a `<source-basename>.m3u8.unsupported`
 //! sentinel on ffmpeg failure. The new pipeline (see
 //! [`crate::video::hls_paths`]) puts everything inside a hash-keyed
 //! subdirectory, so the legacy flat files are orphaned the moment the
 //! upgraded binary boots — they're not served, not refreshed, and not
 //! GC'd by the new orphan cleanup (which deliberately ignores anything
 //! that doesn't sit inside a `<shard>/<hash>/` dir).
 //!
 //! This migration runs once on startup. It walks `$VIDEO_PATH` at depth
 //! 1, deletes every `.m3u8` / `.m3u8.tmp` / `.m3u8.unsupported` / `.ts`
 //! file, and reports a single info line. It is idempotent — a second
 //! run finds nothing and reports zero deletions, so it's safe to leave
 //! wired in across releases until the codebase finally drops the
 //! module.
 //!
 //! Sub-directories under `$VIDEO_PATH` are intentionally left alone:
 //! every legitimate child of `$VIDEO_PATH` in the new layout is a
 //! 2-char shard directory holding hash subdirs, and those are managed
 //! by `cleanup_orphaned_playlists`.
 use std::path::Path;
 use log::{info, warn};
 /// Counters for what the migration did this run.
 #[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]
 pub struct RetireStats {
    pub deleted_playlists: usize,
    pub deleted_segments: usize,
    pub deleted_sentinels: usize,
    pub deleted_tmp: usize,
    pub errors: usize,
 }
 impl RetireStats {
    pub fn total_deleted(&self) -> usize {
        self.deleted_playlists + self.deleted_segments + self.deleted_sentinels + self.deleted_tmp
    }
 }
 /// Delete every legacy basename-keyed HLS artifact at the root of
 /// `video_dir`. Hash dirs (children that are directories) are skipped.
 /// Returns counts so the caller can log a single line summary.
 pub fn retire_legacy_hls_output(video_dir: &Path) -> RetireStats {
    let mut stats = RetireStats::default();
    let read = match std::fs::read_dir(video_dir) {
        Ok(r) => r,
        Err(e) => {
            warn!(
                "Legacy HLS migration: cannot read {} ({}); skipping",
                video_dir.display(),
                e
            );
            return stats;
        }
    };
    for entry in read.flatten() {
        let file_type = match entry.file_type() {
            Ok(t) => t,
            Err(_) => continue,
        };
        if !file_type.is_file() {
            // Hash shard directories live here in the new layout.
            continue;
        }
        let path = entry.path();
        let Some(name) = path.file_name().and_then(|n| n.to_str()) else {
            continue;
        };
        let bucket = classify(name);
        let Some(bucket) = bucket else {
            continue;
        };
        match std::fs::remove_file(&path) {
            Ok(()) => match bucket {
                LegacyKind::Playlist => stats.deleted_playlists += 1,
                LegacyKind::Segment => stats.deleted_segments += 1,
                LegacyKind::Sentinel => stats.deleted_sentinels += 1,
                LegacyKind::Tmp => stats.deleted_tmp += 1,
            },
            Err(e) => {
                warn!(
                    "Legacy HLS migration: failed to remove {}: {}",
                    path.display(),
                    e
                );
                stats.errors += 1;
            }
        }
    }
    if stats.total_deleted() > 0 || stats.errors > 0 {
        info!(
            "Legacy HLS migration: deleted {} playlist(s), {} segment(s), {} sentinel(s), {} tmp; {} error(s)",
            stats.deleted_playlists,
            stats.deleted_segments,
            stats.deleted_sentinels,
            stats.deleted_tmp,
            stats.errors,
        );
    } else {
        info!(
            "Legacy HLS migration: nothing to do under {}",
            video_dir.display()
        );
    }
    stats
 }
 #[derive(Debug, Clone, Copy)]
 enum LegacyKind {
    Playlist,
    Segment,
    Sentinel,
    Tmp,
 }
 /// Decide whether a flat file at `$VIDEO_PATH` root is legacy HLS
 /// output. Returns `None` for anything else — operator-stashed files,
 /// new-layout files (which don't live here), etc. — so we don't rm them.
 fn classify(name: &str) -> Option<LegacyKind> {
    // Order matters: sentinel and tmp are more specific suffixes that
    // sit on top of the .m3u8 / .ts extensions, so check them first.
    if name.ends_with(".m3u8.unsupported") {
        return Some(LegacyKind::Sentinel);
    }
    if name.ends_with(".m3u8.tmp") {
        return Some(LegacyKind::Tmp);
    }
    if name.ends_with(".m3u8") {
        return Some(LegacyKind::Playlist);
    }
    if name.ends_with(".ts") {
        return Some(LegacyKind::Segment);
    }
    None
 }
 #[cfg(test)]
 mod tests {
    use super::*;
    use std::fs;
    use tempfile::tempdir;
    #[test]
    fn classify_recognises_each_legacy_artifact() {
        assert!(matches!(
            classify("IMG_0341.MOV.m3u8"),
            Some(LegacyKind::Playlist)
        ));
        assert!(matches!(
            classify("IMG_0341.MOV_000.ts"),
            Some(LegacyKind::Segment)
        ));
        assert!(matches!(
            classify("IMG_0341.MOV.m3u8.unsupported"),
            Some(LegacyKind::Sentinel)
        ));
        assert!(matches!(
            classify("IMG_0341.MOV.m3u8.tmp"),
            Some(LegacyKind::Tmp)
        ));
        assert!(classify("README.md").is_none());
        assert!(classify("ab").is_none()); // shard dir name
        assert!(classify(".keep").is_none());
    }
    #[test]
    fn retire_deletes_legacy_and_leaves_hash_dirs() {
        let tmp = tempdir().unwrap();
        let root = tmp.path();
        // Legacy artifacts at root.
        fs::write(root.join("IMG_0341.MOV.m3u8"), b"#EXTM3U").unwrap();
        fs::write(root.join("IMG_0341.MOV_000.ts"), b"\x00").unwrap();
        fs::write(root.join("IMG_0341.MOV_001.ts"), b"\x00").unwrap();
        fs::write(root.join("clip.MP4.m3u8.unsupported"), b"").unwrap();
        fs::write(root.join("partial.m3u8.tmp"), b"").unwrap();
        // New-layout hash dir we must NOT touch.
        let hash_dir = root.join("ab").join("a".repeat(64));
        fs::create_dir_all(&hash_dir).unwrap();
        fs::write(hash_dir.join("playlist.m3u8"), b"#EXTM3U").unwrap();
        fs::write(hash_dir.join("segment_000.ts"), b"\x00").unwrap();
        // Unrelated file we must NOT touch.
        fs::write(root.join("README.md"), b"don't touch me").unwrap();
        let stats = retire_legacy_hls_output(root);
        assert_eq!(stats.deleted_playlists, 1);
        assert_eq!(stats.deleted_segments, 2);
        assert_eq!(stats.deleted_sentinels, 1);
        assert_eq!(stats.deleted_tmp, 1);
        assert_eq!(stats.errors, 0);
        // Legacy artifacts gone.
        assert!(!root.join("IMG_0341.MOV.m3u8").exists());
        assert!(!root.join("IMG_0341.MOV_000.ts").exists());
        assert!(!root.join("clip.MP4.m3u8.unsupported").exists());
        assert!(!root.join("partial.m3u8.tmp").exists());
        // Hash dir untouched.
        assert!(hash_dir.join("playlist.m3u8").exists());
        assert!(hash_dir.join("segment_000.ts").exists());
        // Unrelated file untouched.
        assert!(root.join("README.md").exists());
    }
    #[test]
    fn retire_is_idempotent() {
        let tmp = tempdir().unwrap();
        let root = tmp.path();
        fs::write(root.join("video.mp4.m3u8"), b"#EXTM3U").unwrap();
        fs::write(root.join("video.mp4_000.ts"), b"\x00").unwrap();
        let first = retire_legacy_hls_output(root);
        assert_eq!(first.deleted_playlists + first.deleted_segments, 2);
        let second = retire_legacy_hls_output(root);
        assert_eq!(second.total_deleted(), 0);
        assert_eq!(second.errors, 0);
    }
    #[test]
    fn retire_handles_missing_dir() {
        // No panic, no error count blowing up — just a warn + zero stats.
        let tmp = tempdir().unwrap();
        let missing = tmp.path().join("does_not_exist");
        let stats = retire_legacy_hls_output(&missing);
        assert_eq!(stats.total_deleted(), 0);
        assert_eq!(stats.errors, 0);
    }
 }
@@ -1,6 +1,6 @@
 use crate::otel::global_tracer;
 use crate::thumbnails::{is_video, update_media_counts};
 use crate::video::ffmpeg::{Ffmpeg, GifType};
 use crate::{is_video, update_media_counts};
 use log::info;
 use opentelemetry::trace::Tracer;
 use std::fs;
@@ -9,8 +9,6 @@ use walkdir::WalkDir;
 pub mod actors;
 pub mod ffmpeg;
 pub mod hls_paths;
 pub mod legacy_migration;
 #[allow(dead_code)]
 pub async fn generate_video_gifs() {
Author	SHA1	Message	Date
Cameron Cordes	2ff06413c6	ai: restructure agentic user message — facts up top + forcing gate Small models (~8B) were producing generic responses regardless of persona, and bailing out of the agentic loop on iteration 1. Two underlying causes: 1. Photo facts (date, location, contact, tags, visual) were buried between "Please analyze this photo" preamble and "Use the available tools" outro. Small models skim and miss them, which is why outputs weren't anchoring to the actual photo. 2. The user message ended with "write a detailed insight" — small models took the path of least resistance and just wrote, ignoring the soft "aim to use 5 tools" floor in the system prompt. Restructured the user message: - Leads with a "## This photo" bulleted block so the metadata is visible top-down. File path, date+source, contact, location+GPS, tags, and (in hybrid) the visual description are all bullets the model can't skim past. - Replaces the prose body with a numbered "## What to do" recipe: (1) recall_facts_for_photo + recall_entities, (2) ≥3 of the time-window tools, (3) write only after tool results, referencing specific facts. "Generic narration is not acceptable" is explicit. - Ends with a hard forcing line: "YOUR FIRST RESPONSE MUST BE A TOOL CALL. Do not output any final answer text until you have called at least 5 tools." Replaces the soft "aim to" floor with a directive small models actually follow. Tradeoff: big models also follow the recipe literally and may call 5 tools when 3 would do. Optimizing for the small-model floor first; soften once that's working. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 10:59:39 -04:00
Cameron Cordes	66ea8490ab	backfill_date_taken: surface the actual diesel error in warnings The DAO swallowed every diesel::update failure as a flat `anyhow!("Update error")`, then trace_db_call further reduced it to `DbError { kind: UpdateError }`. Operators saw "update failed for lib 2 Snapchat/foo.mp4: DbError { kind: UpdateError }" with no clue why (constraint violation? type mismatch? row vanished mid-flight? DB locked?). Two changes: - Preserve the diesel error in the anyhow chain along with the input params (lib, rel_path, date_taken, source) so the cause is visible. - Log the chain at warn-level inside the DAO before the trace wrapper collapses it to DbErrorKind::UpdateError, so the warning at the call site finally has something diagnosable next to it. - Treat zero-row updates as a debug-level "row likely retired by the missing-file scan" rather than a hard failure — that case is benign and shouldn't poison the drain's error tally. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 10:41:09 -04:00
Cameron Cordes	10ba706b39	ai: reframe iteration budget as capacity, not constraint Small models (~8B) were bailing out of the agentic loop after one or two tool calls under the previous "hard budget … stop when nearly exhausted" phrasing. They read that as a conservation directive and the "trivial photos may need fewer" clause gave them an easy out. Flipped both the agentic and chat-turn prompts to frame the budget as capacity to spend, with a soft floor (≥5 tool calls before writing) and an explicit reserve clause for the final reply. Big models will still deviate when warranted; small models follow the floor. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-07 10:36:05 -04:00
Cameron Cordes	9071d05932	ai: insight tools audit — bug fixes, new tools, prompt structure Bug fixes: - get_sms_messages.days_radius is now actually honored (was hardcoded to ±4d in SmsApiClient::fetch_messages_for_contact). - describe_photo memoized for the lifetime of one agentic loop / one chat turn — re-running mid-loop produced conflicting visual descriptions in the transcript. Agentic user message: - Pre-resolve location via Apollo + Nominatim and emit one Location: line instead of bare GPS, mirroring the non-agentic flow. - Date now formats with weekday + canonical-date source so the model can hedge on fs_time-derived dates. - Hybrid mode visual block tells the model not to call describe_photo (the tool is already gated off in hybrid). System prompt structure: - custom_system_prompt now appends under an explicit "User overrides (these take precedence)" heading instead of prepending — so a custom voice/POV/format prompt actually beats the built-in defaults. - Numbered rules collapsed into bulleted "Tool-use guidance"; merged the contradictory "multiple tools BEFORE" / "after 5 calls" rules. - Chat budget annotation surfaces as its own ## heading. New tools: - recall_facts_for_entity(entity_id\|name) — facts for one entity without needing a photo path. Fills the "tell me about Sarah" chat case where recall_facts_for_photo doesn't apply. - find_photos_with_entity(entity_id\|name) — "when did I last see X / show me photos from the Tahoe trip" via entity_photo_links. - get_exif(file_path) — full EXIF row for any photo, for technical ("what camera was this on?") questions. Tools removed: - get_file_tags duplicated the inline Tags: line on the user message; exposing both gave models an excuse to "confirm" what they had. Tool descriptions tightened: - search_rag now correctly says "per-day, per-contact summaries" and explains the date is for time-decay weighting. - recall_entities warns about empty-filter dumps. - store_entity / store_fact document dedup return + snake_case predicate vocabulary. - reverse_geocode defers to the pre-resolved location and to get_personal_place_at for personal places. - get_current_datetime narrowed to time-since-photo use. Calendar / location: - get_calendar_events accepts query and embeds it for hybrid time + semantic ranking (was always passing None for the embedding). - get_location_history exposes limit; description tells the model there's no semantic ranking on this surface. New disable_writes flag: - POST /insights/generate/agentic and the chat endpoints accept disable_writes: bool. When true, drops store_entity / store_fact from the tool palette and rewrites the system prompt's knowledge- write line. Lets users explore alternate prompts (caption-style, third-person, haiku) without polluting the persistent KB. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-06 22:37:32 -04:00
		`@@ -1,2 +0,0 @@`
			`DROP INDEX IF EXISTS idx_entity_facts_superseded_by;`
			`ALTER TABLE entity_facts DROP COLUMN superseded_by;`
		`@@ -1 +0,0 @@`
			`ALTER TABLE personas DROP COLUMN reviewed_only_facts;`