Merge pull request 'feature/insight-chat-improvements' (#83) from feature/insight-chat-improvements into master

Reviewed-on: #83

CLAUDE.md

@@ -658,7 +658,12 @@ clients whether chat is available for a given insight.
 
 - `POST /insights/chat` runs one turn of the agentic loop against the replayed
   history. Body: `{ file_path, library?, user_message, model?, backend?, num_ctx?,
-  temperature?, top_p?, top_k?, min_p?, max_iterations?, amend? }`.
+  temperature?, top_p?, top_k?, min_p?, max_iterations?, system_prompt?, amend? }`.
+  `system_prompt` is a per-turn override: in append mode (default) it's applied
+  ephemerally — the original system message is restored before persistence so
+  the stored transcript keeps its baked persona. In amend mode the override
+  stays in place and becomes the new insight row's system message. Mirrors the
+  internal `annotate_system_with_budget` swap-and-restore pattern.
 - `POST /insights/chat/stream` is the SSE variant — same request body, response
   is `text/event-stream` with events: `iteration_start`, `text` (delta), `tool_call`,
   `tool_result`, `truncated`, `done`, plus a server-emitted `error_message` on

specs/002-insight-chat-improvements.md

@@ -0,0 +1,392 @@
+# Insight Chat improvements — design
+
+**Date:** 2026-05-07
+**Branch:** `feature/insight-chat-improvements` (in both `ImageApi/` and `FileViewer-React/`)
+**Scope:** ImageApi photo-anchored insight + chat surface, plus the
+FileViewer-React client. Apollo's free/visit chat is **not** in this cycle.
+
+## Problem
+
+Three concrete gaps in today's insight + chat surface:
+
+1. **Tool drift.** ImageApi exposes 13 tools to the LLM. Some are gated on
+   `apollo_enabled` / `has_vision`, but several optional ones
+   (`search_rag`, `get_calendar_events`, `get_location_history`) are
+   registered unconditionally even when their backing tables are empty.
+   Descriptions vary in quality and a couple have outright bugs.
+2. **Inconsistent / incomplete tool descriptions.** Tools like
+   `search_messages` describe their selection rules but omit useful
+   examples; `store_fact` doesn't show the `object_entity_id` vs
+   `object_value` choice; `get_sms_messages` accepts a `days_radius`
+   parameter that the backing client silently ignores. The LLM is being
+   instructed against a slightly wrong reality.
+3. **System prompt fights the persona.** Today's generation prompt
+   prepends the user's `custom_system_prompt` and then immediately asserts
+   `"You are a personal photo memory assistant..."`. The user message
+   demands `"a detailed insight with a title and summary"`. Both
+   contradict whatever voice / shape / POV the persona just established.
+   On chat continuation the persona is baked into the stored transcript at
+   generation time and can't be changed without regenerating.
+
+## Goals
+
+- Tool catalog is **representative** — every tool registered for a turn is
+  backed by data the user actually has.
+- Tool descriptions are **concise but complete**, with examples for any
+  tool whose param choice has multiple modes or non-obvious interactions.
+- Persona / system prompt is **authoritative** for voice, length, and
+  shape — both at generation and during chat continuation.
+- Per-turn system prompt overrides on chat work without surprising
+  side-effects on the stored transcript outside `amend` mode.
+
+## Non-goals
+
+- Apollo backend / frontend changes. Separate cycle.
+- Refactoring the `generate_photo_title` post-hoc title flow. Already
+  takes `custom_system_prompt`.
+- Tool consolidation (e.g. merging `search_messages` + `get_sms_messages`).
+  Considered and deferred — keeps blast radius small.
+- Removing knowledge-memory tools (`recall_*` / `store_*`). Audit
+  confirmed they have a live read path via `knowledge.rs` HTTP routes.
+- Persisting persona changes to the stored transcript outside `amend`
+  mode. Deliberate — re-opens use the persona currently active in the
+  client, not a sticky historical setting.
+
+---
+
+## Design
+
+### A. System prompt — generation
+
+Today (`insight_generator.rs:3305–3326`):
+
+```
+[custom_system_prompt if any] +
+"You are a personal photo memory assistant helping to reconstruct..." +
+{owner_id_note} +
+{fewshot_block} +
+"IMPORTANT INSTRUCTIONS:
+1. You MUST call multiple tools...
+2. When calling get_sms_messages and search_rag...
+3. Use recall_facts_for_photo...
+...
+8. You have a hard budget of {max_iterations} iterations..."
+```
+
+The first concatenation is the bug: `custom` claims one identity, the
+next line asserts another.
+
+**New structure** — two named blocks, in order:
+
+```
+[Identity / voice / format block]    ← persona-controlled (or neutral default)
+[Procedural block]                   ← always identity-free
+```
+
+**Identity block:**
+- When `custom_system_prompt` is supplied: use that string verbatim, no
+  pre/append.
+- When not: a neutral default that doesn't fight a future persona.
+  Working text: `"You are reconstructing a memory from a photo. Use the
+  gathered context to write a thoughtful summary; you decide voice,
+  length, and shape."`
+
+**Procedural block** — identity-free, always emitted:
+
+```
+Tool-use guidance:
+- You have a budget of {max_iterations} tool-calling iterations.
+- Call tools to gather context BEFORE writing your final answer; don't
+  answer after one or two calls.
+- When calling get_sms_messages or search_rag, make at least one call
+  WITHOUT a contact filter — surrounding events matter even when a
+  contact is known.
+- Use recall_facts_for_photo + recall_entities to load any prior
+  knowledge about subjects in the photo.
+- When you identify people / places / events / things, use store_entity
+  + store_fact to grow the persistent memory.
+- A tool returning no results is informative; continue with the others.
+
+{owner_id_note if applicable}
+{fewshot_block if applicable}
+```
+
+Differences from today's "IMPORTANT INSTRUCTIONS" block: removed the
+"you are a personal photo memory assistant" framing and the explicit
+"at least 5 tool calls" floor (replaced with the softer "don't answer
+after one or two"). Few-shot stays — it's pattern-of-tool-use, not
+identity.
+
+### B. User message — generation
+
+Today (line 3357):
+
+```
+{visual_block}Please analyze this photo and gather any relevant context
+from the surrounding weeks.
+
+Photo file path: {file_path}
+Date taken: {date}
+{contact_info}
+{gps_info}
+{tags_info}
+
+Use the available tools to gather more context about this moment
+(messages, calendar events, location history, etc.), then write a
+detailed insight with a title and summary.
+```
+
+Problems: the trailing line bakes in output shape ("title and
+summary"), and the title from the resulting response is **discarded
+anyway** — `generate_photo_title` (line 3494) regenerates the title
+post-hoc from the summary. So the prompt is constraining voice for no
+data-model benefit.
+
+**New payload** — context-only, no output prescription:
+
+```
+{visual_block}Photo file path: {file_path}
+Date taken: {date}
+{contact_info}
+{gps_info}
+{tags_info}
+
+Gather context with the available tools, then respond.
+```
+
+The persona owns shape. If a user wants "title-then-paragraph" output,
+their persona prompt says so.
+
+### C. System prompt — chat continuation
+
+Add `system_prompt: Option<String>` to `ChatTurnRequest` (and to its
+HTTP wrapper `ChatTurnHttpRequest`). It carries through both the
+non-streaming `chat_turn` and the streaming `chat_turn_stream`.
+
+**Append mode (default, `amend=false`)** — ephemeral
+swap-and-restore, mirroring the existing `annotate_system_with_budget`
+pattern:
+
+1. Load stored transcript.
+2. If `system_prompt` is `Some(s)`:
+   - If first message is a `system` role: stash original content,
+     replace with `s`.
+   - Else: prepend a synthetic ephemeral system message with `s` (note
+     it's synthetic so the restore step pops it rather than rewriting).
+3. Run `annotate_system_with_budget` on top (existing per-turn budget
+   note appends to whatever's there now).
+4. Run the agentic loop.
+5. **Before persistence**, restore the original system content (or pop
+   the synthetic one). Run `restore_system_content` for the budget
+   annotation as today.
+6. Save.
+
+Result: the model sees the override; the stored transcript is
+unchanged outside the model's actual reply.
+
+**Amend mode (`amend=true`)**:
+
+- If `system_prompt` is supplied: the override stays in place during
+  the serialization for the new insight row. The new row's
+  `training_messages` system message is the override. `is_current=false`
+  flips on prior rows as today.
+- If not supplied: behaves as today (stored transcript's system message
+  carries forward unchanged).
+
+### D. FileViewer-React — client wiring
+
+`hooks/useInsightChat.tsx`:
+- `SendTurnOptions` gains `systemPromptOverride?: string | null`.
+- Inside `sendTurn`, before issuing the streaming POST:
+  1. Read the active persona's `systemPrompt` from AsyncStorage
+     (already loaded for generation flows — reuse the same accessor).
+  2. If a one-shot `systemPromptOverride` is set, append as a suffix
+     (`${persona}\n\n${override}`) so persona voice survives + override
+     tweaks the turn.
+  3. Include the resulting string as `system_prompt` on the request body.
+- No history-load change. The history endpoint still returns the stored
+  transcript.
+
+`components/InsightChatModal.tsx`:
+- Add a small "Style note" composer affordance — a one-shot text input
+  that, when filled, becomes the `systemPromptOverride` for the next
+  send. Cleared after send.
+- The existing persona chip continues to open `PersonaManagerModal`.
+
+`hooks/usePersonas.tsx` and the bundled defaults:
+- Built-in `assistant` and `journal` prompts get audited and rewritten
+  to **explicitly state voice / shape / length** — since the framework
+  no longer guarantees a default shape, the persona must.
+
+### E. Tool catalog — gating
+
+Widen `build_tool_definitions` from `(has_vision: bool, apollo_enabled:
+bool)` to a single `ToolGateOpts` struct:
+
+```rust
+pub struct ToolGateOpts {
+    pub has_vision: bool,
+    pub apollo_enabled: bool,
+    pub daily_summaries_present: bool,
+    pub calendar_present: bool,
+    pub location_history_present: bool,
+}
+```
+
+The chat / generation services compute the three new fields lazily per
+turn via `SELECT 1 FROM <table> LIMIT 1` (cheap; cached for the turn's
+duration). Lazy because operators import data after launch and we don't
+want to require a restart for the LLM to discover its new capabilities.
+
+Per-tool gating:
+
+| Tool | Existing gate | New gate |
+|---|---|---|
+| `describe_photo` | `has_vision` | unchanged |
+| `get_personal_place_at` | `apollo_enabled` | unchanged |
+| `get_calendar_events` | none | `calendar_present` |
+| `get_location_history` | none | `location_history_present` |
+| `search_rag` | none | `daily_summaries_present` |
+
+All other tools always-on. (`get_sms_messages` and `search_messages`
+fail informatively if SMS-API is unreachable; not worth a startup probe
+since intermittent failures are the same shape.)
+
+### F. Tool descriptions — convention
+
+Every description follows:
+
+1. One sentence: **what** + **when to call**.
+2. Param semantics worth knowing (units, ranges, mode behavior,
+   precedence).
+3. **Example invocation** for tools with multiple modes, optional bands,
+   or non-obvious parameter interactions.
+4. Cross-references when relevant: `prefer X when both apply`.
+
+Banned: all-caps section headers inside descriptions
+(`"CONTENT search"`, `"TIME-BASED fetch"`); persona-prescriptive language
+(`"you are a..."`); behavioral references to other tools by description
+rather than name.
+
+Tools getting examples: `search_messages`, `search_rag`, `store_fact`,
+`get_sms_messages`. Trivial tools (`get_current_datetime`,
+`reverse_geocode`, `get_file_tags`) skip the example.
+
+Sample (`search_messages`):
+
+> Search SMS/MMS message bodies. Modes: `fts5` (keyword + phrase + prefix
+> + AND/OR/NOT + NEAR proximity), `semantic` (embedding similarity,
+> requires generated embeddings), `hybrid` (RRF merge, recommended;
+> degrades to `fts5` when embeddings absent). Optional `start_ts` /
+> `end_ts` (real-UTC unix seconds) and `contact_id` filters. For pure
+> date / contact browsing without keywords, prefer `get_sms_messages`.
+>
+> Examples:
+> - `{query: "trader joe's"}` — phrase across all time.
+> - `{query: "dinner", contact_id: 42, start_ts: 1700000000, end_ts: 1700604800}`
+>   — keyword within a contact and a week.
+> - `{query: "NEAR(meeting work, 5)"}` — proximity search.
+
+### G. SMS tool fixes
+
+#### `get_sms_messages` — honor `days_radius`
+
+Today: `sms_client::fetch_messages_for_contact(contact, center_ts)`
+hardcodes `Duration::days(4)` (lines 31–37). The tool accepts
+`days_radius` and silently ignores it.
+
+**Fix:** widen the signature to
+`fetch_messages_for_contact(contact, center_ts, days_radius)`. Tool
+plumbs through. Default 4 retained for back-compat.
+
+#### `search_messages` — add date and contact_id filters
+
+Today: ImageApi's `search_messages` only forwards `query`, `mode`,
+`limit` to SMS-API.
+
+**Fix:** add `start_ts`, `end_ts`, `contact_id` parameters.
+- `contact_id` forwards directly to SMS-API
+  (`/api/messages/search/?contact_id=`).
+- `start_ts` / `end_ts` are not natively accepted by SMS-API's search
+  endpoint. Apply client-side post-filter on the response (Apollo's
+  pattern: `chat_tools.py:670–680`). Bump the SMS-API `limit` to a
+  larger fetch pool when a date filter is supplied so in-window matches
+  aren't lost to out-of-window FTS rank.
+
+---
+
+## Implementation sequencing
+
+Each step is independently mergeable.
+
+### ImageApi PRs
+
+1. **Split system-prompt assembly + neutralize user message.** Two
+   named blocks; user message context-only. Default identity string
+   added. Tests: golden snapshots of the resulting `system_content`
+   with and without `custom_system_prompt`.
+2. **`system_prompt` field on chat request + swap/restore + amend
+   persistence.** Mirrors `annotate_system_with_budget` pattern. Tests:
+   round-trip system content unchanged in append mode; persisted in
+   amend mode.
+3. **`fetch_messages_for_contact` honors `days_radius`.** Tool wires
+   the param through. Tests: window math at the client level.
+4. **`ToolGateOpts` + per-tool description rewrites.** Description
+   text changes are the bulk of the diff but no behavior change beyond
+   gating.
+
+### FileViewer-React PR
+
+5. **Chat hook sends `system_prompt`; modal gets style-note input;
+   built-in personas updated to specify shape.** The
+   `useInsightChat.sendTurn` call site picks up the persona and
+   includes it on every chat turn body. Style-note input is a one-shot
+   suffix.
+
+## Testing & verification
+
+**Automated:**
+- Unit (Rust): swap-and-restore round-trip preserves stored transcript.
+- Unit (Rust): amend mode persists override into new insight row.
+- Unit (Rust): `fetch_messages_for_contact(days_radius=N)` produces a
+  window of `2N` days centered on `center_ts`.
+- Unit (Rust): `build_tool_definitions(opts)` excludes gated tools when
+  the corresponding flag is false.
+
+**Manual:**
+- Run a chat turn against an existing insight without `system_prompt` →
+  output unchanged from baseline.
+- Same insight, with override → output reflects new voice.
+- Re-open chat → original baked persona still authoritative (override
+  was ephemeral).
+- Regenerate an insight with the journal persona → model's voice
+  matches journal style; no "memory assistant" framing leaks through.
+- Toggle data presence (delete a row from `calendar_events`) → tool
+  drops from the catalog on the next turn.
+
+## Risks
+
+- **Default identity wording matters.** A too-neutral default ("Use the
+  gathered context to write a summary") might produce flatter output
+  than today's "personal photo memory assistant" framing for users
+  who never set a persona. Mitigation: tune the default with a small
+  set of test photos before merging.
+- **Persona-suffix style notes can contradict persona voice.** A user
+  who picks `journal` (first person, warm) and adds the style note
+  "respond in bullet points" will get a tonal collision. Acceptable —
+  the user expressed a per-turn intent and we honor it. Document the
+  composition rule in the persona-manager UI.
+- **Lazy data-presence probes add a per-turn `SELECT 1`.** Negligible
+  on SQLite (sub-millisecond) but adds up across many turns. Cache the
+  result for the turn's duration; don't re-probe per-tool.
+
+## Open questions
+
+None blocking. Items deferred to a possible follow-up cycle:
+
+- Apollo parity for the same per-turn override pattern (already
+  present; just needs RN client wiring on the photo path which is
+  already proxy).
+- Tool consolidation (`search_messages` + `get_sms_messages` →
+  single `search_messages` with optional date filter, Apollo-style).
+  Considered and deferred — separate spec.

src/ai/handlers.rs

@@ -640,6 +640,10 @@ pub struct ChatTurnHttpRequest {
     pub min_p: Option<f32>,
     #[serde(default)]
     pub max_iterations: Option<usize>,
+    /// Per-turn system-prompt override. Ephemeral in append mode,
+    /// persisted in amend mode. See ChatTurnRequest for semantics.
+    #[serde(default)]
+    pub system_prompt: Option<String>,
     #[serde(default)]
     pub amend: bool,
 }
@@ -695,6 +699,7 @@ pub async fn chat_turn_handler(
         top_k: request.top_k,
         min_p: request.min_p,
         max_iterations: request.max_iterations,
+        system_prompt: request.system_prompt.clone(),
         amend: request.amend,
     };
 
@@ -909,6 +914,7 @@ pub async fn chat_stream_handler(
         top_k: request.top_k,
         min_p: request.min_p,
         max_iterations: request.max_iterations,
+        system_prompt: request.system_prompt.clone(),
         amend: request.amend,
     };
 

src/ai/insight_chat.rs

@@ -45,6 +45,11 @@ pub struct ChatTurnRequest {
     pub top_k: Option<i32>,
     pub min_p: Option<f32>,
     pub max_iterations: Option<usize>,
+    /// Per-turn system-prompt override. In append mode (default), applied
+    /// ephemerally — original system message restored before persistence.
+    /// In amend mode, persisted into the new insight row's system message.
+    /// None / empty = no change.
+    pub system_prompt: Option<String>,
     /// When true, write a new insight row (regenerating title) instead of
     /// updating training_messages on the existing row.
     pub amend: bool,
@@ -359,10 +364,12 @@ impl InsightChatService {
             .map(|imgs| !imgs.is_empty())
             .unwrap_or(false);
         let offer_describe_tool = !is_hybrid && local_first_user_has_image;
-        let tools = InsightGenerator::build_tool_definitions(
-            offer_describe_tool,
-            self.generator.apollo_enabled(),
-        );
+        // current_gate_opts(has_vision) sets gate_opts.has_vision = has_vision
+        // and probes the per-table presence flags. Pass `offer_describe_tool`
+        // directly — the `!is_hybrid && local_first_user_has_image` decision
+        // is the chat-path's vision predicate.
+        let gate_opts = self.generator.current_gate_opts(offer_describe_tool);
+        let tools = InsightGenerator::build_tool_definitions(gate_opts);
 
         // Image base64 only needed when describe_photo is on the menu. Load
         // lazily to avoid disk IO when the loop never invokes it.
@@ -385,6 +392,13 @@ impl InsightChatService {
         // 7. Append the new user turn.
         messages.push(ChatMessage::user(req.user_message.clone()));
 
+        // Apply per-turn system-prompt override BEFORE the budget annotation
+        // so the budget note attaches to the override, not the original.
+        // The stash is consumed below before persistence (append mode) or
+        // dropped (amend mode, where the override stays in place).
+        let override_stash =
+            apply_system_prompt_override(&mut messages, req.system_prompt.as_deref());
+
         // Temporarily annotate the system message with this turn's iteration
         // budget so the model knows how many tool-calling rounds it has. We
         // restore the original content before persistence so the note doesn't
@@ -481,6 +495,14 @@ impl InsightChatService {
         // before we persist so it doesn't snowball on each subsequent turn.
         restore_system_content(&mut messages, original_system_content);
 
+        // Append mode: undo the per-turn system-prompt override so the
+        // stored transcript keeps the original baked persona. Amend mode:
+        // keep the override in place — it becomes the new insight row's
+        // system message.
+        if !req.amend {
+            restore_system_prompt_override(&mut messages, override_stash);
+        }
+
         // 9. Persist. Append mode rewrites the JSON blob in place; amend
         //    mode regenerates the title and inserts a new insight row,
         //    relying on store_insight to flip prior rows' is_current=false.
@@ -790,10 +812,12 @@ impl InsightChatService {
             .map(|imgs| !imgs.is_empty())
             .unwrap_or(false);
         let offer_describe_tool = !is_hybrid && local_first_user_has_image;
-        let tools = InsightGenerator::build_tool_definitions(
-            offer_describe_tool,
-            self.generator.apollo_enabled(),
-        );
+        // current_gate_opts(has_vision) sets gate_opts.has_vision = has_vision
+        // and probes the per-table presence flags. Pass `offer_describe_tool`
+        // directly — the `!is_hybrid && local_first_user_has_image` decision
+        // is the chat-path's vision predicate.
+        let gate_opts = self.generator.current_gate_opts(offer_describe_tool);
+        let tools = InsightGenerator::build_tool_definitions(gate_opts);
 
         let image_base64: Option<String> = if offer_describe_tool {
             self.generator.load_image_as_base64(&normalized).ok()
@@ -812,6 +836,10 @@ impl InsightChatService {
 
         messages.push(ChatMessage::user(req.user_message.clone()));
 
+        // Mirror chat_turn: per-turn override goes on first, budget note next.
+        let override_stash =
+            apply_system_prompt_override(&mut messages, req.system_prompt.as_deref());
+
         let original_system_content = annotate_system_with_budget(&mut messages, max_iterations);
 
         let mut tool_calls_made = 0usize;
@@ -946,6 +974,13 @@ impl InsightChatService {
         // before we persist so it doesn't snowball on each subsequent turn.
         restore_system_content(&mut messages, original_system_content);
 
+        // Append mode: undo the per-turn system-prompt override (mirrors
+        // chat_turn). Amend mode: keep the override — it becomes the new
+        // insight row's system message.
+        if !req.amend {
+            restore_system_prompt_override(&mut messages, override_stash);
+        }
+
         // Persist.
         let json = serde_json::to_string(&messages)
             .map_err(|e| anyhow!("failed to serialize chat history: {}", e))?;
@@ -1153,6 +1188,64 @@ fn restore_system_content(messages: &mut [ChatMessage], original: Option<String>
     }
 }
 
+/// Receipt produced by [`apply_system_prompt_override`] so the caller can
+/// undo the override before persistence. Two variants because we either
+/// replaced an existing system message (need its original content) or
+/// prepended a synthetic one (need to pop it).
+#[derive(Debug)]
+pub(crate) enum SystemPromptStash {
+    Replaced { original: String },
+    Prepended,
+}
+
+/// Apply a per-turn `system_prompt` override to `messages` so the model
+/// sees the requested persona for this turn. Returns a stash the caller
+/// must pass to [`restore_system_prompt_override`] before persisting the
+/// transcript — without that step, append-mode chat would silently
+/// rewrite the stored persona.
+///
+/// No-op (returns `None`) when `override_prompt` is `None` or empty.
+pub(crate) fn apply_system_prompt_override(
+    messages: &mut Vec<ChatMessage>,
+    override_prompt: Option<&str>,
+) -> Option<SystemPromptStash> {
+    let prompt = override_prompt
+        .map(str::trim)
+        .filter(|s| !s.is_empty())?
+        .to_string();
+    if let Some(first) = messages.first_mut()
+        && first.role == "system"
+    {
+        let original = std::mem::replace(&mut first.content, prompt);
+        return Some(SystemPromptStash::Replaced { original });
+    }
+    messages.insert(0, ChatMessage::system(prompt));
+    Some(SystemPromptStash::Prepended)
+}
+
+/// Undo an override previously applied by [`apply_system_prompt_override`].
+/// No-op when `stash` is `None`.
+pub(crate) fn restore_system_prompt_override(
+    messages: &mut Vec<ChatMessage>,
+    stash: Option<SystemPromptStash>,
+) {
+    let Some(stash) = stash else { return };
+    match stash {
+        SystemPromptStash::Replaced { original } => {
+            if let Some(first) = messages.first_mut()
+                && first.role == "system"
+            {
+                first.content = original;
+            }
+        }
+        SystemPromptStash::Prepended => {
+            if matches!(messages.first(), Some(m) if m.role == "system") {
+                messages.remove(0);
+            }
+        }
+    }
+}
+
 /// View returned to clients for chat-UI rendering.
 #[derive(Debug)]
 pub struct HistoryView {
@@ -1386,4 +1479,94 @@ mod tests {
         let cut = find_raw_cut(&msgs, 2).expect("boundary cut should succeed");
         assert_eq!(cut, msgs.len());
     }
+
+    #[test]
+    fn apply_override_replaces_existing_system_message() {
+        let mut msgs = vec![
+            ChatMessage::system("original persona"),
+            ChatMessage::user("hi"),
+        ];
+        let stash = apply_system_prompt_override(&mut msgs, Some("new persona"));
+        assert_eq!(msgs[0].content, "new persona");
+        match stash {
+            Some(SystemPromptStash::Replaced { original }) => {
+                assert_eq!(original, "original persona");
+            }
+            other => panic!("expected Replaced, got {:?}", other),
+        }
+    }
+
+    #[test]
+    fn apply_override_prepends_synthetic_when_missing() {
+        let mut msgs = vec![ChatMessage::user("hi")];
+        let stash = apply_system_prompt_override(&mut msgs, Some("new persona"));
+        assert_eq!(msgs.len(), 2);
+        assert_eq!(msgs[0].role, "system");
+        assert_eq!(msgs[0].content, "new persona");
+        assert!(matches!(stash, Some(SystemPromptStash::Prepended)));
+    }
+
+    #[test]
+    fn apply_override_no_op_when_none() {
+        let mut msgs = vec![ChatMessage::system("sys"), ChatMessage::user("hi")];
+        let stash = apply_system_prompt_override(&mut msgs, None);
+        assert!(stash.is_none());
+        assert_eq!(msgs[0].content, "sys");
+    }
+
+    #[test]
+    fn apply_override_no_op_for_empty_string() {
+        let mut msgs = vec![ChatMessage::system("sys")];
+        let stash = apply_system_prompt_override(&mut msgs, Some(""));
+        assert!(stash.is_none());
+        assert_eq!(msgs[0].content, "sys");
+    }
+
+    #[test]
+    fn restore_override_replaces_back() {
+        let mut msgs = vec![ChatMessage::system("new"), ChatMessage::user("hi")];
+        restore_system_prompt_override(
+            &mut msgs,
+            Some(SystemPromptStash::Replaced {
+                original: "original".to_string(),
+            }),
+        );
+        assert_eq!(msgs[0].content, "original");
+        assert_eq!(msgs.len(), 2);
+    }
+
+    #[test]
+    fn restore_override_pops_synthetic() {
+        let mut msgs = vec![ChatMessage::system("new"), ChatMessage::user("hi")];
+        restore_system_prompt_override(&mut msgs, Some(SystemPromptStash::Prepended));
+        assert_eq!(msgs.len(), 1);
+        assert_eq!(msgs[0].role, "user");
+    }
+
+    #[test]
+    fn override_round_trip_preserves_original_system_message() {
+        let mut msgs = vec![
+            ChatMessage::system("original persona"),
+            ChatMessage::user("first user"),
+            assistant_text("first reply"),
+        ];
+        let stash = apply_system_prompt_override(&mut msgs, Some("ephemeral persona"));
+        assert_eq!(msgs[0].content, "ephemeral persona");
+        restore_system_prompt_override(&mut msgs, stash);
+        assert_eq!(msgs[0].content, "original persona");
+        assert_eq!(msgs.len(), 3);
+        assert_eq!(msgs[1].role, "user");
+        assert_eq!(msgs[2].role, "assistant");
+    }
+
+    #[test]
+    fn override_with_synthetic_round_trip_drops_extra_message() {
+        let mut msgs = vec![ChatMessage::user("first user")];
+        let stash = apply_system_prompt_override(&mut msgs, Some("ephemeral"));
+        assert_eq!(msgs.len(), 2);
+        assert_eq!(msgs[0].role, "system");
+        restore_system_prompt_override(&mut msgs, stash);
+        assert_eq!(msgs.len(), 1);
+        assert_eq!(msgs[0].role, "user");
+    }
 }

src/ai/sms_client.rs

@@ -20,31 +20,36 @@ impl SmsApiClient {
         }
     }
 
-    /// Fetch messages for a specific contact within ±4 days of the given timestamp
-    /// Falls back to all contacts if no messages found for the specific contact
-    /// Messages are sorted by proximity to the center timestamp
+    /// Compute a `[start, end]` unix-second window of `2 * radius_days`
+    /// centered on `center_ts`. `radius_days < 1` is clamped to 1 to avoid
+    /// degenerate zero-width windows.
+    pub(crate) fn window_for_radius(center_ts: i64, radius_days: i64) -> (i64, i64) {
+        let r = radius_days.max(1);
+        let span = r * 86400;
+        (center_ts - span, center_ts + span)
+    }
+
+    /// Fetch messages for a specific contact within ±`radius_days` of the
+    /// given timestamp. Falls back to all contacts when no messages found
+    /// for the named contact. Sorted by proximity to the center timestamp.
     pub async fn fetch_messages_for_contact(
         &self,
         contact: Option<&str>,
         center_timestamp: i64,
+        radius_days: i64,
     ) -> Result<Vec<SmsMessage>> {
-        use chrono::Duration;
+        let effective_radius = radius_days.max(1);
+        let (start_ts, end_ts) = Self::window_for_radius(center_timestamp, radius_days);
 
-        // Calculate ±4 days range around the center timestamp
         let center_dt = chrono::DateTime::from_timestamp(center_timestamp, 0)
             .ok_or_else(|| anyhow::anyhow!("Invalid timestamp"))?;
 
-        let start_dt = center_dt - Duration::days(4);
-        let end_dt = center_dt + Duration::days(4);
-
-        let start_ts = start_dt.timestamp();
-        let end_ts = end_dt.timestamp();
-
         // If contact specified, try fetching for that contact first
         if let Some(contact_name) = contact {
             log::info!(
-                "Fetching SMS for contact: {} (±4 days from {})",
+                "Fetching SMS for contact: {} (±{} days from {})",
                 contact_name,
+                effective_radius,
                 center_dt.format("%Y-%m-%d %H:%M:%S")
             );
             let messages = self
@@ -68,7 +73,8 @@ impl SmsApiClient {
 
         // Fallback to all contacts
         log::info!(
-            "Fetching all SMS messages (±4 days from {})",
+            "Fetching all SMS messages (±{} days from {})",
+            effective_radius,
             center_dt.format("%Y-%m-%d %H:%M:%S")
         );
         self.fetch_messages(start_ts, end_ts, None, Some(center_timestamp))
@@ -255,19 +261,26 @@ impl SmsApiClient {
     ///   - "fts5"     keyword-only, supports phrase / prefix / boolean / NEAR
     ///   - "semantic" embedding similarity
     ///   - "hybrid"   both merged via reciprocal rank fusion (recommended)
-    pub async fn search_messages(
+    ///
+    /// The SMS-API endpoint accepts `contact_id` natively; date filtering is
+    /// the caller's responsibility (post-filter on the returned rows).
+    pub async fn search_messages_with_contact(
         &self,
         query: &str,
         mode: &str,
         limit: usize,
+        contact_id: Option<i64>,
     ) -> Result<Vec<SmsSearchHit>> {
-        let url = format!(
+        let mut url = format!(
             "{}/api/messages/search/?q={}&mode={}&limit={}",
             self.base_url,
             urlencoding::encode(query),
             urlencoding::encode(mode),
             limit
         );
+        if let Some(cid) = contact_id {
+            url.push_str(&format!("&contact_id={}", cid));
+        }
 
         let mut request = self.client.get(&url);
         if let Some(token) = &self.token {
@@ -379,3 +392,29 @@ struct SmsSearchResponse {
     #[serde(default)]
     search_method: String,
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn window_for_radius_produces_2n_day_span() {
+        let center: i64 = 1_700_000_000;
+        let (start, end) = SmsApiClient::window_for_radius(center, 7);
+        assert_eq!(end - start, 14 * 86400);
+        assert_eq!(start + 7 * 86400, center);
+        assert_eq!(end - 7 * 86400, center);
+    }
+
+    #[test]
+    fn window_for_radius_clamps_zero_to_one() {
+        let (start, end) = SmsApiClient::window_for_radius(100_000, 0);
+        assert_eq!(end - start, 2 * 86400);
+    }
+
+    #[test]
+    fn window_for_radius_clamps_negative_to_one() {
+        let (start, end) = SmsApiClient::window_for_radius(100_000, -7);
+        assert_eq!(end - start, 2 * 86400);
+    }
+}

src/bin/populate_knowledge.rs

@@ -14,6 +14,7 @@ use image_api::database::{
     SqliteInsightDao, SqliteKnowledgeDao, SqliteLocationHistoryDao, SqliteSearchHistoryDao,
     connect,
 };
+use image_api::faces::{FaceDao, SqliteFaceDao};
 use image_api::file_types::{IMAGE_EXTENSIONS, VIDEO_EXTENSIONS};
 use image_api::libraries::{self, Library};
 use image_api::tags::{SqliteTagDao, TagDao};
@@ -182,6 +183,8 @@ async fn main() -> anyhow::Result<()> {
         Arc::new(Mutex::new(Box::new(SqliteTagDao::default())));
     let knowledge_dao: Arc<Mutex<Box<dyn KnowledgeDao>>> =
         Arc::new(Mutex::new(Box::new(SqliteKnowledgeDao::new())));
+    let face_dao: Arc<Mutex<Box<dyn FaceDao>>> =
+        Arc::new(Mutex::new(Box::new(SqliteFaceDao::new())));
 
     // Pass the full library set so `resolve_full_path` probes every root,
     // even when --library restricts the walk. A rel_path shared across
@@ -198,6 +201,7 @@ async fn main() -> anyhow::Result<()> {
         location_dao,
         search_dao,
         tag_dao,
+        face_dao,
         knowledge_dao,
         all_libs.clone(),
     );

src/database/daily_summary_dao.rs

@@ -75,6 +75,11 @@ pub trait DailySummaryDao: Sync + Send {
         context: &opentelemetry::Context,
         contact: &str,
     ) -> Result<i64, DbError>;
+
+    /// Cheap presence check — returns true iff at least one daily summary row
+    /// exists. Used by gating logic that only needs "is the table empty?",
+    /// avoiding a `COUNT(*)` full scan on large corpora.
+    fn has_any_summaries(&mut self, context: &opentelemetry::Context) -> Result<bool, DbError>;
 }
 
 pub struct SqliteDailySummaryDao {
@@ -454,6 +459,30 @@ impl DailySummaryDao for SqliteDailySummaryDao {
         })
         .map_err(|_| DbError::new(DbErrorKind::QueryError))
     }
+
+    fn has_any_summaries(&mut self, context: &opentelemetry::Context) -> Result<bool, DbError> {
+        trace_db_call(context, "query", "has_any_summaries", |_span| {
+            let mut conn = self
+                .connection
+                .lock()
+                .expect("Unable to get DailySummaryDao");
+
+            #[derive(QueryableByName)]
+            struct ProbeResult {
+                #[diesel(sql_type = diesel::sql_types::Integer)]
+                #[allow(dead_code)]
+                one: i32,
+            }
+
+            let rows: Vec<ProbeResult> =
+                diesel::sql_query("SELECT 1 as one FROM daily_conversation_summaries LIMIT 1")
+                    .load(conn.deref_mut())
+                    .map_err(|e| anyhow::anyhow!("Failed to probe daily summaries: {}", e))?;
+
+            Ok(!rows.is_empty())
+        })
+        .map_err(|_| DbError::new(DbErrorKind::QueryError))
+    }
 }
 
 // Helper structs for raw SQL queries

src/faces.rs

@@ -503,6 +503,10 @@ pub trait FaceDao: Send + Sync {
         into: i32,
     ) -> anyhow::Result<Person>;
 
+    /// Cheap presence probe — returns true iff at least one face has been
+    /// detected (excluding marker rows). Used by chat-tool gating.
+    fn has_any_faces(&mut self, ctx: &opentelemetry::Context) -> anyhow::Result<bool>;
+
     /// Resolve `(library_id, rel_path)` → `content_hash` via image_exif.
     /// Returns None when the photo hasn't been EXIF-indexed yet (no row
     /// in image_exif) or when the row exists but content_hash is NULL.
@@ -1432,6 +1436,19 @@ impl FaceDao for SqliteFaceDao {
         })
     }
 
+    fn has_any_faces(&mut self, ctx: &opentelemetry::Context) -> anyhow::Result<bool> {
+        let mut conn = self.connection.lock().expect("face dao lock");
+        trace_db_call(ctx, "query", "has_any_faces", |_span| {
+            face_detections::table
+                .filter(face_detections::status.eq("detected"))
+                .select(face_detections::id)
+                .first::<i32>(conn.deref_mut())
+                .optional()
+                .map(|x| x.is_some())
+                .with_context(|| "has_any_faces query")
+        })
+    }
+
     fn resolve_content_hash(
         &mut self,
         ctx: &opentelemetry::Context,

src/files.rs

@@ -1718,7 +1718,12 @@ mod tests {
             // Mock — files.rs tests don't exercise the date-override endpoints.
             // Returning a synthetic row keeps the trait satisfied without
             // depending on private DbError constructors.
-            Ok(mock_exif_row(library_id, rel_path, Some(date_taken), Some("manual".to_string())))
+            Ok(mock_exif_row(
+                library_id,
+                rel_path,
+                Some(date_taken),
+                Some("manual".to_string()),
+            ))
         }
 
         fn clear_manual_date_taken(

src/main.rs

@@ -995,10 +995,8 @@ async fn upload_image(
                             }
                         };
                         let perceptual = perceptual_hash::compute(&uploaded_path);
-                        let resolved_date = date_resolver::resolve_date_taken(
-                            &uploaded_path,
-                            exif_data.date_taken,
-                        );
+                        let resolved_date =
+                            date_resolver::resolve_date_taken(&uploaded_path, exif_data.date_taken);
                         let insert_exif = InsertImageExif {
                             library_id: target_library.id,
                             file_path: relative_path.clone(),
@@ -1022,8 +1020,7 @@ async fn upload_image(
                             size_bytes,
                             phash_64: perceptual.map(|h| h.phash_64),
                             dhash_64: perceptual.map(|h| h.dhash_64),
-                            date_taken_source: resolved_date
-                                .map(|r| r.source.as_str().to_string()),
+                            date_taken_source: resolved_date.map(|r| r.source.as_str().to_string()),
                         };
 
                         if let Ok(mut dao) = exif_dao.lock() {
@@ -1687,7 +1684,16 @@ fn create_thumbnails(libs: &[libraries::Library], excluded_dirs: &[String]) {
                     ]);
 
                     debug!("Generating video thumbnail: {:?}", thumb_path);
-                    generate_video_thumbnail(src, &thumb_path);
+                    if let Err(e) = generate_video_thumbnail(src, &thumb_path) {
+                        let sentinel = unsupported_thumbnail_sentinel(&thumb_path);
+                        error!(
+                            "Unable to thumbnail video {:?}: {}. Writing sentinel {:?}",
+                            src, e, sentinel
+                        );
+                        if let Err(se) = std::fs::write(&sentinel, b"") {
+                            warn!("Failed to write sentinel {:?}: {}", sentinel, se);
+                        }
+                    }
                     video_span.end();
                 } else if is_image(&entry) {
                     match generate_image_thumbnail(src, &thumb_path) {

src/memories.rs

@@ -213,10 +213,7 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
         // dispatch on the source-app prefix instead.
         const NON_TIMESTAMP_PREFIXES: &[&str] = &["snapchat-"];
         let lower = filename.to_ascii_lowercase();
-        if NON_TIMESTAMP_PREFIXES
-            .iter()
-            .any(|p| lower.starts_with(p))
-        {
+        if NON_TIMESTAMP_PREFIXES.iter().any(|p| lower.starts_with(p)) {
             return None;
         }
 

src/state.rs

@@ -10,6 +10,7 @@ use crate::database::{
     connect,
 };
 use crate::database::{PreviewDao, SqlitePreviewDao};
+use crate::faces;
 use crate::libraries::{self, Library, LibraryHealthMap};
 use crate::tags::{SqliteTagDao, TagDao};
 use crate::video::actors::{
@@ -206,6 +207,8 @@ impl Default for AppState {
             Arc::new(Mutex::new(Box::new(SqliteTagDao::default())));
         let knowledge_dao: Arc<Mutex<Box<dyn KnowledgeDao>>> =
             Arc::new(Mutex::new(Box::new(SqliteKnowledgeDao::new())));
+        let face_dao: Arc<Mutex<Box<dyn faces::FaceDao>>> =
+            Arc::new(Mutex::new(Box::new(faces::SqliteFaceDao::new())));
 
         // Load base path and ensure the primary library row reflects it.
         let base_path = env::var("BASE_PATH").expect("BASE_PATH was not set in the env");
@@ -231,6 +234,7 @@ impl Default for AppState {
             location_dao.clone(),
             search_dao.clone(),
             tag_dao.clone(),
+            face_dao.clone(),
             knowledge_dao,
             libraries_vec.clone(),
         );
@@ -348,6 +352,8 @@ impl AppState {
             Arc::new(Mutex::new(Box::new(SqliteTagDao::default())));
         let knowledge_dao: Arc<Mutex<Box<dyn KnowledgeDao>>> =
             Arc::new(Mutex::new(Box::new(SqliteKnowledgeDao::new())));
+        let face_dao: Arc<Mutex<Box<dyn faces::FaceDao>>> =
+            Arc::new(Mutex::new(Box::new(faces::SqliteFaceDao::new())));
 
         // Initialize test InsightGenerator with all data sources
         let base_path_str = base_path.to_string_lossy().to_string();
@@ -370,6 +376,7 @@ impl AppState {
             location_dao.clone(),
             search_dao.clone(),
             tag_dao.clone(),
+            face_dao.clone(),
             knowledge_dao,
             vec![test_lib],
         );

src/video/actors.rs

@@ -107,19 +107,39 @@ pub async fn create_playlist(video_path: &str, playlist_file: &str) -> Result<Ch
     result
 }
 
-pub fn generate_video_thumbnail(path: &Path, destination: &Path) {
-    Command::new("ffmpeg")
+pub fn generate_video_thumbnail(path: &Path, destination: &Path) -> std::io::Result<()> {
+    // -vf scale + -c:v mjpeg mirrors `generate_image_thumbnail_ffmpeg`. The
+    // filter chain matters as much as the scale does: without it, ffmpeg
+    // hands the decoded frame straight to the mjpeg encoder, which rejects
+    // any non-yuvj420p source ("Non full-range YUV is non-standard"). The
+    // filter chain lets ffmpeg auto-insert the pix_fmt converter the
+    // encoder needs, which is how the image-thumbnail path already handles
+    // the same class of source.
+    let output = Command::new("ffmpeg")
+        .arg("-y")
         .arg("-ss")
         .arg("3")
         .arg("-i")
-        .arg(path.to_str().unwrap())
+        .arg(path)
         .arg("-vframes")
         .arg("1")
+        .arg("-vf")
+        .arg("scale=200:-1")
         .arg("-f")
         .arg("image2")
+        .arg("-c:v")
+        .arg("mjpeg")
         .arg(destination)
-        .output()
-        .expect("Failure to create video frame");
+        .output()?;
+
+    if !output.status.success() {
+        return Err(std::io::Error::other(format!(
+            "ffmpeg failed ({}): {}",
+            output.status,
+            String::from_utf8_lossy(&output.stderr).trim()
+        )));
+    }
+    Ok(())
 }
 
 /// Use ffmpeg to extract a 200px-wide thumbnail from formats the `image` crate