Persona chats can now start from a message in an SMS thread, in one of two modes. Plus date_info, a hard rule that photos never go to OpenRouter, and local → hybrid switching for photo chats. Needs FileViewer-React feat/sms-anchored-persona-chat for the UI.
POST /persona_chat/conversations
- { persona_id }
+ { persona_id, // optional for as_contact: practice has no persona
+ anchor?: { contact_id, message_id, mode?: "about" | "as_contact" } }
+ → snapshots ~40 SMS-API messages up to message_id into the row (502 if unreadable)
GET /persona_chat/conversations
+ ?contact_id=N → only chats anchored on that thread (SQL filter, indexed)
Tools offered per chat:
open chat
about the thread
practice (as_contact)
messages / calendar / location / places / faces
✓
✓
–
memory (recall_*, store_*)
✓
✓
–
get_thread_messages, look_at_message_image (bound to the chat's own thread)
Practice mode plays the contact and coaches on "out of character". It runs with no persona: it's stored under PRACTICE_PERSONA_ID with a fixed stand-in, so deleting or editing personas can't affect it.
Thread tools read only the chat's own thread: the server supplies the contact, and a different contact_id from the model is refused. MMS images are checked for type and capped at 20 MB before download. The chat loop refuses tool calls that weren't offered on that turn.
Photos and OpenRouter:
OpenRouterClient::messages_to_openai // every OpenRouter request
- images → image_url parts (sent)
+ any image → Err(IMAGES_STAY_LOCAL)
OpenRouterClient::describe_image
+ always Err (hybrid describes with the local model)
insight chat continuation, stored=local, effective=hybrid
- rejected: "switching from local to hybrid mid-chat isn't supported"
+ local vision model describes the photo (cached per image)
+ description replaces the photo in what's sent
+ photo restored before saving; describe failure fails the turn
+ stored model id only reused on the same backend
Migration 2026-10-19-000000_persona_chat_sms_anchor: nullable anchor_* columns plus an index on persona_chat_conversations. Version 1.5.0 → 1.6.0.
Evidence
Before: all of this run on the e2e server with a stub SMS-API and llama-swap gemma4-31. The first two features had no API at all; date maths was left to the model. After:
Anchored "about" chat: "how long ago / what weekday was the beach day?" → one date_info call → "it was a thursday! and wow, 7 years, 3 months and 14 days ago". "6 weeks after the 4th?" → add_days: 42 → "august 15th!"
Practice: "yeah still good 💀 see ya then". Asked about something not in the thread → "what cabin?". On "out of character" it gave honest coaching.
Local → hybrid, against a fake OpenRouter that recorded every request, on all three continuation endpoints (/insights/chat, /chat/turn, /chat/stream):
has_image=False on every request (~17 KB of text vs a 62 KB stored transcript).
Model sent was fake/model, not gemma4-31.
The stored transcript still had the photo.
Switching back to local saw the real photo, with 0 OpenRouter requests.
After review fixes: practice created with no persona works ("yea 9am 💀 see u then") and survives persona deletion. The model called look_at_message_image with just a message id, and the server supplied the bound thread.
Tests:cargo test 906 passed (from 883); clippy clean. New tests cover:
## Summary
Persona chats can now start from a message in an SMS thread, in one of two modes. Plus `date_info`, a hard rule that photos never go to OpenRouter, and local → hybrid switching for photo chats. Needs FileViewer-React `feat/sms-anchored-persona-chat` for the UI.
```diff
POST /persona_chat/conversations
- { persona_id }
+ { persona_id, // optional for as_contact: practice has no persona
+ anchor?: { contact_id, message_id, mode?: "about" | "as_contact" } }
+ → snapshots ~40 SMS-API messages up to message_id into the row (502 if unreadable)
GET /persona_chat/conversations
+ ?contact_id=N → only chats anchored on that thread (SQL filter, indexed)
```
Tools offered per chat:
| | open chat | about the thread | practice (`as_contact`) |
|---|---|---|---|
| messages / calendar / location / places / faces | ✓ | ✓ | – |
| memory (`recall_*`, `store_*`) | ✓ | ✓ | – |
| `get_thread_messages`, `look_at_message_image` (bound to the chat's own thread) | – | ✓ | ✓ |
| `get_photos_near_time`, `look_at_photo` (indexed photos only) | – | ✓ | – |
| `date_info` (replaces `get_current_datetime`) | ✓ | ✓ | – |
| `describe_photo` | – (was offered with no photo) | – | – |
Practice mode plays the contact and coaches on "out of character". It runs with no persona: it's stored under `PRACTICE_PERSONA_ID` with a fixed stand-in, so deleting or editing personas can't affect it.
Thread tools read only the chat's own thread: the server supplies the contact, and a different `contact_id` from the model is refused. MMS images are checked for type and capped at 20 MB before download. The chat loop refuses tool calls that weren't offered on that turn.
Photos and OpenRouter:
```diff
OpenRouterClient::messages_to_openai // every OpenRouter request
- images → image_url parts (sent)
+ any image → Err(IMAGES_STAY_LOCAL)
OpenRouterClient::describe_image
+ always Err (hybrid describes with the local model)
insight chat continuation, stored=local, effective=hybrid
- rejected: "switching from local to hybrid mid-chat isn't supported"
+ local vision model describes the photo (cached per image)
+ description replaces the photo in what's sent
+ photo restored before saving; describe failure fails the turn
+ stored model id only reused on the same backend
```
Migration `2026-10-19-000000_persona_chat_sms_anchor`: nullable `anchor_*` columns plus an index on `persona_chat_conversations`. Version 1.5.0 → 1.6.0.
## Evidence
- **Before:** all of this run on the e2e server with a stub SMS-API and llama-swap `gemma4-31`. The first two features had no API at all; date maths was left to the model.
**After:**
- Anchored "about" chat: "how long ago / what weekday was the beach day?" → one `date_info` call → *"it was a thursday! and wow, 7 years, 3 months and 14 days ago"*. "6 weeks after the 4th?" → `add_days: 42` → *"august 15th!"*
- Practice: *"yeah still good 💀 see ya then"*. Asked about something not in the thread → *"what cabin?"*. On "out of character" it gave honest coaching.
- Local → hybrid, against a fake OpenRouter that recorded every request, on all three continuation endpoints (`/insights/chat`, `/chat/turn`, `/chat/stream`):
- `has_image=False` on every request (~17 KB of text vs a 62 KB stored transcript).
- Model sent was `fake/model`, not `gemma4-31`.
- The stored transcript still had the photo.
- Switching back to local saw the real photo, with 0 OpenRouter requests.
- After review fixes: practice created with no persona works (*"yea 9am 💀 see u then"*) and survives persona deletion. The model called `look_at_message_image` with just a message id, and the server supplied the bound thread.
- **Tests:** `cargo test` 906 passed (from 883); clippy clean. New tests cover:
- the anchored and practice prompts
- the practice-mode tool filter
- the anchor DAO round trip
- SMS thread parsing (MMS media indices)
- `date_info` (weekday, calendar gaps, month-end clamping, display zone)
- thread binding (another thread refused)
- the per-thread DAO query
- the MMS type/size check
- the OpenRouter image refusal
- the image swap and restore
## Merge Danger
**Door:** two-way
The migration only adds nullable columns and has a `down.sql`; older code ignores them. Everything else is code.
**Blast Radius:** AI chat
- The OpenRouter guard fails any path that relied on sending images to OpenRouter. None exist today; hybrid always describes locally.
- Tool-call refusal applies to the photo chat too. It now enforces the existing catalog gates instead of running whatever the model names.
- Old chat transcripts that called `get_current_datetime` replay fine, but a model repeating that name now gets "not available".
- `/insights/chat/stream` continuation now needs the local vision model to be up when the target is hybrid.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Persona conversations can now be anchored on a message in an SMS thread.
Creating one snapshots the thread up to that message (via SMS-API) into the
row; every turn appends it to the persona prompt with guidance to talk like a
text thread rather than write an insight.
Anchored turns get four tools: get_thread_messages (page the thread),
look_at_message_image (local vision on an MMS photo), get_photos_near_time
(library photos with tags and people) and look_at_photo (local vision on a
library photo). The conversation list filters by contact_id.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An anchored conversation now has a mode. 'about' is the existing behaviour (a
persona talks with the user about the thread). 'as_contact' has the model play
the contact so the user can rehearse the conversation: its own prompt (stay in
the contact's voice, don't invent facts, coach on 'out of character'), no
persona prompt, and only the thread tools — no memory, library, calendar or
other-thread access, and no knowledge writes.
The shared persona/file-chat loop now refuses tool calls that weren't in the
turn's catalog, so the gates hold even when a model names a tool it wasn't
offered.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
describe_photo describes the file chat's photo. Persona chats have no photo,
so a vision-capable model was offered a tool that could only answer "No image
available". Persona chats now build their catalog with the vision gate off;
the thread tools still cover looking at images there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Small local models get weekdays, "how long ago" and day counts between dates
wrong, confidently. Persona chats anchored on old SMS threads ask exactly
those questions. date_info answers them exactly, in the owner's display zone:
- no args: now (replaces get_current_datetime)
- date: its weekday and how long ago / until
- date + other_date: the gap in days, weeks and years/months/days, plus
hours when both carry times
- date + add_days: the date N days after (or before)
Checked live on gemma4-26b-A4B and gemma4-31: each date question took one
date_info call and came back correct.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing in the client stopped it: a message carrying images became image_url
parts and went out. Photos stayed local only because callers routed around it.
messages_to_openai, which every OpenRouter request passes through, now refuses
any image (IMAGES_STAY_LOCAL), and describe_image always errors; hybrid mode
describes photos with the local model. A caller bug now fails the turn instead
of uploading a photo.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A local insight's transcript carries the photo itself, so continuing on hybrid
was refused. Each hybrid turn now has the local vision model describe the
photo (cached per image), sends the description in its place, and puts the
photo back before saving, so the stored chat and later local turns keep it.
If the local description fails, the turn fails. The stored model id is only
reused on the same backend, so a llama-swap slot isn't sent to OpenRouter.
Checked end to end against a recording fake OpenRouter on all three
continuation endpoints: no image data left, the stored transcript kept the
photo, and switching back to local saw the photo again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Companion app PR: Apps/FileViewer-React#49 (https://git.crkc.xyz/Apps/FileViewer-React/pulls/49). Merge and deploy this one first: the app's new calls need the API and migration.
- Thread tools are bound server-side to the chat's own thread. The model
used to pass contact_id, so practice mode could read any conversation it
named; a model-supplied contact_id is now only checked, never trusted, and
the prompts no longer hand it one.
- Practice chats need no persona. They're stored under PRACTICE_PERSONA_ID
and run with an inert stand-in, so deleting or editing personas can't hide
or break them. persona_id is optional on create for as_contact.
- The per-thread list filters in SQL (list_persona_chats_for_contact, served
by idx_persona_chat_anchor_contact) instead of decoding every transcript.
- fetch_mms_image checks the content type and a 20 MB cap before reading
the body, so a video attachment isn't downloaded in full.
- get_photos_near_time unions tags over every path sharing the content, per
the multi-library model.
- look_at_photo only opens indexed photos, so an excluded folder can't be
read by guessing a path.
- One date parser: get_photos_near_time uses date_info::parse_center.
- Practice prompt source rewrapped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cameron
merged commit e8aa337442 into master2026-10-21 00:11:23 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Summary
Persona chats can now start from a message in an SMS thread, in one of two modes. Plus
date_info, a hard rule that photos never go to OpenRouter, and local → hybrid switching for photo chats. Needs FileViewer-Reactfeat/sms-anchored-persona-chatfor the UI.Tools offered per chat:
as_contact)recall_*,store_*)get_thread_messages,look_at_message_image(bound to the chat's own thread)get_photos_near_time,look_at_photo(indexed photos only)date_info(replacesget_current_datetime)describe_photoPractice mode plays the contact and coaches on "out of character". It runs with no persona: it's stored under
PRACTICE_PERSONA_IDwith a fixed stand-in, so deleting or editing personas can't affect it.Thread tools read only the chat's own thread: the server supplies the contact, and a different
contact_idfrom the model is refused. MMS images are checked for type and capped at 20 MB before download. The chat loop refuses tool calls that weren't offered on that turn.Photos and OpenRouter:
Migration
2026-10-19-000000_persona_chat_sms_anchor: nullableanchor_*columns plus an index onpersona_chat_conversations. Version 1.5.0 → 1.6.0.Evidence
gemma4-31. The first two features had no API at all; date maths was left to the model.After:
date_infocall → "it was a thursday! and wow, 7 years, 3 months and 14 days ago". "6 weeks after the 4th?" →add_days: 42→ "august 15th!"/insights/chat,/chat/turn,/chat/stream):has_image=Falseon every request (~17 KB of text vs a 62 KB stored transcript).fake/model, notgemma4-31.look_at_message_imagewith just a message id, and the server supplied the bound thread.cargo test906 passed (from 883); clippy clean. New tests cover:date_info(weekday, calendar gaps, month-end clamping, display zone)Merge Danger
Door: two-way
The migration only adds nullable columns and has a
down.sql; older code ignores them. Everything else is code.Blast Radius: AI chat
get_current_datetimereplay fine, but a model repeating that name now gets "not available"./insights/chat/streamcontinuation now needs the local vision model to be up when the target is hybrid.🤖 Generated with Claude Code
Companion app PR: Apps/FileViewer-React#49 (Apps/FileViewer-React#49). Merge and deploy this one first: the app's new calls need the API and migration.