feat(insight-chat): allow switching a local photo chat to hybrid
A local insight's transcript carries the photo itself, so continuing on hybrid was refused. Each hybrid turn now has the local vision model describe the photo (cached per image), sends the description in its place, and puts the photo back before saving, so the stored chat and later local turns keep it. If the local description fails, the turn fails. The stored model id is only reused on the same backend, so a llama-swap slot isn't sent to OpenRouter. Checked end to end against a recording fake OpenRouter on all three continuation endpoints: no image data left, the stored transcript kept the photo, and switching back to local saw the photo again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
cc11884272
commit
ba39628a0d
2 files changed
+292
-43
No files matched your search
@@ -900,9 +900,18 @@ endpoint.
|
||||
a deploy-time decision, not a request-time one).
|
||||
- `hybrid → hybrid` allowed.
|
||||
- `hybrid → local` allowed (the inlined description replays as text).
|
||||
- `local → hybrid` rejected — the stored transcript has raw images in the
|
||||
first user message and OpenRouter providers don't accept that shape
|
||||
consistently. Regenerate the insight in hybrid mode instead.
|
||||
- `local → hybrid` allowed. The stored transcript carries the raw photo in
|
||||
its first user message, so each hybrid turn swaps it for a local vision
|
||||
description (cached per photo) in the copy sent to OpenRouter, then puts
|
||||
the photo back before saving (`InsightChatService::inline_images_as_text`).
|
||||
If the local model can't describe it, the turn fails. A stored model id is
|
||||
only reused on the same backend (`continuation_model`).
|
||||
|
||||
**Photos never go to OpenRouter.** `OpenRouterClient` refuses any request
|
||||
carrying image data (`IMAGES_STAY_LOCAL`) and its `describe_image` always
|
||||
errors — the client is the choke point, so a caller bug fails the turn
|
||||
instead of uploading a photo. Image work in hybrid mode goes through
|
||||
`ResolvedBackend::local()`.
|
||||
|
||||
**Insight Chat Continuation:**
|
||||
|
||||
@@ -933,9 +942,9 @@ clients whether chat is available for a given insight.
|
||||
|
||||
Backend routing rules (matches agentic-insight generation):
|
||||
- Stored `backend` on the insight row is authoritative by default.
|
||||
- `request.backend` may override per-turn. `local -> hybrid` is rejected in
|
||||
v1 (would require on-the-fly visual-description rewrite); `hybrid -> local`
|
||||
replays verbatim since the description is already inlined as text.
|
||||
- `request.backend` may override per-turn, in either direction (see the
|
||||
cross-replay matrix above); `hybrid -> local` replays verbatim since the
|
||||
description is already inlined as text.
|
||||
- `request.model` overrides the chat model (an Ollama id in local mode, an
|
||||
OpenRouter id in hybrid mode).
|
||||
|
||||
|
||||
Reference in new issue
Block a user