Apps/ImageApi
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

ai: mirror chat model on local client to prevent mid-turn model swap

Browse Source 
When the user selects a model from the picker, the local client's
primary_model and vision_model now match the chat model. Prevents
llama-swap exclusive mode from swapping models when describe_photo
or rerank fires during an agentic turn.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Cameron Cordes
2026-05-24 19:27:29 -04:00
parent fb388c29d7
commit 208344ad98
1 changed files with 9 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
src/ai/insight_generator.rs
View File

@@ -3635,8 +3635,16 @@ Return ONLY the summary, nothing else."#,
}; };
// ── local client (utility calls: rerank, describe_image, etc.) ─ // ── local client (utility calls: rerank, describe_image, etc.) ─
// For llamacpp: mirror the chat model selection so rerank /
// describe_image hit the same model that's already loaded —
// avoids a mid-turn model swap in llama-swap exclusive mode.
let local: Box<dyn LlmClient> = if local_via_llamacpp { let local: Box<dyn LlmClient> = if local_via_llamacpp {
Box::new(self.llamacpp.as_ref().unwrap().as_ref().clone()) let mut lc = self.llamacpp.as_ref().unwrap().as_ref().clone();
if let Some(ref m) = overrides.model {
lc.primary_model = m.clone();
lc.set_vision_model(m.clone());
}
Box::new(lc)
} else { } else {
Box::new(self.ollama.clone()) Box::new(self.ollama.clone())
}; };