Long insight/persona chats on llama-swap failed with "exceeds the
available context size" even after raising Context in the app:
- llama-server fixes its window at launch (-c) and ignores num_ctx, but
the history budget trusted the requested num_ctx, so raising it kept
MORE history than the server could hold. LlmClient::context_window()
now reports the served window (llama-swap: /upstream/<model>/props,
cached 10 min) and the budget uses min(requested, served).
- apply_context_budget could only drop tool-call pairs, so plain Q&A
chats never shrank. It now also drops whole earlier exchanges (keeping
user/assistant alternation) and, last, the current turn's tool results,
never the in-progress question.
- The budget only ran once, before the tool loop. ContextFitter now
fits every LLM call (on a copy; the stored transcript is untouched)
and, on a context-overflow error, shrinks the budget from the server's
reported token counts and retries once.
GET /insights/models reports each llama-swap model's context_window when
it's known without loading the model (cached, or currently running), for
the app's Context picker.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cameron CordesandClaude Opus 5.5