AI: add enable_thinking reasoning toggle plumbed to llama.cpp

New optional SamplingOverride forwarded to llama-server as
chat_template_kwargs.enable_thinking (gates Qwen3-style reasoning
blocks). None leaves the template default; other backends ignore it.
Wired through the agentic-insight and chat-turn request bodies/handlers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Opus 4.8 authored and Cameron committed 2026-06-29 12:39:16 -04:00
1 parent 7684220f52
commit 475072810e
8 files changed
+55

No files matched your search

+3
View File
@@ -3933,6 +3933,7 @@ Return ONLY the summary, nothing else."#,
if let Some(ctx) = overrides.num_ctx {
c.set_num_ctx(Some(ctx));
}
c.set_enable_thinking(overrides.enable_thinking);
Box::new(c)
} else {
// Pure Ollama local.
@@ -4064,6 +4065,7 @@ Return ONLY the summary, nothing else."#,
top_p: Option<f32>,
top_k: Option<i32>,
min_p: Option<f32>,
enable_thinking: Option<bool>,
max_iterations: usize,
backend: Option<String>,
fewshot_examples: Vec<Vec<ChatMessage>>,
@@ -4091,6 +4093,7 @@ Return ONLY the summary, nothing else."#,
top_p,
top_k,
min_p,
enable_thinking,
};
let backend = self.resolve_backend(kind, &overrides).await?;
span.set_attribute(KeyValue::new("model", backend.model().to_string()));