Accumulate streamed tool calls across chunks in Ollama streaming
Ollama >=0.8 can stream tool_calls incrementally across NDJSON chunks; chat_with_tools_stream did `tool_calls = Some(tcs)` per chunk, so only the last chunk's calls survived assembly and earlier calls were silently dropped. Append into the accumulator instead. - ollama: append_streamed_tool_calls helper + tests covering two calls arriving in separate chunks and the single-chunk batch case. - llamacpp: the SSE delta assembly was already correct (per-index BTreeMap, same-index argument fragments concatenate, distinct indexes accumulate); extracted it into apply_tool_call_deltas / finalize_tool_calls and added tests pinning that behavior. - llm_client: new shared strip_think_blocks (moved from ollama's private extract_final_answer, which now delegates) so the tool-calling final content paths can reuse it; unit tests for tagged/plain/unclosed/empty cases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
8e4f91561b
commit
592dfcb42c
3 files changed
+223
-66
No files matched your search
@@ -170,3 +170,55 @@ pub struct ModelCapabilities {
|
||||
pub has_vision: bool,
|
||||
pub has_tool_calling: bool,
|
||||
}
|
||||
|
||||
/// Strip a leading `<think>…</think>` reasoning block from model output.
|
||||
///
|
||||
/// Thinking models sometimes emit chain-of-thought inside think tags before
|
||||
/// the real answer. Everything after the first `</think>` is the answer;
|
||||
/// when no tag is present — or the text after it is empty — the trimmed
|
||||
/// input is returned unchanged. Mirrors the behavior Ollama's
|
||||
/// `extract_final_answer` has applied to single-shot generation; shared here
|
||||
/// so the tool-calling final-content paths (agentic generation + chat) can
|
||||
/// apply the identical cleanup before parsing / persisting.
|
||||
pub fn strip_think_blocks(response: &str) -> String {
|
||||
let response = response.trim();
|
||||
|
||||
if let Some(pos) = response.find("</think>") {
|
||||
let answer = response[pos + "</think>".len()..].trim();
|
||||
if !answer.is_empty() {
|
||||
return answer.to_string();
|
||||
}
|
||||
}
|
||||
|
||||
response.to_string()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn strip_think_blocks_removes_leading_think_block() {
|
||||
let raw = "<think>\nLet me reason about this.\n</think>\n\nTitle: A Day Out\n\nThe body.";
|
||||
assert_eq!(strip_think_blocks(raw), "Title: A Day Out\n\nThe body.");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn strip_think_blocks_passes_through_plain_content() {
|
||||
assert_eq!(strip_think_blocks(" just an answer "), "just an answer");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn strip_think_blocks_keeps_content_when_answer_after_tag_is_empty() {
|
||||
// A think block with nothing after it: better to return the trimmed
|
||||
// original than an empty string (matches Ollama's fallback).
|
||||
let raw = "<think>only thoughts</think>";
|
||||
assert_eq!(strip_think_blocks(raw), raw);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn strip_think_blocks_handles_unclosed_tag() {
|
||||
let raw = "<think>thinking forever";
|
||||
assert_eq!(strip_think_blocks(raw), raw);
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user