Tracing: LLM clients + streaming spans on tracing, drop cx threading (#140)
LlmClient methods no longer take an OpenTelemetry Context. llm.request is now a tracing span (otel.kind=client, llm.provider/operation/model) that parents on the caller's ambient span; traceparent injection reads the span's OTel context via OpenTelemetrySpanExt. The src/ai feature spans (handlers, insight generation, chat, persona chat, daily summaries, TTS) move to #[instrument] / info_span!, and every LLM-bearing tokio::spawn carries its parent span with .instrument(): the streaming chat turn (chat_turn_stream_inner), async chat / persona turns, background insight jobs and TTS jobs. Sub-spans in the agentic loops (ai.chat.loop, ai.agentic.loop) instrument each LLM / tool await. DAO calls keep their explicit Context (derived from the tracing span) until the DAO migration (#141). tests/otel_llm_spans.rs pins nesting + traceparent for the blocking and streaming paths across a spawn on a multi-thread runtime. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
7a59afc62b
commit
1c508cdabd
20 files changed
+973
-1030
No files matched your search
+4
-25
@@ -47,24 +47,12 @@ impl LocalLlm {
|
||||
/// pick query vs document — retrieval models treat the two sides
|
||||
/// differently and an unmarked embed invites prefix-mismatch bugs.
|
||||
pub async fn embed_query(&self, text: &str) -> Result<Vec<f32>> {
|
||||
super::embed_query(
|
||||
&opentelemetry::Context::new(),
|
||||
&self.ollama,
|
||||
self.llamacpp.as_deref(),
|
||||
text,
|
||||
)
|
||||
.await
|
||||
super::embed_query(&self.ollama, self.llamacpp.as_deref(), text).await
|
||||
}
|
||||
|
||||
/// Embed corpus text (applies `EMBED_DOCUMENT_PREFIX`).
|
||||
pub async fn embed_document(&self, text: &str) -> Result<Vec<f32>> {
|
||||
super::embed_document(
|
||||
&opentelemetry::Context::new(),
|
||||
&self.ollama,
|
||||
self.llamacpp.as_deref(),
|
||||
text,
|
||||
)
|
||||
.await
|
||||
super::embed_document(&self.ollama, self.llamacpp.as_deref(), text).await
|
||||
}
|
||||
|
||||
/// Single-shot local text generation via the `LLM_BACKEND`-selected
|
||||
@@ -72,23 +60,14 @@ impl LocalLlm {
|
||||
pub async fn generate(&self, prompt: &str, system: Option<&str>) -> Result<String> {
|
||||
if super::local_backend_is_llamacpp() {
|
||||
if let Some(lc) = self.llamacpp.as_deref() {
|
||||
return <LlamaCppClient as LlmClient>::generate(
|
||||
lc,
|
||||
&opentelemetry::Context::new(),
|
||||
prompt,
|
||||
system,
|
||||
None,
|
||||
)
|
||||
.await;
|
||||
return <LlamaCppClient as LlmClient>::generate(lc, prompt, system, None).await;
|
||||
}
|
||||
anyhow::bail!(
|
||||
"LLM_BACKEND=llamacpp but LlamaCppClient is unconfigured — \
|
||||
set LLAMA_SWAP_URL or switch to LLM_BACKEND=ollama"
|
||||
);
|
||||
}
|
||||
self.ollama
|
||||
.generate(&opentelemetry::Context::new(), prompt, system)
|
||||
.await
|
||||
self.ollama.generate(prompt, system).await
|
||||
}
|
||||
|
||||
/// Label identifying which backend + model produces embeddings right
|
||||
|
||||
Reference in new issue
Block a user