Tracing: LLM clients + streaming spans on tracing, drop cx threading (#140)
LlmClient methods no longer take an OpenTelemetry Context. llm.request is now a tracing span (otel.kind=client, llm.provider/operation/model) that parents on the caller's ambient span; traceparent injection reads the span's OTel context via OpenTelemetrySpanExt. The src/ai feature spans (handlers, insight generation, chat, persona chat, daily summaries, TTS) move to #[instrument] / info_span!, and every LLM-bearing tokio::spawn carries its parent span with .instrument(): the streaming chat turn (chat_turn_stream_inner), async chat / persona turns, background insight jobs and TTS jobs. Sub-spans in the agentic loops (ai.chat.loop, ai.agentic.loop) instrument each LLM / tool await. DAO calls keep their explicit Context (derived from the tracing span) until the DAO migration (#141). tests/otel_llm_spans.rs pins nesting + traceparent for the blocking and streaming paths across a spawn on a multi-thread runtime. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
7a59afc62b
commit
1c508cdabd
20 files changed
+973
-1030
No files matched your search
+6
-11
@@ -1,7 +1,6 @@
|
||||
use anyhow::Result;
|
||||
use async_trait::async_trait;
|
||||
use futures::stream::BoxStream;
|
||||
use opentelemetry::Context;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Provider-agnostic surface for LLM backends (Ollama, OpenRouter, …).
|
||||
@@ -12,11 +11,10 @@ use serde::{Deserialize, Serialize};
|
||||
/// stays as base64 strings here and is rewritten into content-parts where
|
||||
/// needed.
|
||||
///
|
||||
/// Every method that performs an HTTP call takes the caller's tracing
|
||||
/// `Context` as its first parameter so the impl can start an `llm.request`
|
||||
/// span parented under the caller's feature span — the codebase threads
|
||||
/// contexts explicitly rather than relying on ambient `Context::current()`
|
||||
/// at deep call sites.
|
||||
/// Every method that performs an HTTP call starts an `llm.request` span
|
||||
/// parented on the caller's ambient `tracing` span, so callers must run
|
||||
/// inside their feature span (`#[instrument]` / `.instrument(span)`) — and
|
||||
/// carry it across any `tokio::spawn` — for the LLM span to nest under it.
|
||||
// First consumer lands in a later PR (OpenRouter impl + hybrid mode routing).
|
||||
#[allow(dead_code)]
|
||||
#[async_trait]
|
||||
@@ -25,7 +23,6 @@ pub trait LlmClient: Send + Sync {
|
||||
/// base64 images (ignored by providers without vision support).
|
||||
async fn generate(
|
||||
&self,
|
||||
cx: &Context,
|
||||
prompt: &str,
|
||||
system: Option<&str>,
|
||||
images: Option<Vec<String>>,
|
||||
@@ -35,7 +32,6 @@ pub trait LlmClient: Send + Sync {
|
||||
/// (which may contain tool_calls) plus optional prompt/eval token counts.
|
||||
async fn chat_with_tools(
|
||||
&self,
|
||||
cx: &Context,
|
||||
messages: Vec<ChatMessage>,
|
||||
tools: Vec<Tool>,
|
||||
) -> Result<(ChatMessage, Option<i32>, Option<i32>)>;
|
||||
@@ -48,17 +44,16 @@ pub trait LlmClient: Send + Sync {
|
||||
/// as one `Done` event.
|
||||
async fn chat_with_tools_stream(
|
||||
&self,
|
||||
cx: &Context,
|
||||
messages: Vec<ChatMessage>,
|
||||
tools: Vec<Tool>,
|
||||
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>>;
|
||||
|
||||
/// Batch embedding generation. Dimensionality is provider/model specific.
|
||||
async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>>;
|
||||
async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>>;
|
||||
|
||||
/// One-shot vision description of an image. Used to convert images into
|
||||
/// plain text for the hybrid-mode conversation flow.
|
||||
async fn describe_image(&self, cx: &Context, image_base64: &str) -> Result<String>;
|
||||
async fn describe_image(&self, image_base64: &str) -> Result<String>;
|
||||
|
||||
/// Enumerate available models with their capabilities.
|
||||
async fn list_models(&self) -> Result<Vec<ModelCapabilities>>;
|
||||
|
||||
Reference in new issue
Block a user