Tracing: LLM clients + streaming spans on tracing, drop cx threading (#140)

LlmClient methods no longer take an OpenTelemetry Context. llm.request is
now a tracing span (otel.kind=client, llm.provider/operation/model) that
parents on the caller's ambient span; traceparent injection reads the
span's OTel context via OpenTelemetrySpanExt.

The src/ai feature spans (handlers, insight generation, chat, persona
chat, daily summaries, TTS) move to #[instrument] / info_span!, and every
LLM-bearing tokio::spawn carries its parent span with .instrument(): the
streaming chat turn (chat_turn_stream_inner), async chat / persona turns,
background insight jobs and TTS jobs. Sub-spans in the agentic loops
(ai.chat.loop, ai.agentic.loop) instrument each LLM / tool await.

DAO calls keep their explicit Context (derived from the tracing span)
until the DAO migration (#141).

tests/otel_llm_spans.rs pins nesting + traceparent for the blocking and
streaming paths across a spawn on a multi-thread runtime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Opus 5.5 committed 2026-09-29 17:02:40 -04:00
1 parent 7a59afc62b
commit 1c508cdabd
20 files changed
+973 -1030

No files matched your search

+6 -11
View File
@@ -1,7 +1,6 @@
use anyhow::Result;
use async_trait::async_trait;
use futures::stream::BoxStream;
use opentelemetry::Context;
use serde::{Deserialize, Serialize};
/// Provider-agnostic surface for LLM backends (Ollama, OpenRouter, …).
@@ -12,11 +11,10 @@ use serde::{Deserialize, Serialize};
/// stays as base64 strings here and is rewritten into content-parts where
/// needed.
///
/// Every method that performs an HTTP call takes the caller's tracing
/// `Context` as its first parameter so the impl can start an `llm.request`
/// span parented under the caller's feature span — the codebase threads
/// contexts explicitly rather than relying on ambient `Context::current()`
/// at deep call sites.
/// Every method that performs an HTTP call starts an `llm.request` span
/// parented on the caller's ambient `tracing` span, so callers must run
/// inside their feature span (`#[instrument]` / `.instrument(span)`) — and
/// carry it across any `tokio::spawn` — for the LLM span to nest under it.
// First consumer lands in a later PR (OpenRouter impl + hybrid mode routing).
#[allow(dead_code)]
#[async_trait]
@@ -25,7 +23,6 @@ pub trait LlmClient: Send + Sync {
/// base64 images (ignored by providers without vision support).
async fn generate(
&self,
cx: &Context,
prompt: &str,
system: Option<&str>,
images: Option<Vec<String>>,
@@ -35,7 +32,6 @@ pub trait LlmClient: Send + Sync {
/// (which may contain tool_calls) plus optional prompt/eval token counts.
async fn chat_with_tools(
&self,
cx: &Context,
messages: Vec<ChatMessage>,
tools: Vec<Tool>,
) -> Result<(ChatMessage, Option<i32>, Option<i32>)>;
@@ -48,17 +44,16 @@ pub trait LlmClient: Send + Sync {
/// as one `Done` event.
async fn chat_with_tools_stream(
&self,
cx: &Context,
messages: Vec<ChatMessage>,
tools: Vec<Tool>,
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>>;
/// Batch embedding generation. Dimensionality is provider/model specific.
async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>>;
async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>>;
/// One-shot vision description of an image. Used to convert images into
/// plain text for the hybrid-mode conversation flow.
async fn describe_image(&self, cx: &Context, image_base64: &str) -> Result<String>;
async fn describe_image(&self, image_base64: &str) -> Result<String>;
/// Enumerate available models with their capabilities.
async fn list_models(&self) -> Result<Vec<ModelCapabilities>>;