Otel: llm.request client spans + traceparent on LLM calls (#130)
Add OpenTelemetry tracing to every LLM client call so an insight/chat turn is a single trace from HTTP entry to each downstream inference. - LlmClient trait: thread a &opentelemetry::Context through all 5 methods (generate, chat_with_tools, chat_with_tools_stream, generate_embeddings, describe_image), matching the DAO convention of explicit context threading rather than ambient Context::current(). - otel::start_llm_request_span: single shared helper that every provider (openrouter / llamacpp / ollama) funnels through, so the span shape is identical: name llm.request, SpanKind::Client, llm.provider / llm.operation / llm.model. - Per-method spans in all three clients with the spec's operation enum (chat | chat_stream | embeddings | describe_image | generate); stream spans are owned and moved into the async_stream generator so they end when the response drains, not when headers arrive. - llm.tokens.input / llm.tokens.output recorded on non-stream and stream completion; llm.cost_usd from usage.cost (OpenRouter); http.status_code + error status on failure. - W3C traceparent injected on outgoing LLM HTTP requests (llama-swap, Ollama, OpenRouter) gated on span_context.is_valid() so debug no-op-tracer builds don't send an all-zero parent. - Turn-level spans (ai.chat.turn.execute, ai.persona_chat.turn.execute) now carry llm.backend + llm.model; with_entry streaming paths thread the turn context (turn_cx) to child describe_image / agentic-loop calls instead of the ambient context. clippy clean; 690 lib + 775 integration + 4 doc tests green.
This commit is contained in:
1 parent
e84ac82f86
commit
696d9fa8f3
17 files changed
+910
-234
No files matched your search
+12
-2
@@ -1,6 +1,7 @@
|
||||
use anyhow::Result;
|
||||
use async_trait::async_trait;
|
||||
use futures::stream::BoxStream;
|
||||
use opentelemetry::Context;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Provider-agnostic surface for LLM backends (Ollama, OpenRouter, …).
|
||||
@@ -10,6 +11,12 @@ use serde::{Deserialize, Serialize};
|
||||
/// when a provider requires it (OpenAI-compatible APIs do), and `images`
|
||||
/// stays as base64 strings here and is rewritten into content-parts where
|
||||
/// needed.
|
||||
///
|
||||
/// Every method that performs an HTTP call takes the caller's tracing
|
||||
/// `Context` as its first parameter so the impl can start an `llm.request`
|
||||
/// span parented under the caller's feature span — the codebase threads
|
||||
/// contexts explicitly rather than relying on ambient `Context::current()`
|
||||
/// at deep call sites.
|
||||
// First consumer lands in a later PR (OpenRouter impl + hybrid mode routing).
|
||||
#[allow(dead_code)]
|
||||
#[async_trait]
|
||||
@@ -18,6 +25,7 @@ pub trait LlmClient: Send + Sync {
|
||||
/// base64 images (ignored by providers without vision support).
|
||||
async fn generate(
|
||||
&self,
|
||||
cx: &Context,
|
||||
prompt: &str,
|
||||
system: Option<&str>,
|
||||
images: Option<Vec<String>>,
|
||||
@@ -27,6 +35,7 @@ pub trait LlmClient: Send + Sync {
|
||||
/// (which may contain tool_calls) plus optional prompt/eval token counts.
|
||||
async fn chat_with_tools(
|
||||
&self,
|
||||
cx: &Context,
|
||||
messages: Vec<ChatMessage>,
|
||||
tools: Vec<Tool>,
|
||||
) -> Result<(ChatMessage, Option<i32>, Option<i32>)>;
|
||||
@@ -39,16 +48,17 @@ pub trait LlmClient: Send + Sync {
|
||||
/// as one `Done` event.
|
||||
async fn chat_with_tools_stream(
|
||||
&self,
|
||||
cx: &Context,
|
||||
messages: Vec<ChatMessage>,
|
||||
tools: Vec<Tool>,
|
||||
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>>;
|
||||
|
||||
/// Batch embedding generation. Dimensionality is provider/model specific.
|
||||
async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>>;
|
||||
async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>>;
|
||||
|
||||
/// One-shot vision description of an image. Used to convert images into
|
||||
/// plain text for the hybrid-mode conversation flow.
|
||||
async fn describe_image(&self, image_base64: &str) -> Result<String>;
|
||||
async fn describe_image(&self, cx: &Context, image_base64: &str) -> Result<String>;
|
||||
|
||||
/// Enumerate available models with their capabilities.
|
||||
async fn list_models(&self) -> Result<Vec<ModelCapabilities>>;
|
||||
|
||||
Reference in new issue
Block a user