Prefs: owner defaults (model, backend, persona_id, reel span/length) #129

Merged
cameron merged 5 commits from feature/prefs-owner-defaults into master 2026-09-29 01:50:09 +00:00
29 changed files with 1838 additions and 326 deletions

No files matched your search

+5
View File
@@ -533,6 +533,11 @@ POST /insights/chat (single-turn reply, non-streaming)
POST /insights/chat/stream (SSE: text / tool_call / tool_result / truncated / done) POST /insights/chat/stream (SSE: text / tool_call / tool_result / truncated / done)
GET /insights/chat/history?path=... (rendered transcript with tool invocations) GET /insights/chat/history?path=... (rendered transcript with tool invocations)
POST /insights/chat/rewind (truncate transcript at a rendered index) POST /insights/chat/rewind (truncate transcript at a rendered index)
// User AI Prefs (owner defaults + passive mirror; src/prefs.rs)
GET /prefs (current row; auth)
PUT /prefs (partial update: omitted=unchanged, null=clear, value=set; auth)
GET /timezones (IANA zone list from chrono-tz; unauthenticated)
``` ```
**Request Types:** **Request Types:**
+22
View File
@@ -62,3 +62,25 @@ Insights and pre-computed artifacts grouped by calendar day (daily message summa
reel pre-generation). Because it is keyed on the day, re-dating a photo across a day reel pre-generation). Because it is keyed on the day, re-dating a photo across a day
boundary invalidates the derived data for that day, which regenerates on its normal schedule. boundary invalidates the derived data for that day, which regenerates on its normal schedule.
_Avoid_: cache, nightly output _Avoid_: cache, nightly output
### Configuration
**Owner default**:
A per-request default the owner sets for the whole deployment — the model, backend,
persona, reel span, or segment cap a client gets when it omits the field. Stored in the
single `user_ai_prefs` row, hot-tunable without a redeploy, and global: every client that
omits the field gets it, on every device. Never holds secrets.
_Avoid_: user prefs, user settings, preferences (ambiguous with device-local client state)
**Deployment config**:
Values the server needs before it can start serving — bind address, database path, media
roots, the URLs used to build clients — plus every secret. Lives in `.env` only; changing
it is a redeploy, by design.
_Avoid_: environment config, system config
**Hard default**:
The env/code fallback a request lands on when neither an explicit param nor a stored
owner default supplies the field (e.g. backend `local`, persona `default`, reel span
day). It is the floor under the owner-default layer: env changes keep working whenever a
default column is NULL.
_Avoid_: fallback value, env default
+78
View File
@@ -0,0 +1,78 @@
# ADR-0002: Owner defaults live in the DB row; .env keeps deployment config and secrets
**Status**: accepted (2026-09-27)
## Context
All configuration lived in `.env`: bind address, media paths, Ollama/llama-swap/OpenRouter
URLs, model names, `LLM_BACKEND`, the embedding contract, TTS voices. Two friction points:
values the owner actually *tunes* (which model, which backend, which persona, how long a
reel is) needed an edit + redeploy to change, and clients had no server-side fallback —
"use the default" was expressed by omitting a field, and the default itself was invisible
and untouchable at runtime. The `user_ai_prefs` single row (`id = 1`) already existed as
a passive mirror of the last client reel request for the nightly pre-gen. #126/#127 gave
it an explicit `PUT /prefs` surface and five owner-default columns (`default_model`,
`default_backend`, `default_persona_id`, `reel_span`, `reel_max_segments`) — the first
batch of non-private configuration moved out of `.env` and into the database.
## Decision
Configuration splits into two homes by one rule — *does the server need the value before
it can start serving?*
- **`.env` (deployment config)**: everything read at boot to bind, connect, or open —
`BIND_URL`, `DATABASE_URL`, media paths, `OLLAMA_*_URL` / `LLAMA_SWAP_URL`,
`LLM_BACKEND`, the embedding contract — and **all secrets** (`OPENROUTER_API_KEY`,
`SECRET_KEY`, `SMS_API_TOKEN`). Secrets never go in the database: the SQLite file gets
backed up, copied, and attached to bug reports.
- **`user_ai_prefs` (owner defaults)**: per-request defaults resolved at request time —
the model/backend/persona a client gets when it omits the field, reel span and
segment cap, display zone, TTS voice. Hot-tunable via `PUT /prefs`, effective on the
next request, no redeploy. This row is the designated home for future non-private,
per-request configuration that outgrows `.env`. The table is a **single row per
deployment, not a per-user profile** — `user_ai_prefs` is a historical name; the
canonical term is *owner defaults* (single-user tool: owner = admin).
Resolution order at every consumer: **explicit request param → stored owner default →
hard default** (the env/code fallback). An explicit request always wins, so any client
can override the owner's default per request. The env value stays the floor under the
DB: a NULL `default_model` means "whatever the Ollama/llama-swap primary is", so env
changes keep working.
**Privacy posture is safe-by-default.** `default_backend` is the one owner default with
network-egress consequences: `hybrid` routes insight generation and chat to OpenRouter.
With no row or a NULL column, the hard default is `local` — data stays on the LAN.
Leaving the network requires an explicit owner action (storing `hybrid`), and any client
can still pin itself by sending `backend` explicitly. Owner defaults therefore split
into two sensitivity classes: **harmless tuning** (span, segments, display zone, voice,
model, persona) and **privacy/security** (`default_backend`), which any admin UI must
surface with a warning, not a quiet dropdown.
## Considered options
- **Everything in `.env`** (status quo): safe, but tuning a model default means edit +
redeploy, and clients have no server-side default to fall back to — a picker's
"server default" is just the env value, invisible and unchangeable at runtime.
Rejected for per-request defaults.
- **Everything in the DB**: boot-time values (bind address, DB path, the URLs used to
build the HTTP clients) become chicken-and-egg, and secrets sit next to tuning values
in a file that gets copied around. Rejected.
- **A separate `system_config` table**: cleaner name, but a second single-row table
would be a synonym with extra migrations; the row and the `PUT /prefs` surface already
exist. Rejected — the row keeps its name, its *term* becomes owner defaults.
## Non-obvious rules worth remembering
- **Owner defaults are global to the deployment.** Every client that omits a field gets
the stored value, on every device. Storing `default_backend=hybrid` changes the
mobile app's behavior even for users who never touch the setting; a client that must
stay local should send `backend` explicitly (tracked for FileViewer-React #33).
- **A stored value that is no longer consumable fails at write time, not request time** —
`PUT /prefs` validates `default_backend` against the canonical `BackendKind::parse`
and `reel_span` against the span enum before writing, so a bad default cannot 500
every request later.
- **The passive reel mirror (`capture_prefs`) may only write the fields it owns** (voice,
library, display zone, effective span/segments); it carries the AI owner-defaults
through unchanged, because the upsert overwrites every column.
- **NULL means "use the hard default", not "unset the env"** — the DB layer sits
*above* env, never instead of it.
@@ -0,0 +1,5 @@
ALTER TABLE user_ai_prefs DROP COLUMN default_model;
ALTER TABLE user_ai_prefs DROP COLUMN default_backend;
ALTER TABLE user_ai_prefs DROP COLUMN default_persona_id;
ALTER TABLE user_ai_prefs DROP COLUMN reel_span;
ALTER TABLE user_ai_prefs DROP COLUMN reel_max_segments;
@@ -0,0 +1,8 @@
-- Owner defaults: recurring per-user picks that were previously re-sent on
-- every request. All nullable — NULL = keep the current per-request /
-- hard-default behavior.
ALTER TABLE user_ai_prefs ADD COLUMN default_model TEXT;
ALTER TABLE user_ai_prefs ADD COLUMN default_backend TEXT;
ALTER TABLE user_ai_prefs ADD COLUMN default_persona_id TEXT;
ALTER TABLE user_ai_prefs ADD COLUMN reel_span TEXT;
ALTER TABLE user_ai_prefs ADD COLUMN reel_max_segments INTEGER;
+10 -2
View File
@@ -360,11 +360,17 @@ async fn generate_and_store_daily_summary(
span.set_attribute(KeyValue::new("date", date.to_string())); span.set_attribute(KeyValue::new("date", date.to_string()));
span.set_attribute(KeyValue::new("contact", contact.to_string())); span.set_attribute(KeyValue::new("contact", contact.to_string()));
span.set_attribute(KeyValue::new("message_count", messages.len() as i64)); span.set_attribute(KeyValue::new("message_count", messages.len() as i64));
// Parent context for the child llm.request spans; the span itself stays
// borrowed here for the attribute updates below.
let llm_cx =
opentelemetry::Context::new().with_remote_span_context(span.span_context().clone());
let (prompt, system_prompt) = build_daily_summary_prompt(contact, date, messages); let (prompt, system_prompt) = build_daily_summary_prompt(contact, date, messages);
// Generate summary with LLM // Generate summary with LLM
let summary = ollama.generate(&prompt, Some(system_prompt)).await?; let summary = ollama
.generate(&llm_cx, &prompt, Some(system_prompt))
.await?;
log::debug!( log::debug!(
"Generated summary for {}: {}", "Generated summary for {}: {}",
@@ -382,7 +388,9 @@ async fn generate_and_store_daily_summary(
); );
// Embed the stripped summary (store original summary in DB) // Embed the stripped summary (store original summary in DB)
let embedding = ollama.generate_embedding(&stripped_summary).await?; let embedding = ollama
.generate_embedding(&llm_cx, &stripped_summary)
.await?;
span.set_attribute(KeyValue::new( span.set_attribute(KeyValue::new(
"embedding_dimensions", "embedding_dimensions",
+175 -1
View File
@@ -384,6 +384,36 @@ pub struct ServerModels {
pub default_model: String, pub default_model: String,
} }
/// Apply the stored owner defaults to any request field the client omitted.
/// Resolution order: request param → stored pref → hard default (applied
/// downstream). The request always wins; the pref only fills gaps. One DAO
/// read per request; a failed read silently falls back to hard defaults.
fn apply_owner_defaults(
app_state: &AppState,
model: &mut Option<String>,
backend: &mut Option<String>,
persona_id: &mut Option<String>,
) {
if model.is_some() && backend.is_some() && persona_id.is_some() {
return; // nothing to fill — skip the DB read
}
let Some(mut dao) = app_state.user_ai_prefs_dao.lock().ok() else {
return;
};
let Ok(Some(prefs)) = dao.get_prefs(&opentelemetry::Context::new()) else {
return;
};
if model.is_none() {
*model = prefs.default_model;
}
if backend.is_none() {
*backend = prefs.default_backend;
}
if persona_id.is_none() {
*persona_id = prefs.default_persona_id;
}
}
/// POST /insights/generate - Generate insight for a specific photo (async) /// POST /insights/generate - Generate insight for a specific photo (async)
#[post("/insights/generate")] #[post("/insights/generate")]
pub async fn generate_insight_handler( pub async fn generate_insight_handler(
@@ -396,6 +426,16 @@ pub async fn generate_insight_handler(
let tracer = global_tracer(); let tracer = global_tracer();
let mut span = tracer.start_with_context("http.insights.generate", &parent_context); let mut span = tracer.start_with_context("http.insights.generate", &parent_context);
let mut request = request.0;
// Stored owner defaults fill what the client omitted (request → pref →
// hard default; the hard default still applies downstream).
apply_owner_defaults(
&app_state,
&mut request.model,
&mut request.backend,
&mut request.persona_id,
);
let normalized_path = normalize_path(&request.file_path); let normalized_path = normalize_path(&request.file_path);
let library = app_state.primary_library(); let library = app_state.primary_library();
let gen_type = InsightGenerationType::Standard; let gen_type = InsightGenerationType::Standard;
@@ -714,6 +754,16 @@ pub async fn generate_agentic_insight_handler(
let tracer = global_tracer(); let tracer = global_tracer();
let mut span = tracer.start_with_context("http.insights.generate_agentic", &parent_context); let mut span = tracer.start_with_context("http.insights.generate_agentic", &parent_context);
let mut request = request.0;
// Stored owner defaults fill what the client omitted (request → pref →
// hard default; the hard default still applies downstream).
apply_owner_defaults(
&app_state,
&mut request.model,
&mut request.backend,
&mut request.persona_id,
);
let normalized_path = normalize_path(&request.file_path); let normalized_path = normalize_path(&request.file_path);
let library = app_state.primary_library(); let library = app_state.primary_library();
let gen_type = InsightGenerationType::Agentic; let gen_type = InsightGenerationType::Agentic;
@@ -1231,6 +1281,16 @@ pub async fn chat_turn_handler(
let mut span = tracer.start_with_context("http.insights.chat", &parent_context); let mut span = tracer.start_with_context("http.insights.chat", &parent_context);
span.set_attribute(KeyValue::new("file_path", request.file_path.clone())); span.set_attribute(KeyValue::new("file_path", request.file_path.clone()));
let mut request = request.0;
// Stored owner defaults fill what the client omitted (request → pref →
// hard default; the hard default still applies downstream).
apply_owner_defaults(
&app_state,
&mut request.model,
&mut request.backend,
&mut request.persona_id,
);
let library = let library =
match libraries::resolve_library_param_state(&app_state, request.library.as_deref()) { match libraries::resolve_library_param_state(&app_state, request.library.as_deref()) {
Ok(Some(lib)) => lib, Ok(Some(lib)) => lib,
@@ -1607,10 +1667,22 @@ pub struct ChatSwitchBranchRequest {
/// Returns `text/event-stream` with one event per chat stream event. /// Returns `text/event-stream` with one event per chat stream event.
#[post("/insights/chat/stream")] #[post("/insights/chat/stream")]
pub async fn chat_stream_handler( pub async fn chat_stream_handler(
http_request: HttpRequest,
claims: Claims, claims: Claims,
request: web::Json<ChatTurnHttpRequest>, request: web::Json<ChatTurnHttpRequest>,
app_state: web::Data<AppState>, app_state: web::Data<AppState>,
) -> HttpResponse { ) -> HttpResponse {
let parent_context = extract_context_from_request(&http_request);
let mut request = request.0;
// Stored owner defaults fill what the client omitted (request → pref →
// hard default; the hard default still applies downstream).
apply_owner_defaults(
&app_state,
&mut request.model,
&mut request.backend,
&mut request.persona_id,
);
let library = let library =
match libraries::resolve_library_param_state(&app_state, request.library.as_deref()) { match libraries::resolve_library_param_state(&app_state, request.library.as_deref()) {
Ok(Some(lib)) => lib, Ok(Some(lib)) => lib,
@@ -1646,7 +1718,7 @@ pub async fn chat_stream_handler(
}; };
let service = app_state.insight_chat.clone(); let service = app_state.insight_chat.clone();
let events = service.chat_turn_stream(chat_req); let events = service.chat_turn_stream(chat_req, parent_context);
// Map ChatStreamEvent → SSE frame bytes. // Map ChatStreamEvent → SSE frame bytes.
let sse_stream = futures::stream::StreamExt::map(events, |ev| { let sse_stream = futures::stream::StreamExt::map(events, |ev| {
@@ -1758,6 +1830,16 @@ pub async fn turn_async_handler(
let mut span = tracer.start_with_context("http.insights.chat_turn_async", &parent_context); let mut span = tracer.start_with_context("http.insights.chat_turn_async", &parent_context);
span.set_attribute(KeyValue::new("file_path", request.file_path.clone())); span.set_attribute(KeyValue::new("file_path", request.file_path.clone()));
let mut request = request.0;
// Stored owner defaults fill what the client omitted (request → pref →
// hard default; the hard default still applies downstream).
apply_owner_defaults(
&app_state,
&mut request.model,
&mut request.backend,
&mut request.persona_id,
);
let library = let library =
match libraries::resolve_library_param_state(&app_state, request.library.as_deref()) { match libraries::resolve_library_param_state(&app_state, request.library.as_deref()) {
Ok(Some(lib)) => lib, Ok(Some(lib)) => lib,
@@ -2277,3 +2359,95 @@ mod turn_replay_tests {
assert_eq!(info.total_events_pushed, 2); assert_eq!(info.total_events_pushed, 2);
} }
} }
#[cfg(test)]
mod owner_defaults_tests {
use super::apply_owner_defaults;
use crate::database::models::UpsertUserAiPrefs;
use crate::database::{SqliteUserAiPrefsDao, UserAiPrefsDao};
use crate::state::AppState;
use std::sync::{Arc, Mutex};
fn prefs_state(prefs: Option<UpsertUserAiPrefs>) -> AppState {
let conn = Arc::new(Mutex::new(crate::database::test::in_memory_db_connection()));
let mut state = AppState::test_state();
state.user_ai_prefs_dao = Arc::new(Mutex::new(Box::new(
SqliteUserAiPrefsDao::from_connection(conn.clone()),
)));
if let Some(p) = prefs {
let mut dao = SqliteUserAiPrefsDao::from_connection(conn);
dao.upsert_prefs(&opentelemetry::Context::new(), &p)
.expect("seed prefs");
}
state
}
fn seeded_defaults() -> UpsertUserAiPrefs {
UpsertUserAiPrefs {
voice: None,
tz_offset_minutes: None,
library: None,
updated_at: 0,
display_tz: None,
default_model: Some("llama3.2:3b".into()),
default_backend: Some("hybrid".into()),
default_persona_id: Some("cabin".into()),
reel_span: None,
reel_max_segments: None,
}
}
#[actix_rt::test]
async fn request_wins_over_stored_pref() {
let state = prefs_state(Some(seeded_defaults()));
let mut model = Some("explicit".to_string());
let mut backend = Some("local".to_string());
let mut persona = Some("explicit-p".to_string());
apply_owner_defaults(&state, &mut model, &mut backend, &mut persona);
assert_eq!(model.as_deref(), Some("explicit"));
assert_eq!(backend.as_deref(), Some("local"));
assert_eq!(persona.as_deref(), Some("explicit-p"));
}
#[actix_rt::test]
async fn pref_fills_omitted_fields() {
let state = prefs_state(Some(seeded_defaults()));
let mut model = None;
let mut backend = None;
let mut persona = None;
apply_owner_defaults(&state, &mut model, &mut backend, &mut persona);
assert_eq!(model.as_deref(), Some("llama3.2:3b"));
assert_eq!(backend.as_deref(), Some("hybrid"));
assert_eq!(persona.as_deref(), Some("cabin"));
}
#[actix_rt::test]
async fn no_pref_row_leaves_fields_none() {
let state = prefs_state(None);
let mut model = None;
let mut backend = None;
let mut persona = None;
apply_owner_defaults(&state, &mut model, &mut backend, &mut persona);
// Hard defaults apply downstream; the pref layer fills nothing.
assert!(model.is_none());
assert!(backend.is_none());
assert!(persona.is_none());
}
#[actix_rt::test]
async fn pref_fills_only_the_fields_it_has() {
let state = prefs_state(Some(UpsertUserAiPrefs {
default_backend: None,
default_persona_id: Some("journal".into()),
..seeded_defaults()
}));
let mut model = None;
let mut backend = Some("local".to_string());
let mut persona = None;
apply_owner_defaults(&state, &mut model, &mut backend, &mut persona);
assert_eq!(model.as_deref(), Some("llama3.2:3b"));
// Explicit request backend is untouched; pref had none anyway.
assert_eq!(backend.as_deref(), Some("local"));
assert_eq!(persona.as_deref(), Some("journal"));
}
}
+45 -17
View File
@@ -377,7 +377,7 @@ impl InsightChatService {
let (response, prompt_tokens, eval_tokens) = backend let (response, prompt_tokens, eval_tokens) = backend
.chat() .chat()
.chat_with_tools(messages.clone(), tools.clone()) .chat_with_tools(&loop_cx, messages.clone(), tools.clone())
.await?; .await?;
last_prompt_eval_count = prompt_tokens; last_prompt_eval_count = prompt_tokens;
last_eval_count = eval_tokens; last_eval_count = eval_tokens;
@@ -434,7 +434,7 @@ impl InsightChatService {
let synthetic_idx = push_synthetic_final_prompt(&mut messages); let synthetic_idx = push_synthetic_final_prompt(&mut messages);
let (final_response, prompt_tokens, eval_tokens) = backend let (final_response, prompt_tokens, eval_tokens) = backend
.chat() .chat()
.chat_with_tools(messages.clone(), vec![]) .chat_with_tools(&loop_cx, messages.clone(), vec![])
.await?; .await?;
last_prompt_eval_count = prompt_tokens; last_prompt_eval_count = prompt_tokens;
last_eval_count = eval_tokens; last_eval_count = eval_tokens;
@@ -492,6 +492,7 @@ impl InsightChatService {
let title_raw = backend let title_raw = backend
.chat() .chat()
.generate( .generate(
&loop_cx,
&title_prompt, &title_prompt,
Some( Some(
"You are my long term memory assistant. Use only the information provided. Do not invent details.", "You are my long term memory assistant. Use only the information provided. Do not invent details.",
@@ -781,10 +782,11 @@ impl InsightChatService {
pub fn chat_turn_stream( pub fn chat_turn_stream(
self: Arc<Self>, self: Arc<Self>,
req: ChatTurnRequest, req: ChatTurnRequest,
cx: opentelemetry::Context,
) -> BoxStream<'static, ChatStreamEvent> { ) -> BoxStream<'static, ChatStreamEvent> {
let svc = self; let svc = self;
let s = async_stream::stream! { let s = async_stream::stream! {
match svc.chat_turn_stream_inner(req, Ok).await { match svc.chat_turn_stream_inner(req, cx, Ok).await {
Ok(mut rx) => { Ok(mut rx) => {
while let Some(ev) = rx.recv().await { while let Some(ev) = rx.recv().await {
yield ev; yield ev;
@@ -806,6 +808,7 @@ impl InsightChatService {
async fn chat_turn_stream_inner<F>( async fn chat_turn_stream_inner<F>(
self: Arc<Self>, self: Arc<Self>,
req: ChatTurnRequest, req: ChatTurnRequest,
cx: opentelemetry::Context,
_ev_mapper: F, _ev_mapper: F,
) -> Result<tokio::sync::mpsc::Receiver<ChatStreamEvent>> ) -> Result<tokio::sync::mpsc::Receiver<ChatStreamEvent>>
where where
@@ -814,7 +817,7 @@ impl InsightChatService {
let (tx, rx) = tokio::sync::mpsc::channel::<ChatStreamEvent>(64); let (tx, rx) = tokio::sync::mpsc::channel::<ChatStreamEvent>(64);
let svc = self.clone(); let svc = self.clone();
tokio::spawn(async move { tokio::spawn(async move {
let result = svc.run_streaming_turn(req, tx.clone()).await; let result = svc.run_streaming_turn(req, cx, tx.clone()).await;
if let Err(e) = result { if let Err(e) = result {
let _ = tx.send(ChatStreamEvent::Error(format!("{}", e))).await; let _ = tx.send(ChatStreamEvent::Error(format!("{}", e))).await;
} }
@@ -852,7 +855,7 @@ impl InsightChatService {
span.set_attribute(KeyValue::new("library_id", library_id as i64)); span.set_attribute(KeyValue::new("library_id", library_id as i64));
let result = svc let result = svc
.run_streaming_turn_with_entry(req, entry_clone.clone()) .run_streaming_turn_with_entry(req, entry_clone.clone(), &mut span)
.await; .await;
if let Err(ref e) = result { if let Err(ref e) = result {
span.set_attribute(KeyValue::new("status", "error")); span.set_attribute(KeyValue::new("status", "error"));
@@ -882,6 +885,7 @@ impl InsightChatService {
self: Arc<Self>, self: Arc<Self>,
req: ChatTurnRequest, req: ChatTurnRequest,
entry: Arc<TurnEntry>, entry: Arc<TurnEntry>,
turn_span: &mut opentelemetry::global::BoxedSpan,
) -> Result<()> { ) -> Result<()> {
if req.user_message.trim().is_empty() { if req.user_message.trim().is_empty() {
bail!("user_message must not be empty"); bail!("user_message must not be empty");
@@ -911,11 +915,11 @@ impl InsightChatService {
if req.regenerate || existing_insight.is_none() { if req.regenerate || existing_insight.is_none() {
return self return self
.run_bootstrap_streaming_with_entry(req, normalized, entry) .run_bootstrap_streaming_with_entry(req, normalized, entry, turn_span)
.await; .await;
} }
let insight = existing_insight.expect("just checked Some above"); let insight = existing_insight.expect("just checked Some above");
self.run_continuation_streaming_with_entry(req, normalized, insight, entry) self.run_continuation_streaming_with_entry(req, normalized, insight, entry, turn_span)
.await .await
} }
@@ -926,6 +930,7 @@ impl InsightChatService {
normalized: String, normalized: String,
insight: crate::database::models::PhotoInsight, insight: crate::database::models::PhotoInsight,
entry: Arc<TurnEntry>, entry: Arc<TurnEntry>,
turn_span: &mut opentelemetry::global::BoxedSpan,
) -> Result<()> { ) -> Result<()> {
let active_persona = req let active_persona = req
.persona_id .persona_id
@@ -981,6 +986,12 @@ impl InsightChatService {
}; };
let backend = self.generator.resolve_backend(kind, &overrides).await?; let backend = self.generator.resolve_backend(kind, &overrides).await?;
let model_used = backend.model().to_string(); let model_used = backend.model().to_string();
// llm.* attributes on the turn-level span (issue #130): clients can't
// know local-vs-hybrid, but the turn knows its resolved backend.
turn_span.set_attribute(KeyValue::new("llm.backend", kind.as_str()));
turn_span.set_attribute(KeyValue::new("llm.model", model_used.clone()));
let turn_cx = opentelemetry::Context::new()
.with_remote_span_context(turn_span.span_context().clone());
let local_first_user_has_image = messages let local_first_user_has_image = messages
.iter() .iter()
@@ -1030,6 +1041,7 @@ impl InsightChatService {
&active_persona, &active_persona,
max_iterations, max_iterations,
&entry, &entry,
&turn_cx,
) )
.await?; .await?;
let AgenticLoopOutcome { let AgenticLoopOutcome {
@@ -1146,6 +1158,7 @@ impl InsightChatService {
req: ChatTurnRequest, req: ChatTurnRequest,
normalized: String, normalized: String,
entry: Arc<TurnEntry>, entry: Arc<TurnEntry>,
turn_span: &mut opentelemetry::global::BoxedSpan,
) -> Result<()> { ) -> Result<()> {
let active_persona = req let active_persona = req
.persona_id .persona_id
@@ -1171,6 +1184,11 @@ impl InsightChatService {
}; };
let backend = self.generator.resolve_backend(kind, &overrides).await?; let backend = self.generator.resolve_backend(kind, &overrides).await?;
let model_used = backend.model().to_string(); let model_used = backend.model().to_string();
// llm.* attributes on the turn-level span (issue #130).
turn_span.set_attribute(KeyValue::new("llm.backend", kind.as_str()));
turn_span.set_attribute(KeyValue::new("llm.model", model_used.clone()));
let turn_cx = opentelemetry::Context::new()
.with_remote_span_context(turn_span.span_context().clone());
let image_base64: Option<String> = self.generator.load_image_as_base64(&normalized).ok(); let image_base64: Option<String> = self.generator.load_image_as_base64(&normalized).ok();
@@ -1190,7 +1208,7 @@ impl InsightChatService {
let visual_block = if !backend.images_inline { let visual_block = if !backend.images_inline {
match image_base64.as_deref() { match image_base64.as_deref() {
Some(b64) => match backend.local().describe_image(b64).await { Some(b64) => match backend.local().describe_image(&turn_cx, b64).await {
Ok(desc) => { Ok(desc) => {
format!("Visual description (from local vision model):\n{}\n", desc) format!("Visual description (from local vision model):\n{}\n", desc)
} }
@@ -1245,6 +1263,7 @@ impl InsightChatService {
&active_persona, &active_persona,
max_iterations, max_iterations,
&entry, &entry,
&turn_cx,
) )
.await?; .await?;
let AgenticLoopOutcome { let AgenticLoopOutcome {
@@ -1333,6 +1352,7 @@ impl InsightChatService {
active_persona: &str, active_persona: &str,
max_iterations: usize, max_iterations: usize,
entry: &Arc<TurnEntry>, entry: &Arc<TurnEntry>,
cx: &opentelemetry::Context,
) -> Result<AgenticLoopOutcome> { ) -> Result<AgenticLoopOutcome> {
crate::ai::insight_chat::run_streaming_agentic_loop_with_entry( crate::ai::insight_chat::run_streaming_agentic_loop_with_entry(
&self.generator, &self.generator,
@@ -1345,6 +1365,7 @@ impl InsightChatService {
active_persona, active_persona,
max_iterations, max_iterations,
entry, entry,
cx,
) )
.await .await
} }
@@ -1352,6 +1373,7 @@ impl InsightChatService {
async fn run_streaming_turn( async fn run_streaming_turn(
self: Arc<Self>, self: Arc<Self>,
req: ChatTurnRequest, req: ChatTurnRequest,
cx: opentelemetry::Context,
tx: tokio::sync::mpsc::Sender<ChatStreamEvent>, tx: tokio::sync::mpsc::Sender<ChatStreamEvent>,
) -> Result<()> { ) -> Result<()> {
if req.user_message.trim().is_empty() { if req.user_message.trim().is_empty() {
@@ -1381,17 +1403,17 @@ impl InsightChatService {
// library bootstraps cleanly, even when another library has // library bootstraps cleanly, even when another library has
// an insight for the same rel_path. // an insight for the same rel_path.
let existing_insight = { let existing_insight = {
let cx = opentelemetry::Context::new(); let db_cx = opentelemetry::Context::new();
let mut dao = self.insight_dao.lock().expect("Unable to lock InsightDao"); let mut dao = self.insight_dao.lock().expect("Unable to lock InsightDao");
dao.get_current_insight_for_library(&cx, req.library_id, &normalized) dao.get_current_insight_for_library(&db_cx, req.library_id, &normalized)
.map_err(|e| anyhow!("failed to load insight: {:?}", e))? .map_err(|e| anyhow!("failed to load insight: {:?}", e))?
}; };
if req.regenerate || existing_insight.is_none() { if req.regenerate || existing_insight.is_none() {
return self.run_bootstrap_streaming(req, normalized, tx).await; return self.run_bootstrap_streaming(req, normalized, cx, tx).await;
} }
let insight = existing_insight.expect("just checked Some above"); let insight = existing_insight.expect("just checked Some above");
self.run_continuation_streaming(req, normalized, insight, tx) self.run_continuation_streaming(req, normalized, cx, insight, tx)
.await .await
} }
@@ -1403,6 +1425,7 @@ impl InsightChatService {
&self, &self,
req: ChatTurnRequest, req: ChatTurnRequest,
normalized: String, normalized: String,
cx: opentelemetry::Context,
insight: crate::database::models::PhotoInsight, insight: crate::database::models::PhotoInsight,
tx: tokio::sync::mpsc::Sender<ChatStreamEvent>, tx: tokio::sync::mpsc::Sender<ChatStreamEvent>,
) -> Result<()> { ) -> Result<()> {
@@ -1500,6 +1523,7 @@ impl InsightChatService {
&active_persona, &active_persona,
max_iterations, max_iterations,
&tx, &tx,
&cx,
) )
.await?; .await?;
let AgenticLoopOutcome { let AgenticLoopOutcome {
@@ -1611,6 +1635,7 @@ impl InsightChatService {
&self, &self,
req: ChatTurnRequest, req: ChatTurnRequest,
normalized: String, normalized: String,
cx: opentelemetry::Context,
tx: tokio::sync::mpsc::Sender<ChatStreamEvent>, tx: tokio::sync::mpsc::Sender<ChatStreamEvent>,
) -> Result<()> { ) -> Result<()> {
let active_persona = req let active_persona = req
@@ -1667,7 +1692,7 @@ impl InsightChatService {
// images_inline backends send images directly to the chat model. // images_inline backends send images directly to the chat model.
let visual_block = if !backend.images_inline { let visual_block = if !backend.images_inline {
match image_base64.as_deref() { match image_base64.as_deref() {
Some(b64) => match backend.local().describe_image(b64).await { Some(b64) => match backend.local().describe_image(&cx, b64).await {
Ok(desc) => { Ok(desc) => {
format!("Visual description (from local vision model):\n{}\n", desc) format!("Visual description (from local vision model):\n{}\n", desc)
} }
@@ -1734,6 +1759,7 @@ impl InsightChatService {
&active_persona, &active_persona,
max_iterations, max_iterations,
&tx, &tx,
&cx,
) )
.await?; .await?;
let AgenticLoopOutcome { let AgenticLoopOutcome {
@@ -1816,6 +1842,7 @@ impl InsightChatService {
active_persona: &str, active_persona: &str,
max_iterations: usize, max_iterations: usize,
tx: &tokio::sync::mpsc::Sender<ChatStreamEvent>, tx: &tokio::sync::mpsc::Sender<ChatStreamEvent>,
cx: &opentelemetry::Context,
) -> Result<AgenticLoopOutcome> { ) -> Result<AgenticLoopOutcome> {
let mut tool_calls_made = 0usize; let mut tool_calls_made = 0usize;
let mut iterations_used = 0usize; let mut iterations_used = 0usize;
@@ -1834,7 +1861,7 @@ impl InsightChatService {
let mut stream = backend let mut stream = backend
.chat() .chat()
.chat_with_tools_stream(messages.clone(), tools.clone()) .chat_with_tools_stream(cx, messages.clone(), tools.clone())
.await?; .await?;
let mut final_message: Option<ChatMessage> = None; let mut final_message: Option<ChatMessage> = None;
@@ -1928,7 +1955,7 @@ impl InsightChatService {
let synthetic_idx = push_synthetic_final_prompt(messages); let synthetic_idx = push_synthetic_final_prompt(messages);
let mut stream = backend let mut stream = backend
.chat() .chat()
.chat_with_tools_stream(messages.clone(), vec![]) .chat_with_tools_stream(cx, messages.clone(), vec![])
.await?; .await?;
let mut final_message: Option<ChatMessage> = None; let mut final_message: Option<ChatMessage> = None;
while let Some(ev) = stream.next().await { while let Some(ev) = stream.next().await {
@@ -2308,6 +2335,7 @@ pub async fn run_streaming_agentic_loop_with_entry(
active_persona: &str, active_persona: &str,
max_iterations: usize, max_iterations: usize,
entry: &Arc<TurnEntry>, entry: &Arc<TurnEntry>,
cx: &opentelemetry::Context,
) -> Result<AgenticLoopOutcome> { ) -> Result<AgenticLoopOutcome> {
let mut tool_calls_made = 0usize; let mut tool_calls_made = 0usize;
let mut iterations_used = 0usize; let mut iterations_used = 0usize;
@@ -2337,7 +2365,7 @@ pub async fn run_streaming_agentic_loop_with_entry(
let mut stream = backend let mut stream = backend
.chat() .chat()
.chat_with_tools_stream(messages.clone(), tools.clone()) .chat_with_tools_stream(cx, messages.clone(), tools.clone())
.await?; .await?;
let mut final_message: Option<ChatMessage> = None; let mut final_message: Option<ChatMessage> = None;
@@ -2420,7 +2448,7 @@ pub async fn run_streaming_agentic_loop_with_entry(
let synthetic_idx = push_synthetic_final_prompt(messages); let synthetic_idx = push_synthetic_final_prompt(messages);
let mut stream = backend let mut stream = backend
.chat() .chat()
.chat_with_tools_stream(messages.clone(), vec![]) .chat_with_tools_stream(cx, messages.clone(), vec![])
.await?; .await?;
let mut final_message: Option<ChatMessage> = None; let mut final_message: Option<ChatMessage> = None;
while let Some(ev) = stream.next().await { while let Some(ev) = stream.next().await {
+64 -36
View File
@@ -646,7 +646,8 @@ impl InsightGenerator {
// (`LLM_BACKEND` switch). Must match the backend that populated the // (`LLM_BACKEND` switch). Must match the backend that populated the
// daily-summary embeddings or similarity search will be garbage. // daily-summary embeddings or similarity search will be garbage.
let query_embedding = let query_embedding =
crate::ai::embed_query(&self.ollama, self.llamacpp.as_deref(), &query).await?; crate::ai::embed_query(&search_cx, &self.ollama, self.llamacpp.as_deref(), &query)
.await?;
// Search for similar daily summaries with time-based weighting // Search for similar daily summaries with time-based weighting
// This prioritizes summaries temporally close to the query date // This prioritizes summaries temporally close to the query date
@@ -712,7 +713,8 @@ impl InsightGenerator {
// Must use the same backend that populated the daily-summary // Must use the same backend that populated the daily-summary
// embeddings or similarity search is garbage (see embed_one docs). // embeddings or similarity search is garbage (see embed_one docs).
let query_embedding = let query_embedding =
crate::ai::embed_query(&self.ollama, self.llamacpp.as_deref(), query).await?; crate::ai::embed_query(&search_cx, &self.ollama, self.llamacpp.as_deref(), query)
.await?;
let mut summary_dao = self let mut summary_dao = self
.daily_summary_dao .daily_summary_dao
@@ -798,7 +800,9 @@ impl InsightGenerator {
let calendar_cx = parent_cx.with_span(span); let calendar_cx = parent_cx.with_span(span);
let query_embedding = if let Some(loc) = location { let query_embedding = if let Some(loc) = location {
match crate::ai::embed_query(&self.ollama, self.llamacpp.as_deref(), loc).await { match crate::ai::embed_query(&calendar_cx, &self.ollama, self.llamacpp.as_deref(), loc)
.await
{
Ok(emb) => Some(emb), Ok(emb) => Some(emb),
Err(e) => { Err(e) => {
log::warn!("Failed to generate embedding for location '{}': {}", loc, e); log::warn!("Failed to generate embedding for location '{}': {}", loc, e);
@@ -969,18 +973,23 @@ impl InsightGenerator {
) )
}; };
let query_embedding = let query_embedding = match crate::ai::embed_query(
match crate::ai::embed_query(&self.ollama, self.llamacpp.as_deref(), &query_text).await &search_cx,
{ &self.ollama,
Ok(emb) => emb, self.llamacpp.as_deref(),
Err(e) => { &query_text,
log::warn!("Failed to generate search embedding: {}", e); )
search_cx.span().set_status(Status::Error { .await
description: e.to_string().into(), {
}); Ok(emb) => emb,
return Ok(None); Err(e) => {
} log::warn!("Failed to generate search embedding: {}", e);
}; search_cx.span().set_status(Status::Error {
description: e.to_string().into(),
});
return Ok(None);
}
};
let searches = { let searches = {
let mut dao = self let mut dao = self
@@ -1254,7 +1263,10 @@ impl InsightGenerator {
// Generate brief photo description for RAG enrichment (vision models only) // Generate brief photo description for RAG enrichment (vision models only)
let photo_description: Option<String> = if let Some(ref img_b64) = image_base64 { let photo_description: Option<String> = if let Some(ref img_b64) = image_base64 {
match ollama_client.generate_photo_description(img_b64).await { match ollama_client
.generate_photo_description(&insight_cx, img_b64)
.await
{
Ok(desc) => { Ok(desc) => {
log::info!("Photo description for RAG enrichment: {}", desc); log::info!("Photo description for RAG enrichment: {}", desc);
Some(desc) Some(desc)
@@ -1323,7 +1335,7 @@ impl InsightGenerator {
if !immediate_messages.is_empty() { if !immediate_messages.is_empty() {
// Step 2: Extract topics from immediate messages to enrich RAG query // Step 2: Extract topics from immediate messages to enrich RAG query
let topics = self let topics = self
.extract_topics_from_messages(&immediate_messages, &ollama_client) .extract_topics_from_messages(&immediate_messages, &ollama_client, &insight_cx)
.await; .await;
log::info!("Extracted topics for query enrichment: {:?}", topics); log::info!("Extracted topics for query enrichment: {:?}", topics);
@@ -1379,6 +1391,7 @@ impl InsightGenerator {
&immediate_messages, &immediate_messages,
&ollama_client, &ollama_client,
custom_system_prompt.as_deref(), custom_system_prompt.as_deref(),
&insight_cx,
) )
.await .await
.unwrap_or_else(|| String::from("No immediate context")); .unwrap_or_else(|| String::from("No immediate context"));
@@ -1388,6 +1401,7 @@ impl InsightGenerator {
&historical_messages, &historical_messages,
&ollama_client, &ollama_client,
custom_system_prompt.as_deref(), custom_system_prompt.as_deref(),
&insight_cx,
) )
.await .await
.unwrap_or_else(|| String::from("No historical context")); .unwrap_or_else(|| String::from("No historical context"));
@@ -1406,6 +1420,7 @@ impl InsightGenerator {
&immediate_messages, &immediate_messages,
&ollama_client, &ollama_client,
custom_system_prompt.as_deref(), custom_system_prompt.as_deref(),
&insight_cx,
) )
.await; .await;
} }
@@ -1416,6 +1431,7 @@ impl InsightGenerator {
&immediate_messages, &immediate_messages,
&ollama_client, &ollama_client,
custom_system_prompt.as_deref(), custom_system_prompt.as_deref(),
&insight_cx,
) )
.await; .await;
} }
@@ -1441,6 +1457,7 @@ impl InsightGenerator {
&rag_messages, &rag_messages,
&ollama_client, &ollama_client,
custom_system_prompt.as_deref(), custom_system_prompt.as_deref(),
&insight_cx,
) )
.await; .await;
} }
@@ -1473,7 +1490,7 @@ impl InsightGenerator {
if !sms_messages.is_empty() { if !sms_messages.is_empty() {
match self match self
.sms_client .sms_client
.summarize_context(&sms_messages, &ollama_client) .summarize_context(&sms_messages, &ollama_client, &insight_cx)
.await .await
{ {
Ok(summary) => { Ok(summary) => {
@@ -1557,6 +1574,7 @@ impl InsightGenerator {
// 10. Generate summary first, then derive title from the summary // 10. Generate summary first, then derive title from the summary
let summary = ollama_client let summary = ollama_client
.generate_photo_summary( .generate_photo_summary(
&insight_cx,
date_taken, date_taken,
location.as_deref(), location.as_deref(),
contact.as_deref(), contact.as_deref(),
@@ -1567,7 +1585,7 @@ impl InsightGenerator {
.await?; .await?;
let title = ollama_client let title = ollama_client
.generate_photo_title(&summary, custom_system_prompt.as_deref()) .generate_photo_title(&insight_cx, &summary, custom_system_prompt.as_deref())
.await?; .await?;
log::info!("Generated title: {}", title); log::info!("Generated title: {}", title);
@@ -1634,6 +1652,7 @@ impl InsightGenerator {
&self, &self,
messages: &[crate::ai::SmsMessage], messages: &[crate::ai::SmsMessage],
ollama: &OllamaClient, ollama: &OllamaClient,
cx: &opentelemetry::Context,
) -> Vec<String> { ) -> Vec<String> {
if messages.is_empty() { if messages.is_empty() {
return Vec::new(); return Vec::new();
@@ -1669,7 +1688,7 @@ Return ONLY the comma-separated list, nothing else."#,
); );
match ollama match ollama
.generate(&prompt, Some("You are an entity extraction assistant. Extract proper nouns, people, places, and domain-specific terms that provide context.")) .generate(cx, &prompt, Some("You are an entity extraction assistant. Extract proper nouns, people, places, and domain-specific terms that provide context."))
.await .await
{ {
Ok(response) => { Ok(response) => {
@@ -1704,6 +1723,7 @@ Return ONLY the comma-separated list, nothing else."#,
messages: &[String], messages: &[String],
ollama: &OllamaClient, ollama: &OllamaClient,
custom_system: Option<&str>, custom_system: Option<&str>,
cx: &opentelemetry::Context,
) -> Option<String> { ) -> Option<String> {
if messages.is_empty() { if messages.is_empty() {
return None; return None;
@@ -1724,7 +1744,7 @@ Return ONLY the summary, nothing else."#,
let system = custom_system let system = custom_system
.unwrap_or("You are a context summarization assistant. Be concise and factual."); .unwrap_or("You are a context summarization assistant. Be concise and factual.");
match ollama.generate(&prompt, Some(system)).await { match ollama.generate(cx, &prompt, Some(system)).await {
Ok(summary) => Some(summary), Ok(summary) => Some(summary),
Err(e) => { Err(e) => {
log::warn!("Failed to summarize messages: {}", e); log::warn!("Failed to summarize messages: {}", e);
@@ -1740,6 +1760,7 @@ Return ONLY the summary, nothing else."#,
messages: &[crate::ai::SmsMessage], messages: &[crate::ai::SmsMessage],
ollama: &OllamaClient, ollama: &OllamaClient,
custom_system: Option<&str>, custom_system: Option<&str>,
cx: &opentelemetry::Context,
) -> Option<String> { ) -> Option<String> {
if messages.is_empty() { if messages.is_empty() {
return None; return None;
@@ -1785,7 +1806,7 @@ Return ONLY the summary, nothing else."#,
"You are a context summarization assistant. Be detailed and factual, preserving important context.", "You are a context summarization assistant. Be detailed and factual, preserving important context.",
); );
match ollama.generate(&prompt, Some(system)).await { match ollama.generate(cx, &prompt, Some(system)).await {
Ok(summary) => Some(summary), Ok(summary) => Some(summary),
Err(e) => { Err(e) => {
log::warn!("Failed to summarize immediate context: {}", e); log::warn!("Failed to summarize immediate context: {}", e);
@@ -1829,7 +1850,7 @@ Return ONLY the summary, nothing else."#,
"get_file_tags" => self.tool_get_file_tags(arguments, cx).await, "get_file_tags" => self.tool_get_file_tags(arguments, cx).await,
"get_faces_in_photo" => self.tool_get_faces_in_photo(arguments, cx).await, "get_faces_in_photo" => self.tool_get_faces_in_photo(arguments, cx).await,
"describe_photo" => { "describe_photo" => {
self.tool_describe_photo(backend.local(), image_base64) self.tool_describe_photo(backend.local(), image_base64, cx)
.await .await
} }
"reverse_geocode" => self.tool_reverse_geocode(arguments).await, "reverse_geocode" => self.tool_reverse_geocode(arguments).await,
@@ -1880,7 +1901,7 @@ Return ONLY the summary, nothing else."#,
&self, &self,
args: &serde_json::Value, args: &serde_json::Value,
local: &dyn LlmClient, local: &dyn LlmClient,
_cx: &opentelemetry::Context, cx: &opentelemetry::Context,
) -> String { ) -> String {
let query = match args.get("query").and_then(|v| v.as_str()) { let query = match args.get("query").and_then(|v| v.as_str()) {
Some(q) => q.to_string(), Some(q) => q.to_string(),
@@ -1944,7 +1965,10 @@ Return ONLY the summary, nothing else."#,
}; };
let final_results = if rerank_enabled && results.len() > limit { let final_results = if rerank_enabled && results.len() > limit {
match self.rerank_with_llm(&query, &results, limit, local).await { match self
.rerank_with_llm(cx, &query, &results, limit, local)
.await
{
Ok(reordered) => reordered, Ok(reordered) => reordered,
Err(e) => { Err(e) => {
log::warn!("rerank failed, using vector order: {}", e); log::warn!("rerank failed, using vector order: {}", e);
@@ -1967,6 +1991,7 @@ Return ONLY the summary, nothing else."#,
/// tool_result. On parse failure we fall back to the input order. /// tool_result. On parse failure we fall back to the input order.
async fn rerank_with_llm( async fn rerank_with_llm(
&self, &self,
cx: &opentelemetry::Context,
query: &str, query: &str,
candidates: &[String], candidates: &[String],
limit: usize, limit: usize,
@@ -2010,7 +2035,7 @@ Return ONLY the summary, nothing else."#,
let started = std::time::Instant::now(); let started = std::time::Instant::now();
let system = let system =
Some("You are a terse relevance ranker. You output only numbers separated by commas."); Some("You are a terse relevance ranker. You output only numbers separated by commas.");
let response = local.generate(&prompt, system, None).await?; let response = local.generate(cx, &prompt, system, None).await?;
log::info!( log::info!(
"rerank: finished in {} ms (prompt={} chars)", "rerank: finished in {} ms (prompt={} chars)",
started.elapsed().as_millis(), started.elapsed().as_millis(),
@@ -2620,10 +2645,11 @@ Return ONLY the summary, nothing else."#,
&self, &self,
local: &dyn LlmClient, local: &dyn LlmClient,
image_base64: &Option<String>, image_base64: &Option<String>,
cx: &opentelemetry::Context,
) -> String { ) -> String {
log::info!("tool_describe_photo: generating visual description"); log::info!("tool_describe_photo: generating visual description");
match image_base64 { match image_base64 {
Some(img) => match local.describe_image(img).await { Some(img) => match local.describe_image(cx, img).await {
Ok(desc) => desc, Ok(desc) => desc,
Err(e) => format!("Error describing photo: {}", e), Err(e) => format!("Error describing photo: {}", e),
}, },
@@ -3031,6 +3057,7 @@ Return ONLY the summary, nothing else."#,
// configured local backend. // configured local backend.
let embed_text = format!("{} {}", name, description); let embed_text = format!("{} {}", name, description);
let embedding: Option<Vec<u8>> = match crate::ai::embed_document( let embedding: Option<Vec<u8>> = match crate::ai::embed_document(
cx,
&self.ollama, &self.ollama,
self.llamacpp.as_deref(), self.llamacpp.as_deref(),
&embed_text, &embed_text,
@@ -4277,7 +4304,7 @@ Return ONLY the summary, nothing else."#,
// the local backend so non-text work stays off OpenRouter. // the local backend so non-text work stays off OpenRouter.
let inlined_visual_description: Option<String> = if !backend.images_inline { let inlined_visual_description: Option<String> = if !backend.images_inline {
match image_base64.as_deref() { match image_base64.as_deref() {
Some(b64) => match backend.local().describe_image(b64).await { Some(b64) => match backend.local().describe_image(&insight_cx, b64).await {
Ok(desc) => { Ok(desc) => {
log::info!("{}: vision describe succeeded ({} chars)", kind, desc.len()); log::info!("{}: vision describe succeeded ({} chars)", kind, desc.len());
Some(desc) Some(desc)
@@ -4397,7 +4424,7 @@ Return ONLY the summary, nothing else."#,
let (response, prompt_tokens, eval_tokens) = backend let (response, prompt_tokens, eval_tokens) = backend
.chat() .chat()
.chat_with_tools(messages.clone(), tools.clone()) .chat_with_tools(&loop_cx, messages.clone(), tools.clone())
.await?; .await?;
last_prompt_eval_count = prompt_tokens; last_prompt_eval_count = prompt_tokens;
@@ -4466,7 +4493,7 @@ Return ONLY the summary, nothing else."#,
))); )));
let (final_response, prompt_tokens, eval_tokens) = backend let (final_response, prompt_tokens, eval_tokens) = backend
.chat() .chat()
.chat_with_tools(messages.clone(), tools.clone()) .chat_with_tools(&loop_cx, messages.clone(), tools.clone())
.await?; .await?;
last_prompt_eval_count = prompt_tokens; last_prompt_eval_count = prompt_tokens;
last_eval_count = eval_tokens; last_eval_count = eval_tokens;
@@ -4499,7 +4526,7 @@ Return ONLY the summary, nothing else."#,
} }
let (resp2, pt2, et2) = backend let (resp2, pt2, et2) = backend
.chat() .chat()
.chat_with_tools(messages.clone(), vec![]) .chat_with_tools(&loop_cx, messages.clone(), vec![])
.await?; .await?;
last_prompt_eval_count = pt2; last_prompt_eval_count = pt2;
last_eval_count = et2; last_eval_count = et2;
@@ -4637,6 +4664,7 @@ Return ONLY the summary, nothing else."#,
tools: Vec<Tool>, tools: Vec<Tool>,
max_iter: usize, max_iter: usize,
) -> Result<String> { ) -> Result<String> {
let loop_cx = opentelemetry::Context::current();
let mut final_content = String::new(); let mut final_content = String::new();
for iteration in 0..max_iter { for iteration in 0..max_iter {
@@ -4644,7 +4672,7 @@ Return ONLY the summary, nothing else."#,
let (response, _prompt_tokens, _eval_tokens) = backend let (response, _prompt_tokens, _eval_tokens) = backend
.chat() .chat()
.chat_with_tools(messages.clone(), tools.clone()) .chat_with_tools(&loop_cx, messages.clone(), tools.clone())
.await?; .await?;
// Sanitize tool call arguments before pushing back into history. // Sanitize tool call arguments before pushing back into history.
@@ -4686,7 +4714,7 @@ Return ONLY the summary, nothing else."#,
"", "",
0, 0,
"", "",
&opentelemetry::Context::new(), &loop_cx,
) )
.await; .await;
messages.push(ChatMessage::tool_result(result)); messages.push(ChatMessage::tool_result(result));
@@ -4710,7 +4738,7 @@ Return ONLY the summary, nothing else."#,
)); ));
let (final_response, _, _) = backend let (final_response, _, _) = backend
.chat() .chat()
.chat_with_tools(messages.clone(), tools.clone()) .chat_with_tools(&loop_cx, messages.clone(), tools.clone())
.await?; .await?;
// Same bounded tool round as the insight loop: execute any last // Same bounded tool round as the insight loop: execute any last
// tool call, then force a tool-less turn for the actual answer. // tool call, then force a tool-less turn for the actual answer.
@@ -4728,14 +4756,14 @@ Return ONLY the summary, nothing else."#,
"", "",
0, 0,
"", "",
&opentelemetry::Context::new(), &loop_cx,
) )
.await; .await;
messages.push(ChatMessage::tool_result(result)); messages.push(ChatMessage::tool_result(result));
} }
let (resp2, _, _) = backend let (resp2, _, _) = backend
.chat() .chat()
.chat_with_tools(messages.clone(), vec![]) .chat_with_tools(&loop_cx, messages.clone(), vec![])
.await?; .await?;
final_content = resp2.content; final_content = resp2.content;
} else { } else {
+111 -26
View File
@@ -20,8 +20,11 @@
// insight_generator / insight_chat. // insight_generator / insight_chat.
#![allow(dead_code)] #![allow(dead_code)]
use anyhow::{Context, Result, anyhow, bail}; use anyhow::Context as _;
use anyhow::{Result, anyhow, bail};
use async_trait::async_trait; use async_trait::async_trait;
use opentelemetry::trace::{Span, Status};
use opentelemetry::{Context, KeyValue};
use reqwest::Client; use reqwest::Client;
use serde::Deserialize; use serde::Deserialize;
use serde_json::{Value, json}; use serde_json::{Value, json};
@@ -486,6 +489,7 @@ impl LlamaCppClient {
async fn chat_completion_with_model( async fn chat_completion_with_model(
&self, &self,
model: &str, model: &str,
mut span: Option<&mut opentelemetry::global::BoxedSpan>,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> { ) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
@@ -517,19 +521,31 @@ impl LlamaCppClient {
// Wait for any TTS synthesis to release the GPU before the request // Wait for any TTS synthesis to release the GPU before the request
// timeout starts (see ai::gpu). // timeout starts (see ai::gpu).
let _gpu = crate::ai::gpu::llm_lease().await; let _gpu = crate::ai::gpu::llm_lease().await;
let resp = self let mut request = self.client.post(&url).json(&Value::Object(body));
.client if let Some(s) = span.as_mut() {
.post(&url) request = crate::otel::with_traceparent(request, s.span_context());
.json(&Value::Object(body)) }
.send() let resp = match request.send().await {
.await Ok(r) => r,
.with_context(|| format!("POST {} failed", url))?; Err(e) => {
if let Some(s) = span.as_mut() {
s.set_status(Status::error("llama-swap request failed"));
}
return Err(e).with_context(|| format!("POST {} failed", url));
}
};
if !resp.status().is_success() { if !resp.status().is_success() {
let status = resp.status(); let status = resp.status();
crate::otel::record_http_failure(
span.as_deref_mut(),
status.as_u16(),
&format!("llama-swap chat request failed: {status}"),
);
let body = resp.text().await.unwrap_or_default(); let body = resp.text().await.unwrap_or_default();
bail!("llama-swap chat request failed: {} — {}", status, body); bail!("llama-swap chat request failed: {} — {}", status, body);
} }
crate::otel::set_http_status(span.as_deref_mut(), resp.status().as_u16());
let parsed: Value = resp.json().await.context("parsing chat response")?; let parsed: Value = resp.json().await.context("parsing chat response")?;
let choice = parsed let choice = parsed
@@ -562,14 +578,35 @@ impl LlamaCppClient {
log_timings(&parsed, prompt_tokens, completion_tokens); log_timings(&parsed, prompt_tokens, completion_tokens);
if let Some(s) = span.as_mut() {
if let Some(pt) = prompt_tokens {
s.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
}
if let Some(ct) = completion_tokens {
s.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
}
s.set_status(Status::Ok);
}
Ok((chat_msg, prompt_tokens, completion_tokens)) Ok((chat_msg, prompt_tokens, completion_tokens))
} }
/// Start the `llm.request` span for an outgoing llama-swap call.
fn start_request_span(
&self,
cx: &Context,
operation: &str,
model: &str,
) -> opentelemetry::global::BoxedSpan {
crate::otel::start_llm_request_span(cx, "llamacpp", operation, model)
}
} }
#[async_trait] #[async_trait]
impl LlmClient for LlamaCppClient { impl LlmClient for LlamaCppClient {
async fn generate( async fn generate(
&self, &self,
cx: &Context,
prompt: &str, prompt: &str,
system: Option<&str>, system: Option<&str>,
images: Option<Vec<String>>, images: Option<Vec<String>>,
@@ -582,12 +619,15 @@ impl LlmClient for LlamaCppClient {
user.images = images; user.images = images;
messages.push(user); messages.push(user);
let (reply, _, _) = self.chat_with_tools(messages, Vec::new()).await?; // Delegates to chat_with_tools, which starts the llm.request span
// (operation=chat) — one span per HTTP request.
let (reply, _, _) = self.chat_with_tools(cx, messages, Vec::new()).await?;
Ok(reply.content) Ok(reply.content)
} }
async fn chat_with_tools( async fn chat_with_tools(
&self, &self,
cx: &Context,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> { ) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
@@ -597,15 +637,25 @@ impl LlmClient for LlamaCppClient {
messages.len(), messages.len(),
tools.len() tools.len()
); );
self.chat_completion_with_model(&self.primary_model.clone(), messages, tools) let mut span = self.start_request_span(cx, "chat", &self.primary_model);
.await self.chat_completion_with_model(
&self.primary_model.clone(),
Some(&mut span),
messages,
tools,
)
.await
} }
async fn chat_with_tools_stream( async fn chat_with_tools_stream(
&self, &self,
cx: &Context,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> { ) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> {
// The span outlives this method — it moves into the stream
// generator and is finalized when the response drains.
let mut span = self.start_request_span(cx, "chat_stream", &self.primary_model);
let url = format!("{}/chat/completions", self.base_url); let url = format!("{}/chat/completions", self.base_url);
let roles: Vec<&str> = messages.iter().map(|m| m.role.as_str()).collect(); let roles: Vec<&str> = messages.iter().map(|m| m.role.as_str()).collect();
log::debug!( log::debug!(
@@ -639,19 +689,24 @@ impl LlmClient for LlamaCppClient {
// timeout starts (see ai::gpu). The guard is moved into the stream // timeout starts (see ai::gpu). The guard is moved into the stream
// below so the lease spans the whole generation, not just the send. // below so the lease spans the whole generation, not just the send.
let gpu = crate::ai::gpu::llm_lease().await; let gpu = crate::ai::gpu::llm_lease().await;
let resp = self let mut request = self.client.post(&url).json(&Value::Object(body));
.client request = crate::otel::with_traceparent(request, span.span_context());
.post(&url) let resp = request
.json(&Value::Object(body))
.send() .send()
.await .await
.with_context(|| format!("POST {} failed", url))?; .with_context(|| format!("POST {} failed", url))?;
if !resp.status().is_success() { if !resp.status().is_success() {
let status = resp.status(); let status = resp.status();
crate::otel::record_http_failure(
Some(&mut span),
status.as_u16(),
&format!("llama-swap stream request failed: {status}"),
);
let body = resp.text().await.unwrap_or_default(); let body = resp.text().await.unwrap_or_default();
bail!("llama-swap stream request failed: {} — {}", status, body); bail!("llama-swap stream request failed: {} — {}", status, body);
} }
crate::otel::set_http_status(Some(&mut span), resp.status().as_u16());
let byte_stream = resp.bytes_stream(); let byte_stream = resp.bytes_stream();
let stream = async_stream::stream! { let stream = async_stream::stream! {
@@ -760,6 +815,13 @@ impl LlmClient for LlamaCppClient {
tool_calls, tool_calls,
images: None, images: None,
}; };
if let Some(pt) = prompt_tokens {
span.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
}
if let Some(ct) = completion_tokens {
span.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
}
span.set_status(Status::Ok);
yield Ok(LlmStreamEvent::Done { yield Ok(LlmStreamEvent::Done {
message, message,
prompt_eval_count: prompt_tokens, prompt_eval_count: prompt_tokens,
@@ -770,7 +832,8 @@ impl LlmClient for LlamaCppClient {
Ok(Box::pin(stream)) Ok(Box::pin(stream))
} }
async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>> { async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
let mut span = self.start_request_span(cx, "embeddings", &self.embedding_model);
// Deliberately NO GPU lease: the embed slot sits in llama-swap's // Deliberately NO GPU lease: the embed slot sits in llama-swap's
// always-resident group and never participates in a model swap, so // always-resident group and never participates in a model swap, so
// leasing here would only stall searches behind a queued synthesis. // leasing here would only stall searches behind a queued synthesis.
@@ -780,19 +843,27 @@ impl LlmClient for LlamaCppClient {
"input": texts, "input": texts,
}); });
let resp = self let mut request = self.client.post(&url).json(&body);
.client request = crate::otel::with_traceparent(request, span.span_context());
.post(&url) let resp = match request.send().await {
.json(&body) Ok(r) => r,
.send() Err(e) => {
.await span.set_status(Status::error("llama-swap embedding request failed"));
.with_context(|| format!("POST {} failed", url))?; return Err(e).with_context(|| format!("POST {} failed", url));
}
};
if !resp.status().is_success() { if !resp.status().is_success() {
let status = resp.status(); let status = resp.status();
crate::otel::record_http_failure(
Some(&mut span),
status.as_u16(),
&format!("llama-swap embedding request failed: {status}"),
);
let body = resp.text().await.unwrap_or_default(); let body = resp.text().await.unwrap_or_default();
bail!("llama-swap embedding request failed: {} — {}", status, body); bail!("llama-swap embedding request failed: {} — {}", status, body);
} }
crate::otel::set_http_status(Some(&mut span), resp.status().as_u16());
#[derive(Deserialize)] #[derive(Deserialize)]
struct EmbedResponse { struct EmbedResponse {
@@ -803,11 +874,20 @@ impl LlmClient for LlamaCppClient {
embedding: Vec<f32>, embedding: Vec<f32>,
} }
let parsed: EmbedResponse = resp.json().await.context("parsing embed response")?; let parsed: EmbedResponse = match resp.json().await {
Ok(p) => p,
Err(e) => {
span.set_status(Status::error("parsing embed response failed"));
return Err(e).context("parsing embed response");
}
};
span.set_status(Status::Ok);
Ok(parsed.data.into_iter().map(|i| i.embedding).collect()) Ok(parsed.data.into_iter().map(|i| i.embedding).collect())
} }
async fn describe_image(&self, image_base64: &str) -> Result<String> { async fn describe_image(&self, cx: &Context, image_base64: &str) -> Result<String> {
// Own span with operation=describe_image — routes through the vision slot.
let mut span = self.start_request_span(cx, "describe_image", &self.vision_model);
let prompt = "Briefly describe what you see in this image in 1-2 sentences. \ let prompt = "Briefly describe what you see in this image in 1-2 sentences. \
Focus on the people, location, and activity."; Focus on the people, location, and activity.";
let system = "You are a scene description assistant. Be concise and factual."; let system = "You are a scene description assistant. Be concise and factual.";
@@ -823,7 +903,12 @@ impl LlmClient for LlamaCppClient {
]; ];
let (reply, _, _) = self let (reply, _, _) = self
.chat_completion_with_model(&self.vision_model.clone(), messages, Vec::new()) .chat_completion_with_model(
&self.vision_model.clone(),
Some(&mut span),
messages,
Vec::new(),
)
.await?; .await?;
Ok(reply.content) Ok(reply.content)
} }
+12 -2
View File
@@ -1,6 +1,7 @@
use anyhow::Result; use anyhow::Result;
use async_trait::async_trait; use async_trait::async_trait;
use futures::stream::BoxStream; use futures::stream::BoxStream;
use opentelemetry::Context;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
/// Provider-agnostic surface for LLM backends (Ollama, OpenRouter, …). /// Provider-agnostic surface for LLM backends (Ollama, OpenRouter, …).
@@ -10,6 +11,12 @@ use serde::{Deserialize, Serialize};
/// when a provider requires it (OpenAI-compatible APIs do), and `images` /// when a provider requires it (OpenAI-compatible APIs do), and `images`
/// stays as base64 strings here and is rewritten into content-parts where /// stays as base64 strings here and is rewritten into content-parts where
/// needed. /// needed.
///
/// Every method that performs an HTTP call takes the caller's tracing
/// `Context` as its first parameter so the impl can start an `llm.request`
/// span parented under the caller's feature span — the codebase threads
/// contexts explicitly rather than relying on ambient `Context::current()`
/// at deep call sites.
// First consumer lands in a later PR (OpenRouter impl + hybrid mode routing). // First consumer lands in a later PR (OpenRouter impl + hybrid mode routing).
#[allow(dead_code)] #[allow(dead_code)]
#[async_trait] #[async_trait]
@@ -18,6 +25,7 @@ pub trait LlmClient: Send + Sync {
/// base64 images (ignored by providers without vision support). /// base64 images (ignored by providers without vision support).
async fn generate( async fn generate(
&self, &self,
cx: &Context,
prompt: &str, prompt: &str,
system: Option<&str>, system: Option<&str>,
images: Option<Vec<String>>, images: Option<Vec<String>>,
@@ -27,6 +35,7 @@ pub trait LlmClient: Send + Sync {
/// (which may contain tool_calls) plus optional prompt/eval token counts. /// (which may contain tool_calls) plus optional prompt/eval token counts.
async fn chat_with_tools( async fn chat_with_tools(
&self, &self,
cx: &Context,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
) -> Result<(ChatMessage, Option<i32>, Option<i32>)>; ) -> Result<(ChatMessage, Option<i32>, Option<i32>)>;
@@ -39,16 +48,17 @@ pub trait LlmClient: Send + Sync {
/// as one `Done` event. /// as one `Done` event.
async fn chat_with_tools_stream( async fn chat_with_tools_stream(
&self, &self,
cx: &Context,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>>; ) -> Result<BoxStream<'static, Result<LlmStreamEvent>>>;
/// Batch embedding generation. Dimensionality is provider/model specific. /// Batch embedding generation. Dimensionality is provider/model specific.
async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>>; async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>>;
/// One-shot vision description of an image. Used to convert images into /// One-shot vision description of an image. Used to convert images into
/// plain text for the hybrid-mode conversation flow. /// plain text for the hybrid-mode conversation flow.
async fn describe_image(&self, image_base64: &str) -> Result<String>; async fn describe_image(&self, cx: &Context, image_base64: &str) -> Result<String>;
/// Enumerate available models with their capabilities. /// Enumerate available models with their capabilities.
async fn list_models(&self) -> Result<Vec<ModelCapabilities>>; async fn list_models(&self) -> Result<Vec<ModelCapabilities>>;
+25 -4
View File
@@ -47,12 +47,24 @@ impl LocalLlm {
/// pick query vs document — retrieval models treat the two sides /// pick query vs document — retrieval models treat the two sides
/// differently and an unmarked embed invites prefix-mismatch bugs. /// differently and an unmarked embed invites prefix-mismatch bugs.
pub async fn embed_query(&self, text: &str) -> Result<Vec<f32>> { pub async fn embed_query(&self, text: &str) -> Result<Vec<f32>> {
super::embed_query(&self.ollama, self.llamacpp.as_deref(), text).await super::embed_query(
&opentelemetry::Context::new(),
&self.ollama,
self.llamacpp.as_deref(),
text,
)
.await
} }
/// Embed corpus text (applies `EMBED_DOCUMENT_PREFIX`). /// Embed corpus text (applies `EMBED_DOCUMENT_PREFIX`).
pub async fn embed_document(&self, text: &str) -> Result<Vec<f32>> { pub async fn embed_document(&self, text: &str) -> Result<Vec<f32>> {
super::embed_document(&self.ollama, self.llamacpp.as_deref(), text).await super::embed_document(
&opentelemetry::Context::new(),
&self.ollama,
self.llamacpp.as_deref(),
text,
)
.await
} }
/// Single-shot local text generation via the `LLM_BACKEND`-selected /// Single-shot local text generation via the `LLM_BACKEND`-selected
@@ -60,14 +72,23 @@ impl LocalLlm {
pub async fn generate(&self, prompt: &str, system: Option<&str>) -> Result<String> { pub async fn generate(&self, prompt: &str, system: Option<&str>) -> Result<String> {
if super::local_backend_is_llamacpp() { if super::local_backend_is_llamacpp() {
if let Some(lc) = self.llamacpp.as_deref() { if let Some(lc) = self.llamacpp.as_deref() {
return <LlamaCppClient as LlmClient>::generate(lc, prompt, system, None).await; return <LlamaCppClient as LlmClient>::generate(
lc,
&opentelemetry::Context::new(),
prompt,
system,
None,
)
.await;
} }
anyhow::bail!( anyhow::bail!(
"LLM_BACKEND=llamacpp but LlamaCppClient is unconfigured — \ "LLM_BACKEND=llamacpp but LlamaCppClient is unconfigured — \
set LLAMA_SWAP_URL or switch to LLM_BACKEND=ollama" set LLAMA_SWAP_URL or switch to LLM_BACKEND=ollama"
); );
} }
self.ollama.generate(prompt, system).await self.ollama
.generate(&opentelemetry::Context::new(), prompt, system)
.await
} }
/// Label identifying which backend + model produces embeddings right /// Label identifying which backend + model produces embeddings right
+9 -5
View File
@@ -115,24 +115,26 @@ fn embed_prefix(key: &str) -> String {
/// `Instruct: <task>\nQuery: `. Must pair with the document prefix the /// `Instruct: <task>\nQuery: `. Must pair with the document prefix the
/// corpus was embedded with or similarity degrades. /// corpus was embedded with or similarity degrades.
pub async fn embed_query( pub async fn embed_query(
cx: &opentelemetry::Context,
ollama: &OllamaClient, ollama: &OllamaClient,
llamacpp: Option<&LlamaCppClient>, llamacpp: Option<&LlamaCppClient>,
text: &str, text: &str,
) -> anyhow::Result<Vec<f32>> { ) -> anyhow::Result<Vec<f32>> {
let prefixed = format!("{}{}", embed_prefix("EMBED_QUERY_PREFIX"), text); let prefixed = format!("{}{}", embed_prefix("EMBED_QUERY_PREFIX"), text);
embed_one(ollama, llamacpp, &prefixed).await embed_one(cx, ollama, llamacpp, &prefixed).await
} }
/// Embed corpus text (the stored side of retrieval). Applies /// Embed corpus text (the stored side of retrieval). Applies
/// `EMBED_DOCUMENT_PREFIX` (default empty; nomic v1.5 wants /// `EMBED_DOCUMENT_PREFIX` (default empty; nomic v1.5 wants
/// `search_document: `, Qwen3-Embedding wants none). /// `search_document: `, Qwen3-Embedding wants none).
pub async fn embed_document( pub async fn embed_document(
cx: &opentelemetry::Context,
ollama: &OllamaClient, ollama: &OllamaClient,
llamacpp: Option<&LlamaCppClient>, llamacpp: Option<&LlamaCppClient>,
text: &str, text: &str,
) -> anyhow::Result<Vec<f32>> { ) -> anyhow::Result<Vec<f32>> {
let prefixed = format!("{}{}", embed_prefix("EMBED_DOCUMENT_PREFIX"), text); let prefixed = format!("{}{}", embed_prefix("EMBED_DOCUMENT_PREFIX"), text);
embed_one(ollama, llamacpp, &prefixed).await embed_one(cx, ollama, llamacpp, &prefixed).await
} }
/// Embed a batch of strings via the configured local backend. Routes /// Embed a batch of strings via the configured local backend. Routes
@@ -140,30 +142,32 @@ pub async fn embed_document(
/// configured), else Ollama. See [`local_backend_is_llamacpp`] for the /// configured), else Ollama. See [`local_backend_is_llamacpp`] for the
/// rationale on consistency. /// rationale on consistency.
pub async fn embed_many( pub async fn embed_many(
cx: &opentelemetry::Context,
ollama: &OllamaClient, ollama: &OllamaClient,
llamacpp: Option<&LlamaCppClient>, llamacpp: Option<&LlamaCppClient>,
texts: &[&str], texts: &[&str],
) -> anyhow::Result<Vec<Vec<f32>>> { ) -> anyhow::Result<Vec<Vec<f32>>> {
if local_backend_is_llamacpp() { if local_backend_is_llamacpp() {
if let Some(lc) = llamacpp { if let Some(lc) = llamacpp {
return <LlamaCppClient as LlmClient>::generate_embeddings(lc, texts).await; return <LlamaCppClient as LlmClient>::generate_embeddings(lc, cx, texts).await;
} }
anyhow::bail!( anyhow::bail!(
"LLM_BACKEND=llamacpp but LlamaCppClient is unconfigured — \ "LLM_BACKEND=llamacpp but LlamaCppClient is unconfigured — \
set LLAMA_SWAP_URL or switch to LLM_BACKEND=ollama" set LLAMA_SWAP_URL or switch to LLM_BACKEND=ollama"
); );
} }
ollama.generate_embeddings(texts).await ollama.generate_embeddings(cx, texts).await
} }
/// Embed one string via the configured local backend. Single-text /// Embed one string via the configured local backend. Single-text
/// convenience over [`embed_many`]. /// convenience over [`embed_many`].
pub async fn embed_one( pub async fn embed_one(
cx: &opentelemetry::Context,
ollama: &OllamaClient, ollama: &OllamaClient,
llamacpp: Option<&LlamaCppClient>, llamacpp: Option<&LlamaCppClient>,
text: &str, text: &str,
) -> anyhow::Result<Vec<f32>> { ) -> anyhow::Result<Vec<f32>> {
let mut vecs = embed_many(ollama, llamacpp, &[text]).await?; let mut vecs = embed_many(cx, ollama, llamacpp, &[text]).await?;
vecs.pop() vecs.pop()
.ok_or_else(|| anyhow::anyhow!("embedding backend returned no embeddings")) .ok_or_else(|| anyhow::anyhow!("embedding backend returned no embeddings"))
} }
+4 -1
View File
@@ -259,10 +259,13 @@ pub async fn translate_nl_query(
nl: &str, nl: &str,
tag_vocab: &[(i32, String)], tag_vocab: &[(i32, String)],
today: chrono::NaiveDate, today: chrono::NaiveDate,
cx: &opentelemetry::Context,
) -> Result<StructuredQuery> { ) -> Result<StructuredQuery> {
let system = build_system_prompt(tag_vocab, today); let system = build_system_prompt(tag_vocab, today);
let messages = vec![ChatMessage::system(system), ChatMessage::user(nl)]; let messages = vec![ChatMessage::system(system), ChatMessage::user(nl)];
let (msg, _, _) = client.chat_with_tools(messages, Vec::<Tool>::new()).await?; let (msg, _, _) = client
.chat_with_tools(cx, messages, Vec::<Tool>::new())
.await?;
parse_response(&msg.content, tag_vocab) parse_response(&msg.content, tag_vocab)
} }
+238 -55
View File
@@ -1,6 +1,9 @@
use anyhow::{Context, Result}; use anyhow::Context as _;
use anyhow::Result;
use async_trait::async_trait; use async_trait::async_trait;
use chrono::NaiveDate; use chrono::NaiveDate;
use opentelemetry::trace::{Span, Status};
use opentelemetry::{Context, KeyValue};
use reqwest::Client; use reqwest::Client;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::collections::HashMap; use std::collections::HashMap;
@@ -365,6 +368,7 @@ impl OllamaClient {
async fn try_generate( async fn try_generate(
&self, &self,
mut span: Option<&mut opentelemetry::global::BoxedSpan>,
url: &str, url: &str,
model: &str, model: &str,
prompt: &str, prompt: &str,
@@ -382,15 +386,30 @@ impl OllamaClient {
think, think,
}; };
let response = self let mut request_builder = self
.client .client
.post(format!("{}/api/generate", url)) .post(format!("{}/api/generate", url))
.json(&request) .json(&request);
.send() if let Some(s) = span.as_mut() {
.await?; request_builder = crate::otel::with_traceparent(request_builder, s.span_context());
}
let response = match request_builder.send().await {
Ok(r) => r,
Err(e) => {
if let Some(s) = span.as_mut() {
s.set_status(Status::error("Ollama request failed"));
}
return Err(e.into());
}
};
if !response.status().is_success() { if !response.status().is_success() {
let status = response.status(); let status = response.status();
crate::otel::record_http_failure(
span.as_deref_mut(),
status.as_u16(),
&format!("Ollama request failed: {status}"),
);
let error_body = response.text().await.unwrap_or_default(); let error_body = response.text().await.unwrap_or_default();
return Err(anyhow::anyhow!( return Err(anyhow::anyhow!(
"Ollama request failed: {} - {}", "Ollama request failed: {} - {}",
@@ -398,8 +417,26 @@ impl OllamaClient {
error_body error_body
)); ));
} }
crate::otel::set_http_status(span.as_deref_mut(), response.status().as_u16());
let result: OllamaResponse = response.json().await?; let result: OllamaResponse = match response.json().await {
Ok(r) => r,
Err(e) => {
if let Some(s) = span.as_mut() {
s.set_status(Status::error("parsing Ollama response failed"));
}
return Err(e.into());
}
};
if let Some(s) = span.as_mut() {
if let Some(pt) = result.prompt_eval_count {
s.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
}
if let Some(ct) = result.eval_count {
s.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
}
s.set_status(Status::Ok);
}
log_chat_metrics( log_chat_metrics(
result.prompt_eval_count, result.prompt_eval_count,
result.prompt_eval_duration, result.prompt_eval_duration,
@@ -409,32 +446,55 @@ impl OllamaClient {
Ok(result.response) Ok(result.response)
} }
pub async fn generate(&self, prompt: &str, system: Option<&str>) -> Result<String> { /// Start the `llm.request` span for an outgoing Ollama call.
self.generate_with_images(prompt, system, None).await fn start_request_span(
&self,
cx: &Context,
operation: &str,
model: &str,
) -> opentelemetry::global::BoxedSpan {
crate::otel::start_llm_request_span(cx, "ollama", operation, model)
}
pub async fn generate(
&self,
cx: &Context,
prompt: &str,
system: Option<&str>,
) -> Result<String> {
self.generate_with_images(cx, prompt, system, None).await
} }
#[allow(dead_code)] #[allow(dead_code)]
pub async fn generate_no_think(&self, prompt: &str, system: Option<&str>) -> Result<String> { pub async fn generate_no_think(
self.generate_with_options(prompt, system, None, Some(false)) &self,
cx: &Context,
prompt: &str,
system: Option<&str>,
) -> Result<String> {
self.generate_with_options(cx, prompt, system, None, Some(false), "generate")
.await .await
} }
pub async fn generate_with_images( pub async fn generate_with_images(
&self, &self,
cx: &Context,
prompt: &str, prompt: &str,
system: Option<&str>, system: Option<&str>,
images: Option<Vec<String>>, images: Option<Vec<String>>,
) -> Result<String> { ) -> Result<String> {
self.generate_with_options(prompt, system, images, None) self.generate_with_options(cx, prompt, system, images, None, "generate")
.await .await
} }
async fn generate_with_options( async fn generate_with_options(
&self, &self,
cx: &Context,
prompt: &str, prompt: &str,
system: Option<&str>, system: Option<&str>,
images: Option<Vec<String>>, images: Option<Vec<String>>,
think: Option<bool>, think: Option<bool>,
operation: &str,
) -> Result<String> { ) -> Result<String> {
log::debug!("=== Ollama Request ==="); log::debug!("=== Ollama Request ===");
log::debug!("Primary model: {}", self.primary_model); log::debug!("Primary model: {}", self.primary_model);
@@ -447,6 +507,10 @@ impl OllamaClient {
} }
log::debug!("====================="); log::debug!("=====================");
// Span for the whole generate attempt; the model attribute
// reflects whichever server actually answered.
let mut span = self.start_request_span(cx, operation, &self.primary_model);
// Try primary server first with primary model // Try primary server first with primary model
log::info!( log::info!(
"Attempting to generate with primary server: {} (model: {})", "Attempting to generate with primary server: {} (model: {})",
@@ -455,6 +519,7 @@ impl OllamaClient {
); );
let primary_result = self let primary_result = self
.try_generate( .try_generate(
Some(&mut span),
&self.primary_url, &self.primary_url,
&self.primary_model, &self.primary_model,
prompt, prompt,
@@ -483,8 +548,10 @@ impl OllamaClient {
fallback_url, fallback_url,
fallback_model fallback_model
); );
span.set_attribute(KeyValue::new("llm.model", fallback_model.to_string()));
match self match self
.try_generate( .try_generate(
Some(&mut span),
fallback_url, fallback_url,
fallback_model, fallback_model,
prompt, prompt,
@@ -531,6 +598,7 @@ impl OllamaClient {
/// Generate a title for a single photo based on its generated summary /// Generate a title for a single photo based on its generated summary
pub async fn generate_photo_title( pub async fn generate_photo_title(
&self, &self,
cx: &Context,
summary: &str, summary: &str,
custom_system: Option<&str>, custom_system: Option<&str>,
) -> Result<String> { ) -> Result<String> {
@@ -546,7 +614,7 @@ Capture the key moment or theme. Return ONLY the title, nothing else."#,
let system = custom_system.unwrap_or("You are my long term memory assistant. Use only the information provided. Do not invent details."); let system = custom_system.unwrap_or("You are my long term memory assistant. Use only the information provided. Do not invent details.");
let title = self let title = self
.generate_with_images(&prompt, Some(system), None) .generate_with_images(cx, &prompt, Some(system), None)
.await?; .await?;
// Models decorate despite "Return ONLY the title": quotes, bold // Models decorate despite "Return ONLY the title": quotes, bold
// markers, sometimes a "Title:" label. // markers, sometimes a "Title:" label.
@@ -563,6 +631,7 @@ Capture the key moment or theme. Return ONLY the title, nothing else."#,
/// Generate a summary for a single photo based on its context /// Generate a summary for a single photo based on its context
pub async fn generate_photo_summary( pub async fn generate_photo_summary(
&self, &self,
cx: &Context,
date: NaiveDate, date: NaiveDate,
location: Option<&str>, location: Option<&str>,
contact: Option<&str>, contact: Option<&str>,
@@ -645,21 +714,32 @@ Analyze the image and use specific details from both the visual content and the
let system = custom_system.unwrap_or("You are a memory refreshing assistant who is able to provide insights through analyzing past conversations. Use only the information provided. Do not invent details."); let system = custom_system.unwrap_or("You are a memory refreshing assistant who is able to provide insights through analyzing past conversations. Use only the information provided. Do not invent details.");
let images = image_base64.map(|img| vec![img]); let images = image_base64.map(|img| vec![img]);
self.generate_with_images(&prompt, Some(system), images) self.generate_with_images(cx, &prompt, Some(system), images)
.await .await
} }
/// Generate a brief visual description of a photo for use in RAG query enrichment. /// Generate a brief visual description of a photo for use in RAG query enrichment.
/// Returns 1-2 sentences describing people, location, and activity visible in the image. /// Returns 1-2 sentences describing people, location, and activity visible in the image.
/// Only called when the model has vision capabilities. /// Only called when the model has vision capabilities.
pub async fn generate_photo_description(&self, image_base64: &str) -> Result<String> { pub async fn generate_photo_description(
&self,
cx: &Context,
image_base64: &str,
) -> Result<String> {
let prompt = "Briefly describe what you see in this image in 1-2 sentences. \ let prompt = "Briefly describe what you see in this image in 1-2 sentences. \
Focus on the people, location, and activity."; Focus on the people, location, and activity.";
let system = "You are a scene description assistant. Be concise and factual."; let system = "You are a scene description assistant. Be concise and factual.";
let images = vec![image_base64.to_string()]; let images = vec![image_base64.to_string()];
let description = self let description = self
.generate_with_images(prompt, Some(system), Some(images)) .generate_with_options(
cx,
prompt,
Some(system),
Some(images),
None,
"describe_image",
)
.await?; .await?;
Ok(description.trim().to_string()) Ok(description.trim().to_string())
@@ -672,9 +752,11 @@ Analyze the image and use specific details from both the visual content and the
/// iteration of the agent loop. /// iteration of the agent loop.
pub async fn chat_with_tools( pub async fn chat_with_tools(
&self, &self,
cx: &Context,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> { ) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
let mut span = self.start_request_span(cx, "chat", &self.primary_model);
let order = self.attempt_order(); let order = self.attempt_order();
let mut errors: Vec<String> = Vec::new(); let mut errors: Vec<String> = Vec::new();
@@ -685,8 +767,11 @@ Analyze the image and use specific details from both the visual content and the
url, url,
model model
); );
if *label == "fallback" {
span.set_attribute(KeyValue::new("llm.model", model.to_string()));
}
match self match self
.try_chat_with_tools(url, messages.clone(), tools.clone()) .try_chat_with_tools(Some(&mut span), url, messages.clone(), tools.clone())
.await .await
{ {
Ok(result) => { Ok(result) => {
@@ -728,18 +813,22 @@ Analyze the image and use specific details from both the visual content and the
/// any, live on the final message). /// any, live on the final message).
pub async fn chat_with_tools_stream( pub async fn chat_with_tools_stream(
&self, &self,
cx: &Context,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> { ) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> {
// Same preference logic as `chat_with_tools`. Only the initial // Same preference logic as `chat_with_tools`. Only the initial
// connection is retried across servers — once the stream begins, // connection is retried across servers — once the stream begins,
// mid-stream errors propagate to the caller. // mid-stream errors propagate to the caller. Each attempt gets its
// own `llm.request` span, created inside
// `try_chat_with_tools_stream` so the winning attempt's span moves
// into the stream and is finalized when the response drains.
let order = self.attempt_order(); let order = self.attempt_order();
let mut last_err: Option<anyhow::Error> = None; let mut last_err: Option<anyhow::Error> = None;
for (label, url, _model) in &order { for (label, url, _model) in &order {
match self match self
.try_chat_with_tools_stream(url, messages.clone(), tools.clone()) .try_chat_with_tools_stream(cx, url, messages.clone(), tools.clone())
.await .await
{ {
Ok(s) => { Ok(s) => {
@@ -759,6 +848,7 @@ Analyze the image and use specific details from both the visual content and the
async fn try_chat_with_tools_stream( async fn try_chat_with_tools_stream(
&self, &self,
cx: &Context,
base_url: &str, base_url: &str,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
@@ -773,6 +863,11 @@ Analyze the image and use specific details from both the visual content and the
}; };
let options = self.build_options(); let options = self.build_options();
// The span outlives this method — the stream yields until the
// response fully drains, so the owned span moves into the
// generator and is finalized there.
let mut span = self.start_request_span(cx, "chat_stream", model);
let request_body = OllamaChatRequest { let request_body = OllamaChatRequest {
model, model,
messages: &messages, messages: &messages,
@@ -781,16 +876,26 @@ Analyze the image and use specific details from both the visual content and the
options, options,
}; };
let response = self let response = {
.client let mut rb = self.client.post(&url).json(&request_body);
.post(&url) rb = crate::otel::with_traceparent(rb, span.span_context());
.json(&request_body) match rb.send().await {
.send() Ok(r) => r,
.await Err(e) => {
.with_context(|| format!("Failed to connect to Ollama at {}", url))?; span.set_status(Status::error("Ollama stream connect failed"));
return Err(e)
.with_context(|| format!("Failed to connect to Ollama at {}", url));
}
}
};
if !response.status().is_success() { if !response.status().is_success() {
let status = response.status(); let status = response.status();
crate::otel::record_http_failure(
Some(&mut span),
status.as_u16(),
&format!("Ollama stream request failed: {status}"),
);
let body = response.text().await.unwrap_or_default(); let body = response.text().await.unwrap_or_default();
anyhow::bail!( anyhow::bail!(
"Ollama stream request failed with status {}: {}", "Ollama stream request failed with status {}: {}",
@@ -798,6 +903,7 @@ Analyze the image and use specific details from both the visual content and the
body body
); );
} }
crate::otel::set_http_status(Some(&mut span), response.status().as_u16());
// Ollama streams NDJSON: each line is a full `OllamaStreamChunk`. // Ollama streams NDJSON: each line is a full `OllamaStreamChunk`.
// We buffer partial lines across chunks from the byte stream. // We buffer partial lines across chunks from the byte stream.
@@ -879,6 +985,13 @@ Analyze the image and use specific details from both the visual content and the
eval_count, eval_count,
eval_duration, eval_duration,
); );
if let Some(pt) = prompt_eval_count {
span.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
}
if let Some(ct) = eval_count {
span.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
}
span.set_status(Status::Ok);
let message = ChatMessage { let message = ChatMessage {
role, role,
content: accumulated, content: accumulated,
@@ -897,6 +1010,7 @@ Analyze the image and use specific details from both the visual content and the
async fn try_chat_with_tools( async fn try_chat_with_tools(
&self, &self,
mut span: Option<&mut opentelemetry::global::BoxedSpan>,
base_url: &str, base_url: &str,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
@@ -924,16 +1038,30 @@ Analyze the image and use specific details from both the visual content and the
.unwrap_or_else(|e| format!("<serialization error: {}>", e)); .unwrap_or_else(|e| format!("<serialization error: {}>", e));
log::debug!("chat_with_tools request body: {}", request_json); log::debug!("chat_with_tools request body: {}", request_json);
let response = self let response = {
.client let mut rb = self.client.post(&url).json(&request_body);
.post(&url) if let Some(s) = span.as_mut() {
.json(&request_body) rb = crate::otel::with_traceparent(rb, s.span_context());
.send() }
.await match rb.send().await {
.with_context(|| format!("Failed to connect to Ollama at {}", url))?; Ok(r) => r,
Err(e) => {
if let Some(s) = span.as_mut() {
s.set_status(Status::error("Ollama chat connect failed"));
}
return Err(e)
.with_context(|| format!("Failed to connect to Ollama at {}", url));
}
}
};
if !response.status().is_success() { if !response.status().is_success() {
let status = response.status(); let status = response.status();
crate::otel::record_http_failure(
span.as_deref_mut(),
status.as_u16(),
&format!("Ollama chat request failed: {status}"),
);
let body = response.text().await.unwrap_or_default(); let body = response.text().await.unwrap_or_default();
// warn, not error — the outer `chat_with_tools` may recover via // warn, not error — the outer `chat_with_tools` may recover via
// the fallback server. When both fail, the outer layer emits the // the fallback server. When both fail, the outer layer emits the
@@ -950,11 +1078,26 @@ Analyze the image and use specific details from both the visual content and the
body body
); );
} }
crate::otel::set_http_status(span.as_deref_mut(), response.status().as_u16());
let chat_response: OllamaChatResponse = response let chat_response: OllamaChatResponse = match response.json().await {
.json() Ok(r) => r,
.await Err(e) => {
.with_context(|| "Failed to parse Ollama chat response")?; if let Some(s) = span.as_mut() {
s.set_status(Status::error("parsing Ollama chat response failed"));
}
return Err(e).with_context(|| "Failed to parse Ollama chat response");
}
};
if let Some(s) = span.as_mut() {
if let Some(pt) = chat_response.prompt_eval_count {
s.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
}
if let Some(ct) = chat_response.eval_count {
s.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
}
s.set_status(Status::Ok);
}
// Log performance counters returned by Ollama. Durations are // Log performance counters returned by Ollama. Durations are
// reported in nanoseconds; we render ms + tokens/sec for skim-ability // reported in nanoseconds; we render ms + tokens/sec for skim-ability
@@ -976,8 +1119,8 @@ Analyze the image and use specific details from both the visual content and the
/// Generate an embedding vector for text using nomic-embed-text:v1.5 /// Generate an embedding vector for text using nomic-embed-text:v1.5
/// Returns a 768-dimensional vector as Vec<f32> /// Returns a 768-dimensional vector as Vec<f32>
pub async fn generate_embedding(&self, text: &str) -> Result<Vec<f32>> { pub async fn generate_embedding(&self, cx: &Context, text: &str) -> Result<Vec<f32>> {
let embeddings = self.generate_embeddings(&[text]).await?; let embeddings = self.generate_embeddings(cx, &[text]).await?;
embeddings embeddings
.into_iter() .into_iter()
.next() .next()
@@ -987,9 +1130,11 @@ Analyze the image and use specific details from both the visual content and the
/// Generate embeddings for multiple texts in a single API call (batch mode) /// Generate embeddings for multiple texts in a single API call (batch mode)
/// Returns a vector of 768-dimensional vectors /// Returns a vector of 768-dimensional vectors
/// This is much more efficient than calling generate_embedding multiple times /// This is much more efficient than calling generate_embedding multiple times
pub async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>> { pub async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
let embedding_model = EMBEDDING_MODEL; let embedding_model = EMBEDDING_MODEL;
let mut span = self.start_request_span(cx, "embeddings", embedding_model);
log::debug!("=== Ollama Batch Embedding Request ==="); log::debug!("=== Ollama Batch Embedding Request ===");
log::debug!("Model: {}", embedding_model); log::debug!("Model: {}", embedding_model);
log::debug!("Batch size: {} texts", texts.len()); log::debug!("Batch size: {} texts", texts.len());
@@ -1003,7 +1148,7 @@ Analyze the image and use specific details from both the visual content and the
embedding_model embedding_model
); );
let primary_result = self let primary_result = self
.try_generate_embeddings(&self.primary_url, embedding_model, texts) .try_generate_embeddings(Some(&mut span), &self.primary_url, embedding_model, texts)
.await; .await;
let embeddings = match primary_result { let embeddings = match primary_result {
@@ -1026,7 +1171,12 @@ Analyze the image and use specific details from both the visual content and the
embedding_model embedding_model
); );
match self match self
.try_generate_embeddings(fallback_url, embedding_model, texts) .try_generate_embeddings(
Some(&mut span),
fallback_url,
embedding_model,
texts,
)
.await .await
{ {
Ok(embeddings) => { Ok(embeddings) => {
@@ -1073,6 +1223,7 @@ Analyze the image and use specific details from both the visual content and the
/// Internal helper to try generating embeddings for multiple texts from a specific server /// Internal helper to try generating embeddings for multiple texts from a specific server
async fn try_generate_embeddings( async fn try_generate_embeddings(
&self, &self,
mut span: Option<&mut opentelemetry::global::BoxedSpan>,
url: &str, url: &str,
model: &str, model: &str,
texts: &[&str], texts: &[&str],
@@ -1082,15 +1233,32 @@ Analyze the image and use specific details from both the visual content and the
input: texts.iter().map(|s| s.to_string()).collect(), input: texts.iter().map(|s| s.to_string()).collect(),
}; };
let response = self let response = {
.client let mut rb = self
.post(format!("{}/api/embed", url)) .client
.json(&request) .post(format!("{}/api/embed", url))
.send() .json(&request);
.await?; if let Some(s) = span.as_mut() {
rb = crate::otel::with_traceparent(rb, s.span_context());
}
match rb.send().await {
Ok(r) => r,
Err(e) => {
if let Some(s) = span.as_mut() {
s.set_status(Status::error("Ollama embedding request failed"));
}
return Err(e.into());
}
}
};
if !response.status().is_success() { if !response.status().is_success() {
let status = response.status(); let status = response.status();
crate::otel::record_http_failure(
span.as_deref_mut(),
status.as_u16(),
&format!("Ollama batch embedding request failed: {status}"),
);
let error_body = response.text().await.unwrap_or_default(); let error_body = response.text().await.unwrap_or_default();
return Err(anyhow::anyhow!( return Err(anyhow::anyhow!(
"Ollama batch embedding request failed: {} - {}", "Ollama batch embedding request failed: {} - {}",
@@ -1098,8 +1266,20 @@ Analyze the image and use specific details from both the visual content and the
error_body error_body
)); ));
} }
crate::otel::set_http_status(span.as_deref_mut(), response.status().as_u16());
let result: OllamaEmbedResponse = response.json().await?; let result: OllamaEmbedResponse = match response.json().await {
Ok(r) => r,
Err(e) => {
if let Some(s) = span.as_mut() {
s.set_status(Status::error("parsing Ollama embed response failed"));
}
return Err(e.into());
}
};
if let Some(s) = span.as_mut() {
s.set_status(Status::Ok);
}
Ok(result.embeddings) Ok(result.embeddings)
} }
} }
@@ -1108,35 +1288,38 @@ Analyze the image and use specific details from both the visual content and the
impl LlmClient for OllamaClient { impl LlmClient for OllamaClient {
async fn generate( async fn generate(
&self, &self,
cx: &Context,
prompt: &str, prompt: &str,
system: Option<&str>, system: Option<&str>,
images: Option<Vec<String>>, images: Option<Vec<String>>,
) -> Result<String> { ) -> Result<String> {
self.generate_with_images(prompt, system, images).await self.generate_with_images(cx, prompt, system, images).await
} }
async fn chat_with_tools( async fn chat_with_tools(
&self, &self,
cx: &Context,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> { ) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
OllamaClient::chat_with_tools(self, messages, tools).await OllamaClient::chat_with_tools(self, cx, messages, tools).await
} }
async fn chat_with_tools_stream( async fn chat_with_tools_stream(
&self, &self,
cx: &Context,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> { ) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> {
OllamaClient::chat_with_tools_stream(self, messages, tools).await OllamaClient::chat_with_tools_stream(self, cx, messages, tools).await
} }
async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>> { async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
OllamaClient::generate_embeddings(self, texts).await OllamaClient::generate_embeddings(self, cx, texts).await
} }
async fn describe_image(&self, image_base64: &str) -> Result<String> { async fn describe_image(&self, cx: &Context, image_base64: &str) -> Result<String> {
self.generate_photo_description(image_base64).await self.generate_photo_description(cx, image_base64).await
} }
async fn list_models(&self) -> Result<Vec<ModelCapabilities>> { async fn list_models(&self) -> Result<Vec<ModelCapabilities>> {
+202 -83
View File
@@ -2,8 +2,11 @@
// the translation helpers directly. // the translation helpers directly.
#![allow(dead_code)] #![allow(dead_code)]
use anyhow::{Context, Result, anyhow, bail}; use anyhow::Context as _;
use anyhow::{Result, anyhow, bail};
use async_trait::async_trait; use async_trait::async_trait;
use opentelemetry::trace::{Span, Status};
use opentelemetry::{Context, KeyValue};
use reqwest::Client; use reqwest::Client;
use serde::Deserialize; use serde::Deserialize;
use serde_json::{Value, json}; use serde_json::{Value, json};
@@ -14,6 +17,7 @@ use std::time::{Duration, Instant};
use crate::ai::llm_client::{ use crate::ai::llm_client::{
ChatMessage, LlmClient, LlmStreamEvent, ModelCapabilities, Tool, ToolCall, ToolCallFunction, ChatMessage, LlmClient, LlmStreamEvent, ModelCapabilities, Tool, ToolCall, ToolCallFunction,
}; };
use futures::stream::{BoxStream, StreamExt}; use futures::stream::{BoxStream, StreamExt};
const DEFAULT_BASE_URL: &str = "https://openrouter.ai/api/v1"; const DEFAULT_BASE_URL: &str = "https://openrouter.ai/api/v1";
@@ -144,6 +148,19 @@ impl OpenRouterClient {
b b
} }
/// Start the `llm.request` span for an outgoing call. `operation` is
/// `chat`, `embeddings`, or `describe` — the call shape, not the entry
/// point (a `describe_image` funneled through `chat_with_tools` is
/// still a `describe`).
fn start_request_span(
&self,
cx: &Context,
operation: &str,
model: &str,
) -> opentelemetry::global::BoxedSpan {
crate::otel::start_llm_request_span(cx, "openrouter", operation, model)
}
/// Translate canonical messages to the OpenAI-compatible wire shape. /// Translate canonical messages to the OpenAI-compatible wire shape.
/// ///
/// Walks in order so it can attach `tool_call_id` to `role=tool` messages /// Walks in order so it can attach `tool_call_id` to `role=tool` messages
@@ -312,6 +329,7 @@ impl OpenRouterClient {
impl LlmClient for OpenRouterClient { impl LlmClient for OpenRouterClient {
async fn generate( async fn generate(
&self, &self,
cx: &Context,
prompt: &str, prompt: &str,
system: Option<&str>, system: Option<&str>,
images: Option<Vec<String>>, images: Option<Vec<String>>,
@@ -324,90 +342,34 @@ impl LlmClient for OpenRouterClient {
user.images = images; user.images = images;
messages.push(user); messages.push(user);
let (reply, _, _) = self.chat_with_tools(messages, Vec::new()).await?; // Delegates to chat_with_tools, which starts the llm.request span
// (operation=chat) — one span per HTTP request.
let (reply, _, _) = self.chat_with_tools(cx, messages, Vec::new()).await?;
Ok(reply.content) Ok(reply.content)
} }
async fn chat_with_tools( async fn chat_with_tools(
&self, &self,
cx: &Context,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> { ) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
let url = format!("{}/chat/completions", self.base_url); let mut span = self.start_request_span(cx, "chat", &self.primary_model);
let mut body = serde_json::Map::new(); self.chat_with_tools_impl(Some(&mut span), messages, tools)
body.insert("model".into(), Value::String(self.primary_model.clone()));
body.insert(
"messages".into(),
Value::Array(Self::messages_to_openai(&messages)),
);
body.insert("stream".into(), Value::Bool(false));
if !tools.is_empty() {
body.insert(
"tools".into(),
serde_json::to_value(&tools).context("serializing tools")?,
);
}
for (k, v) in self.build_options() {
body.insert(k.into(), v);
}
log::info!(
"OpenRouter chat_with_tools: model={} messages={} tools={}",
self.primary_model,
messages.len(),
tools.len()
);
let resp = self
.authed(self.client.post(&url))
.json(&Value::Object(body))
.send()
.await .await
.with_context(|| format!("POST {} failed", url))?;
if !resp.status().is_success() {
let status = resp.status();
let body = resp.text().await.unwrap_or_default();
bail!("OpenRouter chat request failed: {} — {}", status, body);
}
let parsed: Value = resp.json().await.context("parsing chat response")?;
let choice = parsed
.get("choices")
.and_then(|v| v.as_array())
.and_then(|a| a.first())
.ok_or_else(|| {
anyhow!(
"response missing choices[0]: {}",
extract_openrouter_error_detail(&parsed)
)
})?;
let msg = choice.get("message").ok_or_else(|| {
anyhow!(
"choices[0] missing message: {}",
extract_openrouter_error_detail(&parsed)
)
})?;
let chat_msg = Self::openai_message_to_chat(msg)?;
let usage = parsed.get("usage");
let prompt_tokens = usage
.and_then(|u| u.get("prompt_tokens"))
.and_then(|v| v.as_i64())
.map(|n| n as i32);
let completion_tokens = usage
.and_then(|u| u.get("completion_tokens"))
.and_then(|v| v.as_i64())
.map(|n| n as i32);
Ok((chat_msg, prompt_tokens, completion_tokens))
} }
async fn chat_with_tools_stream( async fn chat_with_tools_stream(
&self, &self,
cx: &Context,
messages: Vec<ChatMessage>, messages: Vec<ChatMessage>,
tools: Vec<Tool>, tools: Vec<Tool>,
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> { ) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> {
// The span outlives this method — the stream yields until the
// response fully drains, so the owned span moves into the
// generator and is finalized there.
let mut span = self.start_request_span(cx, "chat_stream", &self.primary_model);
let url = format!("{}/chat/completions", self.base_url); let url = format!("{}/chat/completions", self.base_url);
let mut body = serde_json::Map::new(); let mut body = serde_json::Map::new();
body.insert("model".into(), Value::String(self.primary_model.clone())); body.insert("model".into(), Value::String(self.primary_model.clone()));
@@ -432,18 +394,26 @@ impl LlmClient for OpenRouterClient {
body.insert(k.into(), v); body.insert(k.into(), v);
} }
let resp = self let mut request = self
.authed(self.client.post(&url)) .authed(self.client.post(&url))
.json(&Value::Object(body)) .json(&Value::Object(body));
request = crate::otel::with_traceparent(request, span.span_context());
let resp = request
.send() .send()
.await .await
.with_context(|| format!("POST {} failed", url))?; .with_context(|| format!("POST {} failed", url))?;
if !resp.status().is_success() { if !resp.status().is_success() {
let status = resp.status(); let status = resp.status();
crate::otel::record_http_failure(
Some(&mut span),
status.as_u16(),
&format!("OpenRouter stream request failed: {status}"),
);
let body = resp.text().await.unwrap_or_default(); let body = resp.text().await.unwrap_or_default();
bail!("OpenRouter stream request failed: {} — {}", status, body); bail!("OpenRouter stream request failed: {} — {}", status, body);
} }
crate::otel::set_http_status(Some(&mut span), resp.status().as_u16());
// OpenAI-compat SSE stream. Each event is `data: <json>\n\n`, with // OpenAI-compat SSE stream. Each event is `data: <json>\n\n`, with
// `data: [DONE]` signalling completion. Tool calls arrive as // `data: [DONE]` signalling completion. Tool calls arrive as
@@ -467,6 +437,7 @@ impl LlmClient for OpenRouterClient {
let chunk = match chunk { let chunk = match chunk {
Ok(b) => b, Ok(b) => b,
Err(e) => { Err(e) => {
span.set_status(Status::error(format!("stream read failed: {e}")));
yield Err(anyhow!("stream read failed: {}", e)); yield Err(anyhow!("stream read failed: {}", e));
return; return;
} }
@@ -517,6 +488,9 @@ impl LlmClient for OpenRouterClient {
.get("completion_tokens") .get("completion_tokens")
.and_then(|n| n.as_i64()) .and_then(|n| n.as_i64())
.map(|n| n as i32); .map(|n| n as i32);
if let Some(cost) = usage.get("cost").and_then(|c| c.as_f64()) {
span.set_attribute(KeyValue::new("llm.cost_usd", cost));
}
} }
let Some(choices) = v.get("choices").and_then(|c| c.as_array()) let Some(choices) = v.get("choices").and_then(|c| c.as_array())
@@ -606,6 +580,13 @@ impl LlmClient for OpenRouterClient {
tool_calls, tool_calls,
images: None, images: None,
}; };
if let Some(pt) = prompt_tokens {
span.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
}
if let Some(ct) = completion_tokens {
span.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
}
span.set_status(Status::Ok);
yield Ok(LlmStreamEvent::Done { yield Ok(LlmStreamEvent::Done {
message, message,
prompt_eval_count: prompt_tokens, prompt_eval_count: prompt_tokens,
@@ -616,25 +597,35 @@ impl LlmClient for OpenRouterClient {
Ok(Box::pin(stream)) Ok(Box::pin(stream))
} }
async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>> { async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
let mut span = self.start_request_span(cx, "embeddings", &self.embedding_model);
let url = format!("{}/embeddings", self.base_url); let url = format!("{}/embeddings", self.base_url);
let body = json!({ let body = json!({
"model": self.embedding_model, "model": self.embedding_model,
"input": texts, "input": texts,
}); });
let resp = self let mut request = self.authed(self.client.post(&url)).json(&body);
.authed(self.client.post(&url)) request = crate::otel::with_traceparent(request, span.span_context());
.json(&body) let resp = request
.send() .send()
.await .await
.with_context(|| format!("POST {} failed", url))?; .with_context(|| format!("POST {} failed", url))
.inspect_err(|_| {
span.set_status(Status::error("OpenRouter embedding request failed"));
})?;
if !resp.status().is_success() { if !resp.status().is_success() {
let status = resp.status(); let status = resp.status();
crate::otel::record_http_failure(
Some(&mut span),
status.as_u16(),
&format!("OpenRouter embedding request failed: {status}"),
);
let body = resp.text().await.unwrap_or_default(); let body = resp.text().await.unwrap_or_default();
bail!("OpenRouter embedding request failed: {} — {}", status, body); bail!("OpenRouter embedding request failed: {} — {}", status, body);
} }
crate::otel::set_http_status(Some(&mut span), resp.status().as_u16());
#[derive(Deserialize)] #[derive(Deserialize)]
struct EmbedResponse { struct EmbedResponse {
@@ -646,18 +637,28 @@ impl LlmClient for OpenRouterClient {
} }
let parsed: EmbedResponse = resp.json().await.context("parsing embed response")?; let parsed: EmbedResponse = resp.json().await.context("parsing embed response")?;
span.set_status(Status::Ok);
Ok(parsed.data.into_iter().map(|i| i.embedding).collect()) Ok(parsed.data.into_iter().map(|i| i.embedding).collect())
} }
async fn describe_image(&self, image_base64: &str) -> Result<String> { async fn describe_image(&self, cx: &Context, image_base64: &str) -> Result<String> {
// Own span with operation=describe_image — the funnel into
// chat_with_tools_impl keeps the request on a single span.
let mut span = self.start_request_span(cx, "describe_image", &self.primary_model);
let prompt = "Briefly describe what you see in this image in 1-2 sentences. \ let prompt = "Briefly describe what you see in this image in 1-2 sentences. \
Focus on the people, location, and activity."; Focus on the people, location, and activity.";
self.generate( let mut messages: Vec<ChatMessage> = Vec::new();
prompt, messages.push(ChatMessage::system(
Some("You are a scene description assistant. Be concise and factual."), "You are a scene description assistant. Be concise and factual.",
Some(vec![image_base64.to_string()]), ));
) let mut user = ChatMessage::user(prompt);
.await user.images = Some(vec![image_base64.to_string()]);
messages.push(user);
let (reply, _, _) = self
.chat_with_tools_impl(Some(&mut span), messages, Vec::new())
.await?;
Ok(reply.content)
} }
async fn list_models(&self) -> Result<Vec<ModelCapabilities>> { async fn list_models(&self) -> Result<Vec<ModelCapabilities>> {
@@ -711,6 +712,124 @@ impl LlmClient for OpenRouterClient {
} }
} }
impl OpenRouterClient {
/// HTTP core of `chat_with_tools`. `span` is the `llm.request` span
/// started by the caller (or a funnelling entry point like
/// `describe_image`) so attributes from the response land on the right
/// operation label.
async fn chat_with_tools_impl(
&self,
mut span: Option<&mut opentelemetry::global::BoxedSpan>,
messages: Vec<ChatMessage>,
tools: Vec<Tool>,
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
let url = format!("{}/chat/completions", self.base_url);
let mut body = serde_json::Map::new();
body.insert("model".into(), Value::String(self.primary_model.clone()));
body.insert(
"messages".into(),
Value::Array(Self::messages_to_openai(&messages)),
);
body.insert("stream".into(), Value::Bool(false));
if !tools.is_empty() {
body.insert(
"tools".into(),
serde_json::to_value(&tools).context("serializing tools")?,
);
}
for (k, v) in self.build_options() {
body.insert(k.into(), v);
}
log::info!(
"OpenRouter chat_with_tools: model={} messages={} tools={}",
self.primary_model,
messages.len(),
tools.len()
);
let mut request = self
.authed(self.client.post(&url))
.json(&Value::Object(body));
if let Some(s) = span.as_mut() {
request = crate::otel::with_traceparent(request, s.span_context());
}
let resp = match request.send().await {
Ok(r) => r,
Err(e) => {
if let Some(s) = span.as_mut() {
s.set_status(Status::error("OpenRouter request failed"));
}
return Err(e).with_context(|| format!("POST {} failed", url));
}
};
if !resp.status().is_success() {
let status = resp.status();
crate::otel::record_http_failure(
span.as_deref_mut(),
status.as_u16(),
&format!("OpenRouter chat request failed: {status}"),
);
let body = resp.text().await.unwrap_or_default();
bail!("OpenRouter chat request failed: {} — {}", status, body);
}
crate::otel::set_http_status(span.as_deref_mut(), resp.status().as_u16());
let parsed: Value = match resp.json().await {
Ok(v) => v,
Err(e) => {
if let Some(s) = span.as_mut() {
s.set_status(Status::error("parsing chat response failed"));
}
return Err(e).context("parsing chat response");
}
};
let choice = parsed
.get("choices")
.and_then(|v| v.as_array())
.and_then(|a| a.first())
.ok_or_else(|| {
anyhow!(
"response missing choices[0]: {}",
extract_openrouter_error_detail(&parsed)
)
})?;
let msg = choice.get("message").ok_or_else(|| {
anyhow!(
"choices[0] missing message: {}",
extract_openrouter_error_detail(&parsed)
)
})?;
let chat_msg = Self::openai_message_to_chat(msg)?;
let usage = parsed.get("usage");
let prompt_tokens = usage
.and_then(|u| u.get("prompt_tokens"))
.and_then(|v| v.as_i64())
.map(|n| n as i32);
let completion_tokens = usage
.and_then(|u| u.get("completion_tokens"))
.and_then(|v| v.as_i64())
.map(|n| n as i32);
// OpenRouter reports per-request USD cost in `usage.cost`.
if let Some(s) = span.as_mut() {
if let Some(pt) = prompt_tokens {
s.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
}
if let Some(ct) = completion_tokens {
s.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
}
if let Some(cost) = usage.and_then(|u| u.get("cost")).and_then(|c| c.as_f64()) {
s.set_attribute(KeyValue::new("llm.cost_usd", cost));
}
s.set_status(Status::Ok);
}
Ok((chat_msg, prompt_tokens, completion_tokens))
Review

We may want to return cost too, so a client could also report cost if they want.

We may want to return cost too, so a client could also report cost if they want.
}
}
/// Extract a diagnostic fragment from an OpenRouter response body that /// Extract a diagnostic fragment from an OpenRouter response body that
/// doesn't match the expected `{choices: [...]}` shape. OpenRouter will /// doesn't match the expected `{choices: [...]}` shape. OpenRouter will
/// sometimes return 200 OK with `{"error": {"message": "...", "code": ...}}` /// sometimes return 200 OK with `{"error": {"message": "...", "code": ...}}`
+13 -3
View File
@@ -26,7 +26,7 @@ use actix_web::{HttpRequest, HttpResponse, Responder, delete, get, post, web};
use anyhow::{Context, Result, anyhow, bail}; use anyhow::{Context, Result, anyhow, bail};
use chrono::Utc; use chrono::Utc;
use opentelemetry::KeyValue; use opentelemetry::KeyValue;
use opentelemetry::trace::{Span, Status, Tracer}; use opentelemetry::trace::{Span, Status, TraceContextExt, Tracer};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::collections::HashMap; use std::collections::HashMap;
use std::sync::{Arc, Mutex as StdMutex}; use std::sync::{Arc, Mutex as StdMutex};
@@ -685,6 +685,7 @@ impl PersonaChatSession {
entry_clone.clone(), entry_clone.clone(),
user_id, user_id,
library_id, library_id,
&mut span,
) )
.await; .await;
match result { match result {
@@ -721,6 +722,7 @@ impl PersonaChatSession {
entry: Arc<TurnEntry>, entry: Arc<TurnEntry>,
user_id: i32, user_id: i32,
_library_id: i32, _library_id: i32,
turn_span: &mut opentelemetry::global::BoxedSpan,
) -> Result<()> { ) -> Result<()> {
// Per-conversation mutex — serializes this turn against a // Per-conversation mutex — serializes this turn against a
// concurrent rewind or branch switch on the same transcript. // concurrent rewind or branch switch on the same transcript.
@@ -808,6 +810,12 @@ impl PersonaChatSession {
.resolve_backend(backend_kind, &overrides) .resolve_backend(backend_kind, &overrides)
.await?; .await?;
let model_used = backend.model().to_string(); let model_used = backend.model().to_string();
// llm.* attributes on the turn-level span (issue #130), plus the
// parent context for the child llm.request spans.
turn_span.set_attribute(KeyValue::new("llm.backend", backend.kind.as_str()));
turn_span.set_attribute(KeyValue::new("llm.model", model_used.clone()));
let turn_cx = opentelemetry::Context::new()
.with_remote_span_context(turn_span.span_context().clone());
// Build a no-photo tool catalog. `current_gate_opts_for_persona` // Build a no-photo tool catalog. `current_gate_opts_for_persona`
// already probes each tool's backing table for presence and // already probes each tool's backing table for presence and
@@ -874,6 +882,7 @@ impl PersonaChatSession {
&persona.persona_id, &persona.persona_id,
max_iterations, max_iterations,
&entry, &entry,
&turn_cx,
) )
.await .await
.map_err(|e| anyhow!("persona chat loop failed: {e:?}"))?; .map_err(|e| anyhow!("persona chat loop failed: {e:?}"))?;
@@ -942,7 +951,7 @@ impl PersonaChatSession {
// list falls back to the persona name while the title is empty. // list falls back to the persona name while the title is empty.
if needs_title { if needs_title {
let title = self let title = self
.generate_title(&backend, &messages) .generate_title(&backend, &messages, &turn_cx)
.await .await
.unwrap_or_else(|| fallback_title(&messages)); .unwrap_or_else(|| fallback_title(&messages));
if !title.is_empty() { if !title.is_empty() {
@@ -1047,6 +1056,7 @@ impl PersonaChatSession {
&self, &self,
backend: &crate::ai::backend::ResolvedBackend, backend: &crate::ai::backend::ResolvedBackend,
messages: &[ChatMessage], messages: &[ChatMessage],
cx: &opentelemetry::Context,
) -> Option<String> { ) -> Option<String> {
let user_message = messages.iter().find(|m| m.role == "user")?; let user_message = messages.iter().find(|m| m.role == "user")?;
// The last assistant message with actual prose — earlier ones may be // The last assistant message with actual prose — earlier ones may be
@@ -1057,7 +1067,7 @@ impl PersonaChatSession {
.find(|m| m.role == "assistant" && !m.content.trim().is_empty())?; .find(|m| m.role == "assistant" && !m.content.trim().is_empty())?;
let prompt = title_prompt(&user_message.content, &reply.content); let prompt = title_prompt(&user_message.content, &reply.content);
match backend.chat().generate(&prompt, None, None).await { match backend.chat().generate(cx, &prompt, None, None).await {
Ok(raw) => { Ok(raw) => {
let title = sanitize_title(&crate::ai::llm_client::strip_think_blocks(&raw)); let title = sanitize_title(&crate::ai::llm_client::strip_think_blocks(&raw));
if title.is_empty() { None } else { Some(title) } if title.is_empty() { None } else { Some(title) }
+2
View File
@@ -324,6 +324,7 @@ impl SmsApiClient {
&self, &self,
messages: &[SmsMessage], messages: &[SmsMessage],
ollama: &OllamaClient, ollama: &OllamaClient,
cx: &opentelemetry::Context,
) -> Result<String> { ) -> Result<String> {
if messages.is_empty() { if messages.is_empty() {
return Ok(String::from("No messages on this day")); return Ok(String::from("No messages on this day"));
@@ -356,6 +357,7 @@ Summary:"#,
ollama ollama
.generate( .generate(
cx,
&prompt, &prompt,
// Some("You are a summarizer for the purposes of jogging my memory and highlighting events and situations."), // Some("You are a summarizer for the purposes of jogging my memory and highlighting events and situations."),
Some("You are the keeper of memories, ingest the context and give me a casual summary of the moment."), Some("You are the keeper of memories, ingest the context and give me a casual summary of the moment."),
+6 -2
View File
@@ -220,7 +220,9 @@ async fn main() -> Result<()> {
println!("Generating summary..."); println!("Generating summary...");
let summary = ollama.generate(&prompt, Some(system_prompt)).await?; let summary = ollama
.generate(&opentelemetry::Context::new(), &prompt, Some(system_prompt))
.await?;
println!("\n📝 GENERATED SUMMARY:"); println!("\n📝 GENERATED SUMMARY:");
println!("─────────────────────────────────────────"); println!("─────────────────────────────────────────");
@@ -236,7 +238,9 @@ async fn main() -> Result<()> {
); );
println!("\nGenerating embedding..."); println!("\nGenerating embedding...");
let embedding = ollama.generate_embedding(&stripped).await?; let embedding = ollama
.generate_embedding(&opentelemetry::Context::new(), &stripped)
.await?;
println!("✓ Embedding generated ({} dimensions)", embedding.len()); println!("✓ Embedding generated ({} dimensions)", embedding.len());
println!("Saving to database..."); println!("Saving to database...");
+18
View File
@@ -579,6 +579,19 @@ pub struct UserAiPrefs {
/// default (`camera_tz::DEFAULT_DISPLAY_TZ`). The legacy /// default (`camera_tz::DEFAULT_DISPLAY_TZ`). The legacy
/// `tz_offset_minutes` column stays frozen for rollback. /// `tz_offset_minutes` column stays frozen for rollback.
pub display_tz: Option<String>, pub display_tz: Option<String>,
/// Last/explicit insight + chat model choice (local id or OpenRouter id).
/// NULL = current per-request / env default behavior.
pub default_model: Option<String>,
/// `"local"` | `"hybrid"` — insight/chat backend. NULL = per-request /
/// hard default ("local").
pub default_backend: Option<String>,
/// Active persona for persona chat. NULL = hard default ("default").
pub default_persona_id: Option<String>,
/// Reel span: `"day"` | `"week"` | `"month"`. NULL = per-request /
/// hard default (day).
pub reel_span: Option<String>,
/// Reel segment cap. NULL = per-request / hard default.
pub reel_max_segments: Option<i32>,
} }
#[derive(Insertable, Debug, Clone, serde::Deserialize, serde::Serialize)] #[derive(Insertable, Debug, Clone, serde::Deserialize, serde::Serialize)]
@@ -589,4 +602,9 @@ pub struct UpsertUserAiPrefs {
pub library: Option<String>, pub library: Option<String>,
pub updated_at: i64, pub updated_at: i64,
pub display_tz: Option<String>, pub display_tz: Option<String>,
pub default_model: Option<String>,
pub default_backend: Option<String>,
pub default_persona_id: Option<String>,
pub reel_span: Option<String>,
pub reel_max_segments: Option<i32>,
} }
+5
View File
@@ -293,6 +293,11 @@ diesel::table! {
library -> Nullable<Text>, library -> Nullable<Text>,
updated_at -> BigInt, updated_at -> BigInt,
display_tz -> Nullable<Text>, display_tz -> Nullable<Text>,
default_model -> Nullable<Text>,
default_backend -> Nullable<Text>,
default_persona_id -> Nullable<Text>,
reel_span -> Nullable<Text>,
reel_max_segments -> Nullable<Integer>,
} }
} }
+39
View File
@@ -100,6 +100,11 @@ impl UserAiPrefsDao for SqliteUserAiPrefsDao {
dsl::tz_offset_minutes.eq(&prefs.tz_offset_minutes), dsl::tz_offset_minutes.eq(&prefs.tz_offset_minutes),
dsl::library.eq(&prefs.library), dsl::library.eq(&prefs.library),
dsl::display_tz.eq(&prefs.display_tz), dsl::display_tz.eq(&prefs.display_tz),
dsl::default_model.eq(&prefs.default_model),
dsl::default_backend.eq(&prefs.default_backend),
dsl::default_persona_id.eq(&prefs.default_persona_id),
dsl::reel_span.eq(&prefs.reel_span),
dsl::reel_max_segments.eq(&prefs.reel_max_segments),
dsl::updated_at.eq(&prefs.updated_at), dsl::updated_at.eq(&prefs.updated_at),
)) ))
.execute(connection.deref_mut()) .execute(connection.deref_mut())
@@ -147,6 +152,11 @@ mod tests {
library: Some("1".to_string()), library: Some("1".to_string()),
updated_at: now, updated_at: now,
display_tz: Some("America/New_York".to_string()), display_tz: Some("America/New_York".to_string()),
default_model: Some("llama3.2:3b".to_string()),
default_backend: Some("hybrid".to_string()),
default_persona_id: Some("cabin".to_string()),
reel_span: Some("week".to_string()),
reel_max_segments: Some(24),
}; };
dao.upsert_prefs(&ctx(), &prefs).unwrap(); dao.upsert_prefs(&ctx(), &prefs).unwrap();
@@ -156,6 +166,11 @@ mod tests {
assert_eq!(row.tz_offset_minutes, Some(-480)); assert_eq!(row.tz_offset_minutes, Some(-480));
assert_eq!(row.library, Some("1".to_string())); assert_eq!(row.library, Some("1".to_string()));
assert_eq!(row.display_tz, Some("America/New_York".to_string())); assert_eq!(row.display_tz, Some("America/New_York".to_string()));
assert_eq!(row.default_model, Some("llama3.2:3b".to_string()));
assert_eq!(row.default_backend, Some("hybrid".to_string()));
assert_eq!(row.default_persona_id, Some("cabin".to_string()));
assert_eq!(row.reel_span, Some("week".to_string()));
assert_eq!(row.reel_max_segments, Some(24));
assert_eq!(row.updated_at, now); assert_eq!(row.updated_at, now);
} }
@@ -171,6 +186,11 @@ mod tests {
library: Some("1".to_string()), library: Some("1".to_string()),
updated_at: now1, updated_at: now1,
display_tz: Some("America/New_York".to_string()), display_tz: Some("America/New_York".to_string()),
default_model: Some("llama3.2:3b".to_string()),
default_backend: Some("local".to_string()),
default_persona_id: None,
reel_span: Some("week".to_string()),
reel_max_segments: Some(24),
}; };
dao.upsert_prefs(&ctx(), &prefs1).unwrap(); dao.upsert_prefs(&ctx(), &prefs1).unwrap();
@@ -180,6 +200,11 @@ mod tests {
library: None, library: None,
updated_at: now2, updated_at: now2,
display_tz: None, display_tz: None,
default_model: None,
default_backend: None,
default_persona_id: None,
reel_span: None,
reel_max_segments: None,
}; };
dao.upsert_prefs(&ctx(), &prefs2).unwrap(); dao.upsert_prefs(&ctx(), &prefs2).unwrap();
@@ -188,6 +213,11 @@ mod tests {
assert_eq!(row.tz_offset_minutes, Some(-300)); assert_eq!(row.tz_offset_minutes, Some(-300));
assert!(row.library.is_none()); assert!(row.library.is_none());
assert!(row.display_tz.is_none()); assert!(row.display_tz.is_none());
// Owner-default columns are overwritten to NULL like the rest.
assert!(row.default_model.is_none());
assert!(row.default_backend.is_none());
assert!(row.reel_span.is_none());
assert!(row.reel_max_segments.is_none());
assert_eq!(row.updated_at, now2); assert_eq!(row.updated_at, now2);
} }
@@ -202,6 +232,11 @@ mod tests {
library: None, library: None,
updated_at: now, updated_at: now,
display_tz: Some("Europe/Berlin".to_string()), display_tz: Some("Europe/Berlin".to_string()),
default_model: None,
default_backend: None,
default_persona_id: Some("default".to_string()),
reel_span: Some("month".to_string()),
reel_max_segments: None,
}; };
dao.upsert_prefs(&ctx(), &prefs).unwrap(); dao.upsert_prefs(&ctx(), &prefs).unwrap();
@@ -210,5 +245,9 @@ mod tests {
assert!(row.voice.is_none()); assert!(row.voice.is_none());
assert!(row.library.is_none()); assert!(row.library.is_none());
assert_eq!(row.display_tz, Some("Europe/Berlin".to_string())); assert_eq!(row.display_tz, Some("Europe/Berlin".to_string()));
assert!(row.default_model.is_none());
assert_eq!(row.default_persona_id, Some("default".to_string()));
assert_eq!(row.reel_span, Some("month".to_string()));
assert!(row.reel_max_segments.is_none());
} }
} }
+2 -1
View File
@@ -859,7 +859,8 @@ async fn synthesize_merge<D: KnowledgeDao + 'static>(
let ollama = app_state.ollama.clone(); let ollama = app_state.ollama.clone();
let model_used = ollama.primary_model.clone(); let model_used = ollama.primary_model.clone();
let proposed = match ollama.generate(&prompt, Some(system)).await { let cx = opentelemetry::Context::current();
let proposed = match ollama.generate(&cx, &prompt, Some(system)).await {
Ok(out) => { Ok(out) => {
// Strip the framing models reach for even with explicit // Strip the framing models reach for even with explicit
// "no preamble" guidance: leading "Merged description:" // "no preamble" guidance: leading "Merged description:"
+25
View File
@@ -108,6 +108,31 @@ pub enum MemoriesSpan {
Month, Month,
} }
/// Wire/storage string for the span (`"day"` | `"week"` | `"month"`) — the
/// same lowercase names as the serde rename, so a stored pref round-trips
/// through `MemoriesRequest.span` unchanged.
impl std::fmt::Display for MemoriesSpan {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(match self {
MemoriesSpan::Day => "day",
MemoriesSpan::Week => "week",
MemoriesSpan::Month => "month",
})
}
}
impl std::str::FromStr for MemoriesSpan {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s.trim().to_ascii_lowercase().as_str() {
"day" => Ok(MemoriesSpan::Day),
"week" => Ok(MemoriesSpan::Week),
"month" => Ok(MemoriesSpan::Month),
other => Err(format!("invalid span {other:?} (expected day|week|month)")),
}
}
}
#[derive(Deserialize)] #[derive(Deserialize)]
pub struct MemoriesRequest { pub struct MemoriesRequest {
pub span: Option<MemoriesSpan>, pub span: Option<MemoriesSpan>,
+126 -1
View File
@@ -2,7 +2,9 @@ use actix_web::HttpRequest;
use actix_web::http::header::HeaderMap; use actix_web::http::header::HeaderMap;
use opentelemetry::global::{BoxedSpan, BoxedTracer}; use opentelemetry::global::{BoxedSpan, BoxedTracer};
use opentelemetry::propagation::TextMapPropagator; use opentelemetry::propagation::TextMapPropagator;
use opentelemetry::trace::{Span, Status, Tracer}; use opentelemetry::trace::{
Span, SpanBuilder, SpanContext, SpanKind, Status, TraceContextExt, Tracer,
};
use opentelemetry::{Context, KeyValue, global}; use opentelemetry::{Context, KeyValue, global};
use opentelemetry_appender_log::OpenTelemetryLogBridge; use opentelemetry_appender_log::OpenTelemetryLogBridge;
use opentelemetry_otlp::WithExportConfig; use opentelemetry_otlp::WithExportConfig;
@@ -77,11 +79,78 @@ impl<'a> opentelemetry::propagation::Extractor for HeaderExtractor<'a> {
self.0.keys().map(|k| k.as_str()).collect() self.0.keys().map(|k| k.as_str()).collect()
} }
} }
/// Start a client span for a single LLM call. Every provider funnels its
/// LlmClient methods through this so the span shape (name, `SpanKind::Client`,
/// provider / operation / model attributes) is identical across
/// openrouter / llamacpp / ollama. `operation` is one of `chat`,
/// `chat_stream`, `embeddings`, `describe_image`, `generate`.
pub fn start_llm_request_span(
cx: &Context,
provider: &str,
operation: &str,
model: &str,
) -> BoxedSpan {
let builder = SpanBuilder::from_name("llm.request").with_kind(SpanKind::Client);
let mut span = global_tracer().build_with_context(builder, cx);
span.set_attribute(KeyValue::new("llm.provider", provider.to_string()));
span.set_attribute(KeyValue::new("llm.operation", operation.to_string()));
span.set_attribute(KeyValue::new("llm.model", model.to_string()));
span
}
pub fn extract_context_from_request(req: &HttpRequest) -> Context { pub fn extract_context_from_request(req: &HttpRequest) -> Context {
let propagator = TraceContextPropagator::new(); let propagator = TraceContextPropagator::new();
propagator.extract(&HeaderExtractor(req.headers())) propagator.extract(&HeaderExtractor(req.headers()))
} }
/// Build the W3C trace-context headers to attach to an outgoing HTTP
/// request so a downstream service (llama-swap, Ollama) can parent its own
/// spans onto this trace. Returns an empty vec when the context carries no
/// valid span — e.g. debug builds, where the global tracer is a no-op and
/// injecting an all-zero traceparent would produce garbage parent spans.
pub fn traceparent_headers(context: &Context) -> Vec<(String, String)> {
if !context.span().span_context().is_valid() {
return Vec::new();
}
let mut headers: std::collections::HashMap<String, String> = std::collections::HashMap::new();
TraceContextPropagator::new().inject_context(context, &mut headers);
// The propagator writes an empty tracestate header when there is no
// state — drop empty-valued headers rather than sending noise.
headers.into_iter().filter(|(_, v)| !v.is_empty()).collect()
}
/// Attach the W3C traceparent header to an outgoing request so a downstream
/// service parents its own spans onto this trace. No-op when the span context
/// is not valid — e.g. debug builds, where the global tracer is a no-op and an
/// all-zero traceparent would produce garbage parent spans.
pub fn with_traceparent(
builder: reqwest::RequestBuilder,
span_context: &SpanContext,
) -> reqwest::RequestBuilder {
let mut b = builder;
for (k, v) in
traceparent_headers(&Context::new().with_remote_span_context(span_context.clone()))
{
b = b.header(k, v);
}
b
}
/// Record the HTTP status code on the span (success or failure path).
pub fn set_http_status(span: Option<&mut BoxedSpan>, status: u16) {
if let Some(s) = span {
s.set_attribute(KeyValue::new("http.status_code", status as i64));
}
}
/// Record an HTTP failure on the span: status code plus an error status.
pub fn record_http_failure(span: Option<&mut BoxedSpan>, status: u16, message: &str) {
if let Some(s) = span {
s.set_attribute(KeyValue::new("http.status_code", status as i64));
s.set_status(Status::error(message.to_string()));
}
}
pub fn trace_db_call<F, O>( pub fn trace_db_call<F, O>(
context: &Context, context: &Context,
query_type: &str, query_type: &str,
@@ -110,3 +179,59 @@ where
result result
} }
#[cfg(test)]
mod traceparent_tests {
use super::*;
use opentelemetry::trace::{
SpanContext, SpanId, TraceContextExt, TraceFlags, TraceId, TraceState,
};
/// A context carrying a span context with known ids — no global
/// provider needed, so tests can't pollute process state.
fn context_with_flags(flags: TraceFlags) -> Context {
let trace_id = TraceId::from_bytes([
0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e,
0x0f, 0x10,
]);
let span_id = SpanId::from_bytes([0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08]);
let span_context = SpanContext::new(trace_id, span_id, flags, false, TraceState::default());
Context::new().with_remote_span_context(span_context)
}
#[test]
fn traceparent_headers_sampled_context() {
let cx = context_with_flags(TraceFlags::SAMPLED);
let headers = traceparent_headers(&cx);
assert_eq!(
headers,
vec![(
"traceparent".to_string(),
"00-0102030405060708090a0b0c0d0e0f10-0102030405060708-01".to_string(),
)]
);
}
#[test]
fn traceparent_headers_not_sampled_context() {
// A valid-but-unsampled context still propagates; the flags byte
// carries the state.
let cx = context_with_flags(TraceFlags::NOT_SAMPLED);
let headers = traceparent_headers(&cx);
assert_eq!(
headers,
vec![(
"traceparent".to_string(),
"00-0102030405060708090a0b0c0d0e0f10-0102030405060708-00".to_string(),
)]
);
}
#[test]
fn traceparent_headers_invalid_context_is_empty() {
// No span (debug-build no-op tracer) → no header, never an
// all-zero traceparent.
let cx = Context::new();
assert!(traceparent_headers(&cx).is_empty());
}
}
+412 -70
View File
@@ -8,15 +8,19 @@
//! `GET /timezones` — the IANA zone list from chrono-tz, for picker UIs //! `GET /timezones` — the IANA zone list from chrono-tz, for picker UIs
//! (unauthenticated; static data). //! (unauthenticated; static data).
//! //!
//! Scope: existing columns only (voice, library, display_tz) — no migration. //! Surface: all persisted prefs — the passive mirror (voice, library,
//! The legacy `tz_offset_minutes` column is preserved verbatim on every //! display_tz) and the owner defaults (default_model, default_backend,
//! write. `POST /reels`'s `capture_prefs` stays as the incidental writer; //! default_persona_id, reel_span, reel_max_segments). Consumers resolve
//! both writers hit the same single row, so they're idempotent. Owner //! `request param → stored pref → hard default`; this endpoint is the only
//! defaults (model/backend/persona/reels) are a separate follow-on ticket //! place the owner-default fields are written. `POST /reels`'s
//! that adds the columns. //! `capture_prefs` stays as the incidental writer for the mirror fields
//! (plus the effective reel span/segments); both writers hit the same
//! single row, so they're idempotent. The legacy `tz_offset_minutes`
//! column is preserved verbatim on every write.
use actix_web::{HttpResponse, web}; use actix_web::{HttpResponse, web};
use chrono_tz::Tz; use chrono_tz::Tz;
use serde::de::IntoDeserializer;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use serde_json::json; use serde_json::json;
use std::str::FromStr; use std::str::FromStr;
@@ -24,6 +28,7 @@ use std::time::{SystemTime, UNIX_EPOCH};
use crate::data::Claims; use crate::data::Claims;
use crate::database::models::{UpsertUserAiPrefs, UserAiPrefs}; use crate::database::models::{UpsertUserAiPrefs, UserAiPrefs};
use crate::memories::MemoriesSpan;
use crate::state::AppState; use crate::state::AppState;
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -35,6 +40,18 @@ pub struct PrefsResponse {
pub voice: Option<String>, pub voice: Option<String>,
pub library: Option<String>, pub library: Option<String>,
pub display_tz: Option<String>, pub display_tz: Option<String>,
/// Last/explicit insight + chat model choice. NULL = per-request /
/// env default behavior.
pub default_model: Option<String>,
/// `"local"` | `"hybrid"`. NULL = per-request / hard default.
pub default_backend: Option<String>,
/// Active persona for persona chat. NULL = hard default ("default").
pub default_persona_id: Option<String>,
/// Reel span: `"day"` | `"week"` | `"month"`. NULL = per-request /
/// hard default (day).
pub reel_span: Option<String>,
/// Reel segment cap. NULL = per-request / hard default.
pub reel_max_segments: Option<i32>,
pub updated_at: Option<i64>, pub updated_at: Option<i64>,
} }
@@ -43,41 +60,59 @@ pub struct PrefsResponse {
/// explicit null and a missing key apart on `Option<Option<T>>` (both come /// explicit null and a missing key apart on `Option<Option<T>>` (both come
/// out as outer `None`), so this carries a hand-rolled visitor. /// out as outer `None`), so this carries a hand-rolled visitor.
#[derive(Clone, Debug, Default)] #[derive(Clone, Debug, Default)]
pub enum TriState { pub enum TriState<T> {
#[default] #[default]
Absent, Absent,
Null, Null,
Val(String), Val(T),
} }
impl<'de> Deserialize<'de> for TriState { impl<'de, T: Deserialize<'de>> Deserialize<'de> for TriState<T> {
fn deserialize<D>(deserializer: D) -> Result<TriState, D::Error> fn deserialize<D>(deserializer: D) -> Result<TriState<T>, D::Error>
where where
D: serde::Deserializer<'de>, D: serde::Deserializer<'de>,
{ {
struct Visitor; struct Visitor<T>(std::marker::PhantomData<T>);
impl<'de> serde::de::Visitor<'de> for Visitor { impl<'de, T: Deserialize<'de>> serde::de::Visitor<'de> for Visitor<T> {
type Value = TriState; type Value = TriState<T>;
fn expecting(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result { fn expecting(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
f.write_str("a string or null") f.write_str("a value or null")
} }
// serde_json routes an explicit JSON null to visit_unit/visit_none; // serde_json routes an explicit JSON null to visit_unit/visit_none;
// a missing key never calls deserialize — `#[serde(default)]` // a missing key never calls deserialize — `#[serde(default)]`
// supplies Absent instead. // supplies Absent instead. Concrete values are re-deserialized into
fn visit_unit<E: serde::de::Error>(self) -> Result<TriState, E> { // T through single-value deserializers (serde_json's
// deserialize_any dispatches directly to visit_str/visit_i64/…
// — visit_some only fires on the deserialize_option path).
fn visit_unit<E: serde::de::Error>(self) -> Result<TriState<T>, E> {
Ok(TriState::Null) Ok(TriState::Null)
} }
fn visit_none<E: serde::de::Error>(self) -> Result<TriState, E> { fn visit_none<E: serde::de::Error>(self) -> Result<TriState<T>, E> {
Ok(TriState::Null) Ok(TriState::Null)
} }
fn visit_str<E: serde::de::Error>(self, v: &str) -> Result<TriState, E> { fn visit_some<D2>(self, d: D2) -> Result<TriState<T>, D2::Error>
Ok(TriState::Val(v.to_string())) where
D2: serde::Deserializer<'de>,
{
Ok(TriState::Val(T::deserialize(d)?))
} }
fn visit_string<E: serde::de::Error>(self, v: String) -> Result<TriState, E> { // `into_deserializer()` gives a single-value deserializer whose
Ok(TriState::Val(v)) // error type is the visitor's E, so T::deserialize lands on
// Result<T, E> with no conversion.
fn visit_str<E: serde::de::Error>(self, v: &str) -> Result<TriState<T>, E> {
T::deserialize(v.into_deserializer()).map(TriState::Val)
}
fn visit_string<E: serde::de::Error>(self, v: String) -> Result<TriState<T>, E> {
T::deserialize(v.as_str().into_deserializer()).map(TriState::Val)
}
fn visit_i64<E: serde::de::Error>(self, v: i64) -> Result<TriState<T>, E> {
T::deserialize(v.into_deserializer()).map(TriState::Val)
}
fn visit_u64<E: serde::de::Error>(self, v: u64) -> Result<TriState<T>, E> {
T::deserialize(v.into_deserializer()).map(TriState::Val)
} }
} }
deserializer.deserialize_any(Visitor) deserializer.deserialize_any(Visitor(std::marker::PhantomData))
} }
} }
@@ -86,11 +121,21 @@ impl<'de> Deserialize<'de> for TriState {
#[derive(Deserialize, Debug, Default)] #[derive(Deserialize, Debug, Default)]
pub struct UpdatePrefsRequest { pub struct UpdatePrefsRequest {
#[serde(default)] #[serde(default)]
pub voice: TriState, pub voice: TriState<String>,
#[serde(default)] #[serde(default)]
pub library: TriState, pub library: TriState<String>,
#[serde(default)] #[serde(default)]
pub display_tz: TriState, pub display_tz: TriState<String>,
#[serde(default)]
pub default_model: TriState<String>,
#[serde(default)]
pub default_backend: TriState<String>,
#[serde(default)]
pub default_persona_id: TriState<String>,
#[serde(default)]
pub reel_span: TriState<String>,
#[serde(default)]
pub reel_max_segments: TriState<i32>,
} }
#[derive(Serialize, Debug)] #[derive(Serialize, Debug)]
@@ -109,6 +154,18 @@ fn is_valid_display_tz(name: &str) -> bool {
Tz::from_str(name.trim()).is_ok() Tz::from_str(name.trim()).is_ok()
} }
/// A stored `default_backend` must be consumable downstream, i.e. accepted
/// by the canonical `BackendKind::parse` ("local"/"hybrid", case-insensitive).
/// Delegating to that parser keeps the write-time validator and the read-time
/// consumer from drifting apart; anything else stored here would 500 on every
/// insight/chat request instead of failing once at write time. The blank
/// string the parser tolerates (empty request field = app default) is
/// rejected here — a stored default is an explicit choice, and NULL already
/// means "no default".
fn is_valid_backend(name: &str) -> bool {
!name.trim().is_empty() && crate::ai::backend::BackendKind::parse(name).is_ok()
}
/// The full set of zone names `PUT /prefs` will accept — `chrono_tz::TZ_VARIANTS` /// The full set of zone names `PUT /prefs` will accept — `chrono_tz::TZ_VARIANTS`
/// is exactly the domain of `Tz::from_str`, so the picker list and the /// is exactly the domain of `Tz::from_str`, so the picker list and the
/// validation check can never drift apart. /// validation check can never drift apart.
@@ -119,14 +176,13 @@ pub fn all_timezones() -> Vec<String> {
.collect() .collect()
} }
/// Merge a partial update onto the existing row. Returns /// Merge a partial update onto the existing row. An omitted field keeps
/// `(voice, library, display_tz, changed)`. An omitted field keeps the /// the stored value (or `None` when there is no row); a present field wins.
/// stored value (or `None` when there is no row); a present field wins. fn tri_value<T: Clone>(
fn tri_value( field: &TriState<T>,
field: &TriState,
existing: Option<&UserAiPrefs>, existing: Option<&UserAiPrefs>,
keep: impl FnOnce(&UserAiPrefs) -> Option<String>, keep: impl FnOnce(&UserAiPrefs) -> Option<T>,
) -> Option<String> { ) -> Option<T> {
match field { match field {
TriState::Absent => existing.and_then(keep), TriState::Absent => existing.and_then(keep),
TriState::Null => None, TriState::Null => None,
@@ -134,17 +190,43 @@ fn tri_value(
} }
} }
fn merge_patch( /// The eight partial-update fields after a merge (everything except the
existing: Option<&UserAiPrefs>, /// frozen `tz_offset_minutes`, which the handler copies through verbatim).
patch: &UpdatePrefsRequest, struct MergedPrefs {
) -> (Option<String>, Option<String>, Option<String>, bool) { voice: Option<String>,
let voice = tri_value(&patch.voice, existing, |e| e.voice.clone()); library: Option<String>,
let library = tri_value(&patch.library, existing, |e| e.library.clone()); display_tz: Option<String>,
let display_tz = tri_value(&patch.display_tz, existing, |e| e.display_tz.clone()); default_model: Option<String>,
default_backend: Option<String>,
default_persona_id: Option<String>,
reel_span: Option<String>,
reel_max_segments: Option<i32>,
}
fn merge_patch(existing: Option<&UserAiPrefs>, patch: &UpdatePrefsRequest) -> (MergedPrefs, bool) {
let merged = MergedPrefs {
voice: tri_value(&patch.voice, existing, |e| e.voice.clone()),
library: tri_value(&patch.library, existing, |e| e.library.clone()),
display_tz: tri_value(&patch.display_tz, existing, |e| e.display_tz.clone()),
default_model: tri_value(&patch.default_model, existing, |e| e.default_model.clone()),
default_backend: tri_value(&patch.default_backend, existing, |e| {
e.default_backend.clone()
}),
default_persona_id: tri_value(&patch.default_persona_id, existing, |e| {
e.default_persona_id.clone()
}),
reel_span: tri_value(&patch.reel_span, existing, |e| e.reel_span.clone()),
reel_max_segments: tri_value(&patch.reel_max_segments, existing, |e| e.reel_max_segments),
};
let changed = !matches!(patch.voice, TriState::Absent) let changed = !matches!(patch.voice, TriState::Absent)
|| !matches!(patch.library, TriState::Absent) || !matches!(patch.library, TriState::Absent)
|| !matches!(patch.display_tz, TriState::Absent); || !matches!(patch.display_tz, TriState::Absent)
(voice, library, display_tz, changed) || !matches!(patch.default_model, TriState::Absent)
|| !matches!(patch.default_backend, TriState::Absent)
|| !matches!(patch.default_persona_id, TriState::Absent)
|| !matches!(patch.reel_span, TriState::Absent)
|| !matches!(patch.reel_max_segments, TriState::Absent);
(merged, changed)
} }
fn row_to_response(row: Option<&UserAiPrefs>) -> PrefsResponse { fn row_to_response(row: Option<&UserAiPrefs>) -> PrefsResponse {
@@ -153,12 +235,22 @@ fn row_to_response(row: Option<&UserAiPrefs>) -> PrefsResponse {
voice: row.voice.clone(), voice: row.voice.clone(),
library: row.library.clone(), library: row.library.clone(),
display_tz: row.display_tz.clone(), display_tz: row.display_tz.clone(),
default_model: row.default_model.clone(),
default_backend: row.default_backend.clone(),
default_persona_id: row.default_persona_id.clone(),
reel_span: row.reel_span.clone(),
reel_max_segments: row.reel_max_segments,
updated_at: Some(row.updated_at), updated_at: Some(row.updated_at),
}, },
None => PrefsResponse { None => PrefsResponse {
voice: None, voice: None,
library: None, library: None,
display_tz: None, display_tz: None,
default_model: None,
default_backend: None,
default_persona_id: None,
reel_span: None,
reel_max_segments: None,
updated_at: None, updated_at: None,
}, },
} }
@@ -192,7 +284,8 @@ pub async fn put_prefs(
) -> HttpResponse { ) -> HttpResponse {
let patch = body.0; let patch = body.0;
// Validate before any write: an invalid zone name → 400, nothing written. // Validate before any write: an invalid value → 400, nothing written
// (not even the other fields in the same body).
if let TriState::Val(name) = &patch.display_tz if let TriState::Val(name) = &patch.display_tz
&& !is_valid_display_tz(name) && !is_valid_display_tz(name)
{ {
@@ -200,6 +293,20 @@ pub async fn put_prefs(
"error": format!("invalid display_tz: {name}") "error": format!("invalid display_tz: {name}")
})); }));
} }
if let TriState::Val(name) = &patch.default_backend
&& !is_valid_backend(name)
{
return HttpResponse::BadRequest().json(json!({
"error": format!("invalid default_backend: {name} (expected local|hybrid)")
}));
}
if let TriState::Val(name) = &patch.reel_span
&& MemoriesSpan::from_str(name).is_err()
{
return HttpResponse::BadRequest().json(json!({
"error": format!("invalid reel_span: {name} (expected day|week|month)")
}));
}
let mut dao = match app_state.user_ai_prefs_dao.lock() { let mut dao = match app_state.user_ai_prefs_dao.lock() {
Ok(dao) => dao, Ok(dao) => dao,
@@ -219,7 +326,7 @@ pub async fn put_prefs(
} }
}; };
let (voice, library, display_tz, changed) = merge_patch(existing.as_ref(), &patch); let (merged, changed) = merge_patch(existing.as_ref(), &patch);
if !changed { if !changed {
// Empty body: no-op, return the current row without bumping updated_at. // Empty body: no-op, return the current row without bumping updated_at.
return HttpResponse::Ok().json(row_to_response(existing.as_ref())); return HttpResponse::Ok().json(row_to_response(existing.as_ref()));
@@ -231,18 +338,28 @@ pub async fn put_prefs(
.as_secs() as i64; .as_secs() as i64;
let resp = PrefsResponse { let resp = PrefsResponse {
voice: voice.clone(), voice: merged.voice.clone(),
library: library.clone(), library: merged.library.clone(),
display_tz: display_tz.clone(), display_tz: merged.display_tz.clone(),
default_model: merged.default_model.clone(),
default_backend: merged.default_backend.clone(),
default_persona_id: merged.default_persona_id.clone(),
reel_span: merged.reel_span.clone(),
reel_max_segments: merged.reel_max_segments,
updated_at: Some(now), updated_at: Some(now),
}; };
let upsert = UpsertUserAiPrefs { let upsert = UpsertUserAiPrefs {
voice, voice: merged.voice,
// Frozen legacy column: preserved verbatim, never written by this // Frozen legacy column: preserved verbatim, never written by this
// surface. // surface.
tz_offset_minutes: existing.as_ref().and_then(|e| e.tz_offset_minutes), tz_offset_minutes: existing.as_ref().and_then(|e| e.tz_offset_minutes),
library, library: merged.library,
display_tz, display_tz: merged.display_tz,
default_model: merged.default_model,
default_backend: merged.default_backend,
default_persona_id: merged.default_persona_id,
reel_span: merged.reel_span,
reel_max_segments: merged.reel_max_segments,
updated_at: now, updated_at: now,
}; };
match dao.upsert_prefs(&ctx, &upsert) { match dao.upsert_prefs(&ctx, &upsert) {
@@ -311,6 +428,27 @@ mod tests {
library: library.map(str::to_string), library: library.map(str::to_string),
updated_at: 1_700_000_000, updated_at: 1_700_000_000,
display_tz: display_tz.map(str::to_string), display_tz: display_tz.map(str::to_string),
default_model: None,
default_backend: None,
default_persona_id: None,
reel_span: None,
reel_max_segments: None,
}
}
/// A row with owner defaults populated, for the owner-default tests.
fn seeded_row_with_defaults() -> UpsertUserAiPrefs {
UpsertUserAiPrefs {
voice: Some("grandma".to_string()),
tz_offset_minutes: Some(-240),
library: Some("1".to_string()),
updated_at: 1_700_000_000,
display_tz: Some("America/New_York".to_string()),
default_model: Some("llama3.2:3b".to_string()),
default_backend: Some("hybrid".to_string()),
default_persona_id: Some("cabin".to_string()),
reel_span: Some("week".to_string()),
reel_max_segments: Some(24),
} }
} }
@@ -363,6 +501,11 @@ mod tests {
"voice": null, "voice": null,
"library": null, "library": null,
"display_tz": null, "display_tz": null,
"default_model": null,
"default_backend": null,
"default_persona_id": null,
"reel_span": null,
"reel_max_segments": null,
"updated_at": null "updated_at": null
}) })
); );
@@ -682,43 +825,242 @@ mod tests {
// --- merge_patch (pure) -------------------------------------------------- // --- merge_patch (pure) --------------------------------------------------
#[test] fn row_with_defaults() -> UserAiPrefs {
fn merge_patch_omitted_keeps_existing() { UserAiPrefs {
let row = UserAiPrefs {
id: 1, id: 1,
voice: Some("a".into()), voice: Some("a".into()),
tz_offset_minutes: None, tz_offset_minutes: None,
library: Some("b".into()), library: Some("b".into()),
updated_at: 0, updated_at: 0,
display_tz: Some("c".into()), display_tz: Some("c".into()),
}; default_model: Some("m1".into()),
let (voice, library, display_tz, changed) = default_backend: Some("local".into()),
merge_patch(Some(&row), &UpdatePrefsRequest::default()); default_persona_id: Some("p1".into()),
assert_eq!(voice.as_deref(), Some("a")); reel_span: Some("week".into()),
assert_eq!(library.as_deref(), Some("b")); reel_max_segments: Some(24),
assert_eq!(display_tz.as_deref(), Some("c")); }
}
#[test]
fn merge_patch_omitted_keeps_existing() {
let row = row_with_defaults();
let (merged, changed) = merge_patch(Some(&row), &UpdatePrefsRequest::default());
assert_eq!(merged.voice.as_deref(), Some("a"));
assert_eq!(merged.library.as_deref(), Some("b"));
assert_eq!(merged.display_tz.as_deref(), Some("c"));
assert_eq!(merged.default_model.as_deref(), Some("m1"));
assert_eq!(merged.default_backend.as_deref(), Some("local"));
assert_eq!(merged.default_persona_id.as_deref(), Some("p1"));
assert_eq!(merged.reel_span.as_deref(), Some("week"));
assert_eq!(merged.reel_max_segments, Some(24));
assert!(!changed); assert!(!changed);
} }
#[test] #[test]
fn merge_patch_present_field_wins_including_null() { fn merge_patch_present_field_wins_including_null() {
let row = UserAiPrefs { let row = row_with_defaults();
id: 1,
voice: Some("a".into()),
tz_offset_minutes: None,
library: None,
updated_at: 0,
display_tz: None,
};
let patch = UpdatePrefsRequest { let patch = UpdatePrefsRequest {
voice: TriState::Null, // clear voice: TriState::Null, // clear
library: TriState::Val(String::from("x")), // set library: TriState::Val(String::from("x")), // set
display_tz: TriState::Absent, display_tz: TriState::Absent,
default_backend: TriState::Null, // clear an owner default
reel_span: TriState::Val(String::from("month")),
reel_max_segments: TriState::Val(12),
..UpdatePrefsRequest::default()
}; };
let (voice, library, display_tz, changed) = merge_patch(Some(&row), &patch); let (merged, changed) = merge_patch(Some(&row), &patch);
assert!(voice.is_none()); assert!(merged.voice.is_none());
assert_eq!(library.as_deref(), Some("x")); assert_eq!(merged.library.as_deref(), Some("x"));
assert!(display_tz.is_none()); // Absent field keeps the stored value.
assert_eq!(merged.display_tz.as_deref(), Some("c"));
assert!(merged.default_backend.is_none());
assert_eq!(merged.reel_span.as_deref(), Some("month"));
assert_eq!(merged.reel_max_segments, Some(12));
// Untouched owner defaults survive.
assert_eq!(merged.default_model.as_deref(), Some("m1"));
assert_eq!(merged.default_persona_id.as_deref(), Some("p1"));
assert!(changed); assert!(changed);
} }
// --- PUT /prefs: owner defaults ------------------------------------------
#[actix_rt::test]
async fn put_prefs_owner_defaults_set_and_persist() {
let (state, mut seed_dao) = prefs_state();
seed(
&mut seed_dao,
seeded_row(Some("grandma"), Some("1"), Some("America/New_York")),
);
let app = init_service(
App::new()
.service(
web::resource("/prefs")
.route(web::get().to(get_prefs))
.route(web::put().to(put_prefs)),
)
.service(web::resource("/timezones").route(web::get().to(list_timezones)))
.app_data(state.clone()),
)
.await;
let (header, value) = auth();
let req = TestRequest::put()
.uri("/prefs")
.insert_header((header, value))
.set_json(serde_json::json!({
"default_model": "llama3.2:3b",
"default_backend": "hybrid",
"default_persona_id": "cabin",
"reel_span": "week",
"reel_max_segments": 24
}))
.to_request();
let resp = call_service(&app, req).await;
assert_eq!(resp.status(), 200);
let mut dao = state.user_ai_prefs_dao.lock().unwrap();
let row = dao
.get_prefs(&opentelemetry::Context::new())
.unwrap()
.unwrap();
assert_eq!(row.default_model.as_deref(), Some("llama3.2:3b"));
assert_eq!(row.default_backend.as_deref(), Some("hybrid"));
assert_eq!(row.default_persona_id.as_deref(), Some("cabin"));
assert_eq!(row.reel_span.as_deref(), Some("week"));
assert_eq!(row.reel_max_segments, Some(24));
// Mirror fields untouched by the owner-default patch.
assert_eq!(row.voice.as_deref(), Some("grandma"));
assert_eq!(row.library.as_deref(), Some("1"));
}
#[actix_rt::test]
async fn put_prefs_owner_defaults_null_clears() {
let (state, mut seed_dao) = prefs_state();
seed(&mut seed_dao, seeded_row_with_defaults());
let app = init_service(
App::new()
.service(
web::resource("/prefs")
.route(web::get().to(get_prefs))
.route(web::put().to(put_prefs)),
)
.service(web::resource("/timezones").route(web::get().to(list_timezones)))
.app_data(state.clone()),
)
.await;
let (header, value) = auth();
let req = TestRequest::put()
.uri("/prefs")
.insert_header((header, value))
.set_json(serde_json::json!({
"default_model": null,
"default_backend": null,
"default_persona_id": null,
"reel_span": null,
"reel_max_segments": null
}))
.to_request();
let resp = call_service(&app, req).await;
assert_eq!(resp.status(), 200);
let mut dao = state.user_ai_prefs_dao.lock().unwrap();
let row = dao
.get_prefs(&opentelemetry::Context::new())
.unwrap()
.unwrap();
assert!(row.default_model.is_none());
assert!(row.default_backend.is_none());
assert!(row.default_persona_id.is_none());
assert!(row.reel_span.is_none());
assert!(row.reel_max_segments.is_none());
// Sibling fields survive a clear patch.
assert_eq!(row.voice.as_deref(), Some("grandma"));
}
#[actix_rt::test]
async fn put_prefs_invalid_backend_returns_400_and_writes_nothing() {
let (state, mut seed_dao) = prefs_state();
seed(
&mut seed_dao,
seeded_row(Some("grandma"), Some("1"), Some("America/New_York")),
);
let app = init_service(
App::new()
.service(
web::resource("/prefs")
.route(web::get().to(get_prefs))
.route(web::put().to(put_prefs)),
)
.service(web::resource("/timezones").route(web::get().to(list_timezones)))
.app_data(state.clone()),
)
.await;
let (header, value) = auth();
let req = TestRequest::put()
.uri("/prefs")
.insert_header((header, value))
.set_json(serde_json::json!({"default_backend": "quantum", "voice": "dad"}))
.to_request();
let resp = call_service(&app, req).await;
assert_eq!(resp.status(), 400);
let mut dao = state.user_ai_prefs_dao.lock().unwrap();
let row = dao
.get_prefs(&opentelemetry::Context::new())
.unwrap()
.unwrap();
// Nothing written — not even the valid field in the same body.
assert_eq!(row.voice.as_deref(), Some("grandma"));
assert!(row.default_backend.is_none());
assert_eq!(row.updated_at, 1_700_000_000);
}
#[test]
fn backend_validator_delegates_to_canonical_parser() {
// Case-insensitive like BackendKind::parse (a stored pref must always
// be consumable downstream), but the blank string the parser tolerates
// for request fields is rejected — NULL already means "no default".
assert!(is_valid_backend("local"));
assert!(is_valid_backend("hybrid"));
assert!(is_valid_backend("Hybrid"));
assert!(is_valid_backend(" LOCAL "));
assert!(!is_valid_backend("quantum"));
assert!(!is_valid_backend(""));
assert!(!is_valid_backend(" "));
}
#[actix_rt::test]
async fn put_prefs_invalid_reel_span_returns_400_and_writes_nothing() {
let (state, mut seed_dao) = prefs_state();
seed(
&mut seed_dao,
seeded_row(Some("grandma"), Some("1"), Some("America/New_York")),
);
let app = init_service(
App::new()
.service(
web::resource("/prefs")
.route(web::get().to(get_prefs))
.route(web::put().to(put_prefs)),
)
.service(web::resource("/timezones").route(web::get().to(list_timezones)))
.app_data(state.clone()),
)
.await;
let (header, value) = auth();
let req = TestRequest::put()
.uri("/prefs")
.insert_header((header, value))
.set_json(serde_json::json!({"reel_span": "decade"}))
.to_request();
let resp = call_service(&app, req).await;
assert_eq!(resp.status(), 400);
let mut dao = state.user_ai_prefs_dao.lock().unwrap();
let row = dao
.get_prefs(&opentelemetry::Context::new())
.unwrap()
.unwrap();
assert!(row.reel_span.is_none());
assert_eq!(row.updated_at, 1_700_000_000);
}
} }
+174 -15
View File
@@ -18,6 +18,7 @@ pub mod selector;
use std::collections::HashMap; use std::collections::HashMap;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::str::FromStr;
use std::sync::{LazyLock, Mutex, Mutex as StdMutex}; use std::sync::{LazyLock, Mutex, Mutex as StdMutex};
use std::time::{Duration, Instant}; use std::time::{Duration, Instant};
@@ -71,13 +72,38 @@ pub fn normalize_library_key(libs: &[Library], param: Option<&str>) -> String {
} }
} }
/// Reel span / segment-cap resolution: explicit request param → stored
/// owner pref → hard default. A stored span that no longer parses (e.g.
/// hand-edited) is skipped rather than erroring — the hard default still
/// applies. Non-positive stored segment caps are ignored the same way.
fn resolve_reel_params(
req_span: Option<MemoriesSpan>,
req_max_segments: Option<usize>,
prefs: Option<&crate::database::models::UserAiPrefs>,
) -> (MemoriesSpan, usize) {
let pref_span = prefs
.and_then(|p| p.reel_span.as_deref())
.and_then(|s| MemoriesSpan::from_str(s).ok());
let pref_max_segments = prefs
.and_then(|p| p.reel_max_segments)
.filter(|v| *v > 0)
.map(|v| v as usize);
let span = req_span.or(pref_span).unwrap_or(MemoriesSpan::Day);
let max_segments = req_max_segments
.or(pref_max_segments)
.unwrap_or(selector::DEFAULT_MAX_SEGMENTS);
(span, max_segments)
}
/// Best-effort: mirror the latest client reel params into `user_ai_prefs` /// Best-effort: mirror the latest client reel params into `user_ai_prefs`
/// so the nightly pre-gen scheduler can pick them up. Never fails the /// so the nightly pre-gen scheduler can pick them up. Never fails the
/// caller regardless of DB errors. /// caller regardless of DB errors.
fn capture_prefs( fn capture_prefs(
app_state: &AppState, app_state: &AppState,
req: &web::Json<CreateReelRequest>, req: &CreateReelRequest,
library_param: Option<&str>, library_param: Option<&str>,
effective_span: MemoriesSpan,
effective_max_segments: usize,
) -> Result<(), anyhow::Error> { ) -> Result<(), anyhow::Error> {
use crate::database::models::UpsertUserAiPrefs; use crate::database::models::UpsertUserAiPrefs;
let now = std::time::SystemTime::now() let now = std::time::SystemTime::now()
@@ -93,6 +119,21 @@ fn capture_prefs(
}; };
let mut dao = app_state.user_ai_prefs_dao.lock().expect("lock"); let mut dao = app_state.user_ai_prefs_dao.lock().expect("lock");
let ctx = opentelemetry::Context::new(); let ctx = opentelemetry::Context::new();
// The row passively mirrors the latest request's EFFECTIVE reel params
// (after the request → pref → default resolution done by the caller), so
// the stored reel_span / reel_max_segments track what was last used —
// idempotent when the pref itself supplied the effective value.
//
// The owner-default fields (model/backend/persona) are owned by PUT /prefs,
// not by reel requests. The upsert overwrites every column, so we must
// carry the existing values through unchanged — writing None here would
// silently wipe the owner's stored defaults on every reel creation. If the
// read fails we bail without writing: this is best-effort, and clobbering
// owner defaults on a transient error is worse than skipping a mirror.
let existing = match dao.get_prefs(&ctx) {
Ok(row) => row,
Err(e) => return Err(anyhow::anyhow!("failed to read prefs before mirror: {e}")),
};
dao.upsert_prefs( dao.upsert_prefs(
&ctx, &ctx,
&UpsertUserAiPrefs { &UpsertUserAiPrefs {
@@ -104,6 +145,11 @@ fn capture_prefs(
library: Some(library), library: Some(library),
updated_at: now as i64, updated_at: now as i64,
display_tz: req.display_tz.clone().filter(|s| !s.is_empty()), display_tz: req.display_tz.clone().filter(|s| !s.is_empty()),
default_model: existing.as_ref().and_then(|e| e.default_model.clone()),
default_backend: existing.as_ref().and_then(|e| e.default_backend.clone()),
default_persona_id: existing.as_ref().and_then(|e| e.default_persona_id.clone()),
reel_span: Some(effective_span.to_string()),
reel_max_segments: Some(effective_max_segments as i32),
}, },
) )
.map_err(|e| anyhow::anyhow!("failed to upsert user_ai_prefs: {e}")) .map_err(|e| anyhow::anyhow!("failed to upsert user_ai_prefs: {e}"))
@@ -415,20 +461,21 @@ pub async fn create_reel_handler(
})); }));
} }
let span = req.span.unwrap_or(MemoriesSpan::Day); // Owner defaults: explicit request param → stored pref → hard default.
let max_segments = req.max_segments.unwrap_or(selector::DEFAULT_MAX_SEGMENTS); // The stored row is read once and reused for the span, the segment cap,
// Grouping zone: explicit request param → stored pref → app default // and the display zone. The legacy device-offset param is still mirrored
// (ADR-0001). The legacy device-offset param is still mirrored into the // into the frozen prefs column by `capture_prefs` for rollback but no
// frozen prefs column by `capture_prefs` for rollback but no longer // longer drives selection.
// drives selection. let stored_prefs = app_state
.user_ai_prefs_dao
.lock()
.ok()
.and_then(|mut dao| dao.get_prefs(&opentelemetry::Context::new()).ok().flatten());
let (span, max_segments) =
resolve_reel_params(req.span, req.max_segments, stored_prefs.as_ref());
let display_zone_name = { let display_zone_name = {
let req_tz = req.display_tz.clone().filter(|s| !s.trim().is_empty()); let req_tz = req.display_tz.clone().filter(|s| !s.trim().is_empty());
let stored = app_state let stored = stored_prefs.as_ref().and_then(|p| p.display_tz.clone());
.user_ai_prefs_dao
.lock()
.ok()
.and_then(|mut dao| dao.get_prefs(&opentelemetry::Context::new()).ok().flatten())
.and_then(|p| p.display_tz);
crate::camera_tz::display_zone_for(req_tz.as_deref().or(stored.as_deref())) crate::camera_tz::display_zone_for(req_tz.as_deref().or(stored.as_deref()))
}; };
let selector = ReelSelector::Memories { let selector = ReelSelector::Memories {
@@ -489,7 +536,7 @@ pub async fn create_reel_handler(
}, },
); );
// Capture params for passive prefs mirror (best-effort, never fails). // Capture params for passive prefs mirror (best-effort, never fails).
let _ = capture_prefs(&app_state, &req, req.library.as_deref()); let _ = capture_prefs(&app_state, &req, req.library.as_deref(), span, max_segments);
return HttpResponse::Accepted().json(ReelJobCreatedResponse { return HttpResponse::Accepted().json(ReelJobCreatedResponse {
job_id: job_id.to_string(), job_id: job_id.to_string(),
status: ReelJobStatus::Done, status: ReelJobStatus::Done,
@@ -549,7 +596,7 @@ pub async fn create_reel_handler(
with_job(job_id, |job| job.abort = Some(handle.abort_handle())); with_job(job_id, |job| job.abort = Some(handle.abort_handle()));
// Capture params for passive prefs mirror (best-effort, never fails). // Capture params for passive prefs mirror (best-effort, never fails).
let _ = capture_prefs(&app_state, &req, req.library.as_deref()); let _ = capture_prefs(&app_state, &req, req.library.as_deref(), span, max_segments);
HttpResponse::Accepted().json(ReelJobCreatedResponse { HttpResponse::Accepted().json(ReelJobCreatedResponse {
job_id: job_id.to_string(), job_id: job_id.to_string(),
@@ -1333,9 +1380,121 @@ fn sweep_reel_cache(app_state: &AppState) -> usize {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use crate::database::models::UserAiPrefs;
use crate::libraries::Library; use crate::libraries::Library;
use chrono::TimeZone; use chrono::TimeZone;
fn prefs_row(span: Option<&str>, max_segments: Option<i32>) -> UserAiPrefs {
UserAiPrefs {
id: 1,
voice: None,
tz_offset_minutes: None,
library: None,
updated_at: 0,
display_tz: None,
default_model: None,
default_backend: None,
default_persona_id: None,
reel_span: span.map(str::to_string),
reel_max_segments: max_segments,
}
}
#[test]
fn resolve_reel_params_request_wins_over_pref() {
let prefs = prefs_row(Some("month"), Some(10));
let (span, max) = resolve_reel_params(Some(MemoriesSpan::Day), Some(5), Some(&prefs));
assert_eq!(span, MemoriesSpan::Day);
assert_eq!(max, 5);
}
#[test]
fn resolve_reel_params_pref_fills_omitted_request() {
let prefs = prefs_row(Some("week"), Some(24));
let (span, max) = resolve_reel_params(None, None, Some(&prefs));
assert_eq!(span, MemoriesSpan::Week);
assert_eq!(max, 24);
}
#[test]
fn resolve_reel_params_no_pref_falls_to_hard_defaults() {
let (span, max) = resolve_reel_params(None, None, None);
assert_eq!(span, MemoriesSpan::Day);
assert_eq!(max, selector::DEFAULT_MAX_SEGMENTS);
}
#[test]
fn resolve_reel_params_invalid_or_nonpositive_pref_skipped() {
// Hand-edited / corrupt stored values fall through to hard defaults
// instead of erroring the request.
let bad_span = prefs_row(Some("decade"), Some(0));
let (span, max) = resolve_reel_params(None, None, Some(&bad_span));
assert_eq!(span, MemoriesSpan::Day);
assert_eq!(max, selector::DEFAULT_MAX_SEGMENTS);
}
/// Regression: reel creation is a PASSIVE mirror. The upsert overwrites
/// every column, so `capture_prefs` must carry the PUT /prefs-owned
/// default_model / default_backend / default_persona_id through unchanged
/// — a reel request has no business wiping the owner's AI defaults.
#[actix_rt::test]
async fn reel_capture_preserves_owner_defaults() {
use crate::database::models::UpsertUserAiPrefs;
use crate::database::{SqliteUserAiPrefsDao, UserAiPrefsDao};
use crate::state::AppState;
use std::sync::{Arc, Mutex};
let conn = Arc::new(Mutex::new(crate::database::test::in_memory_db_connection()));
let mut state = AppState::test_state();
state.user_ai_prefs_dao = Arc::new(Mutex::new(Box::new(
SqliteUserAiPrefsDao::from_connection(conn.clone()),
)));
{
let mut dao = SqliteUserAiPrefsDao::from_connection(conn.clone());
dao.upsert_prefs(
&opentelemetry::Context::new(),
&UpsertUserAiPrefs {
voice: None,
tz_offset_minutes: None,
library: None,
updated_at: 0,
display_tz: None,
default_model: Some("llama3.2:3b".into()),
default_backend: Some("hybrid".into()),
default_persona_id: Some("cabin".into()),
reel_span: None,
reel_max_segments: None,
},
)
.expect("seed owner defaults");
}
let req = CreateReelRequest {
span: Some(MemoriesSpan::Week),
timezone_offset_minutes: Some(-240),
display_tz: None,
library: None,
voice: Some("grandma".into()),
max_segments: Some(12),
};
capture_prefs(&state, &req, None, MemoriesSpan::Week, 12).expect("capture");
let row = state
.user_ai_prefs_dao
.lock()
.expect("lock")
.get_prefs(&opentelemetry::Context::new())
.expect("read")
.expect("row");
// The mirrored reel params landed...
assert_eq!(row.reel_span.as_deref(), Some("week"));
assert_eq!(row.reel_max_segments, Some(12));
// ...and the owner defaults survived untouched.
assert_eq!(row.default_model.as_deref(), Some("llama3.2:3b"));
assert_eq!(row.default_backend.as_deref(), Some("hybrid"));
assert_eq!(row.default_persona_id.as_deref(), Some("cabin"));
}
fn photo(p: &str, lib: i32) -> SegmentMedia { fn photo(p: &str, lib: i32) -> SegmentMedia {
SegmentMedia::Photo { SegmentMedia::Photo {
rel_path: p.to_string(), rel_path: p.to_string(),
+2 -1
View File
@@ -281,8 +281,9 @@ pub async fn generate_script(
beats: &[PlannedBeat], beats: &[PlannedBeat],
) -> Result<ReelScript> { ) -> Result<ReelScript> {
let (system, user) = build_script_messages(meta, beats); let (system, user) = build_script_messages(meta, beats);
let cx = opentelemetry::Context::current();
let raw = client let raw = client
.generate(&user, Some(&system), None) .generate(&cx, &user, Some(&system), None)
.await .await
.context("LLM script generation failed")?; .context("LLM script generation failed")?;
Ok(parse_script_response(&raw, beats.len())) Ok(parse_script_response(&raw, beats.len()))
+1 -1
View File
@@ -211,7 +211,7 @@ pub async fn unified_search<TagD: TagDao>(
log::info!("unified_search: translating with model={}", backend.model()); log::info!("unified_search: translating with model={}", backend.model());
let today = chrono::Utc::now().date_naive(); let today = chrono::Utc::now().date_naive();
let sq = match translate_nl_query(backend.chat(), &nl, &tag_vocab, today).await { let sq = match translate_nl_query(backend.chat(), &nl, &tag_vocab, today, &ctx).await {
Ok(sq) => sq, Ok(sq) => sq,
Err(e) => { Err(e) => {
log::warn!("unified_search: translate_nl_query failed: {e:?}"); log::warn!("unified_search: translate_nl_query failed: {e:?}");