Merge pull request 'Prefs: owner defaults (model, backend, persona_id, reel span/length)' (#129) from feature/prefs-owner-defaults into master
Reviewed-on: #129
This commit was merged in pull request #129.
This commit is contained in:
commit
da3901c564
29 files changed
+1838
-326
No files matched your search
@@ -533,6 +533,11 @@ POST /insights/chat (single-turn reply, non-streaming)
|
|||||||
POST /insights/chat/stream (SSE: text / tool_call / tool_result / truncated / done)
|
POST /insights/chat/stream (SSE: text / tool_call / tool_result / truncated / done)
|
||||||
GET /insights/chat/history?path=... (rendered transcript with tool invocations)
|
GET /insights/chat/history?path=... (rendered transcript with tool invocations)
|
||||||
POST /insights/chat/rewind (truncate transcript at a rendered index)
|
POST /insights/chat/rewind (truncate transcript at a rendered index)
|
||||||
|
|
||||||
|
// User AI Prefs (owner defaults + passive mirror; src/prefs.rs)
|
||||||
|
GET /prefs (current row; auth)
|
||||||
|
PUT /prefs (partial update: omitted=unchanged, null=clear, value=set; auth)
|
||||||
|
GET /timezones (IANA zone list from chrono-tz; unauthenticated)
|
||||||
```
|
```
|
||||||
|
|
||||||
**Request Types:**
|
**Request Types:**
|
||||||
|
|||||||
+22
@@ -62,3 +62,25 @@ Insights and pre-computed artifacts grouped by calendar day (daily message summa
|
|||||||
reel pre-generation). Because it is keyed on the day, re-dating a photo across a day
|
reel pre-generation). Because it is keyed on the day, re-dating a photo across a day
|
||||||
boundary invalidates the derived data for that day, which regenerates on its normal schedule.
|
boundary invalidates the derived data for that day, which regenerates on its normal schedule.
|
||||||
_Avoid_: cache, nightly output
|
_Avoid_: cache, nightly output
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
**Owner default**:
|
||||||
|
A per-request default the owner sets for the whole deployment — the model, backend,
|
||||||
|
persona, reel span, or segment cap a client gets when it omits the field. Stored in the
|
||||||
|
single `user_ai_prefs` row, hot-tunable without a redeploy, and global: every client that
|
||||||
|
omits the field gets it, on every device. Never holds secrets.
|
||||||
|
_Avoid_: user prefs, user settings, preferences (ambiguous with device-local client state)
|
||||||
|
|
||||||
|
**Deployment config**:
|
||||||
|
Values the server needs before it can start serving — bind address, database path, media
|
||||||
|
roots, the URLs used to build clients — plus every secret. Lives in `.env` only; changing
|
||||||
|
it is a redeploy, by design.
|
||||||
|
_Avoid_: environment config, system config
|
||||||
|
|
||||||
|
**Hard default**:
|
||||||
|
The env/code fallback a request lands on when neither an explicit param nor a stored
|
||||||
|
owner default supplies the field (e.g. backend `local`, persona `default`, reel span
|
||||||
|
day). It is the floor under the owner-default layer: env changes keep working whenever a
|
||||||
|
default column is NULL.
|
||||||
|
_Avoid_: fallback value, env default
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
# ADR-0002: Owner defaults live in the DB row; .env keeps deployment config and secrets
|
||||||
|
|
||||||
|
**Status**: accepted (2026-09-27)
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
All configuration lived in `.env`: bind address, media paths, Ollama/llama-swap/OpenRouter
|
||||||
|
URLs, model names, `LLM_BACKEND`, the embedding contract, TTS voices. Two friction points:
|
||||||
|
values the owner actually *tunes* (which model, which backend, which persona, how long a
|
||||||
|
reel is) needed an edit + redeploy to change, and clients had no server-side fallback —
|
||||||
|
"use the default" was expressed by omitting a field, and the default itself was invisible
|
||||||
|
and untouchable at runtime. The `user_ai_prefs` single row (`id = 1`) already existed as
|
||||||
|
a passive mirror of the last client reel request for the nightly pre-gen. #126/#127 gave
|
||||||
|
it an explicit `PUT /prefs` surface and five owner-default columns (`default_model`,
|
||||||
|
`default_backend`, `default_persona_id`, `reel_span`, `reel_max_segments`) — the first
|
||||||
|
batch of non-private configuration moved out of `.env` and into the database.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Configuration splits into two homes by one rule — *does the server need the value before
|
||||||
|
it can start serving?*
|
||||||
|
|
||||||
|
- **`.env` (deployment config)**: everything read at boot to bind, connect, or open —
|
||||||
|
`BIND_URL`, `DATABASE_URL`, media paths, `OLLAMA_*_URL` / `LLAMA_SWAP_URL`,
|
||||||
|
`LLM_BACKEND`, the embedding contract — and **all secrets** (`OPENROUTER_API_KEY`,
|
||||||
|
`SECRET_KEY`, `SMS_API_TOKEN`). Secrets never go in the database: the SQLite file gets
|
||||||
|
backed up, copied, and attached to bug reports.
|
||||||
|
- **`user_ai_prefs` (owner defaults)**: per-request defaults resolved at request time —
|
||||||
|
the model/backend/persona a client gets when it omits the field, reel span and
|
||||||
|
segment cap, display zone, TTS voice. Hot-tunable via `PUT /prefs`, effective on the
|
||||||
|
next request, no redeploy. This row is the designated home for future non-private,
|
||||||
|
per-request configuration that outgrows `.env`. The table is a **single row per
|
||||||
|
deployment, not a per-user profile** — `user_ai_prefs` is a historical name; the
|
||||||
|
canonical term is *owner defaults* (single-user tool: owner = admin).
|
||||||
|
|
||||||
|
Resolution order at every consumer: **explicit request param → stored owner default →
|
||||||
|
hard default** (the env/code fallback). An explicit request always wins, so any client
|
||||||
|
can override the owner's default per request. The env value stays the floor under the
|
||||||
|
DB: a NULL `default_model` means "whatever the Ollama/llama-swap primary is", so env
|
||||||
|
changes keep working.
|
||||||
|
|
||||||
|
**Privacy posture is safe-by-default.** `default_backend` is the one owner default with
|
||||||
|
network-egress consequences: `hybrid` routes insight generation and chat to OpenRouter.
|
||||||
|
With no row or a NULL column, the hard default is `local` — data stays on the LAN.
|
||||||
|
Leaving the network requires an explicit owner action (storing `hybrid`), and any client
|
||||||
|
can still pin itself by sending `backend` explicitly. Owner defaults therefore split
|
||||||
|
into two sensitivity classes: **harmless tuning** (span, segments, display zone, voice,
|
||||||
|
model, persona) and **privacy/security** (`default_backend`), which any admin UI must
|
||||||
|
surface with a warning, not a quiet dropdown.
|
||||||
|
|
||||||
|
## Considered options
|
||||||
|
|
||||||
|
- **Everything in `.env`** (status quo): safe, but tuning a model default means edit +
|
||||||
|
redeploy, and clients have no server-side default to fall back to — a picker's
|
||||||
|
"server default" is just the env value, invisible and unchangeable at runtime.
|
||||||
|
Rejected for per-request defaults.
|
||||||
|
- **Everything in the DB**: boot-time values (bind address, DB path, the URLs used to
|
||||||
|
build the HTTP clients) become chicken-and-egg, and secrets sit next to tuning values
|
||||||
|
in a file that gets copied around. Rejected.
|
||||||
|
- **A separate `system_config` table**: cleaner name, but a second single-row table
|
||||||
|
would be a synonym with extra migrations; the row and the `PUT /prefs` surface already
|
||||||
|
exist. Rejected — the row keeps its name, its *term* becomes owner defaults.
|
||||||
|
|
||||||
|
## Non-obvious rules worth remembering
|
||||||
|
|
||||||
|
- **Owner defaults are global to the deployment.** Every client that omits a field gets
|
||||||
|
the stored value, on every device. Storing `default_backend=hybrid` changes the
|
||||||
|
mobile app's behavior even for users who never touch the setting; a client that must
|
||||||
|
stay local should send `backend` explicitly (tracked for FileViewer-React #33).
|
||||||
|
- **A stored value that is no longer consumable fails at write time, not request time** —
|
||||||
|
`PUT /prefs` validates `default_backend` against the canonical `BackendKind::parse`
|
||||||
|
and `reel_span` against the span enum before writing, so a bad default cannot 500
|
||||||
|
every request later.
|
||||||
|
- **The passive reel mirror (`capture_prefs`) may only write the fields it owns** (voice,
|
||||||
|
library, display zone, effective span/segments); it carries the AI owner-defaults
|
||||||
|
through unchanged, because the upsert overwrites every column.
|
||||||
|
- **NULL means "use the hard default", not "unset the env"** — the DB layer sits
|
||||||
|
*above* env, never instead of it.
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
ALTER TABLE user_ai_prefs DROP COLUMN default_model;
|
||||||
|
ALTER TABLE user_ai_prefs DROP COLUMN default_backend;
|
||||||
|
ALTER TABLE user_ai_prefs DROP COLUMN default_persona_id;
|
||||||
|
ALTER TABLE user_ai_prefs DROP COLUMN reel_span;
|
||||||
|
ALTER TABLE user_ai_prefs DROP COLUMN reel_max_segments;
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
-- Owner defaults: recurring per-user picks that were previously re-sent on
|
||||||
|
-- every request. All nullable — NULL = keep the current per-request /
|
||||||
|
-- hard-default behavior.
|
||||||
|
ALTER TABLE user_ai_prefs ADD COLUMN default_model TEXT;
|
||||||
|
ALTER TABLE user_ai_prefs ADD COLUMN default_backend TEXT;
|
||||||
|
ALTER TABLE user_ai_prefs ADD COLUMN default_persona_id TEXT;
|
||||||
|
ALTER TABLE user_ai_prefs ADD COLUMN reel_span TEXT;
|
||||||
|
ALTER TABLE user_ai_prefs ADD COLUMN reel_max_segments INTEGER;
|
||||||
@@ -360,11 +360,17 @@ async fn generate_and_store_daily_summary(
|
|||||||
span.set_attribute(KeyValue::new("date", date.to_string()));
|
span.set_attribute(KeyValue::new("date", date.to_string()));
|
||||||
span.set_attribute(KeyValue::new("contact", contact.to_string()));
|
span.set_attribute(KeyValue::new("contact", contact.to_string()));
|
||||||
span.set_attribute(KeyValue::new("message_count", messages.len() as i64));
|
span.set_attribute(KeyValue::new("message_count", messages.len() as i64));
|
||||||
|
// Parent context for the child llm.request spans; the span itself stays
|
||||||
|
// borrowed here for the attribute updates below.
|
||||||
|
let llm_cx =
|
||||||
|
opentelemetry::Context::new().with_remote_span_context(span.span_context().clone());
|
||||||
|
|
||||||
let (prompt, system_prompt) = build_daily_summary_prompt(contact, date, messages);
|
let (prompt, system_prompt) = build_daily_summary_prompt(contact, date, messages);
|
||||||
|
|
||||||
// Generate summary with LLM
|
// Generate summary with LLM
|
||||||
let summary = ollama.generate(&prompt, Some(system_prompt)).await?;
|
let summary = ollama
|
||||||
|
.generate(&llm_cx, &prompt, Some(system_prompt))
|
||||||
|
.await?;
|
||||||
|
|
||||||
log::debug!(
|
log::debug!(
|
||||||
"Generated summary for {}: {}",
|
"Generated summary for {}: {}",
|
||||||
@@ -382,7 +388,9 @@ async fn generate_and_store_daily_summary(
|
|||||||
);
|
);
|
||||||
|
|
||||||
// Embed the stripped summary (store original summary in DB)
|
// Embed the stripped summary (store original summary in DB)
|
||||||
let embedding = ollama.generate_embedding(&stripped_summary).await?;
|
let embedding = ollama
|
||||||
|
.generate_embedding(&llm_cx, &stripped_summary)
|
||||||
|
.await?;
|
||||||
|
|
||||||
span.set_attribute(KeyValue::new(
|
span.set_attribute(KeyValue::new(
|
||||||
"embedding_dimensions",
|
"embedding_dimensions",
|
||||||
|
|||||||
+175
-1
@@ -384,6 +384,36 @@ pub struct ServerModels {
|
|||||||
pub default_model: String,
|
pub default_model: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Apply the stored owner defaults to any request field the client omitted.
|
||||||
|
/// Resolution order: request param → stored pref → hard default (applied
|
||||||
|
/// downstream). The request always wins; the pref only fills gaps. One DAO
|
||||||
|
/// read per request; a failed read silently falls back to hard defaults.
|
||||||
|
fn apply_owner_defaults(
|
||||||
|
app_state: &AppState,
|
||||||
|
model: &mut Option<String>,
|
||||||
|
backend: &mut Option<String>,
|
||||||
|
persona_id: &mut Option<String>,
|
||||||
|
) {
|
||||||
|
if model.is_some() && backend.is_some() && persona_id.is_some() {
|
||||||
|
return; // nothing to fill — skip the DB read
|
||||||
|
}
|
||||||
|
let Some(mut dao) = app_state.user_ai_prefs_dao.lock().ok() else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let Ok(Some(prefs)) = dao.get_prefs(&opentelemetry::Context::new()) else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
if model.is_none() {
|
||||||
|
*model = prefs.default_model;
|
||||||
|
}
|
||||||
|
if backend.is_none() {
|
||||||
|
*backend = prefs.default_backend;
|
||||||
|
}
|
||||||
|
if persona_id.is_none() {
|
||||||
|
*persona_id = prefs.default_persona_id;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// POST /insights/generate - Generate insight for a specific photo (async)
|
/// POST /insights/generate - Generate insight for a specific photo (async)
|
||||||
#[post("/insights/generate")]
|
#[post("/insights/generate")]
|
||||||
pub async fn generate_insight_handler(
|
pub async fn generate_insight_handler(
|
||||||
@@ -396,6 +426,16 @@ pub async fn generate_insight_handler(
|
|||||||
let tracer = global_tracer();
|
let tracer = global_tracer();
|
||||||
let mut span = tracer.start_with_context("http.insights.generate", &parent_context);
|
let mut span = tracer.start_with_context("http.insights.generate", &parent_context);
|
||||||
|
|
||||||
|
let mut request = request.0;
|
||||||
|
// Stored owner defaults fill what the client omitted (request → pref →
|
||||||
|
// hard default; the hard default still applies downstream).
|
||||||
|
apply_owner_defaults(
|
||||||
|
&app_state,
|
||||||
|
&mut request.model,
|
||||||
|
&mut request.backend,
|
||||||
|
&mut request.persona_id,
|
||||||
|
);
|
||||||
|
|
||||||
let normalized_path = normalize_path(&request.file_path);
|
let normalized_path = normalize_path(&request.file_path);
|
||||||
let library = app_state.primary_library();
|
let library = app_state.primary_library();
|
||||||
let gen_type = InsightGenerationType::Standard;
|
let gen_type = InsightGenerationType::Standard;
|
||||||
@@ -714,6 +754,16 @@ pub async fn generate_agentic_insight_handler(
|
|||||||
let tracer = global_tracer();
|
let tracer = global_tracer();
|
||||||
let mut span = tracer.start_with_context("http.insights.generate_agentic", &parent_context);
|
let mut span = tracer.start_with_context("http.insights.generate_agentic", &parent_context);
|
||||||
|
|
||||||
|
let mut request = request.0;
|
||||||
|
// Stored owner defaults fill what the client omitted (request → pref →
|
||||||
|
// hard default; the hard default still applies downstream).
|
||||||
|
apply_owner_defaults(
|
||||||
|
&app_state,
|
||||||
|
&mut request.model,
|
||||||
|
&mut request.backend,
|
||||||
|
&mut request.persona_id,
|
||||||
|
);
|
||||||
|
|
||||||
let normalized_path = normalize_path(&request.file_path);
|
let normalized_path = normalize_path(&request.file_path);
|
||||||
let library = app_state.primary_library();
|
let library = app_state.primary_library();
|
||||||
let gen_type = InsightGenerationType::Agentic;
|
let gen_type = InsightGenerationType::Agentic;
|
||||||
@@ -1231,6 +1281,16 @@ pub async fn chat_turn_handler(
|
|||||||
let mut span = tracer.start_with_context("http.insights.chat", &parent_context);
|
let mut span = tracer.start_with_context("http.insights.chat", &parent_context);
|
||||||
span.set_attribute(KeyValue::new("file_path", request.file_path.clone()));
|
span.set_attribute(KeyValue::new("file_path", request.file_path.clone()));
|
||||||
|
|
||||||
|
let mut request = request.0;
|
||||||
|
// Stored owner defaults fill what the client omitted (request → pref →
|
||||||
|
// hard default; the hard default still applies downstream).
|
||||||
|
apply_owner_defaults(
|
||||||
|
&app_state,
|
||||||
|
&mut request.model,
|
||||||
|
&mut request.backend,
|
||||||
|
&mut request.persona_id,
|
||||||
|
);
|
||||||
|
|
||||||
let library =
|
let library =
|
||||||
match libraries::resolve_library_param_state(&app_state, request.library.as_deref()) {
|
match libraries::resolve_library_param_state(&app_state, request.library.as_deref()) {
|
||||||
Ok(Some(lib)) => lib,
|
Ok(Some(lib)) => lib,
|
||||||
@@ -1607,10 +1667,22 @@ pub struct ChatSwitchBranchRequest {
|
|||||||
/// Returns `text/event-stream` with one event per chat stream event.
|
/// Returns `text/event-stream` with one event per chat stream event.
|
||||||
#[post("/insights/chat/stream")]
|
#[post("/insights/chat/stream")]
|
||||||
pub async fn chat_stream_handler(
|
pub async fn chat_stream_handler(
|
||||||
|
http_request: HttpRequest,
|
||||||
claims: Claims,
|
claims: Claims,
|
||||||
request: web::Json<ChatTurnHttpRequest>,
|
request: web::Json<ChatTurnHttpRequest>,
|
||||||
app_state: web::Data<AppState>,
|
app_state: web::Data<AppState>,
|
||||||
) -> HttpResponse {
|
) -> HttpResponse {
|
||||||
|
let parent_context = extract_context_from_request(&http_request);
|
||||||
|
let mut request = request.0;
|
||||||
|
// Stored owner defaults fill what the client omitted (request → pref →
|
||||||
|
// hard default; the hard default still applies downstream).
|
||||||
|
apply_owner_defaults(
|
||||||
|
&app_state,
|
||||||
|
&mut request.model,
|
||||||
|
&mut request.backend,
|
||||||
|
&mut request.persona_id,
|
||||||
|
);
|
||||||
|
|
||||||
let library =
|
let library =
|
||||||
match libraries::resolve_library_param_state(&app_state, request.library.as_deref()) {
|
match libraries::resolve_library_param_state(&app_state, request.library.as_deref()) {
|
||||||
Ok(Some(lib)) => lib,
|
Ok(Some(lib)) => lib,
|
||||||
@@ -1646,7 +1718,7 @@ pub async fn chat_stream_handler(
|
|||||||
};
|
};
|
||||||
|
|
||||||
let service = app_state.insight_chat.clone();
|
let service = app_state.insight_chat.clone();
|
||||||
let events = service.chat_turn_stream(chat_req);
|
let events = service.chat_turn_stream(chat_req, parent_context);
|
||||||
|
|
||||||
// Map ChatStreamEvent → SSE frame bytes.
|
// Map ChatStreamEvent → SSE frame bytes.
|
||||||
let sse_stream = futures::stream::StreamExt::map(events, |ev| {
|
let sse_stream = futures::stream::StreamExt::map(events, |ev| {
|
||||||
@@ -1758,6 +1830,16 @@ pub async fn turn_async_handler(
|
|||||||
let mut span = tracer.start_with_context("http.insights.chat_turn_async", &parent_context);
|
let mut span = tracer.start_with_context("http.insights.chat_turn_async", &parent_context);
|
||||||
span.set_attribute(KeyValue::new("file_path", request.file_path.clone()));
|
span.set_attribute(KeyValue::new("file_path", request.file_path.clone()));
|
||||||
|
|
||||||
|
let mut request = request.0;
|
||||||
|
// Stored owner defaults fill what the client omitted (request → pref →
|
||||||
|
// hard default; the hard default still applies downstream).
|
||||||
|
apply_owner_defaults(
|
||||||
|
&app_state,
|
||||||
|
&mut request.model,
|
||||||
|
&mut request.backend,
|
||||||
|
&mut request.persona_id,
|
||||||
|
);
|
||||||
|
|
||||||
let library =
|
let library =
|
||||||
match libraries::resolve_library_param_state(&app_state, request.library.as_deref()) {
|
match libraries::resolve_library_param_state(&app_state, request.library.as_deref()) {
|
||||||
Ok(Some(lib)) => lib,
|
Ok(Some(lib)) => lib,
|
||||||
@@ -2277,3 +2359,95 @@ mod turn_replay_tests {
|
|||||||
assert_eq!(info.total_events_pushed, 2);
|
assert_eq!(info.total_events_pushed, 2);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod owner_defaults_tests {
|
||||||
|
use super::apply_owner_defaults;
|
||||||
|
use crate::database::models::UpsertUserAiPrefs;
|
||||||
|
use crate::database::{SqliteUserAiPrefsDao, UserAiPrefsDao};
|
||||||
|
use crate::state::AppState;
|
||||||
|
use std::sync::{Arc, Mutex};
|
||||||
|
|
||||||
|
fn prefs_state(prefs: Option<UpsertUserAiPrefs>) -> AppState {
|
||||||
|
let conn = Arc::new(Mutex::new(crate::database::test::in_memory_db_connection()));
|
||||||
|
let mut state = AppState::test_state();
|
||||||
|
state.user_ai_prefs_dao = Arc::new(Mutex::new(Box::new(
|
||||||
|
SqliteUserAiPrefsDao::from_connection(conn.clone()),
|
||||||
|
)));
|
||||||
|
if let Some(p) = prefs {
|
||||||
|
let mut dao = SqliteUserAiPrefsDao::from_connection(conn);
|
||||||
|
dao.upsert_prefs(&opentelemetry::Context::new(), &p)
|
||||||
|
.expect("seed prefs");
|
||||||
|
}
|
||||||
|
state
|
||||||
|
}
|
||||||
|
|
||||||
|
fn seeded_defaults() -> UpsertUserAiPrefs {
|
||||||
|
UpsertUserAiPrefs {
|
||||||
|
voice: None,
|
||||||
|
tz_offset_minutes: None,
|
||||||
|
library: None,
|
||||||
|
updated_at: 0,
|
||||||
|
display_tz: None,
|
||||||
|
default_model: Some("llama3.2:3b".into()),
|
||||||
|
default_backend: Some("hybrid".into()),
|
||||||
|
default_persona_id: Some("cabin".into()),
|
||||||
|
reel_span: None,
|
||||||
|
reel_max_segments: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[actix_rt::test]
|
||||||
|
async fn request_wins_over_stored_pref() {
|
||||||
|
let state = prefs_state(Some(seeded_defaults()));
|
||||||
|
let mut model = Some("explicit".to_string());
|
||||||
|
let mut backend = Some("local".to_string());
|
||||||
|
let mut persona = Some("explicit-p".to_string());
|
||||||
|
apply_owner_defaults(&state, &mut model, &mut backend, &mut persona);
|
||||||
|
assert_eq!(model.as_deref(), Some("explicit"));
|
||||||
|
assert_eq!(backend.as_deref(), Some("local"));
|
||||||
|
assert_eq!(persona.as_deref(), Some("explicit-p"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[actix_rt::test]
|
||||||
|
async fn pref_fills_omitted_fields() {
|
||||||
|
let state = prefs_state(Some(seeded_defaults()));
|
||||||
|
let mut model = None;
|
||||||
|
let mut backend = None;
|
||||||
|
let mut persona = None;
|
||||||
|
apply_owner_defaults(&state, &mut model, &mut backend, &mut persona);
|
||||||
|
assert_eq!(model.as_deref(), Some("llama3.2:3b"));
|
||||||
|
assert_eq!(backend.as_deref(), Some("hybrid"));
|
||||||
|
assert_eq!(persona.as_deref(), Some("cabin"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[actix_rt::test]
|
||||||
|
async fn no_pref_row_leaves_fields_none() {
|
||||||
|
let state = prefs_state(None);
|
||||||
|
let mut model = None;
|
||||||
|
let mut backend = None;
|
||||||
|
let mut persona = None;
|
||||||
|
apply_owner_defaults(&state, &mut model, &mut backend, &mut persona);
|
||||||
|
// Hard defaults apply downstream; the pref layer fills nothing.
|
||||||
|
assert!(model.is_none());
|
||||||
|
assert!(backend.is_none());
|
||||||
|
assert!(persona.is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[actix_rt::test]
|
||||||
|
async fn pref_fills_only_the_fields_it_has() {
|
||||||
|
let state = prefs_state(Some(UpsertUserAiPrefs {
|
||||||
|
default_backend: None,
|
||||||
|
default_persona_id: Some("journal".into()),
|
||||||
|
..seeded_defaults()
|
||||||
|
}));
|
||||||
|
let mut model = None;
|
||||||
|
let mut backend = Some("local".to_string());
|
||||||
|
let mut persona = None;
|
||||||
|
apply_owner_defaults(&state, &mut model, &mut backend, &mut persona);
|
||||||
|
assert_eq!(model.as_deref(), Some("llama3.2:3b"));
|
||||||
|
// Explicit request backend is untouched; pref had none anyway.
|
||||||
|
assert_eq!(backend.as_deref(), Some("local"));
|
||||||
|
assert_eq!(persona.as_deref(), Some("journal"));
|
||||||
|
}
|
||||||
|
}
|
||||||
+45
-17
@@ -377,7 +377,7 @@ impl InsightChatService {
|
|||||||
|
|
||||||
let (response, prompt_tokens, eval_tokens) = backend
|
let (response, prompt_tokens, eval_tokens) = backend
|
||||||
.chat()
|
.chat()
|
||||||
.chat_with_tools(messages.clone(), tools.clone())
|
.chat_with_tools(&loop_cx, messages.clone(), tools.clone())
|
||||||
.await?;
|
.await?;
|
||||||
last_prompt_eval_count = prompt_tokens;
|
last_prompt_eval_count = prompt_tokens;
|
||||||
last_eval_count = eval_tokens;
|
last_eval_count = eval_tokens;
|
||||||
@@ -434,7 +434,7 @@ impl InsightChatService {
|
|||||||
let synthetic_idx = push_synthetic_final_prompt(&mut messages);
|
let synthetic_idx = push_synthetic_final_prompt(&mut messages);
|
||||||
let (final_response, prompt_tokens, eval_tokens) = backend
|
let (final_response, prompt_tokens, eval_tokens) = backend
|
||||||
.chat()
|
.chat()
|
||||||
.chat_with_tools(messages.clone(), vec![])
|
.chat_with_tools(&loop_cx, messages.clone(), vec![])
|
||||||
.await?;
|
.await?;
|
||||||
last_prompt_eval_count = prompt_tokens;
|
last_prompt_eval_count = prompt_tokens;
|
||||||
last_eval_count = eval_tokens;
|
last_eval_count = eval_tokens;
|
||||||
@@ -492,6 +492,7 @@ impl InsightChatService {
|
|||||||
let title_raw = backend
|
let title_raw = backend
|
||||||
.chat()
|
.chat()
|
||||||
.generate(
|
.generate(
|
||||||
|
&loop_cx,
|
||||||
&title_prompt,
|
&title_prompt,
|
||||||
Some(
|
Some(
|
||||||
"You are my long term memory assistant. Use only the information provided. Do not invent details.",
|
"You are my long term memory assistant. Use only the information provided. Do not invent details.",
|
||||||
@@ -781,10 +782,11 @@ impl InsightChatService {
|
|||||||
pub fn chat_turn_stream(
|
pub fn chat_turn_stream(
|
||||||
self: Arc<Self>,
|
self: Arc<Self>,
|
||||||
req: ChatTurnRequest,
|
req: ChatTurnRequest,
|
||||||
|
cx: opentelemetry::Context,
|
||||||
) -> BoxStream<'static, ChatStreamEvent> {
|
) -> BoxStream<'static, ChatStreamEvent> {
|
||||||
let svc = self;
|
let svc = self;
|
||||||
let s = async_stream::stream! {
|
let s = async_stream::stream! {
|
||||||
match svc.chat_turn_stream_inner(req, Ok).await {
|
match svc.chat_turn_stream_inner(req, cx, Ok).await {
|
||||||
Ok(mut rx) => {
|
Ok(mut rx) => {
|
||||||
while let Some(ev) = rx.recv().await {
|
while let Some(ev) = rx.recv().await {
|
||||||
yield ev;
|
yield ev;
|
||||||
@@ -806,6 +808,7 @@ impl InsightChatService {
|
|||||||
async fn chat_turn_stream_inner<F>(
|
async fn chat_turn_stream_inner<F>(
|
||||||
self: Arc<Self>,
|
self: Arc<Self>,
|
||||||
req: ChatTurnRequest,
|
req: ChatTurnRequest,
|
||||||
|
cx: opentelemetry::Context,
|
||||||
_ev_mapper: F,
|
_ev_mapper: F,
|
||||||
) -> Result<tokio::sync::mpsc::Receiver<ChatStreamEvent>>
|
) -> Result<tokio::sync::mpsc::Receiver<ChatStreamEvent>>
|
||||||
where
|
where
|
||||||
@@ -814,7 +817,7 @@ impl InsightChatService {
|
|||||||
let (tx, rx) = tokio::sync::mpsc::channel::<ChatStreamEvent>(64);
|
let (tx, rx) = tokio::sync::mpsc::channel::<ChatStreamEvent>(64);
|
||||||
let svc = self.clone();
|
let svc = self.clone();
|
||||||
tokio::spawn(async move {
|
tokio::spawn(async move {
|
||||||
let result = svc.run_streaming_turn(req, tx.clone()).await;
|
let result = svc.run_streaming_turn(req, cx, tx.clone()).await;
|
||||||
if let Err(e) = result {
|
if let Err(e) = result {
|
||||||
let _ = tx.send(ChatStreamEvent::Error(format!("{}", e))).await;
|
let _ = tx.send(ChatStreamEvent::Error(format!("{}", e))).await;
|
||||||
}
|
}
|
||||||
@@ -852,7 +855,7 @@ impl InsightChatService {
|
|||||||
span.set_attribute(KeyValue::new("library_id", library_id as i64));
|
span.set_attribute(KeyValue::new("library_id", library_id as i64));
|
||||||
|
|
||||||
let result = svc
|
let result = svc
|
||||||
.run_streaming_turn_with_entry(req, entry_clone.clone())
|
.run_streaming_turn_with_entry(req, entry_clone.clone(), &mut span)
|
||||||
.await;
|
.await;
|
||||||
if let Err(ref e) = result {
|
if let Err(ref e) = result {
|
||||||
span.set_attribute(KeyValue::new("status", "error"));
|
span.set_attribute(KeyValue::new("status", "error"));
|
||||||
@@ -882,6 +885,7 @@ impl InsightChatService {
|
|||||||
self: Arc<Self>,
|
self: Arc<Self>,
|
||||||
req: ChatTurnRequest,
|
req: ChatTurnRequest,
|
||||||
entry: Arc<TurnEntry>,
|
entry: Arc<TurnEntry>,
|
||||||
|
turn_span: &mut opentelemetry::global::BoxedSpan,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
if req.user_message.trim().is_empty() {
|
if req.user_message.trim().is_empty() {
|
||||||
bail!("user_message must not be empty");
|
bail!("user_message must not be empty");
|
||||||
@@ -911,11 +915,11 @@ impl InsightChatService {
|
|||||||
|
|
||||||
if req.regenerate || existing_insight.is_none() {
|
if req.regenerate || existing_insight.is_none() {
|
||||||
return self
|
return self
|
||||||
.run_bootstrap_streaming_with_entry(req, normalized, entry)
|
.run_bootstrap_streaming_with_entry(req, normalized, entry, turn_span)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
let insight = existing_insight.expect("just checked Some above");
|
let insight = existing_insight.expect("just checked Some above");
|
||||||
self.run_continuation_streaming_with_entry(req, normalized, insight, entry)
|
self.run_continuation_streaming_with_entry(req, normalized, insight, entry, turn_span)
|
||||||
.await
|
.await
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -926,6 +930,7 @@ impl InsightChatService {
|
|||||||
normalized: String,
|
normalized: String,
|
||||||
insight: crate::database::models::PhotoInsight,
|
insight: crate::database::models::PhotoInsight,
|
||||||
entry: Arc<TurnEntry>,
|
entry: Arc<TurnEntry>,
|
||||||
|
turn_span: &mut opentelemetry::global::BoxedSpan,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
let active_persona = req
|
let active_persona = req
|
||||||
.persona_id
|
.persona_id
|
||||||
@@ -981,6 +986,12 @@ impl InsightChatService {
|
|||||||
};
|
};
|
||||||
let backend = self.generator.resolve_backend(kind, &overrides).await?;
|
let backend = self.generator.resolve_backend(kind, &overrides).await?;
|
||||||
let model_used = backend.model().to_string();
|
let model_used = backend.model().to_string();
|
||||||
|
// llm.* attributes on the turn-level span (issue #130): clients can't
|
||||||
|
// know local-vs-hybrid, but the turn knows its resolved backend.
|
||||||
|
turn_span.set_attribute(KeyValue::new("llm.backend", kind.as_str()));
|
||||||
|
turn_span.set_attribute(KeyValue::new("llm.model", model_used.clone()));
|
||||||
|
let turn_cx = opentelemetry::Context::new()
|
||||||
|
.with_remote_span_context(turn_span.span_context().clone());
|
||||||
|
|
||||||
let local_first_user_has_image = messages
|
let local_first_user_has_image = messages
|
||||||
.iter()
|
.iter()
|
||||||
@@ -1030,6 +1041,7 @@ impl InsightChatService {
|
|||||||
&active_persona,
|
&active_persona,
|
||||||
max_iterations,
|
max_iterations,
|
||||||
&entry,
|
&entry,
|
||||||
|
&turn_cx,
|
||||||
)
|
)
|
||||||
.await?;
|
.await?;
|
||||||
let AgenticLoopOutcome {
|
let AgenticLoopOutcome {
|
||||||
@@ -1146,6 +1158,7 @@ impl InsightChatService {
|
|||||||
req: ChatTurnRequest,
|
req: ChatTurnRequest,
|
||||||
normalized: String,
|
normalized: String,
|
||||||
entry: Arc<TurnEntry>,
|
entry: Arc<TurnEntry>,
|
||||||
|
turn_span: &mut opentelemetry::global::BoxedSpan,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
let active_persona = req
|
let active_persona = req
|
||||||
.persona_id
|
.persona_id
|
||||||
@@ -1171,6 +1184,11 @@ impl InsightChatService {
|
|||||||
};
|
};
|
||||||
let backend = self.generator.resolve_backend(kind, &overrides).await?;
|
let backend = self.generator.resolve_backend(kind, &overrides).await?;
|
||||||
let model_used = backend.model().to_string();
|
let model_used = backend.model().to_string();
|
||||||
|
// llm.* attributes on the turn-level span (issue #130).
|
||||||
|
turn_span.set_attribute(KeyValue::new("llm.backend", kind.as_str()));
|
||||||
|
turn_span.set_attribute(KeyValue::new("llm.model", model_used.clone()));
|
||||||
|
let turn_cx = opentelemetry::Context::new()
|
||||||
|
.with_remote_span_context(turn_span.span_context().clone());
|
||||||
|
|
||||||
let image_base64: Option<String> = self.generator.load_image_as_base64(&normalized).ok();
|
let image_base64: Option<String> = self.generator.load_image_as_base64(&normalized).ok();
|
||||||
|
|
||||||
@@ -1190,7 +1208,7 @@ impl InsightChatService {
|
|||||||
|
|
||||||
let visual_block = if !backend.images_inline {
|
let visual_block = if !backend.images_inline {
|
||||||
match image_base64.as_deref() {
|
match image_base64.as_deref() {
|
||||||
Some(b64) => match backend.local().describe_image(b64).await {
|
Some(b64) => match backend.local().describe_image(&turn_cx, b64).await {
|
||||||
Ok(desc) => {
|
Ok(desc) => {
|
||||||
format!("Visual description (from local vision model):\n{}\n", desc)
|
format!("Visual description (from local vision model):\n{}\n", desc)
|
||||||
}
|
}
|
||||||
@@ -1245,6 +1263,7 @@ impl InsightChatService {
|
|||||||
&active_persona,
|
&active_persona,
|
||||||
max_iterations,
|
max_iterations,
|
||||||
&entry,
|
&entry,
|
||||||
|
&turn_cx,
|
||||||
)
|
)
|
||||||
.await?;
|
.await?;
|
||||||
let AgenticLoopOutcome {
|
let AgenticLoopOutcome {
|
||||||
@@ -1333,6 +1352,7 @@ impl InsightChatService {
|
|||||||
active_persona: &str,
|
active_persona: &str,
|
||||||
max_iterations: usize,
|
max_iterations: usize,
|
||||||
entry: &Arc<TurnEntry>,
|
entry: &Arc<TurnEntry>,
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
) -> Result<AgenticLoopOutcome> {
|
) -> Result<AgenticLoopOutcome> {
|
||||||
crate::ai::insight_chat::run_streaming_agentic_loop_with_entry(
|
crate::ai::insight_chat::run_streaming_agentic_loop_with_entry(
|
||||||
&self.generator,
|
&self.generator,
|
||||||
@@ -1345,6 +1365,7 @@ impl InsightChatService {
|
|||||||
active_persona,
|
active_persona,
|
||||||
max_iterations,
|
max_iterations,
|
||||||
entry,
|
entry,
|
||||||
|
cx,
|
||||||
)
|
)
|
||||||
.await
|
.await
|
||||||
}
|
}
|
||||||
@@ -1352,6 +1373,7 @@ impl InsightChatService {
|
|||||||
async fn run_streaming_turn(
|
async fn run_streaming_turn(
|
||||||
self: Arc<Self>,
|
self: Arc<Self>,
|
||||||
req: ChatTurnRequest,
|
req: ChatTurnRequest,
|
||||||
|
cx: opentelemetry::Context,
|
||||||
tx: tokio::sync::mpsc::Sender<ChatStreamEvent>,
|
tx: tokio::sync::mpsc::Sender<ChatStreamEvent>,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
if req.user_message.trim().is_empty() {
|
if req.user_message.trim().is_empty() {
|
||||||
@@ -1381,17 +1403,17 @@ impl InsightChatService {
|
|||||||
// library bootstraps cleanly, even when another library has
|
// library bootstraps cleanly, even when another library has
|
||||||
// an insight for the same rel_path.
|
// an insight for the same rel_path.
|
||||||
let existing_insight = {
|
let existing_insight = {
|
||||||
let cx = opentelemetry::Context::new();
|
let db_cx = opentelemetry::Context::new();
|
||||||
let mut dao = self.insight_dao.lock().expect("Unable to lock InsightDao");
|
let mut dao = self.insight_dao.lock().expect("Unable to lock InsightDao");
|
||||||
dao.get_current_insight_for_library(&cx, req.library_id, &normalized)
|
dao.get_current_insight_for_library(&db_cx, req.library_id, &normalized)
|
||||||
.map_err(|e| anyhow!("failed to load insight: {:?}", e))?
|
.map_err(|e| anyhow!("failed to load insight: {:?}", e))?
|
||||||
};
|
};
|
||||||
|
|
||||||
if req.regenerate || existing_insight.is_none() {
|
if req.regenerate || existing_insight.is_none() {
|
||||||
return self.run_bootstrap_streaming(req, normalized, tx).await;
|
return self.run_bootstrap_streaming(req, normalized, cx, tx).await;
|
||||||
}
|
}
|
||||||
let insight = existing_insight.expect("just checked Some above");
|
let insight = existing_insight.expect("just checked Some above");
|
||||||
self.run_continuation_streaming(req, normalized, insight, tx)
|
self.run_continuation_streaming(req, normalized, cx, insight, tx)
|
||||||
.await
|
.await
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1403,6 +1425,7 @@ impl InsightChatService {
|
|||||||
&self,
|
&self,
|
||||||
req: ChatTurnRequest,
|
req: ChatTurnRequest,
|
||||||
normalized: String,
|
normalized: String,
|
||||||
|
cx: opentelemetry::Context,
|
||||||
insight: crate::database::models::PhotoInsight,
|
insight: crate::database::models::PhotoInsight,
|
||||||
tx: tokio::sync::mpsc::Sender<ChatStreamEvent>,
|
tx: tokio::sync::mpsc::Sender<ChatStreamEvent>,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
@@ -1500,6 +1523,7 @@ impl InsightChatService {
|
|||||||
&active_persona,
|
&active_persona,
|
||||||
max_iterations,
|
max_iterations,
|
||||||
&tx,
|
&tx,
|
||||||
|
&cx,
|
||||||
)
|
)
|
||||||
.await?;
|
.await?;
|
||||||
let AgenticLoopOutcome {
|
let AgenticLoopOutcome {
|
||||||
@@ -1611,6 +1635,7 @@ impl InsightChatService {
|
|||||||
&self,
|
&self,
|
||||||
req: ChatTurnRequest,
|
req: ChatTurnRequest,
|
||||||
normalized: String,
|
normalized: String,
|
||||||
|
cx: opentelemetry::Context,
|
||||||
tx: tokio::sync::mpsc::Sender<ChatStreamEvent>,
|
tx: tokio::sync::mpsc::Sender<ChatStreamEvent>,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
let active_persona = req
|
let active_persona = req
|
||||||
@@ -1667,7 +1692,7 @@ impl InsightChatService {
|
|||||||
// images_inline backends send images directly to the chat model.
|
// images_inline backends send images directly to the chat model.
|
||||||
let visual_block = if !backend.images_inline {
|
let visual_block = if !backend.images_inline {
|
||||||
match image_base64.as_deref() {
|
match image_base64.as_deref() {
|
||||||
Some(b64) => match backend.local().describe_image(b64).await {
|
Some(b64) => match backend.local().describe_image(&cx, b64).await {
|
||||||
Ok(desc) => {
|
Ok(desc) => {
|
||||||
format!("Visual description (from local vision model):\n{}\n", desc)
|
format!("Visual description (from local vision model):\n{}\n", desc)
|
||||||
}
|
}
|
||||||
@@ -1734,6 +1759,7 @@ impl InsightChatService {
|
|||||||
&active_persona,
|
&active_persona,
|
||||||
max_iterations,
|
max_iterations,
|
||||||
&tx,
|
&tx,
|
||||||
|
&cx,
|
||||||
)
|
)
|
||||||
.await?;
|
.await?;
|
||||||
let AgenticLoopOutcome {
|
let AgenticLoopOutcome {
|
||||||
@@ -1816,6 +1842,7 @@ impl InsightChatService {
|
|||||||
active_persona: &str,
|
active_persona: &str,
|
||||||
max_iterations: usize,
|
max_iterations: usize,
|
||||||
tx: &tokio::sync::mpsc::Sender<ChatStreamEvent>,
|
tx: &tokio::sync::mpsc::Sender<ChatStreamEvent>,
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
) -> Result<AgenticLoopOutcome> {
|
) -> Result<AgenticLoopOutcome> {
|
||||||
let mut tool_calls_made = 0usize;
|
let mut tool_calls_made = 0usize;
|
||||||
let mut iterations_used = 0usize;
|
let mut iterations_used = 0usize;
|
||||||
@@ -1834,7 +1861,7 @@ impl InsightChatService {
|
|||||||
|
|
||||||
let mut stream = backend
|
let mut stream = backend
|
||||||
.chat()
|
.chat()
|
||||||
.chat_with_tools_stream(messages.clone(), tools.clone())
|
.chat_with_tools_stream(cx, messages.clone(), tools.clone())
|
||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
let mut final_message: Option<ChatMessage> = None;
|
let mut final_message: Option<ChatMessage> = None;
|
||||||
@@ -1928,7 +1955,7 @@ impl InsightChatService {
|
|||||||
let synthetic_idx = push_synthetic_final_prompt(messages);
|
let synthetic_idx = push_synthetic_final_prompt(messages);
|
||||||
let mut stream = backend
|
let mut stream = backend
|
||||||
.chat()
|
.chat()
|
||||||
.chat_with_tools_stream(messages.clone(), vec![])
|
.chat_with_tools_stream(cx, messages.clone(), vec![])
|
||||||
.await?;
|
.await?;
|
||||||
let mut final_message: Option<ChatMessage> = None;
|
let mut final_message: Option<ChatMessage> = None;
|
||||||
while let Some(ev) = stream.next().await {
|
while let Some(ev) = stream.next().await {
|
||||||
@@ -2308,6 +2335,7 @@ pub async fn run_streaming_agentic_loop_with_entry(
|
|||||||
active_persona: &str,
|
active_persona: &str,
|
||||||
max_iterations: usize,
|
max_iterations: usize,
|
||||||
entry: &Arc<TurnEntry>,
|
entry: &Arc<TurnEntry>,
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
) -> Result<AgenticLoopOutcome> {
|
) -> Result<AgenticLoopOutcome> {
|
||||||
let mut tool_calls_made = 0usize;
|
let mut tool_calls_made = 0usize;
|
||||||
let mut iterations_used = 0usize;
|
let mut iterations_used = 0usize;
|
||||||
@@ -2337,7 +2365,7 @@ pub async fn run_streaming_agentic_loop_with_entry(
|
|||||||
|
|
||||||
let mut stream = backend
|
let mut stream = backend
|
||||||
.chat()
|
.chat()
|
||||||
.chat_with_tools_stream(messages.clone(), tools.clone())
|
.chat_with_tools_stream(cx, messages.clone(), tools.clone())
|
||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
let mut final_message: Option<ChatMessage> = None;
|
let mut final_message: Option<ChatMessage> = None;
|
||||||
@@ -2420,7 +2448,7 @@ pub async fn run_streaming_agentic_loop_with_entry(
|
|||||||
let synthetic_idx = push_synthetic_final_prompt(messages);
|
let synthetic_idx = push_synthetic_final_prompt(messages);
|
||||||
let mut stream = backend
|
let mut stream = backend
|
||||||
.chat()
|
.chat()
|
||||||
.chat_with_tools_stream(messages.clone(), vec![])
|
.chat_with_tools_stream(cx, messages.clone(), vec![])
|
||||||
.await?;
|
.await?;
|
||||||
let mut final_message: Option<ChatMessage> = None;
|
let mut final_message: Option<ChatMessage> = None;
|
||||||
while let Some(ev) = stream.next().await {
|
while let Some(ev) = stream.next().await {
|
||||||
|
|||||||
+64
-36
@@ -646,7 +646,8 @@ impl InsightGenerator {
|
|||||||
// (`LLM_BACKEND` switch). Must match the backend that populated the
|
// (`LLM_BACKEND` switch). Must match the backend that populated the
|
||||||
// daily-summary embeddings or similarity search will be garbage.
|
// daily-summary embeddings or similarity search will be garbage.
|
||||||
let query_embedding =
|
let query_embedding =
|
||||||
crate::ai::embed_query(&self.ollama, self.llamacpp.as_deref(), &query).await?;
|
crate::ai::embed_query(&search_cx, &self.ollama, self.llamacpp.as_deref(), &query)
|
||||||
|
.await?;
|
||||||
|
|
||||||
// Search for similar daily summaries with time-based weighting
|
// Search for similar daily summaries with time-based weighting
|
||||||
// This prioritizes summaries temporally close to the query date
|
// This prioritizes summaries temporally close to the query date
|
||||||
@@ -712,7 +713,8 @@ impl InsightGenerator {
|
|||||||
// Must use the same backend that populated the daily-summary
|
// Must use the same backend that populated the daily-summary
|
||||||
// embeddings or similarity search is garbage (see embed_one docs).
|
// embeddings or similarity search is garbage (see embed_one docs).
|
||||||
let query_embedding =
|
let query_embedding =
|
||||||
crate::ai::embed_query(&self.ollama, self.llamacpp.as_deref(), query).await?;
|
crate::ai::embed_query(&search_cx, &self.ollama, self.llamacpp.as_deref(), query)
|
||||||
|
.await?;
|
||||||
|
|
||||||
let mut summary_dao = self
|
let mut summary_dao = self
|
||||||
.daily_summary_dao
|
.daily_summary_dao
|
||||||
@@ -798,7 +800,9 @@ impl InsightGenerator {
|
|||||||
let calendar_cx = parent_cx.with_span(span);
|
let calendar_cx = parent_cx.with_span(span);
|
||||||
|
|
||||||
let query_embedding = if let Some(loc) = location {
|
let query_embedding = if let Some(loc) = location {
|
||||||
match crate::ai::embed_query(&self.ollama, self.llamacpp.as_deref(), loc).await {
|
match crate::ai::embed_query(&calendar_cx, &self.ollama, self.llamacpp.as_deref(), loc)
|
||||||
|
.await
|
||||||
|
{
|
||||||
Ok(emb) => Some(emb),
|
Ok(emb) => Some(emb),
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
log::warn!("Failed to generate embedding for location '{}': {}", loc, e);
|
log::warn!("Failed to generate embedding for location '{}': {}", loc, e);
|
||||||
@@ -969,18 +973,23 @@ impl InsightGenerator {
|
|||||||
)
|
)
|
||||||
};
|
};
|
||||||
|
|
||||||
let query_embedding =
|
let query_embedding = match crate::ai::embed_query(
|
||||||
match crate::ai::embed_query(&self.ollama, self.llamacpp.as_deref(), &query_text).await
|
&search_cx,
|
||||||
{
|
&self.ollama,
|
||||||
Ok(emb) => emb,
|
self.llamacpp.as_deref(),
|
||||||
Err(e) => {
|
&query_text,
|
||||||
log::warn!("Failed to generate search embedding: {}", e);
|
)
|
||||||
search_cx.span().set_status(Status::Error {
|
.await
|
||||||
description: e.to_string().into(),
|
{
|
||||||
});
|
Ok(emb) => emb,
|
||||||
return Ok(None);
|
Err(e) => {
|
||||||
}
|
log::warn!("Failed to generate search embedding: {}", e);
|
||||||
};
|
search_cx.span().set_status(Status::Error {
|
||||||
|
description: e.to_string().into(),
|
||||||
|
});
|
||||||
|
return Ok(None);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
let searches = {
|
let searches = {
|
||||||
let mut dao = self
|
let mut dao = self
|
||||||
@@ -1254,7 +1263,10 @@ impl InsightGenerator {
|
|||||||
|
|
||||||
// Generate brief photo description for RAG enrichment (vision models only)
|
// Generate brief photo description for RAG enrichment (vision models only)
|
||||||
let photo_description: Option<String> = if let Some(ref img_b64) = image_base64 {
|
let photo_description: Option<String> = if let Some(ref img_b64) = image_base64 {
|
||||||
match ollama_client.generate_photo_description(img_b64).await {
|
match ollama_client
|
||||||
|
.generate_photo_description(&insight_cx, img_b64)
|
||||||
|
.await
|
||||||
|
{
|
||||||
Ok(desc) => {
|
Ok(desc) => {
|
||||||
log::info!("Photo description for RAG enrichment: {}", desc);
|
log::info!("Photo description for RAG enrichment: {}", desc);
|
||||||
Some(desc)
|
Some(desc)
|
||||||
@@ -1323,7 +1335,7 @@ impl InsightGenerator {
|
|||||||
if !immediate_messages.is_empty() {
|
if !immediate_messages.is_empty() {
|
||||||
// Step 2: Extract topics from immediate messages to enrich RAG query
|
// Step 2: Extract topics from immediate messages to enrich RAG query
|
||||||
let topics = self
|
let topics = self
|
||||||
.extract_topics_from_messages(&immediate_messages, &ollama_client)
|
.extract_topics_from_messages(&immediate_messages, &ollama_client, &insight_cx)
|
||||||
.await;
|
.await;
|
||||||
|
|
||||||
log::info!("Extracted topics for query enrichment: {:?}", topics);
|
log::info!("Extracted topics for query enrichment: {:?}", topics);
|
||||||
@@ -1379,6 +1391,7 @@ impl InsightGenerator {
|
|||||||
&immediate_messages,
|
&immediate_messages,
|
||||||
&ollama_client,
|
&ollama_client,
|
||||||
custom_system_prompt.as_deref(),
|
custom_system_prompt.as_deref(),
|
||||||
|
&insight_cx,
|
||||||
)
|
)
|
||||||
.await
|
.await
|
||||||
.unwrap_or_else(|| String::from("No immediate context"));
|
.unwrap_or_else(|| String::from("No immediate context"));
|
||||||
@@ -1388,6 +1401,7 @@ impl InsightGenerator {
|
|||||||
&historical_messages,
|
&historical_messages,
|
||||||
&ollama_client,
|
&ollama_client,
|
||||||
custom_system_prompt.as_deref(),
|
custom_system_prompt.as_deref(),
|
||||||
|
&insight_cx,
|
||||||
)
|
)
|
||||||
.await
|
.await
|
||||||
.unwrap_or_else(|| String::from("No historical context"));
|
.unwrap_or_else(|| String::from("No historical context"));
|
||||||
@@ -1406,6 +1420,7 @@ impl InsightGenerator {
|
|||||||
&immediate_messages,
|
&immediate_messages,
|
||||||
&ollama_client,
|
&ollama_client,
|
||||||
custom_system_prompt.as_deref(),
|
custom_system_prompt.as_deref(),
|
||||||
|
&insight_cx,
|
||||||
)
|
)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
@@ -1416,6 +1431,7 @@ impl InsightGenerator {
|
|||||||
&immediate_messages,
|
&immediate_messages,
|
||||||
&ollama_client,
|
&ollama_client,
|
||||||
custom_system_prompt.as_deref(),
|
custom_system_prompt.as_deref(),
|
||||||
|
&insight_cx,
|
||||||
)
|
)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
@@ -1441,6 +1457,7 @@ impl InsightGenerator {
|
|||||||
&rag_messages,
|
&rag_messages,
|
||||||
&ollama_client,
|
&ollama_client,
|
||||||
custom_system_prompt.as_deref(),
|
custom_system_prompt.as_deref(),
|
||||||
|
&insight_cx,
|
||||||
)
|
)
|
||||||
.await;
|
.await;
|
||||||
}
|
}
|
||||||
@@ -1473,7 +1490,7 @@ impl InsightGenerator {
|
|||||||
if !sms_messages.is_empty() {
|
if !sms_messages.is_empty() {
|
||||||
match self
|
match self
|
||||||
.sms_client
|
.sms_client
|
||||||
.summarize_context(&sms_messages, &ollama_client)
|
.summarize_context(&sms_messages, &ollama_client, &insight_cx)
|
||||||
.await
|
.await
|
||||||
{
|
{
|
||||||
Ok(summary) => {
|
Ok(summary) => {
|
||||||
@@ -1557,6 +1574,7 @@ impl InsightGenerator {
|
|||||||
// 10. Generate summary first, then derive title from the summary
|
// 10. Generate summary first, then derive title from the summary
|
||||||
let summary = ollama_client
|
let summary = ollama_client
|
||||||
.generate_photo_summary(
|
.generate_photo_summary(
|
||||||
|
&insight_cx,
|
||||||
date_taken,
|
date_taken,
|
||||||
location.as_deref(),
|
location.as_deref(),
|
||||||
contact.as_deref(),
|
contact.as_deref(),
|
||||||
@@ -1567,7 +1585,7 @@ impl InsightGenerator {
|
|||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
let title = ollama_client
|
let title = ollama_client
|
||||||
.generate_photo_title(&summary, custom_system_prompt.as_deref())
|
.generate_photo_title(&insight_cx, &summary, custom_system_prompt.as_deref())
|
||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
log::info!("Generated title: {}", title);
|
log::info!("Generated title: {}", title);
|
||||||
@@ -1634,6 +1652,7 @@ impl InsightGenerator {
|
|||||||
&self,
|
&self,
|
||||||
messages: &[crate::ai::SmsMessage],
|
messages: &[crate::ai::SmsMessage],
|
||||||
ollama: &OllamaClient,
|
ollama: &OllamaClient,
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
) -> Vec<String> {
|
) -> Vec<String> {
|
||||||
if messages.is_empty() {
|
if messages.is_empty() {
|
||||||
return Vec::new();
|
return Vec::new();
|
||||||
@@ -1669,7 +1688,7 @@ Return ONLY the comma-separated list, nothing else."#,
|
|||||||
);
|
);
|
||||||
|
|
||||||
match ollama
|
match ollama
|
||||||
.generate(&prompt, Some("You are an entity extraction assistant. Extract proper nouns, people, places, and domain-specific terms that provide context."))
|
.generate(cx, &prompt, Some("You are an entity extraction assistant. Extract proper nouns, people, places, and domain-specific terms that provide context."))
|
||||||
.await
|
.await
|
||||||
{
|
{
|
||||||
Ok(response) => {
|
Ok(response) => {
|
||||||
@@ -1704,6 +1723,7 @@ Return ONLY the comma-separated list, nothing else."#,
|
|||||||
messages: &[String],
|
messages: &[String],
|
||||||
ollama: &OllamaClient,
|
ollama: &OllamaClient,
|
||||||
custom_system: Option<&str>,
|
custom_system: Option<&str>,
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
) -> Option<String> {
|
) -> Option<String> {
|
||||||
if messages.is_empty() {
|
if messages.is_empty() {
|
||||||
return None;
|
return None;
|
||||||
@@ -1724,7 +1744,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
let system = custom_system
|
let system = custom_system
|
||||||
.unwrap_or("You are a context summarization assistant. Be concise and factual.");
|
.unwrap_or("You are a context summarization assistant. Be concise and factual.");
|
||||||
|
|
||||||
match ollama.generate(&prompt, Some(system)).await {
|
match ollama.generate(cx, &prompt, Some(system)).await {
|
||||||
Ok(summary) => Some(summary),
|
Ok(summary) => Some(summary),
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
log::warn!("Failed to summarize messages: {}", e);
|
log::warn!("Failed to summarize messages: {}", e);
|
||||||
@@ -1740,6 +1760,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
messages: &[crate::ai::SmsMessage],
|
messages: &[crate::ai::SmsMessage],
|
||||||
ollama: &OllamaClient,
|
ollama: &OllamaClient,
|
||||||
custom_system: Option<&str>,
|
custom_system: Option<&str>,
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
) -> Option<String> {
|
) -> Option<String> {
|
||||||
if messages.is_empty() {
|
if messages.is_empty() {
|
||||||
return None;
|
return None;
|
||||||
@@ -1785,7 +1806,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
"You are a context summarization assistant. Be detailed and factual, preserving important context.",
|
"You are a context summarization assistant. Be detailed and factual, preserving important context.",
|
||||||
);
|
);
|
||||||
|
|
||||||
match ollama.generate(&prompt, Some(system)).await {
|
match ollama.generate(cx, &prompt, Some(system)).await {
|
||||||
Ok(summary) => Some(summary),
|
Ok(summary) => Some(summary),
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
log::warn!("Failed to summarize immediate context: {}", e);
|
log::warn!("Failed to summarize immediate context: {}", e);
|
||||||
@@ -1829,7 +1850,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
"get_file_tags" => self.tool_get_file_tags(arguments, cx).await,
|
"get_file_tags" => self.tool_get_file_tags(arguments, cx).await,
|
||||||
"get_faces_in_photo" => self.tool_get_faces_in_photo(arguments, cx).await,
|
"get_faces_in_photo" => self.tool_get_faces_in_photo(arguments, cx).await,
|
||||||
"describe_photo" => {
|
"describe_photo" => {
|
||||||
self.tool_describe_photo(backend.local(), image_base64)
|
self.tool_describe_photo(backend.local(), image_base64, cx)
|
||||||
.await
|
.await
|
||||||
}
|
}
|
||||||
"reverse_geocode" => self.tool_reverse_geocode(arguments).await,
|
"reverse_geocode" => self.tool_reverse_geocode(arguments).await,
|
||||||
@@ -1880,7 +1901,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
&self,
|
&self,
|
||||||
args: &serde_json::Value,
|
args: &serde_json::Value,
|
||||||
local: &dyn LlmClient,
|
local: &dyn LlmClient,
|
||||||
_cx: &opentelemetry::Context,
|
cx: &opentelemetry::Context,
|
||||||
) -> String {
|
) -> String {
|
||||||
let query = match args.get("query").and_then(|v| v.as_str()) {
|
let query = match args.get("query").and_then(|v| v.as_str()) {
|
||||||
Some(q) => q.to_string(),
|
Some(q) => q.to_string(),
|
||||||
@@ -1944,7 +1965,10 @@ Return ONLY the summary, nothing else."#,
|
|||||||
};
|
};
|
||||||
|
|
||||||
let final_results = if rerank_enabled && results.len() > limit {
|
let final_results = if rerank_enabled && results.len() > limit {
|
||||||
match self.rerank_with_llm(&query, &results, limit, local).await {
|
match self
|
||||||
|
.rerank_with_llm(cx, &query, &results, limit, local)
|
||||||
|
.await
|
||||||
|
{
|
||||||
Ok(reordered) => reordered,
|
Ok(reordered) => reordered,
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
log::warn!("rerank failed, using vector order: {}", e);
|
log::warn!("rerank failed, using vector order: {}", e);
|
||||||
@@ -1967,6 +1991,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
/// tool_result. On parse failure we fall back to the input order.
|
/// tool_result. On parse failure we fall back to the input order.
|
||||||
async fn rerank_with_llm(
|
async fn rerank_with_llm(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
query: &str,
|
query: &str,
|
||||||
candidates: &[String],
|
candidates: &[String],
|
||||||
limit: usize,
|
limit: usize,
|
||||||
@@ -2010,7 +2035,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
let started = std::time::Instant::now();
|
let started = std::time::Instant::now();
|
||||||
let system =
|
let system =
|
||||||
Some("You are a terse relevance ranker. You output only numbers separated by commas.");
|
Some("You are a terse relevance ranker. You output only numbers separated by commas.");
|
||||||
let response = local.generate(&prompt, system, None).await?;
|
let response = local.generate(cx, &prompt, system, None).await?;
|
||||||
log::info!(
|
log::info!(
|
||||||
"rerank: finished in {} ms (prompt={} chars)",
|
"rerank: finished in {} ms (prompt={} chars)",
|
||||||
started.elapsed().as_millis(),
|
started.elapsed().as_millis(),
|
||||||
@@ -2620,10 +2645,11 @@ Return ONLY the summary, nothing else."#,
|
|||||||
&self,
|
&self,
|
||||||
local: &dyn LlmClient,
|
local: &dyn LlmClient,
|
||||||
image_base64: &Option<String>,
|
image_base64: &Option<String>,
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
) -> String {
|
) -> String {
|
||||||
log::info!("tool_describe_photo: generating visual description");
|
log::info!("tool_describe_photo: generating visual description");
|
||||||
match image_base64 {
|
match image_base64 {
|
||||||
Some(img) => match local.describe_image(img).await {
|
Some(img) => match local.describe_image(cx, img).await {
|
||||||
Ok(desc) => desc,
|
Ok(desc) => desc,
|
||||||
Err(e) => format!("Error describing photo: {}", e),
|
Err(e) => format!("Error describing photo: {}", e),
|
||||||
},
|
},
|
||||||
@@ -3031,6 +3057,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
// configured local backend.
|
// configured local backend.
|
||||||
let embed_text = format!("{} {}", name, description);
|
let embed_text = format!("{} {}", name, description);
|
||||||
let embedding: Option<Vec<u8>> = match crate::ai::embed_document(
|
let embedding: Option<Vec<u8>> = match crate::ai::embed_document(
|
||||||
|
cx,
|
||||||
&self.ollama,
|
&self.ollama,
|
||||||
self.llamacpp.as_deref(),
|
self.llamacpp.as_deref(),
|
||||||
&embed_text,
|
&embed_text,
|
||||||
@@ -4277,7 +4304,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
// the local backend so non-text work stays off OpenRouter.
|
// the local backend so non-text work stays off OpenRouter.
|
||||||
let inlined_visual_description: Option<String> = if !backend.images_inline {
|
let inlined_visual_description: Option<String> = if !backend.images_inline {
|
||||||
match image_base64.as_deref() {
|
match image_base64.as_deref() {
|
||||||
Some(b64) => match backend.local().describe_image(b64).await {
|
Some(b64) => match backend.local().describe_image(&insight_cx, b64).await {
|
||||||
Ok(desc) => {
|
Ok(desc) => {
|
||||||
log::info!("{}: vision describe succeeded ({} chars)", kind, desc.len());
|
log::info!("{}: vision describe succeeded ({} chars)", kind, desc.len());
|
||||||
Some(desc)
|
Some(desc)
|
||||||
@@ -4397,7 +4424,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
|
|
||||||
let (response, prompt_tokens, eval_tokens) = backend
|
let (response, prompt_tokens, eval_tokens) = backend
|
||||||
.chat()
|
.chat()
|
||||||
.chat_with_tools(messages.clone(), tools.clone())
|
.chat_with_tools(&loop_cx, messages.clone(), tools.clone())
|
||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
last_prompt_eval_count = prompt_tokens;
|
last_prompt_eval_count = prompt_tokens;
|
||||||
@@ -4466,7 +4493,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
)));
|
)));
|
||||||
let (final_response, prompt_tokens, eval_tokens) = backend
|
let (final_response, prompt_tokens, eval_tokens) = backend
|
||||||
.chat()
|
.chat()
|
||||||
.chat_with_tools(messages.clone(), tools.clone())
|
.chat_with_tools(&loop_cx, messages.clone(), tools.clone())
|
||||||
.await?;
|
.await?;
|
||||||
last_prompt_eval_count = prompt_tokens;
|
last_prompt_eval_count = prompt_tokens;
|
||||||
last_eval_count = eval_tokens;
|
last_eval_count = eval_tokens;
|
||||||
@@ -4499,7 +4526,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
}
|
}
|
||||||
let (resp2, pt2, et2) = backend
|
let (resp2, pt2, et2) = backend
|
||||||
.chat()
|
.chat()
|
||||||
.chat_with_tools(messages.clone(), vec![])
|
.chat_with_tools(&loop_cx, messages.clone(), vec![])
|
||||||
.await?;
|
.await?;
|
||||||
last_prompt_eval_count = pt2;
|
last_prompt_eval_count = pt2;
|
||||||
last_eval_count = et2;
|
last_eval_count = et2;
|
||||||
@@ -4637,6 +4664,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
max_iter: usize,
|
max_iter: usize,
|
||||||
) -> Result<String> {
|
) -> Result<String> {
|
||||||
|
let loop_cx = opentelemetry::Context::current();
|
||||||
let mut final_content = String::new();
|
let mut final_content = String::new();
|
||||||
|
|
||||||
for iteration in 0..max_iter {
|
for iteration in 0..max_iter {
|
||||||
@@ -4644,7 +4672,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
|
|
||||||
let (response, _prompt_tokens, _eval_tokens) = backend
|
let (response, _prompt_tokens, _eval_tokens) = backend
|
||||||
.chat()
|
.chat()
|
||||||
.chat_with_tools(messages.clone(), tools.clone())
|
.chat_with_tools(&loop_cx, messages.clone(), tools.clone())
|
||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
// Sanitize tool call arguments before pushing back into history.
|
// Sanitize tool call arguments before pushing back into history.
|
||||||
@@ -4686,7 +4714,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
"",
|
"",
|
||||||
0,
|
0,
|
||||||
"",
|
"",
|
||||||
&opentelemetry::Context::new(),
|
&loop_cx,
|
||||||
)
|
)
|
||||||
.await;
|
.await;
|
||||||
messages.push(ChatMessage::tool_result(result));
|
messages.push(ChatMessage::tool_result(result));
|
||||||
@@ -4710,7 +4738,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
));
|
));
|
||||||
let (final_response, _, _) = backend
|
let (final_response, _, _) = backend
|
||||||
.chat()
|
.chat()
|
||||||
.chat_with_tools(messages.clone(), tools.clone())
|
.chat_with_tools(&loop_cx, messages.clone(), tools.clone())
|
||||||
.await?;
|
.await?;
|
||||||
// Same bounded tool round as the insight loop: execute any last
|
// Same bounded tool round as the insight loop: execute any last
|
||||||
// tool call, then force a tool-less turn for the actual answer.
|
// tool call, then force a tool-less turn for the actual answer.
|
||||||
@@ -4728,14 +4756,14 @@ Return ONLY the summary, nothing else."#,
|
|||||||
"",
|
"",
|
||||||
0,
|
0,
|
||||||
"",
|
"",
|
||||||
&opentelemetry::Context::new(),
|
&loop_cx,
|
||||||
)
|
)
|
||||||
.await;
|
.await;
|
||||||
messages.push(ChatMessage::tool_result(result));
|
messages.push(ChatMessage::tool_result(result));
|
||||||
}
|
}
|
||||||
let (resp2, _, _) = backend
|
let (resp2, _, _) = backend
|
||||||
.chat()
|
.chat()
|
||||||
.chat_with_tools(messages.clone(), vec![])
|
.chat_with_tools(&loop_cx, messages.clone(), vec![])
|
||||||
.await?;
|
.await?;
|
||||||
final_content = resp2.content;
|
final_content = resp2.content;
|
||||||
} else {
|
} else {
|
||||||
|
|||||||
+111
-26
@@ -20,8 +20,11 @@
|
|||||||
// insight_generator / insight_chat.
|
// insight_generator / insight_chat.
|
||||||
#![allow(dead_code)]
|
#![allow(dead_code)]
|
||||||
|
|
||||||
use anyhow::{Context, Result, anyhow, bail};
|
use anyhow::Context as _;
|
||||||
|
use anyhow::{Result, anyhow, bail};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
|
use opentelemetry::trace::{Span, Status};
|
||||||
|
use opentelemetry::{Context, KeyValue};
|
||||||
use reqwest::Client;
|
use reqwest::Client;
|
||||||
use serde::Deserialize;
|
use serde::Deserialize;
|
||||||
use serde_json::{Value, json};
|
use serde_json::{Value, json};
|
||||||
@@ -486,6 +489,7 @@ impl LlamaCppClient {
|
|||||||
async fn chat_completion_with_model(
|
async fn chat_completion_with_model(
|
||||||
&self,
|
&self,
|
||||||
model: &str,
|
model: &str,
|
||||||
|
mut span: Option<&mut opentelemetry::global::BoxedSpan>,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
|
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
|
||||||
@@ -517,19 +521,31 @@ impl LlamaCppClient {
|
|||||||
// Wait for any TTS synthesis to release the GPU before the request
|
// Wait for any TTS synthesis to release the GPU before the request
|
||||||
// timeout starts (see ai::gpu).
|
// timeout starts (see ai::gpu).
|
||||||
let _gpu = crate::ai::gpu::llm_lease().await;
|
let _gpu = crate::ai::gpu::llm_lease().await;
|
||||||
let resp = self
|
let mut request = self.client.post(&url).json(&Value::Object(body));
|
||||||
.client
|
if let Some(s) = span.as_mut() {
|
||||||
.post(&url)
|
request = crate::otel::with_traceparent(request, s.span_context());
|
||||||
.json(&Value::Object(body))
|
}
|
||||||
.send()
|
let resp = match request.send().await {
|
||||||
.await
|
Ok(r) => r,
|
||||||
.with_context(|| format!("POST {} failed", url))?;
|
Err(e) => {
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
s.set_status(Status::error("llama-swap request failed"));
|
||||||
|
}
|
||||||
|
return Err(e).with_context(|| format!("POST {} failed", url));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
if !resp.status().is_success() {
|
if !resp.status().is_success() {
|
||||||
let status = resp.status();
|
let status = resp.status();
|
||||||
|
crate::otel::record_http_failure(
|
||||||
|
span.as_deref_mut(),
|
||||||
|
status.as_u16(),
|
||||||
|
&format!("llama-swap chat request failed: {status}"),
|
||||||
|
);
|
||||||
let body = resp.text().await.unwrap_or_default();
|
let body = resp.text().await.unwrap_or_default();
|
||||||
bail!("llama-swap chat request failed: {} — {}", status, body);
|
bail!("llama-swap chat request failed: {} — {}", status, body);
|
||||||
}
|
}
|
||||||
|
crate::otel::set_http_status(span.as_deref_mut(), resp.status().as_u16());
|
||||||
|
|
||||||
let parsed: Value = resp.json().await.context("parsing chat response")?;
|
let parsed: Value = resp.json().await.context("parsing chat response")?;
|
||||||
let choice = parsed
|
let choice = parsed
|
||||||
@@ -562,14 +578,35 @@ impl LlamaCppClient {
|
|||||||
|
|
||||||
log_timings(&parsed, prompt_tokens, completion_tokens);
|
log_timings(&parsed, prompt_tokens, completion_tokens);
|
||||||
|
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
if let Some(pt) = prompt_tokens {
|
||||||
|
s.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
|
||||||
|
}
|
||||||
|
if let Some(ct) = completion_tokens {
|
||||||
|
s.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
|
||||||
|
}
|
||||||
|
s.set_status(Status::Ok);
|
||||||
|
}
|
||||||
|
|
||||||
Ok((chat_msg, prompt_tokens, completion_tokens))
|
Ok((chat_msg, prompt_tokens, completion_tokens))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Start the `llm.request` span for an outgoing llama-swap call.
|
||||||
|
fn start_request_span(
|
||||||
|
&self,
|
||||||
|
cx: &Context,
|
||||||
|
operation: &str,
|
||||||
|
model: &str,
|
||||||
|
) -> opentelemetry::global::BoxedSpan {
|
||||||
|
crate::otel::start_llm_request_span(cx, "llamacpp", operation, model)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
#[async_trait]
|
#[async_trait]
|
||||||
impl LlmClient for LlamaCppClient {
|
impl LlmClient for LlamaCppClient {
|
||||||
async fn generate(
|
async fn generate(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
prompt: &str,
|
prompt: &str,
|
||||||
system: Option<&str>,
|
system: Option<&str>,
|
||||||
images: Option<Vec<String>>,
|
images: Option<Vec<String>>,
|
||||||
@@ -582,12 +619,15 @@ impl LlmClient for LlamaCppClient {
|
|||||||
user.images = images;
|
user.images = images;
|
||||||
messages.push(user);
|
messages.push(user);
|
||||||
|
|
||||||
let (reply, _, _) = self.chat_with_tools(messages, Vec::new()).await?;
|
// Delegates to chat_with_tools, which starts the llm.request span
|
||||||
|
// (operation=chat) — one span per HTTP request.
|
||||||
|
let (reply, _, _) = self.chat_with_tools(cx, messages, Vec::new()).await?;
|
||||||
Ok(reply.content)
|
Ok(reply.content)
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn chat_with_tools(
|
async fn chat_with_tools(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
|
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
|
||||||
@@ -597,15 +637,25 @@ impl LlmClient for LlamaCppClient {
|
|||||||
messages.len(),
|
messages.len(),
|
||||||
tools.len()
|
tools.len()
|
||||||
);
|
);
|
||||||
self.chat_completion_with_model(&self.primary_model.clone(), messages, tools)
|
let mut span = self.start_request_span(cx, "chat", &self.primary_model);
|
||||||
.await
|
self.chat_completion_with_model(
|
||||||
|
&self.primary_model.clone(),
|
||||||
|
Some(&mut span),
|
||||||
|
messages,
|
||||||
|
tools,
|
||||||
|
)
|
||||||
|
.await
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn chat_with_tools_stream(
|
async fn chat_with_tools_stream(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> {
|
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> {
|
||||||
|
// The span outlives this method — it moves into the stream
|
||||||
|
// generator and is finalized when the response drains.
|
||||||
|
let mut span = self.start_request_span(cx, "chat_stream", &self.primary_model);
|
||||||
let url = format!("{}/chat/completions", self.base_url);
|
let url = format!("{}/chat/completions", self.base_url);
|
||||||
let roles: Vec<&str> = messages.iter().map(|m| m.role.as_str()).collect();
|
let roles: Vec<&str> = messages.iter().map(|m| m.role.as_str()).collect();
|
||||||
log::debug!(
|
log::debug!(
|
||||||
@@ -639,19 +689,24 @@ impl LlmClient for LlamaCppClient {
|
|||||||
// timeout starts (see ai::gpu). The guard is moved into the stream
|
// timeout starts (see ai::gpu). The guard is moved into the stream
|
||||||
// below so the lease spans the whole generation, not just the send.
|
// below so the lease spans the whole generation, not just the send.
|
||||||
let gpu = crate::ai::gpu::llm_lease().await;
|
let gpu = crate::ai::gpu::llm_lease().await;
|
||||||
let resp = self
|
let mut request = self.client.post(&url).json(&Value::Object(body));
|
||||||
.client
|
request = crate::otel::with_traceparent(request, span.span_context());
|
||||||
.post(&url)
|
let resp = request
|
||||||
.json(&Value::Object(body))
|
|
||||||
.send()
|
.send()
|
||||||
.await
|
.await
|
||||||
.with_context(|| format!("POST {} failed", url))?;
|
.with_context(|| format!("POST {} failed", url))?;
|
||||||
|
|
||||||
if !resp.status().is_success() {
|
if !resp.status().is_success() {
|
||||||
let status = resp.status();
|
let status = resp.status();
|
||||||
|
crate::otel::record_http_failure(
|
||||||
|
Some(&mut span),
|
||||||
|
status.as_u16(),
|
||||||
|
&format!("llama-swap stream request failed: {status}"),
|
||||||
|
);
|
||||||
let body = resp.text().await.unwrap_or_default();
|
let body = resp.text().await.unwrap_or_default();
|
||||||
bail!("llama-swap stream request failed: {} — {}", status, body);
|
bail!("llama-swap stream request failed: {} — {}", status, body);
|
||||||
}
|
}
|
||||||
|
crate::otel::set_http_status(Some(&mut span), resp.status().as_u16());
|
||||||
|
|
||||||
let byte_stream = resp.bytes_stream();
|
let byte_stream = resp.bytes_stream();
|
||||||
let stream = async_stream::stream! {
|
let stream = async_stream::stream! {
|
||||||
@@ -760,6 +815,13 @@ impl LlmClient for LlamaCppClient {
|
|||||||
tool_calls,
|
tool_calls,
|
||||||
images: None,
|
images: None,
|
||||||
};
|
};
|
||||||
|
if let Some(pt) = prompt_tokens {
|
||||||
|
span.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
|
||||||
|
}
|
||||||
|
if let Some(ct) = completion_tokens {
|
||||||
|
span.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
|
||||||
|
}
|
||||||
|
span.set_status(Status::Ok);
|
||||||
yield Ok(LlmStreamEvent::Done {
|
yield Ok(LlmStreamEvent::Done {
|
||||||
message,
|
message,
|
||||||
prompt_eval_count: prompt_tokens,
|
prompt_eval_count: prompt_tokens,
|
||||||
@@ -770,7 +832,8 @@ impl LlmClient for LlamaCppClient {
|
|||||||
Ok(Box::pin(stream))
|
Ok(Box::pin(stream))
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
|
async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
|
||||||
|
let mut span = self.start_request_span(cx, "embeddings", &self.embedding_model);
|
||||||
// Deliberately NO GPU lease: the embed slot sits in llama-swap's
|
// Deliberately NO GPU lease: the embed slot sits in llama-swap's
|
||||||
// always-resident group and never participates in a model swap, so
|
// always-resident group and never participates in a model swap, so
|
||||||
// leasing here would only stall searches behind a queued synthesis.
|
// leasing here would only stall searches behind a queued synthesis.
|
||||||
@@ -780,19 +843,27 @@ impl LlmClient for LlamaCppClient {
|
|||||||
"input": texts,
|
"input": texts,
|
||||||
});
|
});
|
||||||
|
|
||||||
let resp = self
|
let mut request = self.client.post(&url).json(&body);
|
||||||
.client
|
request = crate::otel::with_traceparent(request, span.span_context());
|
||||||
.post(&url)
|
let resp = match request.send().await {
|
||||||
.json(&body)
|
Ok(r) => r,
|
||||||
.send()
|
Err(e) => {
|
||||||
.await
|
span.set_status(Status::error("llama-swap embedding request failed"));
|
||||||
.with_context(|| format!("POST {} failed", url))?;
|
return Err(e).with_context(|| format!("POST {} failed", url));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
if !resp.status().is_success() {
|
if !resp.status().is_success() {
|
||||||
let status = resp.status();
|
let status = resp.status();
|
||||||
|
crate::otel::record_http_failure(
|
||||||
|
Some(&mut span),
|
||||||
|
status.as_u16(),
|
||||||
|
&format!("llama-swap embedding request failed: {status}"),
|
||||||
|
);
|
||||||
let body = resp.text().await.unwrap_or_default();
|
let body = resp.text().await.unwrap_or_default();
|
||||||
bail!("llama-swap embedding request failed: {} — {}", status, body);
|
bail!("llama-swap embedding request failed: {} — {}", status, body);
|
||||||
}
|
}
|
||||||
|
crate::otel::set_http_status(Some(&mut span), resp.status().as_u16());
|
||||||
|
|
||||||
#[derive(Deserialize)]
|
#[derive(Deserialize)]
|
||||||
struct EmbedResponse {
|
struct EmbedResponse {
|
||||||
@@ -803,11 +874,20 @@ impl LlmClient for LlamaCppClient {
|
|||||||
embedding: Vec<f32>,
|
embedding: Vec<f32>,
|
||||||
}
|
}
|
||||||
|
|
||||||
let parsed: EmbedResponse = resp.json().await.context("parsing embed response")?;
|
let parsed: EmbedResponse = match resp.json().await {
|
||||||
|
Ok(p) => p,
|
||||||
|
Err(e) => {
|
||||||
|
span.set_status(Status::error("parsing embed response failed"));
|
||||||
|
return Err(e).context("parsing embed response");
|
||||||
|
}
|
||||||
|
};
|
||||||
|
span.set_status(Status::Ok);
|
||||||
Ok(parsed.data.into_iter().map(|i| i.embedding).collect())
|
Ok(parsed.data.into_iter().map(|i| i.embedding).collect())
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn describe_image(&self, image_base64: &str) -> Result<String> {
|
async fn describe_image(&self, cx: &Context, image_base64: &str) -> Result<String> {
|
||||||
|
// Own span with operation=describe_image — routes through the vision slot.
|
||||||
|
let mut span = self.start_request_span(cx, "describe_image", &self.vision_model);
|
||||||
let prompt = "Briefly describe what you see in this image in 1-2 sentences. \
|
let prompt = "Briefly describe what you see in this image in 1-2 sentences. \
|
||||||
Focus on the people, location, and activity.";
|
Focus on the people, location, and activity.";
|
||||||
let system = "You are a scene description assistant. Be concise and factual.";
|
let system = "You are a scene description assistant. Be concise and factual.";
|
||||||
@@ -823,7 +903,12 @@ impl LlmClient for LlamaCppClient {
|
|||||||
];
|
];
|
||||||
|
|
||||||
let (reply, _, _) = self
|
let (reply, _, _) = self
|
||||||
.chat_completion_with_model(&self.vision_model.clone(), messages, Vec::new())
|
.chat_completion_with_model(
|
||||||
|
&self.vision_model.clone(),
|
||||||
|
Some(&mut span),
|
||||||
|
messages,
|
||||||
|
Vec::new(),
|
||||||
|
)
|
||||||
.await?;
|
.await?;
|
||||||
Ok(reply.content)
|
Ok(reply.content)
|
||||||
}
|
}
|
||||||
|
|||||||
+12
-2
@@ -1,6 +1,7 @@
|
|||||||
use anyhow::Result;
|
use anyhow::Result;
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
use futures::stream::BoxStream;
|
use futures::stream::BoxStream;
|
||||||
|
use opentelemetry::Context;
|
||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
|
|
||||||
/// Provider-agnostic surface for LLM backends (Ollama, OpenRouter, …).
|
/// Provider-agnostic surface for LLM backends (Ollama, OpenRouter, …).
|
||||||
@@ -10,6 +11,12 @@ use serde::{Deserialize, Serialize};
|
|||||||
/// when a provider requires it (OpenAI-compatible APIs do), and `images`
|
/// when a provider requires it (OpenAI-compatible APIs do), and `images`
|
||||||
/// stays as base64 strings here and is rewritten into content-parts where
|
/// stays as base64 strings here and is rewritten into content-parts where
|
||||||
/// needed.
|
/// needed.
|
||||||
|
///
|
||||||
|
/// Every method that performs an HTTP call takes the caller's tracing
|
||||||
|
/// `Context` as its first parameter so the impl can start an `llm.request`
|
||||||
|
/// span parented under the caller's feature span — the codebase threads
|
||||||
|
/// contexts explicitly rather than relying on ambient `Context::current()`
|
||||||
|
/// at deep call sites.
|
||||||
// First consumer lands in a later PR (OpenRouter impl + hybrid mode routing).
|
// First consumer lands in a later PR (OpenRouter impl + hybrid mode routing).
|
||||||
#[allow(dead_code)]
|
#[allow(dead_code)]
|
||||||
#[async_trait]
|
#[async_trait]
|
||||||
@@ -18,6 +25,7 @@ pub trait LlmClient: Send + Sync {
|
|||||||
/// base64 images (ignored by providers without vision support).
|
/// base64 images (ignored by providers without vision support).
|
||||||
async fn generate(
|
async fn generate(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
prompt: &str,
|
prompt: &str,
|
||||||
system: Option<&str>,
|
system: Option<&str>,
|
||||||
images: Option<Vec<String>>,
|
images: Option<Vec<String>>,
|
||||||
@@ -27,6 +35,7 @@ pub trait LlmClient: Send + Sync {
|
|||||||
/// (which may contain tool_calls) plus optional prompt/eval token counts.
|
/// (which may contain tool_calls) plus optional prompt/eval token counts.
|
||||||
async fn chat_with_tools(
|
async fn chat_with_tools(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
) -> Result<(ChatMessage, Option<i32>, Option<i32>)>;
|
) -> Result<(ChatMessage, Option<i32>, Option<i32>)>;
|
||||||
@@ -39,16 +48,17 @@ pub trait LlmClient: Send + Sync {
|
|||||||
/// as one `Done` event.
|
/// as one `Done` event.
|
||||||
async fn chat_with_tools_stream(
|
async fn chat_with_tools_stream(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>>;
|
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>>;
|
||||||
|
|
||||||
/// Batch embedding generation. Dimensionality is provider/model specific.
|
/// Batch embedding generation. Dimensionality is provider/model specific.
|
||||||
async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>>;
|
async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>>;
|
||||||
|
|
||||||
/// One-shot vision description of an image. Used to convert images into
|
/// One-shot vision description of an image. Used to convert images into
|
||||||
/// plain text for the hybrid-mode conversation flow.
|
/// plain text for the hybrid-mode conversation flow.
|
||||||
async fn describe_image(&self, image_base64: &str) -> Result<String>;
|
async fn describe_image(&self, cx: &Context, image_base64: &str) -> Result<String>;
|
||||||
|
|
||||||
/// Enumerate available models with their capabilities.
|
/// Enumerate available models with their capabilities.
|
||||||
async fn list_models(&self) -> Result<Vec<ModelCapabilities>>;
|
async fn list_models(&self) -> Result<Vec<ModelCapabilities>>;
|
||||||
|
|||||||
+25
-4
@@ -47,12 +47,24 @@ impl LocalLlm {
|
|||||||
/// pick query vs document — retrieval models treat the two sides
|
/// pick query vs document — retrieval models treat the two sides
|
||||||
/// differently and an unmarked embed invites prefix-mismatch bugs.
|
/// differently and an unmarked embed invites prefix-mismatch bugs.
|
||||||
pub async fn embed_query(&self, text: &str) -> Result<Vec<f32>> {
|
pub async fn embed_query(&self, text: &str) -> Result<Vec<f32>> {
|
||||||
super::embed_query(&self.ollama, self.llamacpp.as_deref(), text).await
|
super::embed_query(
|
||||||
|
&opentelemetry::Context::new(),
|
||||||
|
&self.ollama,
|
||||||
|
self.llamacpp.as_deref(),
|
||||||
|
text,
|
||||||
|
)
|
||||||
|
.await
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Embed corpus text (applies `EMBED_DOCUMENT_PREFIX`).
|
/// Embed corpus text (applies `EMBED_DOCUMENT_PREFIX`).
|
||||||
pub async fn embed_document(&self, text: &str) -> Result<Vec<f32>> {
|
pub async fn embed_document(&self, text: &str) -> Result<Vec<f32>> {
|
||||||
super::embed_document(&self.ollama, self.llamacpp.as_deref(), text).await
|
super::embed_document(
|
||||||
|
&opentelemetry::Context::new(),
|
||||||
|
&self.ollama,
|
||||||
|
self.llamacpp.as_deref(),
|
||||||
|
text,
|
||||||
|
)
|
||||||
|
.await
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Single-shot local text generation via the `LLM_BACKEND`-selected
|
/// Single-shot local text generation via the `LLM_BACKEND`-selected
|
||||||
@@ -60,14 +72,23 @@ impl LocalLlm {
|
|||||||
pub async fn generate(&self, prompt: &str, system: Option<&str>) -> Result<String> {
|
pub async fn generate(&self, prompt: &str, system: Option<&str>) -> Result<String> {
|
||||||
if super::local_backend_is_llamacpp() {
|
if super::local_backend_is_llamacpp() {
|
||||||
if let Some(lc) = self.llamacpp.as_deref() {
|
if let Some(lc) = self.llamacpp.as_deref() {
|
||||||
return <LlamaCppClient as LlmClient>::generate(lc, prompt, system, None).await;
|
return <LlamaCppClient as LlmClient>::generate(
|
||||||
|
lc,
|
||||||
|
&opentelemetry::Context::new(),
|
||||||
|
prompt,
|
||||||
|
system,
|
||||||
|
None,
|
||||||
|
)
|
||||||
|
.await;
|
||||||
}
|
}
|
||||||
anyhow::bail!(
|
anyhow::bail!(
|
||||||
"LLM_BACKEND=llamacpp but LlamaCppClient is unconfigured — \
|
"LLM_BACKEND=llamacpp but LlamaCppClient is unconfigured — \
|
||||||
set LLAMA_SWAP_URL or switch to LLM_BACKEND=ollama"
|
set LLAMA_SWAP_URL or switch to LLM_BACKEND=ollama"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
self.ollama.generate(prompt, system).await
|
self.ollama
|
||||||
|
.generate(&opentelemetry::Context::new(), prompt, system)
|
||||||
|
.await
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Label identifying which backend + model produces embeddings right
|
/// Label identifying which backend + model produces embeddings right
|
||||||
|
|||||||
+9
-5
@@ -115,24 +115,26 @@ fn embed_prefix(key: &str) -> String {
|
|||||||
/// `Instruct: <task>\nQuery: `. Must pair with the document prefix the
|
/// `Instruct: <task>\nQuery: `. Must pair with the document prefix the
|
||||||
/// corpus was embedded with or similarity degrades.
|
/// corpus was embedded with or similarity degrades.
|
||||||
pub async fn embed_query(
|
pub async fn embed_query(
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
ollama: &OllamaClient,
|
ollama: &OllamaClient,
|
||||||
llamacpp: Option<&LlamaCppClient>,
|
llamacpp: Option<&LlamaCppClient>,
|
||||||
text: &str,
|
text: &str,
|
||||||
) -> anyhow::Result<Vec<f32>> {
|
) -> anyhow::Result<Vec<f32>> {
|
||||||
let prefixed = format!("{}{}", embed_prefix("EMBED_QUERY_PREFIX"), text);
|
let prefixed = format!("{}{}", embed_prefix("EMBED_QUERY_PREFIX"), text);
|
||||||
embed_one(ollama, llamacpp, &prefixed).await
|
embed_one(cx, ollama, llamacpp, &prefixed).await
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Embed corpus text (the stored side of retrieval). Applies
|
/// Embed corpus text (the stored side of retrieval). Applies
|
||||||
/// `EMBED_DOCUMENT_PREFIX` (default empty; nomic v1.5 wants
|
/// `EMBED_DOCUMENT_PREFIX` (default empty; nomic v1.5 wants
|
||||||
/// `search_document: `, Qwen3-Embedding wants none).
|
/// `search_document: `, Qwen3-Embedding wants none).
|
||||||
pub async fn embed_document(
|
pub async fn embed_document(
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
ollama: &OllamaClient,
|
ollama: &OllamaClient,
|
||||||
llamacpp: Option<&LlamaCppClient>,
|
llamacpp: Option<&LlamaCppClient>,
|
||||||
text: &str,
|
text: &str,
|
||||||
) -> anyhow::Result<Vec<f32>> {
|
) -> anyhow::Result<Vec<f32>> {
|
||||||
let prefixed = format!("{}{}", embed_prefix("EMBED_DOCUMENT_PREFIX"), text);
|
let prefixed = format!("{}{}", embed_prefix("EMBED_DOCUMENT_PREFIX"), text);
|
||||||
embed_one(ollama, llamacpp, &prefixed).await
|
embed_one(cx, ollama, llamacpp, &prefixed).await
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Embed a batch of strings via the configured local backend. Routes
|
/// Embed a batch of strings via the configured local backend. Routes
|
||||||
@@ -140,30 +142,32 @@ pub async fn embed_document(
|
|||||||
/// configured), else Ollama. See [`local_backend_is_llamacpp`] for the
|
/// configured), else Ollama. See [`local_backend_is_llamacpp`] for the
|
||||||
/// rationale on consistency.
|
/// rationale on consistency.
|
||||||
pub async fn embed_many(
|
pub async fn embed_many(
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
ollama: &OllamaClient,
|
ollama: &OllamaClient,
|
||||||
llamacpp: Option<&LlamaCppClient>,
|
llamacpp: Option<&LlamaCppClient>,
|
||||||
texts: &[&str],
|
texts: &[&str],
|
||||||
) -> anyhow::Result<Vec<Vec<f32>>> {
|
) -> anyhow::Result<Vec<Vec<f32>>> {
|
||||||
if local_backend_is_llamacpp() {
|
if local_backend_is_llamacpp() {
|
||||||
if let Some(lc) = llamacpp {
|
if let Some(lc) = llamacpp {
|
||||||
return <LlamaCppClient as LlmClient>::generate_embeddings(lc, texts).await;
|
return <LlamaCppClient as LlmClient>::generate_embeddings(lc, cx, texts).await;
|
||||||
}
|
}
|
||||||
anyhow::bail!(
|
anyhow::bail!(
|
||||||
"LLM_BACKEND=llamacpp but LlamaCppClient is unconfigured — \
|
"LLM_BACKEND=llamacpp but LlamaCppClient is unconfigured — \
|
||||||
set LLAMA_SWAP_URL or switch to LLM_BACKEND=ollama"
|
set LLAMA_SWAP_URL or switch to LLM_BACKEND=ollama"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
ollama.generate_embeddings(texts).await
|
ollama.generate_embeddings(cx, texts).await
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Embed one string via the configured local backend. Single-text
|
/// Embed one string via the configured local backend. Single-text
|
||||||
/// convenience over [`embed_many`].
|
/// convenience over [`embed_many`].
|
||||||
pub async fn embed_one(
|
pub async fn embed_one(
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
ollama: &OllamaClient,
|
ollama: &OllamaClient,
|
||||||
llamacpp: Option<&LlamaCppClient>,
|
llamacpp: Option<&LlamaCppClient>,
|
||||||
text: &str,
|
text: &str,
|
||||||
) -> anyhow::Result<Vec<f32>> {
|
) -> anyhow::Result<Vec<f32>> {
|
||||||
let mut vecs = embed_many(ollama, llamacpp, &[text]).await?;
|
let mut vecs = embed_many(cx, ollama, llamacpp, &[text]).await?;
|
||||||
vecs.pop()
|
vecs.pop()
|
||||||
.ok_or_else(|| anyhow::anyhow!("embedding backend returned no embeddings"))
|
.ok_or_else(|| anyhow::anyhow!("embedding backend returned no embeddings"))
|
||||||
}
|
}
|
||||||
|
|||||||
+4
-1
@@ -259,10 +259,13 @@ pub async fn translate_nl_query(
|
|||||||
nl: &str,
|
nl: &str,
|
||||||
tag_vocab: &[(i32, String)],
|
tag_vocab: &[(i32, String)],
|
||||||
today: chrono::NaiveDate,
|
today: chrono::NaiveDate,
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
) -> Result<StructuredQuery> {
|
) -> Result<StructuredQuery> {
|
||||||
let system = build_system_prompt(tag_vocab, today);
|
let system = build_system_prompt(tag_vocab, today);
|
||||||
let messages = vec![ChatMessage::system(system), ChatMessage::user(nl)];
|
let messages = vec![ChatMessage::system(system), ChatMessage::user(nl)];
|
||||||
let (msg, _, _) = client.chat_with_tools(messages, Vec::<Tool>::new()).await?;
|
let (msg, _, _) = client
|
||||||
|
.chat_with_tools(cx, messages, Vec::<Tool>::new())
|
||||||
|
.await?;
|
||||||
parse_response(&msg.content, tag_vocab)
|
parse_response(&msg.content, tag_vocab)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+238
-55
@@ -1,6 +1,9 @@
|
|||||||
use anyhow::{Context, Result};
|
use anyhow::Context as _;
|
||||||
|
use anyhow::Result;
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
use chrono::NaiveDate;
|
use chrono::NaiveDate;
|
||||||
|
use opentelemetry::trace::{Span, Status};
|
||||||
|
use opentelemetry::{Context, KeyValue};
|
||||||
use reqwest::Client;
|
use reqwest::Client;
|
||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
@@ -365,6 +368,7 @@ impl OllamaClient {
|
|||||||
|
|
||||||
async fn try_generate(
|
async fn try_generate(
|
||||||
&self,
|
&self,
|
||||||
|
mut span: Option<&mut opentelemetry::global::BoxedSpan>,
|
||||||
url: &str,
|
url: &str,
|
||||||
model: &str,
|
model: &str,
|
||||||
prompt: &str,
|
prompt: &str,
|
||||||
@@ -382,15 +386,30 @@ impl OllamaClient {
|
|||||||
think,
|
think,
|
||||||
};
|
};
|
||||||
|
|
||||||
let response = self
|
let mut request_builder = self
|
||||||
.client
|
.client
|
||||||
.post(format!("{}/api/generate", url))
|
.post(format!("{}/api/generate", url))
|
||||||
.json(&request)
|
.json(&request);
|
||||||
.send()
|
if let Some(s) = span.as_mut() {
|
||||||
.await?;
|
request_builder = crate::otel::with_traceparent(request_builder, s.span_context());
|
||||||
|
}
|
||||||
|
let response = match request_builder.send().await {
|
||||||
|
Ok(r) => r,
|
||||||
|
Err(e) => {
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
s.set_status(Status::error("Ollama request failed"));
|
||||||
|
}
|
||||||
|
return Err(e.into());
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
if !response.status().is_success() {
|
if !response.status().is_success() {
|
||||||
let status = response.status();
|
let status = response.status();
|
||||||
|
crate::otel::record_http_failure(
|
||||||
|
span.as_deref_mut(),
|
||||||
|
status.as_u16(),
|
||||||
|
&format!("Ollama request failed: {status}"),
|
||||||
|
);
|
||||||
let error_body = response.text().await.unwrap_or_default();
|
let error_body = response.text().await.unwrap_or_default();
|
||||||
return Err(anyhow::anyhow!(
|
return Err(anyhow::anyhow!(
|
||||||
"Ollama request failed: {} - {}",
|
"Ollama request failed: {} - {}",
|
||||||
@@ -398,8 +417,26 @@ impl OllamaClient {
|
|||||||
error_body
|
error_body
|
||||||
));
|
));
|
||||||
}
|
}
|
||||||
|
crate::otel::set_http_status(span.as_deref_mut(), response.status().as_u16());
|
||||||
|
|
||||||
let result: OllamaResponse = response.json().await?;
|
let result: OllamaResponse = match response.json().await {
|
||||||
|
Ok(r) => r,
|
||||||
|
Err(e) => {
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
s.set_status(Status::error("parsing Ollama response failed"));
|
||||||
|
}
|
||||||
|
return Err(e.into());
|
||||||
|
}
|
||||||
|
};
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
if let Some(pt) = result.prompt_eval_count {
|
||||||
|
s.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
|
||||||
|
}
|
||||||
|
if let Some(ct) = result.eval_count {
|
||||||
|
s.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
|
||||||
|
}
|
||||||
|
s.set_status(Status::Ok);
|
||||||
|
}
|
||||||
log_chat_metrics(
|
log_chat_metrics(
|
||||||
result.prompt_eval_count,
|
result.prompt_eval_count,
|
||||||
result.prompt_eval_duration,
|
result.prompt_eval_duration,
|
||||||
@@ -409,32 +446,55 @@ impl OllamaClient {
|
|||||||
Ok(result.response)
|
Ok(result.response)
|
||||||
}
|
}
|
||||||
|
|
||||||
pub async fn generate(&self, prompt: &str, system: Option<&str>) -> Result<String> {
|
/// Start the `llm.request` span for an outgoing Ollama call.
|
||||||
self.generate_with_images(prompt, system, None).await
|
fn start_request_span(
|
||||||
|
&self,
|
||||||
|
cx: &Context,
|
||||||
|
operation: &str,
|
||||||
|
model: &str,
|
||||||
|
) -> opentelemetry::global::BoxedSpan {
|
||||||
|
crate::otel::start_llm_request_span(cx, "ollama", operation, model)
|
||||||
|
}
|
||||||
|
|
||||||
|
pub async fn generate(
|
||||||
|
&self,
|
||||||
|
cx: &Context,
|
||||||
|
prompt: &str,
|
||||||
|
system: Option<&str>,
|
||||||
|
) -> Result<String> {
|
||||||
|
self.generate_with_images(cx, prompt, system, None).await
|
||||||
}
|
}
|
||||||
|
|
||||||
#[allow(dead_code)]
|
#[allow(dead_code)]
|
||||||
pub async fn generate_no_think(&self, prompt: &str, system: Option<&str>) -> Result<String> {
|
pub async fn generate_no_think(
|
||||||
self.generate_with_options(prompt, system, None, Some(false))
|
&self,
|
||||||
|
cx: &Context,
|
||||||
|
prompt: &str,
|
||||||
|
system: Option<&str>,
|
||||||
|
) -> Result<String> {
|
||||||
|
self.generate_with_options(cx, prompt, system, None, Some(false), "generate")
|
||||||
.await
|
.await
|
||||||
}
|
}
|
||||||
|
|
||||||
pub async fn generate_with_images(
|
pub async fn generate_with_images(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
prompt: &str,
|
prompt: &str,
|
||||||
system: Option<&str>,
|
system: Option<&str>,
|
||||||
images: Option<Vec<String>>,
|
images: Option<Vec<String>>,
|
||||||
) -> Result<String> {
|
) -> Result<String> {
|
||||||
self.generate_with_options(prompt, system, images, None)
|
self.generate_with_options(cx, prompt, system, images, None, "generate")
|
||||||
.await
|
.await
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn generate_with_options(
|
async fn generate_with_options(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
prompt: &str,
|
prompt: &str,
|
||||||
system: Option<&str>,
|
system: Option<&str>,
|
||||||
images: Option<Vec<String>>,
|
images: Option<Vec<String>>,
|
||||||
think: Option<bool>,
|
think: Option<bool>,
|
||||||
|
operation: &str,
|
||||||
) -> Result<String> {
|
) -> Result<String> {
|
||||||
log::debug!("=== Ollama Request ===");
|
log::debug!("=== Ollama Request ===");
|
||||||
log::debug!("Primary model: {}", self.primary_model);
|
log::debug!("Primary model: {}", self.primary_model);
|
||||||
@@ -447,6 +507,10 @@ impl OllamaClient {
|
|||||||
}
|
}
|
||||||
log::debug!("=====================");
|
log::debug!("=====================");
|
||||||
|
|
||||||
|
// Span for the whole generate attempt; the model attribute
|
||||||
|
// reflects whichever server actually answered.
|
||||||
|
let mut span = self.start_request_span(cx, operation, &self.primary_model);
|
||||||
|
|
||||||
// Try primary server first with primary model
|
// Try primary server first with primary model
|
||||||
log::info!(
|
log::info!(
|
||||||
"Attempting to generate with primary server: {} (model: {})",
|
"Attempting to generate with primary server: {} (model: {})",
|
||||||
@@ -455,6 +519,7 @@ impl OllamaClient {
|
|||||||
);
|
);
|
||||||
let primary_result = self
|
let primary_result = self
|
||||||
.try_generate(
|
.try_generate(
|
||||||
|
Some(&mut span),
|
||||||
&self.primary_url,
|
&self.primary_url,
|
||||||
&self.primary_model,
|
&self.primary_model,
|
||||||
prompt,
|
prompt,
|
||||||
@@ -483,8 +548,10 @@ impl OllamaClient {
|
|||||||
fallback_url,
|
fallback_url,
|
||||||
fallback_model
|
fallback_model
|
||||||
);
|
);
|
||||||
|
span.set_attribute(KeyValue::new("llm.model", fallback_model.to_string()));
|
||||||
match self
|
match self
|
||||||
.try_generate(
|
.try_generate(
|
||||||
|
Some(&mut span),
|
||||||
fallback_url,
|
fallback_url,
|
||||||
fallback_model,
|
fallback_model,
|
||||||
prompt,
|
prompt,
|
||||||
@@ -531,6 +598,7 @@ impl OllamaClient {
|
|||||||
/// Generate a title for a single photo based on its generated summary
|
/// Generate a title for a single photo based on its generated summary
|
||||||
pub async fn generate_photo_title(
|
pub async fn generate_photo_title(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
summary: &str,
|
summary: &str,
|
||||||
custom_system: Option<&str>,
|
custom_system: Option<&str>,
|
||||||
) -> Result<String> {
|
) -> Result<String> {
|
||||||
@@ -546,7 +614,7 @@ Capture the key moment or theme. Return ONLY the title, nothing else."#,
|
|||||||
let system = custom_system.unwrap_or("You are my long term memory assistant. Use only the information provided. Do not invent details.");
|
let system = custom_system.unwrap_or("You are my long term memory assistant. Use only the information provided. Do not invent details.");
|
||||||
|
|
||||||
let title = self
|
let title = self
|
||||||
.generate_with_images(&prompt, Some(system), None)
|
.generate_with_images(cx, &prompt, Some(system), None)
|
||||||
.await?;
|
.await?;
|
||||||
// Models decorate despite "Return ONLY the title": quotes, bold
|
// Models decorate despite "Return ONLY the title": quotes, bold
|
||||||
// markers, sometimes a "Title:" label.
|
// markers, sometimes a "Title:" label.
|
||||||
@@ -563,6 +631,7 @@ Capture the key moment or theme. Return ONLY the title, nothing else."#,
|
|||||||
/// Generate a summary for a single photo based on its context
|
/// Generate a summary for a single photo based on its context
|
||||||
pub async fn generate_photo_summary(
|
pub async fn generate_photo_summary(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
date: NaiveDate,
|
date: NaiveDate,
|
||||||
location: Option<&str>,
|
location: Option<&str>,
|
||||||
contact: Option<&str>,
|
contact: Option<&str>,
|
||||||
@@ -645,21 +714,32 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
let system = custom_system.unwrap_or("You are a memory refreshing assistant who is able to provide insights through analyzing past conversations. Use only the information provided. Do not invent details.");
|
let system = custom_system.unwrap_or("You are a memory refreshing assistant who is able to provide insights through analyzing past conversations. Use only the information provided. Do not invent details.");
|
||||||
|
|
||||||
let images = image_base64.map(|img| vec![img]);
|
let images = image_base64.map(|img| vec![img]);
|
||||||
self.generate_with_images(&prompt, Some(system), images)
|
self.generate_with_images(cx, &prompt, Some(system), images)
|
||||||
.await
|
.await
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Generate a brief visual description of a photo for use in RAG query enrichment.
|
/// Generate a brief visual description of a photo for use in RAG query enrichment.
|
||||||
/// Returns 1-2 sentences describing people, location, and activity visible in the image.
|
/// Returns 1-2 sentences describing people, location, and activity visible in the image.
|
||||||
/// Only called when the model has vision capabilities.
|
/// Only called when the model has vision capabilities.
|
||||||
pub async fn generate_photo_description(&self, image_base64: &str) -> Result<String> {
|
pub async fn generate_photo_description(
|
||||||
|
&self,
|
||||||
|
cx: &Context,
|
||||||
|
image_base64: &str,
|
||||||
|
) -> Result<String> {
|
||||||
let prompt = "Briefly describe what you see in this image in 1-2 sentences. \
|
let prompt = "Briefly describe what you see in this image in 1-2 sentences. \
|
||||||
Focus on the people, location, and activity.";
|
Focus on the people, location, and activity.";
|
||||||
let system = "You are a scene description assistant. Be concise and factual.";
|
let system = "You are a scene description assistant. Be concise and factual.";
|
||||||
let images = vec![image_base64.to_string()];
|
let images = vec![image_base64.to_string()];
|
||||||
|
|
||||||
let description = self
|
let description = self
|
||||||
.generate_with_images(prompt, Some(system), Some(images))
|
.generate_with_options(
|
||||||
|
cx,
|
||||||
|
prompt,
|
||||||
|
Some(system),
|
||||||
|
Some(images),
|
||||||
|
None,
|
||||||
|
"describe_image",
|
||||||
|
)
|
||||||
.await?;
|
.await?;
|
||||||
|
|
||||||
Ok(description.trim().to_string())
|
Ok(description.trim().to_string())
|
||||||
@@ -672,9 +752,11 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
/// iteration of the agent loop.
|
/// iteration of the agent loop.
|
||||||
pub async fn chat_with_tools(
|
pub async fn chat_with_tools(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
|
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
|
||||||
|
let mut span = self.start_request_span(cx, "chat", &self.primary_model);
|
||||||
let order = self.attempt_order();
|
let order = self.attempt_order();
|
||||||
let mut errors: Vec<String> = Vec::new();
|
let mut errors: Vec<String> = Vec::new();
|
||||||
|
|
||||||
@@ -685,8 +767,11 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
url,
|
url,
|
||||||
model
|
model
|
||||||
);
|
);
|
||||||
|
if *label == "fallback" {
|
||||||
|
span.set_attribute(KeyValue::new("llm.model", model.to_string()));
|
||||||
|
}
|
||||||
match self
|
match self
|
||||||
.try_chat_with_tools(url, messages.clone(), tools.clone())
|
.try_chat_with_tools(Some(&mut span), url, messages.clone(), tools.clone())
|
||||||
.await
|
.await
|
||||||
{
|
{
|
||||||
Ok(result) => {
|
Ok(result) => {
|
||||||
@@ -728,18 +813,22 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
/// any, live on the final message).
|
/// any, live on the final message).
|
||||||
pub async fn chat_with_tools_stream(
|
pub async fn chat_with_tools_stream(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> {
|
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> {
|
||||||
// Same preference logic as `chat_with_tools`. Only the initial
|
// Same preference logic as `chat_with_tools`. Only the initial
|
||||||
// connection is retried across servers — once the stream begins,
|
// connection is retried across servers — once the stream begins,
|
||||||
// mid-stream errors propagate to the caller.
|
// mid-stream errors propagate to the caller. Each attempt gets its
|
||||||
|
// own `llm.request` span, created inside
|
||||||
|
// `try_chat_with_tools_stream` so the winning attempt's span moves
|
||||||
|
// into the stream and is finalized when the response drains.
|
||||||
let order = self.attempt_order();
|
let order = self.attempt_order();
|
||||||
let mut last_err: Option<anyhow::Error> = None;
|
let mut last_err: Option<anyhow::Error> = None;
|
||||||
|
|
||||||
for (label, url, _model) in &order {
|
for (label, url, _model) in &order {
|
||||||
match self
|
match self
|
||||||
.try_chat_with_tools_stream(url, messages.clone(), tools.clone())
|
.try_chat_with_tools_stream(cx, url, messages.clone(), tools.clone())
|
||||||
.await
|
.await
|
||||||
{
|
{
|
||||||
Ok(s) => {
|
Ok(s) => {
|
||||||
@@ -759,6 +848,7 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
|
|
||||||
async fn try_chat_with_tools_stream(
|
async fn try_chat_with_tools_stream(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
base_url: &str,
|
base_url: &str,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
@@ -773,6 +863,11 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
};
|
};
|
||||||
let options = self.build_options();
|
let options = self.build_options();
|
||||||
|
|
||||||
|
// The span outlives this method — the stream yields until the
|
||||||
|
// response fully drains, so the owned span moves into the
|
||||||
|
// generator and is finalized there.
|
||||||
|
let mut span = self.start_request_span(cx, "chat_stream", model);
|
||||||
|
|
||||||
let request_body = OllamaChatRequest {
|
let request_body = OllamaChatRequest {
|
||||||
model,
|
model,
|
||||||
messages: &messages,
|
messages: &messages,
|
||||||
@@ -781,16 +876,26 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
options,
|
options,
|
||||||
};
|
};
|
||||||
|
|
||||||
let response = self
|
let response = {
|
||||||
.client
|
let mut rb = self.client.post(&url).json(&request_body);
|
||||||
.post(&url)
|
rb = crate::otel::with_traceparent(rb, span.span_context());
|
||||||
.json(&request_body)
|
match rb.send().await {
|
||||||
.send()
|
Ok(r) => r,
|
||||||
.await
|
Err(e) => {
|
||||||
.with_context(|| format!("Failed to connect to Ollama at {}", url))?;
|
span.set_status(Status::error("Ollama stream connect failed"));
|
||||||
|
return Err(e)
|
||||||
|
.with_context(|| format!("Failed to connect to Ollama at {}", url));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
if !response.status().is_success() {
|
if !response.status().is_success() {
|
||||||
let status = response.status();
|
let status = response.status();
|
||||||
|
crate::otel::record_http_failure(
|
||||||
|
Some(&mut span),
|
||||||
|
status.as_u16(),
|
||||||
|
&format!("Ollama stream request failed: {status}"),
|
||||||
|
);
|
||||||
let body = response.text().await.unwrap_or_default();
|
let body = response.text().await.unwrap_or_default();
|
||||||
anyhow::bail!(
|
anyhow::bail!(
|
||||||
"Ollama stream request failed with status {}: {}",
|
"Ollama stream request failed with status {}: {}",
|
||||||
@@ -798,6 +903,7 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
body
|
body
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
crate::otel::set_http_status(Some(&mut span), response.status().as_u16());
|
||||||
|
|
||||||
// Ollama streams NDJSON: each line is a full `OllamaStreamChunk`.
|
// Ollama streams NDJSON: each line is a full `OllamaStreamChunk`.
|
||||||
// We buffer partial lines across chunks from the byte stream.
|
// We buffer partial lines across chunks from the byte stream.
|
||||||
@@ -879,6 +985,13 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
eval_count,
|
eval_count,
|
||||||
eval_duration,
|
eval_duration,
|
||||||
);
|
);
|
||||||
|
if let Some(pt) = prompt_eval_count {
|
||||||
|
span.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
|
||||||
|
}
|
||||||
|
if let Some(ct) = eval_count {
|
||||||
|
span.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
|
||||||
|
}
|
||||||
|
span.set_status(Status::Ok);
|
||||||
let message = ChatMessage {
|
let message = ChatMessage {
|
||||||
role,
|
role,
|
||||||
content: accumulated,
|
content: accumulated,
|
||||||
@@ -897,6 +1010,7 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
|
|
||||||
async fn try_chat_with_tools(
|
async fn try_chat_with_tools(
|
||||||
&self,
|
&self,
|
||||||
|
mut span: Option<&mut opentelemetry::global::BoxedSpan>,
|
||||||
base_url: &str,
|
base_url: &str,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
@@ -924,16 +1038,30 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
.unwrap_or_else(|e| format!("<serialization error: {}>", e));
|
.unwrap_or_else(|e| format!("<serialization error: {}>", e));
|
||||||
log::debug!("chat_with_tools request body: {}", request_json);
|
log::debug!("chat_with_tools request body: {}", request_json);
|
||||||
|
|
||||||
let response = self
|
let response = {
|
||||||
.client
|
let mut rb = self.client.post(&url).json(&request_body);
|
||||||
.post(&url)
|
if let Some(s) = span.as_mut() {
|
||||||
.json(&request_body)
|
rb = crate::otel::with_traceparent(rb, s.span_context());
|
||||||
.send()
|
}
|
||||||
.await
|
match rb.send().await {
|
||||||
.with_context(|| format!("Failed to connect to Ollama at {}", url))?;
|
Ok(r) => r,
|
||||||
|
Err(e) => {
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
s.set_status(Status::error("Ollama chat connect failed"));
|
||||||
|
}
|
||||||
|
return Err(e)
|
||||||
|
.with_context(|| format!("Failed to connect to Ollama at {}", url));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
if !response.status().is_success() {
|
if !response.status().is_success() {
|
||||||
let status = response.status();
|
let status = response.status();
|
||||||
|
crate::otel::record_http_failure(
|
||||||
|
span.as_deref_mut(),
|
||||||
|
status.as_u16(),
|
||||||
|
&format!("Ollama chat request failed: {status}"),
|
||||||
|
);
|
||||||
let body = response.text().await.unwrap_or_default();
|
let body = response.text().await.unwrap_or_default();
|
||||||
// warn, not error — the outer `chat_with_tools` may recover via
|
// warn, not error — the outer `chat_with_tools` may recover via
|
||||||
// the fallback server. When both fail, the outer layer emits the
|
// the fallback server. When both fail, the outer layer emits the
|
||||||
@@ -950,11 +1078,26 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
body
|
body
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
crate::otel::set_http_status(span.as_deref_mut(), response.status().as_u16());
|
||||||
|
|
||||||
let chat_response: OllamaChatResponse = response
|
let chat_response: OllamaChatResponse = match response.json().await {
|
||||||
.json()
|
Ok(r) => r,
|
||||||
.await
|
Err(e) => {
|
||||||
.with_context(|| "Failed to parse Ollama chat response")?;
|
if let Some(s) = span.as_mut() {
|
||||||
|
s.set_status(Status::error("parsing Ollama chat response failed"));
|
||||||
|
}
|
||||||
|
return Err(e).with_context(|| "Failed to parse Ollama chat response");
|
||||||
|
}
|
||||||
|
};
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
if let Some(pt) = chat_response.prompt_eval_count {
|
||||||
|
s.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
|
||||||
|
}
|
||||||
|
if let Some(ct) = chat_response.eval_count {
|
||||||
|
s.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
|
||||||
|
}
|
||||||
|
s.set_status(Status::Ok);
|
||||||
|
}
|
||||||
|
|
||||||
// Log performance counters returned by Ollama. Durations are
|
// Log performance counters returned by Ollama. Durations are
|
||||||
// reported in nanoseconds; we render ms + tokens/sec for skim-ability
|
// reported in nanoseconds; we render ms + tokens/sec for skim-ability
|
||||||
@@ -976,8 +1119,8 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
|
|
||||||
/// Generate an embedding vector for text using nomic-embed-text:v1.5
|
/// Generate an embedding vector for text using nomic-embed-text:v1.5
|
||||||
/// Returns a 768-dimensional vector as Vec<f32>
|
/// Returns a 768-dimensional vector as Vec<f32>
|
||||||
pub async fn generate_embedding(&self, text: &str) -> Result<Vec<f32>> {
|
pub async fn generate_embedding(&self, cx: &Context, text: &str) -> Result<Vec<f32>> {
|
||||||
let embeddings = self.generate_embeddings(&[text]).await?;
|
let embeddings = self.generate_embeddings(cx, &[text]).await?;
|
||||||
embeddings
|
embeddings
|
||||||
.into_iter()
|
.into_iter()
|
||||||
.next()
|
.next()
|
||||||
@@ -987,9 +1130,11 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
/// Generate embeddings for multiple texts in a single API call (batch mode)
|
/// Generate embeddings for multiple texts in a single API call (batch mode)
|
||||||
/// Returns a vector of 768-dimensional vectors
|
/// Returns a vector of 768-dimensional vectors
|
||||||
/// This is much more efficient than calling generate_embedding multiple times
|
/// This is much more efficient than calling generate_embedding multiple times
|
||||||
pub async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
|
pub async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
|
||||||
let embedding_model = EMBEDDING_MODEL;
|
let embedding_model = EMBEDDING_MODEL;
|
||||||
|
|
||||||
|
let mut span = self.start_request_span(cx, "embeddings", embedding_model);
|
||||||
|
|
||||||
log::debug!("=== Ollama Batch Embedding Request ===");
|
log::debug!("=== Ollama Batch Embedding Request ===");
|
||||||
log::debug!("Model: {}", embedding_model);
|
log::debug!("Model: {}", embedding_model);
|
||||||
log::debug!("Batch size: {} texts", texts.len());
|
log::debug!("Batch size: {} texts", texts.len());
|
||||||
@@ -1003,7 +1148,7 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
embedding_model
|
embedding_model
|
||||||
);
|
);
|
||||||
let primary_result = self
|
let primary_result = self
|
||||||
.try_generate_embeddings(&self.primary_url, embedding_model, texts)
|
.try_generate_embeddings(Some(&mut span), &self.primary_url, embedding_model, texts)
|
||||||
.await;
|
.await;
|
||||||
|
|
||||||
let embeddings = match primary_result {
|
let embeddings = match primary_result {
|
||||||
@@ -1026,7 +1171,12 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
embedding_model
|
embedding_model
|
||||||
);
|
);
|
||||||
match self
|
match self
|
||||||
.try_generate_embeddings(fallback_url, embedding_model, texts)
|
.try_generate_embeddings(
|
||||||
|
Some(&mut span),
|
||||||
|
fallback_url,
|
||||||
|
embedding_model,
|
||||||
|
texts,
|
||||||
|
)
|
||||||
.await
|
.await
|
||||||
{
|
{
|
||||||
Ok(embeddings) => {
|
Ok(embeddings) => {
|
||||||
@@ -1073,6 +1223,7 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
/// Internal helper to try generating embeddings for multiple texts from a specific server
|
/// Internal helper to try generating embeddings for multiple texts from a specific server
|
||||||
async fn try_generate_embeddings(
|
async fn try_generate_embeddings(
|
||||||
&self,
|
&self,
|
||||||
|
mut span: Option<&mut opentelemetry::global::BoxedSpan>,
|
||||||
url: &str,
|
url: &str,
|
||||||
model: &str,
|
model: &str,
|
||||||
texts: &[&str],
|
texts: &[&str],
|
||||||
@@ -1082,15 +1233,32 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
input: texts.iter().map(|s| s.to_string()).collect(),
|
input: texts.iter().map(|s| s.to_string()).collect(),
|
||||||
};
|
};
|
||||||
|
|
||||||
let response = self
|
let response = {
|
||||||
.client
|
let mut rb = self
|
||||||
.post(format!("{}/api/embed", url))
|
.client
|
||||||
.json(&request)
|
.post(format!("{}/api/embed", url))
|
||||||
.send()
|
.json(&request);
|
||||||
.await?;
|
if let Some(s) = span.as_mut() {
|
||||||
|
rb = crate::otel::with_traceparent(rb, s.span_context());
|
||||||
|
}
|
||||||
|
match rb.send().await {
|
||||||
|
Ok(r) => r,
|
||||||
|
Err(e) => {
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
s.set_status(Status::error("Ollama embedding request failed"));
|
||||||
|
}
|
||||||
|
return Err(e.into());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
if !response.status().is_success() {
|
if !response.status().is_success() {
|
||||||
let status = response.status();
|
let status = response.status();
|
||||||
|
crate::otel::record_http_failure(
|
||||||
|
span.as_deref_mut(),
|
||||||
|
status.as_u16(),
|
||||||
|
&format!("Ollama batch embedding request failed: {status}"),
|
||||||
|
);
|
||||||
let error_body = response.text().await.unwrap_or_default();
|
let error_body = response.text().await.unwrap_or_default();
|
||||||
return Err(anyhow::anyhow!(
|
return Err(anyhow::anyhow!(
|
||||||
"Ollama batch embedding request failed: {} - {}",
|
"Ollama batch embedding request failed: {} - {}",
|
||||||
@@ -1098,8 +1266,20 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
error_body
|
error_body
|
||||||
));
|
));
|
||||||
}
|
}
|
||||||
|
crate::otel::set_http_status(span.as_deref_mut(), response.status().as_u16());
|
||||||
|
|
||||||
let result: OllamaEmbedResponse = response.json().await?;
|
let result: OllamaEmbedResponse = match response.json().await {
|
||||||
|
Ok(r) => r,
|
||||||
|
Err(e) => {
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
s.set_status(Status::error("parsing Ollama embed response failed"));
|
||||||
|
}
|
||||||
|
return Err(e.into());
|
||||||
|
}
|
||||||
|
};
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
s.set_status(Status::Ok);
|
||||||
|
}
|
||||||
Ok(result.embeddings)
|
Ok(result.embeddings)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1108,35 +1288,38 @@ Analyze the image and use specific details from both the visual content and the
|
|||||||
impl LlmClient for OllamaClient {
|
impl LlmClient for OllamaClient {
|
||||||
async fn generate(
|
async fn generate(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
prompt: &str,
|
prompt: &str,
|
||||||
system: Option<&str>,
|
system: Option<&str>,
|
||||||
images: Option<Vec<String>>,
|
images: Option<Vec<String>>,
|
||||||
) -> Result<String> {
|
) -> Result<String> {
|
||||||
self.generate_with_images(prompt, system, images).await
|
self.generate_with_images(cx, prompt, system, images).await
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn chat_with_tools(
|
async fn chat_with_tools(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
|
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
|
||||||
OllamaClient::chat_with_tools(self, messages, tools).await
|
OllamaClient::chat_with_tools(self, cx, messages, tools).await
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn chat_with_tools_stream(
|
async fn chat_with_tools_stream(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> {
|
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> {
|
||||||
OllamaClient::chat_with_tools_stream(self, messages, tools).await
|
OllamaClient::chat_with_tools_stream(self, cx, messages, tools).await
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
|
async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
|
||||||
OllamaClient::generate_embeddings(self, texts).await
|
OllamaClient::generate_embeddings(self, cx, texts).await
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn describe_image(&self, image_base64: &str) -> Result<String> {
|
async fn describe_image(&self, cx: &Context, image_base64: &str) -> Result<String> {
|
||||||
self.generate_photo_description(image_base64).await
|
self.generate_photo_description(cx, image_base64).await
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn list_models(&self) -> Result<Vec<ModelCapabilities>> {
|
async fn list_models(&self) -> Result<Vec<ModelCapabilities>> {
|
||||||
|
|||||||
+202
-83
@@ -2,8 +2,11 @@
|
|||||||
// the translation helpers directly.
|
// the translation helpers directly.
|
||||||
#![allow(dead_code)]
|
#![allow(dead_code)]
|
||||||
|
|
||||||
use anyhow::{Context, Result, anyhow, bail};
|
use anyhow::Context as _;
|
||||||
|
use anyhow::{Result, anyhow, bail};
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
|
use opentelemetry::trace::{Span, Status};
|
||||||
|
use opentelemetry::{Context, KeyValue};
|
||||||
use reqwest::Client;
|
use reqwest::Client;
|
||||||
use serde::Deserialize;
|
use serde::Deserialize;
|
||||||
use serde_json::{Value, json};
|
use serde_json::{Value, json};
|
||||||
@@ -14,6 +17,7 @@ use std::time::{Duration, Instant};
|
|||||||
use crate::ai::llm_client::{
|
use crate::ai::llm_client::{
|
||||||
ChatMessage, LlmClient, LlmStreamEvent, ModelCapabilities, Tool, ToolCall, ToolCallFunction,
|
ChatMessage, LlmClient, LlmStreamEvent, ModelCapabilities, Tool, ToolCall, ToolCallFunction,
|
||||||
};
|
};
|
||||||
|
|
||||||
use futures::stream::{BoxStream, StreamExt};
|
use futures::stream::{BoxStream, StreamExt};
|
||||||
|
|
||||||
const DEFAULT_BASE_URL: &str = "https://openrouter.ai/api/v1";
|
const DEFAULT_BASE_URL: &str = "https://openrouter.ai/api/v1";
|
||||||
@@ -144,6 +148,19 @@ impl OpenRouterClient {
|
|||||||
b
|
b
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Start the `llm.request` span for an outgoing call. `operation` is
|
||||||
|
/// `chat`, `embeddings`, or `describe` — the call shape, not the entry
|
||||||
|
/// point (a `describe_image` funneled through `chat_with_tools` is
|
||||||
|
/// still a `describe`).
|
||||||
|
fn start_request_span(
|
||||||
|
&self,
|
||||||
|
cx: &Context,
|
||||||
|
operation: &str,
|
||||||
|
model: &str,
|
||||||
|
) -> opentelemetry::global::BoxedSpan {
|
||||||
|
crate::otel::start_llm_request_span(cx, "openrouter", operation, model)
|
||||||
|
}
|
||||||
|
|
||||||
/// Translate canonical messages to the OpenAI-compatible wire shape.
|
/// Translate canonical messages to the OpenAI-compatible wire shape.
|
||||||
///
|
///
|
||||||
/// Walks in order so it can attach `tool_call_id` to `role=tool` messages
|
/// Walks in order so it can attach `tool_call_id` to `role=tool` messages
|
||||||
@@ -312,6 +329,7 @@ impl OpenRouterClient {
|
|||||||
impl LlmClient for OpenRouterClient {
|
impl LlmClient for OpenRouterClient {
|
||||||
async fn generate(
|
async fn generate(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
prompt: &str,
|
prompt: &str,
|
||||||
system: Option<&str>,
|
system: Option<&str>,
|
||||||
images: Option<Vec<String>>,
|
images: Option<Vec<String>>,
|
||||||
@@ -324,90 +342,34 @@ impl LlmClient for OpenRouterClient {
|
|||||||
user.images = images;
|
user.images = images;
|
||||||
messages.push(user);
|
messages.push(user);
|
||||||
|
|
||||||
let (reply, _, _) = self.chat_with_tools(messages, Vec::new()).await?;
|
// Delegates to chat_with_tools, which starts the llm.request span
|
||||||
|
// (operation=chat) — one span per HTTP request.
|
||||||
|
let (reply, _, _) = self.chat_with_tools(cx, messages, Vec::new()).await?;
|
||||||
Ok(reply.content)
|
Ok(reply.content)
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn chat_with_tools(
|
async fn chat_with_tools(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
|
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
|
||||||
let url = format!("{}/chat/completions", self.base_url);
|
let mut span = self.start_request_span(cx, "chat", &self.primary_model);
|
||||||
let mut body = serde_json::Map::new();
|
self.chat_with_tools_impl(Some(&mut span), messages, tools)
|
||||||
body.insert("model".into(), Value::String(self.primary_model.clone()));
|
|
||||||
body.insert(
|
|
||||||
"messages".into(),
|
|
||||||
Value::Array(Self::messages_to_openai(&messages)),
|
|
||||||
);
|
|
||||||
body.insert("stream".into(), Value::Bool(false));
|
|
||||||
if !tools.is_empty() {
|
|
||||||
body.insert(
|
|
||||||
"tools".into(),
|
|
||||||
serde_json::to_value(&tools).context("serializing tools")?,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
for (k, v) in self.build_options() {
|
|
||||||
body.insert(k.into(), v);
|
|
||||||
}
|
|
||||||
|
|
||||||
log::info!(
|
|
||||||
"OpenRouter chat_with_tools: model={} messages={} tools={}",
|
|
||||||
self.primary_model,
|
|
||||||
messages.len(),
|
|
||||||
tools.len()
|
|
||||||
);
|
|
||||||
|
|
||||||
let resp = self
|
|
||||||
.authed(self.client.post(&url))
|
|
||||||
.json(&Value::Object(body))
|
|
||||||
.send()
|
|
||||||
.await
|
.await
|
||||||
.with_context(|| format!("POST {} failed", url))?;
|
|
||||||
|
|
||||||
if !resp.status().is_success() {
|
|
||||||
let status = resp.status();
|
|
||||||
let body = resp.text().await.unwrap_or_default();
|
|
||||||
bail!("OpenRouter chat request failed: {} — {}", status, body);
|
|
||||||
}
|
|
||||||
|
|
||||||
let parsed: Value = resp.json().await.context("parsing chat response")?;
|
|
||||||
let choice = parsed
|
|
||||||
.get("choices")
|
|
||||||
.and_then(|v| v.as_array())
|
|
||||||
.and_then(|a| a.first())
|
|
||||||
.ok_or_else(|| {
|
|
||||||
anyhow!(
|
|
||||||
"response missing choices[0]: {}",
|
|
||||||
extract_openrouter_error_detail(&parsed)
|
|
||||||
)
|
|
||||||
})?;
|
|
||||||
let msg = choice.get("message").ok_or_else(|| {
|
|
||||||
anyhow!(
|
|
||||||
"choices[0] missing message: {}",
|
|
||||||
extract_openrouter_error_detail(&parsed)
|
|
||||||
)
|
|
||||||
})?;
|
|
||||||
let chat_msg = Self::openai_message_to_chat(msg)?;
|
|
||||||
|
|
||||||
let usage = parsed.get("usage");
|
|
||||||
let prompt_tokens = usage
|
|
||||||
.and_then(|u| u.get("prompt_tokens"))
|
|
||||||
.and_then(|v| v.as_i64())
|
|
||||||
.map(|n| n as i32);
|
|
||||||
let completion_tokens = usage
|
|
||||||
.and_then(|u| u.get("completion_tokens"))
|
|
||||||
.and_then(|v| v.as_i64())
|
|
||||||
.map(|n| n as i32);
|
|
||||||
|
|
||||||
Ok((chat_msg, prompt_tokens, completion_tokens))
|
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn chat_with_tools_stream(
|
async fn chat_with_tools_stream(
|
||||||
&self,
|
&self,
|
||||||
|
cx: &Context,
|
||||||
messages: Vec<ChatMessage>,
|
messages: Vec<ChatMessage>,
|
||||||
tools: Vec<Tool>,
|
tools: Vec<Tool>,
|
||||||
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> {
|
) -> Result<BoxStream<'static, Result<LlmStreamEvent>>> {
|
||||||
|
// The span outlives this method — the stream yields until the
|
||||||
|
// response fully drains, so the owned span moves into the
|
||||||
|
// generator and is finalized there.
|
||||||
|
let mut span = self.start_request_span(cx, "chat_stream", &self.primary_model);
|
||||||
|
|
||||||
let url = format!("{}/chat/completions", self.base_url);
|
let url = format!("{}/chat/completions", self.base_url);
|
||||||
let mut body = serde_json::Map::new();
|
let mut body = serde_json::Map::new();
|
||||||
body.insert("model".into(), Value::String(self.primary_model.clone()));
|
body.insert("model".into(), Value::String(self.primary_model.clone()));
|
||||||
@@ -432,18 +394,26 @@ impl LlmClient for OpenRouterClient {
|
|||||||
body.insert(k.into(), v);
|
body.insert(k.into(), v);
|
||||||
}
|
}
|
||||||
|
|
||||||
let resp = self
|
let mut request = self
|
||||||
.authed(self.client.post(&url))
|
.authed(self.client.post(&url))
|
||||||
.json(&Value::Object(body))
|
.json(&Value::Object(body));
|
||||||
|
request = crate::otel::with_traceparent(request, span.span_context());
|
||||||
|
let resp = request
|
||||||
.send()
|
.send()
|
||||||
.await
|
.await
|
||||||
.with_context(|| format!("POST {} failed", url))?;
|
.with_context(|| format!("POST {} failed", url))?;
|
||||||
|
|
||||||
if !resp.status().is_success() {
|
if !resp.status().is_success() {
|
||||||
let status = resp.status();
|
let status = resp.status();
|
||||||
|
crate::otel::record_http_failure(
|
||||||
|
Some(&mut span),
|
||||||
|
status.as_u16(),
|
||||||
|
&format!("OpenRouter stream request failed: {status}"),
|
||||||
|
);
|
||||||
let body = resp.text().await.unwrap_or_default();
|
let body = resp.text().await.unwrap_or_default();
|
||||||
bail!("OpenRouter stream request failed: {} — {}", status, body);
|
bail!("OpenRouter stream request failed: {} — {}", status, body);
|
||||||
}
|
}
|
||||||
|
crate::otel::set_http_status(Some(&mut span), resp.status().as_u16());
|
||||||
|
|
||||||
// OpenAI-compat SSE stream. Each event is `data: <json>\n\n`, with
|
// OpenAI-compat SSE stream. Each event is `data: <json>\n\n`, with
|
||||||
// `data: [DONE]` signalling completion. Tool calls arrive as
|
// `data: [DONE]` signalling completion. Tool calls arrive as
|
||||||
@@ -467,6 +437,7 @@ impl LlmClient for OpenRouterClient {
|
|||||||
let chunk = match chunk {
|
let chunk = match chunk {
|
||||||
Ok(b) => b,
|
Ok(b) => b,
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
|
span.set_status(Status::error(format!("stream read failed: {e}")));
|
||||||
yield Err(anyhow!("stream read failed: {}", e));
|
yield Err(anyhow!("stream read failed: {}", e));
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -517,6 +488,9 @@ impl LlmClient for OpenRouterClient {
|
|||||||
.get("completion_tokens")
|
.get("completion_tokens")
|
||||||
.and_then(|n| n.as_i64())
|
.and_then(|n| n.as_i64())
|
||||||
.map(|n| n as i32);
|
.map(|n| n as i32);
|
||||||
|
if let Some(cost) = usage.get("cost").and_then(|c| c.as_f64()) {
|
||||||
|
span.set_attribute(KeyValue::new("llm.cost_usd", cost));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
let Some(choices) = v.get("choices").and_then(|c| c.as_array())
|
let Some(choices) = v.get("choices").and_then(|c| c.as_array())
|
||||||
@@ -606,6 +580,13 @@ impl LlmClient for OpenRouterClient {
|
|||||||
tool_calls,
|
tool_calls,
|
||||||
images: None,
|
images: None,
|
||||||
};
|
};
|
||||||
|
if let Some(pt) = prompt_tokens {
|
||||||
|
span.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
|
||||||
|
}
|
||||||
|
if let Some(ct) = completion_tokens {
|
||||||
|
span.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
|
||||||
|
}
|
||||||
|
span.set_status(Status::Ok);
|
||||||
yield Ok(LlmStreamEvent::Done {
|
yield Ok(LlmStreamEvent::Done {
|
||||||
message,
|
message,
|
||||||
prompt_eval_count: prompt_tokens,
|
prompt_eval_count: prompt_tokens,
|
||||||
@@ -616,25 +597,35 @@ impl LlmClient for OpenRouterClient {
|
|||||||
Ok(Box::pin(stream))
|
Ok(Box::pin(stream))
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn generate_embeddings(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
|
async fn generate_embeddings(&self, cx: &Context, texts: &[&str]) -> Result<Vec<Vec<f32>>> {
|
||||||
|
let mut span = self.start_request_span(cx, "embeddings", &self.embedding_model);
|
||||||
let url = format!("{}/embeddings", self.base_url);
|
let url = format!("{}/embeddings", self.base_url);
|
||||||
let body = json!({
|
let body = json!({
|
||||||
"model": self.embedding_model,
|
"model": self.embedding_model,
|
||||||
"input": texts,
|
"input": texts,
|
||||||
});
|
});
|
||||||
|
|
||||||
let resp = self
|
let mut request = self.authed(self.client.post(&url)).json(&body);
|
||||||
.authed(self.client.post(&url))
|
request = crate::otel::with_traceparent(request, span.span_context());
|
||||||
.json(&body)
|
let resp = request
|
||||||
.send()
|
.send()
|
||||||
.await
|
.await
|
||||||
.with_context(|| format!("POST {} failed", url))?;
|
.with_context(|| format!("POST {} failed", url))
|
||||||
|
.inspect_err(|_| {
|
||||||
|
span.set_status(Status::error("OpenRouter embedding request failed"));
|
||||||
|
})?;
|
||||||
|
|
||||||
if !resp.status().is_success() {
|
if !resp.status().is_success() {
|
||||||
let status = resp.status();
|
let status = resp.status();
|
||||||
|
crate::otel::record_http_failure(
|
||||||
|
Some(&mut span),
|
||||||
|
status.as_u16(),
|
||||||
|
&format!("OpenRouter embedding request failed: {status}"),
|
||||||
|
);
|
||||||
let body = resp.text().await.unwrap_or_default();
|
let body = resp.text().await.unwrap_or_default();
|
||||||
bail!("OpenRouter embedding request failed: {} — {}", status, body);
|
bail!("OpenRouter embedding request failed: {} — {}", status, body);
|
||||||
}
|
}
|
||||||
|
crate::otel::set_http_status(Some(&mut span), resp.status().as_u16());
|
||||||
|
|
||||||
#[derive(Deserialize)]
|
#[derive(Deserialize)]
|
||||||
struct EmbedResponse {
|
struct EmbedResponse {
|
||||||
@@ -646,18 +637,28 @@ impl LlmClient for OpenRouterClient {
|
|||||||
}
|
}
|
||||||
|
|
||||||
let parsed: EmbedResponse = resp.json().await.context("parsing embed response")?;
|
let parsed: EmbedResponse = resp.json().await.context("parsing embed response")?;
|
||||||
|
span.set_status(Status::Ok);
|
||||||
Ok(parsed.data.into_iter().map(|i| i.embedding).collect())
|
Ok(parsed.data.into_iter().map(|i| i.embedding).collect())
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn describe_image(&self, image_base64: &str) -> Result<String> {
|
async fn describe_image(&self, cx: &Context, image_base64: &str) -> Result<String> {
|
||||||
|
// Own span with operation=describe_image — the funnel into
|
||||||
|
// chat_with_tools_impl keeps the request on a single span.
|
||||||
|
let mut span = self.start_request_span(cx, "describe_image", &self.primary_model);
|
||||||
let prompt = "Briefly describe what you see in this image in 1-2 sentences. \
|
let prompt = "Briefly describe what you see in this image in 1-2 sentences. \
|
||||||
Focus on the people, location, and activity.";
|
Focus on the people, location, and activity.";
|
||||||
self.generate(
|
let mut messages: Vec<ChatMessage> = Vec::new();
|
||||||
prompt,
|
messages.push(ChatMessage::system(
|
||||||
Some("You are a scene description assistant. Be concise and factual."),
|
"You are a scene description assistant. Be concise and factual.",
|
||||||
Some(vec![image_base64.to_string()]),
|
));
|
||||||
)
|
let mut user = ChatMessage::user(prompt);
|
||||||
.await
|
user.images = Some(vec![image_base64.to_string()]);
|
||||||
|
messages.push(user);
|
||||||
|
|
||||||
|
let (reply, _, _) = self
|
||||||
|
.chat_with_tools_impl(Some(&mut span), messages, Vec::new())
|
||||||
|
.await?;
|
||||||
|
Ok(reply.content)
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn list_models(&self) -> Result<Vec<ModelCapabilities>> {
|
async fn list_models(&self) -> Result<Vec<ModelCapabilities>> {
|
||||||
@@ -711,6 +712,124 @@ impl LlmClient for OpenRouterClient {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
impl OpenRouterClient {
|
||||||
|
/// HTTP core of `chat_with_tools`. `span` is the `llm.request` span
|
||||||
|
/// started by the caller (or a funnelling entry point like
|
||||||
|
/// `describe_image`) so attributes from the response land on the right
|
||||||
|
/// operation label.
|
||||||
|
async fn chat_with_tools_impl(
|
||||||
|
&self,
|
||||||
|
mut span: Option<&mut opentelemetry::global::BoxedSpan>,
|
||||||
|
messages: Vec<ChatMessage>,
|
||||||
|
tools: Vec<Tool>,
|
||||||
|
) -> Result<(ChatMessage, Option<i32>, Option<i32>)> {
|
||||||
|
let url = format!("{}/chat/completions", self.base_url);
|
||||||
|
let mut body = serde_json::Map::new();
|
||||||
|
body.insert("model".into(), Value::String(self.primary_model.clone()));
|
||||||
|
body.insert(
|
||||||
|
"messages".into(),
|
||||||
|
Value::Array(Self::messages_to_openai(&messages)),
|
||||||
|
);
|
||||||
|
body.insert("stream".into(), Value::Bool(false));
|
||||||
|
if !tools.is_empty() {
|
||||||
|
body.insert(
|
||||||
|
"tools".into(),
|
||||||
|
serde_json::to_value(&tools).context("serializing tools")?,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
for (k, v) in self.build_options() {
|
||||||
|
body.insert(k.into(), v);
|
||||||
|
}
|
||||||
|
|
||||||
|
log::info!(
|
||||||
|
"OpenRouter chat_with_tools: model={} messages={} tools={}",
|
||||||
|
self.primary_model,
|
||||||
|
messages.len(),
|
||||||
|
tools.len()
|
||||||
|
);
|
||||||
|
|
||||||
|
let mut request = self
|
||||||
|
.authed(self.client.post(&url))
|
||||||
|
.json(&Value::Object(body));
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
request = crate::otel::with_traceparent(request, s.span_context());
|
||||||
|
}
|
||||||
|
let resp = match request.send().await {
|
||||||
|
Ok(r) => r,
|
||||||
|
Err(e) => {
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
s.set_status(Status::error("OpenRouter request failed"));
|
||||||
|
}
|
||||||
|
return Err(e).with_context(|| format!("POST {} failed", url));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
if !resp.status().is_success() {
|
||||||
|
let status = resp.status();
|
||||||
|
crate::otel::record_http_failure(
|
||||||
|
span.as_deref_mut(),
|
||||||
|
status.as_u16(),
|
||||||
|
&format!("OpenRouter chat request failed: {status}"),
|
||||||
|
);
|
||||||
|
let body = resp.text().await.unwrap_or_default();
|
||||||
|
bail!("OpenRouter chat request failed: {} — {}", status, body);
|
||||||
|
}
|
||||||
|
crate::otel::set_http_status(span.as_deref_mut(), resp.status().as_u16());
|
||||||
|
|
||||||
|
let parsed: Value = match resp.json().await {
|
||||||
|
Ok(v) => v,
|
||||||
|
Err(e) => {
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
s.set_status(Status::error("parsing chat response failed"));
|
||||||
|
}
|
||||||
|
return Err(e).context("parsing chat response");
|
||||||
|
}
|
||||||
|
};
|
||||||
|
let choice = parsed
|
||||||
|
.get("choices")
|
||||||
|
.and_then(|v| v.as_array())
|
||||||
|
.and_then(|a| a.first())
|
||||||
|
.ok_or_else(|| {
|
||||||
|
anyhow!(
|
||||||
|
"response missing choices[0]: {}",
|
||||||
|
extract_openrouter_error_detail(&parsed)
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
let msg = choice.get("message").ok_or_else(|| {
|
||||||
|
anyhow!(
|
||||||
|
"choices[0] missing message: {}",
|
||||||
|
extract_openrouter_error_detail(&parsed)
|
||||||
|
)
|
||||||
|
})?;
|
||||||
|
let chat_msg = Self::openai_message_to_chat(msg)?;
|
||||||
|
|
||||||
|
let usage = parsed.get("usage");
|
||||||
|
let prompt_tokens = usage
|
||||||
|
.and_then(|u| u.get("prompt_tokens"))
|
||||||
|
.and_then(|v| v.as_i64())
|
||||||
|
.map(|n| n as i32);
|
||||||
|
let completion_tokens = usage
|
||||||
|
.and_then(|u| u.get("completion_tokens"))
|
||||||
|
.and_then(|v| v.as_i64())
|
||||||
|
.map(|n| n as i32);
|
||||||
|
// OpenRouter reports per-request USD cost in `usage.cost`.
|
||||||
|
if let Some(s) = span.as_mut() {
|
||||||
|
if let Some(pt) = prompt_tokens {
|
||||||
|
s.set_attribute(KeyValue::new("llm.tokens.input", pt as i64));
|
||||||
|
}
|
||||||
|
if let Some(ct) = completion_tokens {
|
||||||
|
s.set_attribute(KeyValue::new("llm.tokens.output", ct as i64));
|
||||||
|
}
|
||||||
|
if let Some(cost) = usage.and_then(|u| u.get("cost")).and_then(|c| c.as_f64()) {
|
||||||
|
s.set_attribute(KeyValue::new("llm.cost_usd", cost));
|
||||||
|
}
|
||||||
|
s.set_status(Status::Ok);
|
||||||
|
}
|
||||||
|
|
||||||
|
Ok((chat_msg, prompt_tokens, completion_tokens))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Extract a diagnostic fragment from an OpenRouter response body that
|
/// Extract a diagnostic fragment from an OpenRouter response body that
|
||||||
/// doesn't match the expected `{choices: [...]}` shape. OpenRouter will
|
/// doesn't match the expected `{choices: [...]}` shape. OpenRouter will
|
||||||
/// sometimes return 200 OK with `{"error": {"message": "...", "code": ...}}`
|
/// sometimes return 200 OK with `{"error": {"message": "...", "code": ...}}`
|
||||||
|
|||||||
+13
-3
@@ -26,7 +26,7 @@ use actix_web::{HttpRequest, HttpResponse, Responder, delete, get, post, web};
|
|||||||
use anyhow::{Context, Result, anyhow, bail};
|
use anyhow::{Context, Result, anyhow, bail};
|
||||||
use chrono::Utc;
|
use chrono::Utc;
|
||||||
use opentelemetry::KeyValue;
|
use opentelemetry::KeyValue;
|
||||||
use opentelemetry::trace::{Span, Status, Tracer};
|
use opentelemetry::trace::{Span, Status, TraceContextExt, Tracer};
|
||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
use std::sync::{Arc, Mutex as StdMutex};
|
use std::sync::{Arc, Mutex as StdMutex};
|
||||||
@@ -685,6 +685,7 @@ impl PersonaChatSession {
|
|||||||
entry_clone.clone(),
|
entry_clone.clone(),
|
||||||
user_id,
|
user_id,
|
||||||
library_id,
|
library_id,
|
||||||
|
&mut span,
|
||||||
)
|
)
|
||||||
.await;
|
.await;
|
||||||
match result {
|
match result {
|
||||||
@@ -721,6 +722,7 @@ impl PersonaChatSession {
|
|||||||
entry: Arc<TurnEntry>,
|
entry: Arc<TurnEntry>,
|
||||||
user_id: i32,
|
user_id: i32,
|
||||||
_library_id: i32,
|
_library_id: i32,
|
||||||
|
turn_span: &mut opentelemetry::global::BoxedSpan,
|
||||||
) -> Result<()> {
|
) -> Result<()> {
|
||||||
// Per-conversation mutex — serializes this turn against a
|
// Per-conversation mutex — serializes this turn against a
|
||||||
// concurrent rewind or branch switch on the same transcript.
|
// concurrent rewind or branch switch on the same transcript.
|
||||||
@@ -808,6 +810,12 @@ impl PersonaChatSession {
|
|||||||
.resolve_backend(backend_kind, &overrides)
|
.resolve_backend(backend_kind, &overrides)
|
||||||
.await?;
|
.await?;
|
||||||
let model_used = backend.model().to_string();
|
let model_used = backend.model().to_string();
|
||||||
|
// llm.* attributes on the turn-level span (issue #130), plus the
|
||||||
|
// parent context for the child llm.request spans.
|
||||||
|
turn_span.set_attribute(KeyValue::new("llm.backend", backend.kind.as_str()));
|
||||||
|
turn_span.set_attribute(KeyValue::new("llm.model", model_used.clone()));
|
||||||
|
let turn_cx = opentelemetry::Context::new()
|
||||||
|
.with_remote_span_context(turn_span.span_context().clone());
|
||||||
|
|
||||||
// Build a no-photo tool catalog. `current_gate_opts_for_persona`
|
// Build a no-photo tool catalog. `current_gate_opts_for_persona`
|
||||||
// already probes each tool's backing table for presence and
|
// already probes each tool's backing table for presence and
|
||||||
@@ -874,6 +882,7 @@ impl PersonaChatSession {
|
|||||||
&persona.persona_id,
|
&persona.persona_id,
|
||||||
max_iterations,
|
max_iterations,
|
||||||
&entry,
|
&entry,
|
||||||
|
&turn_cx,
|
||||||
)
|
)
|
||||||
.await
|
.await
|
||||||
.map_err(|e| anyhow!("persona chat loop failed: {e:?}"))?;
|
.map_err(|e| anyhow!("persona chat loop failed: {e:?}"))?;
|
||||||
@@ -942,7 +951,7 @@ impl PersonaChatSession {
|
|||||||
// list falls back to the persona name while the title is empty.
|
// list falls back to the persona name while the title is empty.
|
||||||
if needs_title {
|
if needs_title {
|
||||||
let title = self
|
let title = self
|
||||||
.generate_title(&backend, &messages)
|
.generate_title(&backend, &messages, &turn_cx)
|
||||||
.await
|
.await
|
||||||
.unwrap_or_else(|| fallback_title(&messages));
|
.unwrap_or_else(|| fallback_title(&messages));
|
||||||
if !title.is_empty() {
|
if !title.is_empty() {
|
||||||
@@ -1047,6 +1056,7 @@ impl PersonaChatSession {
|
|||||||
&self,
|
&self,
|
||||||
backend: &crate::ai::backend::ResolvedBackend,
|
backend: &crate::ai::backend::ResolvedBackend,
|
||||||
messages: &[ChatMessage],
|
messages: &[ChatMessage],
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
) -> Option<String> {
|
) -> Option<String> {
|
||||||
let user_message = messages.iter().find(|m| m.role == "user")?;
|
let user_message = messages.iter().find(|m| m.role == "user")?;
|
||||||
// The last assistant message with actual prose — earlier ones may be
|
// The last assistant message with actual prose — earlier ones may be
|
||||||
@@ -1057,7 +1067,7 @@ impl PersonaChatSession {
|
|||||||
.find(|m| m.role == "assistant" && !m.content.trim().is_empty())?;
|
.find(|m| m.role == "assistant" && !m.content.trim().is_empty())?;
|
||||||
|
|
||||||
let prompt = title_prompt(&user_message.content, &reply.content);
|
let prompt = title_prompt(&user_message.content, &reply.content);
|
||||||
match backend.chat().generate(&prompt, None, None).await {
|
match backend.chat().generate(cx, &prompt, None, None).await {
|
||||||
Ok(raw) => {
|
Ok(raw) => {
|
||||||
let title = sanitize_title(&crate::ai::llm_client::strip_think_blocks(&raw));
|
let title = sanitize_title(&crate::ai::llm_client::strip_think_blocks(&raw));
|
||||||
if title.is_empty() { None } else { Some(title) }
|
if title.is_empty() { None } else { Some(title) }
|
||||||
|
|||||||
@@ -324,6 +324,7 @@ impl SmsApiClient {
|
|||||||
&self,
|
&self,
|
||||||
messages: &[SmsMessage],
|
messages: &[SmsMessage],
|
||||||
ollama: &OllamaClient,
|
ollama: &OllamaClient,
|
||||||
|
cx: &opentelemetry::Context,
|
||||||
) -> Result<String> {
|
) -> Result<String> {
|
||||||
if messages.is_empty() {
|
if messages.is_empty() {
|
||||||
return Ok(String::from("No messages on this day"));
|
return Ok(String::from("No messages on this day"));
|
||||||
@@ -356,6 +357,7 @@ Summary:"#,
|
|||||||
|
|
||||||
ollama
|
ollama
|
||||||
.generate(
|
.generate(
|
||||||
|
cx,
|
||||||
&prompt,
|
&prompt,
|
||||||
// Some("You are a summarizer for the purposes of jogging my memory and highlighting events and situations."),
|
// Some("You are a summarizer for the purposes of jogging my memory and highlighting events and situations."),
|
||||||
Some("You are the keeper of memories, ingest the context and give me a casual summary of the moment."),
|
Some("You are the keeper of memories, ingest the context and give me a casual summary of the moment."),
|
||||||
|
|||||||
@@ -220,7 +220,9 @@ async fn main() -> Result<()> {
|
|||||||
|
|
||||||
println!("Generating summary...");
|
println!("Generating summary...");
|
||||||
|
|
||||||
let summary = ollama.generate(&prompt, Some(system_prompt)).await?;
|
let summary = ollama
|
||||||
|
.generate(&opentelemetry::Context::new(), &prompt, Some(system_prompt))
|
||||||
|
.await?;
|
||||||
|
|
||||||
println!("\n📝 GENERATED SUMMARY:");
|
println!("\n📝 GENERATED SUMMARY:");
|
||||||
println!("─────────────────────────────────────────");
|
println!("─────────────────────────────────────────");
|
||||||
@@ -236,7 +238,9 @@ async fn main() -> Result<()> {
|
|||||||
);
|
);
|
||||||
|
|
||||||
println!("\nGenerating embedding...");
|
println!("\nGenerating embedding...");
|
||||||
let embedding = ollama.generate_embedding(&stripped).await?;
|
let embedding = ollama
|
||||||
|
.generate_embedding(&opentelemetry::Context::new(), &stripped)
|
||||||
|
.await?;
|
||||||
println!("✓ Embedding generated ({} dimensions)", embedding.len());
|
println!("✓ Embedding generated ({} dimensions)", embedding.len());
|
||||||
|
|
||||||
println!("Saving to database...");
|
println!("Saving to database...");
|
||||||
|
|||||||
@@ -579,6 +579,19 @@ pub struct UserAiPrefs {
|
|||||||
/// default (`camera_tz::DEFAULT_DISPLAY_TZ`). The legacy
|
/// default (`camera_tz::DEFAULT_DISPLAY_TZ`). The legacy
|
||||||
/// `tz_offset_minutes` column stays frozen for rollback.
|
/// `tz_offset_minutes` column stays frozen for rollback.
|
||||||
pub display_tz: Option<String>,
|
pub display_tz: Option<String>,
|
||||||
|
/// Last/explicit insight + chat model choice (local id or OpenRouter id).
|
||||||
|
/// NULL = current per-request / env default behavior.
|
||||||
|
pub default_model: Option<String>,
|
||||||
|
/// `"local"` | `"hybrid"` — insight/chat backend. NULL = per-request /
|
||||||
|
/// hard default ("local").
|
||||||
|
pub default_backend: Option<String>,
|
||||||
|
/// Active persona for persona chat. NULL = hard default ("default").
|
||||||
|
pub default_persona_id: Option<String>,
|
||||||
|
/// Reel span: `"day"` | `"week"` | `"month"`. NULL = per-request /
|
||||||
|
/// hard default (day).
|
||||||
|
pub reel_span: Option<String>,
|
||||||
|
/// Reel segment cap. NULL = per-request / hard default.
|
||||||
|
pub reel_max_segments: Option<i32>,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Insertable, Debug, Clone, serde::Deserialize, serde::Serialize)]
|
#[derive(Insertable, Debug, Clone, serde::Deserialize, serde::Serialize)]
|
||||||
@@ -589,4 +602,9 @@ pub struct UpsertUserAiPrefs {
|
|||||||
pub library: Option<String>,
|
pub library: Option<String>,
|
||||||
pub updated_at: i64,
|
pub updated_at: i64,
|
||||||
pub display_tz: Option<String>,
|
pub display_tz: Option<String>,
|
||||||
|
pub default_model: Option<String>,
|
||||||
|
pub default_backend: Option<String>,
|
||||||
|
pub default_persona_id: Option<String>,
|
||||||
|
pub reel_span: Option<String>,
|
||||||
|
pub reel_max_segments: Option<i32>,
|
||||||
}
|
}
|
||||||
@@ -293,6 +293,11 @@ diesel::table! {
|
|||||||
library -> Nullable<Text>,
|
library -> Nullable<Text>,
|
||||||
updated_at -> BigInt,
|
updated_at -> BigInt,
|
||||||
display_tz -> Nullable<Text>,
|
display_tz -> Nullable<Text>,
|
||||||
|
default_model -> Nullable<Text>,
|
||||||
|
default_backend -> Nullable<Text>,
|
||||||
|
default_persona_id -> Nullable<Text>,
|
||||||
|
reel_span -> Nullable<Text>,
|
||||||
|
reel_max_segments -> Nullable<Integer>,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -100,6 +100,11 @@ impl UserAiPrefsDao for SqliteUserAiPrefsDao {
|
|||||||
dsl::tz_offset_minutes.eq(&prefs.tz_offset_minutes),
|
dsl::tz_offset_minutes.eq(&prefs.tz_offset_minutes),
|
||||||
dsl::library.eq(&prefs.library),
|
dsl::library.eq(&prefs.library),
|
||||||
dsl::display_tz.eq(&prefs.display_tz),
|
dsl::display_tz.eq(&prefs.display_tz),
|
||||||
|
dsl::default_model.eq(&prefs.default_model),
|
||||||
|
dsl::default_backend.eq(&prefs.default_backend),
|
||||||
|
dsl::default_persona_id.eq(&prefs.default_persona_id),
|
||||||
|
dsl::reel_span.eq(&prefs.reel_span),
|
||||||
|
dsl::reel_max_segments.eq(&prefs.reel_max_segments),
|
||||||
dsl::updated_at.eq(&prefs.updated_at),
|
dsl::updated_at.eq(&prefs.updated_at),
|
||||||
))
|
))
|
||||||
.execute(connection.deref_mut())
|
.execute(connection.deref_mut())
|
||||||
@@ -147,6 +152,11 @@ mod tests {
|
|||||||
library: Some("1".to_string()),
|
library: Some("1".to_string()),
|
||||||
updated_at: now,
|
updated_at: now,
|
||||||
display_tz: Some("America/New_York".to_string()),
|
display_tz: Some("America/New_York".to_string()),
|
||||||
|
default_model: Some("llama3.2:3b".to_string()),
|
||||||
|
default_backend: Some("hybrid".to_string()),
|
||||||
|
default_persona_id: Some("cabin".to_string()),
|
||||||
|
reel_span: Some("week".to_string()),
|
||||||
|
reel_max_segments: Some(24),
|
||||||
};
|
};
|
||||||
dao.upsert_prefs(&ctx(), &prefs).unwrap();
|
dao.upsert_prefs(&ctx(), &prefs).unwrap();
|
||||||
|
|
||||||
@@ -156,6 +166,11 @@ mod tests {
|
|||||||
assert_eq!(row.tz_offset_minutes, Some(-480));
|
assert_eq!(row.tz_offset_minutes, Some(-480));
|
||||||
assert_eq!(row.library, Some("1".to_string()));
|
assert_eq!(row.library, Some("1".to_string()));
|
||||||
assert_eq!(row.display_tz, Some("America/New_York".to_string()));
|
assert_eq!(row.display_tz, Some("America/New_York".to_string()));
|
||||||
|
assert_eq!(row.default_model, Some("llama3.2:3b".to_string()));
|
||||||
|
assert_eq!(row.default_backend, Some("hybrid".to_string()));
|
||||||
|
assert_eq!(row.default_persona_id, Some("cabin".to_string()));
|
||||||
|
assert_eq!(row.reel_span, Some("week".to_string()));
|
||||||
|
assert_eq!(row.reel_max_segments, Some(24));
|
||||||
assert_eq!(row.updated_at, now);
|
assert_eq!(row.updated_at, now);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -171,6 +186,11 @@ mod tests {
|
|||||||
library: Some("1".to_string()),
|
library: Some("1".to_string()),
|
||||||
updated_at: now1,
|
updated_at: now1,
|
||||||
display_tz: Some("America/New_York".to_string()),
|
display_tz: Some("America/New_York".to_string()),
|
||||||
|
default_model: Some("llama3.2:3b".to_string()),
|
||||||
|
default_backend: Some("local".to_string()),
|
||||||
|
default_persona_id: None,
|
||||||
|
reel_span: Some("week".to_string()),
|
||||||
|
reel_max_segments: Some(24),
|
||||||
};
|
};
|
||||||
dao.upsert_prefs(&ctx(), &prefs1).unwrap();
|
dao.upsert_prefs(&ctx(), &prefs1).unwrap();
|
||||||
|
|
||||||
@@ -180,6 +200,11 @@ mod tests {
|
|||||||
library: None,
|
library: None,
|
||||||
updated_at: now2,
|
updated_at: now2,
|
||||||
display_tz: None,
|
display_tz: None,
|
||||||
|
default_model: None,
|
||||||
|
default_backend: None,
|
||||||
|
default_persona_id: None,
|
||||||
|
reel_span: None,
|
||||||
|
reel_max_segments: None,
|
||||||
};
|
};
|
||||||
dao.upsert_prefs(&ctx(), &prefs2).unwrap();
|
dao.upsert_prefs(&ctx(), &prefs2).unwrap();
|
||||||
|
|
||||||
@@ -188,6 +213,11 @@ mod tests {
|
|||||||
assert_eq!(row.tz_offset_minutes, Some(-300));
|
assert_eq!(row.tz_offset_minutes, Some(-300));
|
||||||
assert!(row.library.is_none());
|
assert!(row.library.is_none());
|
||||||
assert!(row.display_tz.is_none());
|
assert!(row.display_tz.is_none());
|
||||||
|
// Owner-default columns are overwritten to NULL like the rest.
|
||||||
|
assert!(row.default_model.is_none());
|
||||||
|
assert!(row.default_backend.is_none());
|
||||||
|
assert!(row.reel_span.is_none());
|
||||||
|
assert!(row.reel_max_segments.is_none());
|
||||||
assert_eq!(row.updated_at, now2);
|
assert_eq!(row.updated_at, now2);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -202,6 +232,11 @@ mod tests {
|
|||||||
library: None,
|
library: None,
|
||||||
updated_at: now,
|
updated_at: now,
|
||||||
display_tz: Some("Europe/Berlin".to_string()),
|
display_tz: Some("Europe/Berlin".to_string()),
|
||||||
|
default_model: None,
|
||||||
|
default_backend: None,
|
||||||
|
default_persona_id: Some("default".to_string()),
|
||||||
|
reel_span: Some("month".to_string()),
|
||||||
|
reel_max_segments: None,
|
||||||
};
|
};
|
||||||
dao.upsert_prefs(&ctx(), &prefs).unwrap();
|
dao.upsert_prefs(&ctx(), &prefs).unwrap();
|
||||||
|
|
||||||
@@ -210,5 +245,9 @@ mod tests {
|
|||||||
assert!(row.voice.is_none());
|
assert!(row.voice.is_none());
|
||||||
assert!(row.library.is_none());
|
assert!(row.library.is_none());
|
||||||
assert_eq!(row.display_tz, Some("Europe/Berlin".to_string()));
|
assert_eq!(row.display_tz, Some("Europe/Berlin".to_string()));
|
||||||
|
assert!(row.default_model.is_none());
|
||||||
|
assert_eq!(row.default_persona_id, Some("default".to_string()));
|
||||||
|
assert_eq!(row.reel_span, Some("month".to_string()));
|
||||||
|
assert!(row.reel_max_segments.is_none());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
+2
-1
@@ -859,7 +859,8 @@ async fn synthesize_merge<D: KnowledgeDao + 'static>(
|
|||||||
|
|
||||||
let ollama = app_state.ollama.clone();
|
let ollama = app_state.ollama.clone();
|
||||||
let model_used = ollama.primary_model.clone();
|
let model_used = ollama.primary_model.clone();
|
||||||
let proposed = match ollama.generate(&prompt, Some(system)).await {
|
let cx = opentelemetry::Context::current();
|
||||||
|
let proposed = match ollama.generate(&cx, &prompt, Some(system)).await {
|
||||||
Ok(out) => {
|
Ok(out) => {
|
||||||
// Strip the framing models reach for even with explicit
|
// Strip the framing models reach for even with explicit
|
||||||
// "no preamble" guidance: leading "Merged description:"
|
// "no preamble" guidance: leading "Merged description:"
|
||||||
|
|||||||
@@ -108,6 +108,31 @@ pub enum MemoriesSpan {
|
|||||||
Month,
|
Month,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Wire/storage string for the span (`"day"` | `"week"` | `"month"`) — the
|
||||||
|
/// same lowercase names as the serde rename, so a stored pref round-trips
|
||||||
|
/// through `MemoriesRequest.span` unchanged.
|
||||||
|
impl std::fmt::Display for MemoriesSpan {
|
||||||
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||||
|
f.write_str(match self {
|
||||||
|
MemoriesSpan::Day => "day",
|
||||||
|
MemoriesSpan::Week => "week",
|
||||||
|
MemoriesSpan::Month => "month",
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl std::str::FromStr for MemoriesSpan {
|
||||||
|
type Err = String;
|
||||||
|
fn from_str(s: &str) -> Result<Self, Self::Err> {
|
||||||
|
match s.trim().to_ascii_lowercase().as_str() {
|
||||||
|
"day" => Ok(MemoriesSpan::Day),
|
||||||
|
"week" => Ok(MemoriesSpan::Week),
|
||||||
|
"month" => Ok(MemoriesSpan::Month),
|
||||||
|
other => Err(format!("invalid span {other:?} (expected day|week|month)")),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#[derive(Deserialize)]
|
#[derive(Deserialize)]
|
||||||
pub struct MemoriesRequest {
|
pub struct MemoriesRequest {
|
||||||
pub span: Option<MemoriesSpan>,
|
pub span: Option<MemoriesSpan>,
|
||||||
|
|||||||
+126
-1
@@ -2,7 +2,9 @@ use actix_web::HttpRequest;
|
|||||||
use actix_web::http::header::HeaderMap;
|
use actix_web::http::header::HeaderMap;
|
||||||
use opentelemetry::global::{BoxedSpan, BoxedTracer};
|
use opentelemetry::global::{BoxedSpan, BoxedTracer};
|
||||||
use opentelemetry::propagation::TextMapPropagator;
|
use opentelemetry::propagation::TextMapPropagator;
|
||||||
use opentelemetry::trace::{Span, Status, Tracer};
|
use opentelemetry::trace::{
|
||||||
|
Span, SpanBuilder, SpanContext, SpanKind, Status, TraceContextExt, Tracer,
|
||||||
|
};
|
||||||
use opentelemetry::{Context, KeyValue, global};
|
use opentelemetry::{Context, KeyValue, global};
|
||||||
use opentelemetry_appender_log::OpenTelemetryLogBridge;
|
use opentelemetry_appender_log::OpenTelemetryLogBridge;
|
||||||
use opentelemetry_otlp::WithExportConfig;
|
use opentelemetry_otlp::WithExportConfig;
|
||||||
@@ -77,11 +79,78 @@ impl<'a> opentelemetry::propagation::Extractor for HeaderExtractor<'a> {
|
|||||||
self.0.keys().map(|k| k.as_str()).collect()
|
self.0.keys().map(|k| k.as_str()).collect()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
/// Start a client span for a single LLM call. Every provider funnels its
|
||||||
|
/// LlmClient methods through this so the span shape (name, `SpanKind::Client`,
|
||||||
|
/// provider / operation / model attributes) is identical across
|
||||||
|
/// openrouter / llamacpp / ollama. `operation` is one of `chat`,
|
||||||
|
/// `chat_stream`, `embeddings`, `describe_image`, `generate`.
|
||||||
|
pub fn start_llm_request_span(
|
||||||
|
cx: &Context,
|
||||||
|
provider: &str,
|
||||||
|
operation: &str,
|
||||||
|
model: &str,
|
||||||
|
) -> BoxedSpan {
|
||||||
|
let builder = SpanBuilder::from_name("llm.request").with_kind(SpanKind::Client);
|
||||||
|
let mut span = global_tracer().build_with_context(builder, cx);
|
||||||
|
span.set_attribute(KeyValue::new("llm.provider", provider.to_string()));
|
||||||
|
span.set_attribute(KeyValue::new("llm.operation", operation.to_string()));
|
||||||
|
span.set_attribute(KeyValue::new("llm.model", model.to_string()));
|
||||||
|
span
|
||||||
|
}
|
||||||
|
|
||||||
pub fn extract_context_from_request(req: &HttpRequest) -> Context {
|
pub fn extract_context_from_request(req: &HttpRequest) -> Context {
|
||||||
let propagator = TraceContextPropagator::new();
|
let propagator = TraceContextPropagator::new();
|
||||||
propagator.extract(&HeaderExtractor(req.headers()))
|
propagator.extract(&HeaderExtractor(req.headers()))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Build the W3C trace-context headers to attach to an outgoing HTTP
|
||||||
|
/// request so a downstream service (llama-swap, Ollama) can parent its own
|
||||||
|
/// spans onto this trace. Returns an empty vec when the context carries no
|
||||||
|
/// valid span — e.g. debug builds, where the global tracer is a no-op and
|
||||||
|
/// injecting an all-zero traceparent would produce garbage parent spans.
|
||||||
|
pub fn traceparent_headers(context: &Context) -> Vec<(String, String)> {
|
||||||
|
if !context.span().span_context().is_valid() {
|
||||||
|
return Vec::new();
|
||||||
|
}
|
||||||
|
let mut headers: std::collections::HashMap<String, String> = std::collections::HashMap::new();
|
||||||
|
TraceContextPropagator::new().inject_context(context, &mut headers);
|
||||||
|
// The propagator writes an empty tracestate header when there is no
|
||||||
|
// state — drop empty-valued headers rather than sending noise.
|
||||||
|
headers.into_iter().filter(|(_, v)| !v.is_empty()).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Attach the W3C traceparent header to an outgoing request so a downstream
|
||||||
|
/// service parents its own spans onto this trace. No-op when the span context
|
||||||
|
/// is not valid — e.g. debug builds, where the global tracer is a no-op and an
|
||||||
|
/// all-zero traceparent would produce garbage parent spans.
|
||||||
|
pub fn with_traceparent(
|
||||||
|
builder: reqwest::RequestBuilder,
|
||||||
|
span_context: &SpanContext,
|
||||||
|
) -> reqwest::RequestBuilder {
|
||||||
|
let mut b = builder;
|
||||||
|
for (k, v) in
|
||||||
|
traceparent_headers(&Context::new().with_remote_span_context(span_context.clone()))
|
||||||
|
{
|
||||||
|
b = b.header(k, v);
|
||||||
|
}
|
||||||
|
b
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Record the HTTP status code on the span (success or failure path).
|
||||||
|
pub fn set_http_status(span: Option<&mut BoxedSpan>, status: u16) {
|
||||||
|
if let Some(s) = span {
|
||||||
|
s.set_attribute(KeyValue::new("http.status_code", status as i64));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Record an HTTP failure on the span: status code plus an error status.
|
||||||
|
pub fn record_http_failure(span: Option<&mut BoxedSpan>, status: u16, message: &str) {
|
||||||
|
if let Some(s) = span {
|
||||||
|
s.set_attribute(KeyValue::new("http.status_code", status as i64));
|
||||||
|
s.set_status(Status::error(message.to_string()));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
pub fn trace_db_call<F, O>(
|
pub fn trace_db_call<F, O>(
|
||||||
context: &Context,
|
context: &Context,
|
||||||
query_type: &str,
|
query_type: &str,
|
||||||
@@ -110,3 +179,59 @@ where
|
|||||||
|
|
||||||
result
|
result
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod traceparent_tests {
|
||||||
|
use super::*;
|
||||||
|
use opentelemetry::trace::{
|
||||||
|
SpanContext, SpanId, TraceContextExt, TraceFlags, TraceId, TraceState,
|
||||||
|
};
|
||||||
|
|
||||||
|
/// A context carrying a span context with known ids — no global
|
||||||
|
/// provider needed, so tests can't pollute process state.
|
||||||
|
fn context_with_flags(flags: TraceFlags) -> Context {
|
||||||
|
let trace_id = TraceId::from_bytes([
|
||||||
|
0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e,
|
||||||
|
0x0f, 0x10,
|
||||||
|
]);
|
||||||
|
let span_id = SpanId::from_bytes([0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08]);
|
||||||
|
let span_context = SpanContext::new(trace_id, span_id, flags, false, TraceState::default());
|
||||||
|
Context::new().with_remote_span_context(span_context)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn traceparent_headers_sampled_context() {
|
||||||
|
let cx = context_with_flags(TraceFlags::SAMPLED);
|
||||||
|
let headers = traceparent_headers(&cx);
|
||||||
|
assert_eq!(
|
||||||
|
headers,
|
||||||
|
vec![(
|
||||||
|
"traceparent".to_string(),
|
||||||
|
"00-0102030405060708090a0b0c0d0e0f10-0102030405060708-01".to_string(),
|
||||||
|
)]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn traceparent_headers_not_sampled_context() {
|
||||||
|
// A valid-but-unsampled context still propagates; the flags byte
|
||||||
|
// carries the state.
|
||||||
|
let cx = context_with_flags(TraceFlags::NOT_SAMPLED);
|
||||||
|
let headers = traceparent_headers(&cx);
|
||||||
|
assert_eq!(
|
||||||
|
headers,
|
||||||
|
vec![(
|
||||||
|
"traceparent".to_string(),
|
||||||
|
"00-0102030405060708090a0b0c0d0e0f10-0102030405060708-00".to_string(),
|
||||||
|
)]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn traceparent_headers_invalid_context_is_empty() {
|
||||||
|
// No span (debug-build no-op tracer) → no header, never an
|
||||||
|
// all-zero traceparent.
|
||||||
|
let cx = Context::new();
|
||||||
|
assert!(traceparent_headers(&cx).is_empty());
|
||||||
|
}
|
||||||
|
}
|
||||||
+412
-70
@@ -8,15 +8,19 @@
|
|||||||
//! `GET /timezones` — the IANA zone list from chrono-tz, for picker UIs
|
//! `GET /timezones` — the IANA zone list from chrono-tz, for picker UIs
|
||||||
//! (unauthenticated; static data).
|
//! (unauthenticated; static data).
|
||||||
//!
|
//!
|
||||||
//! Scope: existing columns only (voice, library, display_tz) — no migration.
|
//! Surface: all persisted prefs — the passive mirror (voice, library,
|
||||||
//! The legacy `tz_offset_minutes` column is preserved verbatim on every
|
//! display_tz) and the owner defaults (default_model, default_backend,
|
||||||
//! write. `POST /reels`'s `capture_prefs` stays as the incidental writer;
|
//! default_persona_id, reel_span, reel_max_segments). Consumers resolve
|
||||||
//! both writers hit the same single row, so they're idempotent. Owner
|
//! `request param → stored pref → hard default`; this endpoint is the only
|
||||||
//! defaults (model/backend/persona/reels) are a separate follow-on ticket
|
//! place the owner-default fields are written. `POST /reels`'s
|
||||||
//! that adds the columns.
|
//! `capture_prefs` stays as the incidental writer for the mirror fields
|
||||||
|
//! (plus the effective reel span/segments); both writers hit the same
|
||||||
|
//! single row, so they're idempotent. The legacy `tz_offset_minutes`
|
||||||
|
//! column is preserved verbatim on every write.
|
||||||
|
|
||||||
use actix_web::{HttpResponse, web};
|
use actix_web::{HttpResponse, web};
|
||||||
use chrono_tz::Tz;
|
use chrono_tz::Tz;
|
||||||
|
use serde::de::IntoDeserializer;
|
||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
use serde_json::json;
|
use serde_json::json;
|
||||||
use std::str::FromStr;
|
use std::str::FromStr;
|
||||||
@@ -24,6 +28,7 @@ use std::time::{SystemTime, UNIX_EPOCH};
|
|||||||
|
|
||||||
use crate::data::Claims;
|
use crate::data::Claims;
|
||||||
use crate::database::models::{UpsertUserAiPrefs, UserAiPrefs};
|
use crate::database::models::{UpsertUserAiPrefs, UserAiPrefs};
|
||||||
|
use crate::memories::MemoriesSpan;
|
||||||
use crate::state::AppState;
|
use crate::state::AppState;
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -35,6 +40,18 @@ pub struct PrefsResponse {
|
|||||||
pub voice: Option<String>,
|
pub voice: Option<String>,
|
||||||
pub library: Option<String>,
|
pub library: Option<String>,
|
||||||
pub display_tz: Option<String>,
|
pub display_tz: Option<String>,
|
||||||
|
/// Last/explicit insight + chat model choice. NULL = per-request /
|
||||||
|
/// env default behavior.
|
||||||
|
pub default_model: Option<String>,
|
||||||
|
/// `"local"` | `"hybrid"`. NULL = per-request / hard default.
|
||||||
|
pub default_backend: Option<String>,
|
||||||
|
/// Active persona for persona chat. NULL = hard default ("default").
|
||||||
|
pub default_persona_id: Option<String>,
|
||||||
|
/// Reel span: `"day"` | `"week"` | `"month"`. NULL = per-request /
|
||||||
|
/// hard default (day).
|
||||||
|
pub reel_span: Option<String>,
|
||||||
|
/// Reel segment cap. NULL = per-request / hard default.
|
||||||
|
pub reel_max_segments: Option<i32>,
|
||||||
pub updated_at: Option<i64>,
|
pub updated_at: Option<i64>,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -43,41 +60,59 @@ pub struct PrefsResponse {
|
|||||||
/// explicit null and a missing key apart on `Option<Option<T>>` (both come
|
/// explicit null and a missing key apart on `Option<Option<T>>` (both come
|
||||||
/// out as outer `None`), so this carries a hand-rolled visitor.
|
/// out as outer `None`), so this carries a hand-rolled visitor.
|
||||||
#[derive(Clone, Debug, Default)]
|
#[derive(Clone, Debug, Default)]
|
||||||
pub enum TriState {
|
pub enum TriState<T> {
|
||||||
#[default]
|
#[default]
|
||||||
Absent,
|
Absent,
|
||||||
Null,
|
Null,
|
||||||
Val(String),
|
Val(T),
|
||||||
}
|
}
|
||||||
|
|
||||||
impl<'de> Deserialize<'de> for TriState {
|
impl<'de, T: Deserialize<'de>> Deserialize<'de> for TriState<T> {
|
||||||
fn deserialize<D>(deserializer: D) -> Result<TriState, D::Error>
|
fn deserialize<D>(deserializer: D) -> Result<TriState<T>, D::Error>
|
||||||
where
|
where
|
||||||
D: serde::Deserializer<'de>,
|
D: serde::Deserializer<'de>,
|
||||||
{
|
{
|
||||||
struct Visitor;
|
struct Visitor<T>(std::marker::PhantomData<T>);
|
||||||
impl<'de> serde::de::Visitor<'de> for Visitor {
|
impl<'de, T: Deserialize<'de>> serde::de::Visitor<'de> for Visitor<T> {
|
||||||
type Value = TriState;
|
type Value = TriState<T>;
|
||||||
fn expecting(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
|
fn expecting(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
|
||||||
f.write_str("a string or null")
|
f.write_str("a value or null")
|
||||||
}
|
}
|
||||||
// serde_json routes an explicit JSON null to visit_unit/visit_none;
|
// serde_json routes an explicit JSON null to visit_unit/visit_none;
|
||||||
// a missing key never calls deserialize — `#[serde(default)]`
|
// a missing key never calls deserialize — `#[serde(default)]`
|
||||||
// supplies Absent instead.
|
// supplies Absent instead. Concrete values are re-deserialized into
|
||||||
fn visit_unit<E: serde::de::Error>(self) -> Result<TriState, E> {
|
// T through single-value deserializers (serde_json's
|
||||||
|
// deserialize_any dispatches directly to visit_str/visit_i64/…
|
||||||
|
// — visit_some only fires on the deserialize_option path).
|
||||||
|
fn visit_unit<E: serde::de::Error>(self) -> Result<TriState<T>, E> {
|
||||||
Ok(TriState::Null)
|
Ok(TriState::Null)
|
||||||
}
|
}
|
||||||
fn visit_none<E: serde::de::Error>(self) -> Result<TriState, E> {
|
fn visit_none<E: serde::de::Error>(self) -> Result<TriState<T>, E> {
|
||||||
Ok(TriState::Null)
|
Ok(TriState::Null)
|
||||||
}
|
}
|
||||||
fn visit_str<E: serde::de::Error>(self, v: &str) -> Result<TriState, E> {
|
fn visit_some<D2>(self, d: D2) -> Result<TriState<T>, D2::Error>
|
||||||
Ok(TriState::Val(v.to_string()))
|
where
|
||||||
|
D2: serde::Deserializer<'de>,
|
||||||
|
{
|
||||||
|
Ok(TriState::Val(T::deserialize(d)?))
|
||||||
}
|
}
|
||||||
fn visit_string<E: serde::de::Error>(self, v: String) -> Result<TriState, E> {
|
// `into_deserializer()` gives a single-value deserializer whose
|
||||||
Ok(TriState::Val(v))
|
// error type is the visitor's E, so T::deserialize lands on
|
||||||
|
// Result<T, E> with no conversion.
|
||||||
|
fn visit_str<E: serde::de::Error>(self, v: &str) -> Result<TriState<T>, E> {
|
||||||
|
T::deserialize(v.into_deserializer()).map(TriState::Val)
|
||||||
|
}
|
||||||
|
fn visit_string<E: serde::de::Error>(self, v: String) -> Result<TriState<T>, E> {
|
||||||
|
T::deserialize(v.as_str().into_deserializer()).map(TriState::Val)
|
||||||
|
}
|
||||||
|
fn visit_i64<E: serde::de::Error>(self, v: i64) -> Result<TriState<T>, E> {
|
||||||
|
T::deserialize(v.into_deserializer()).map(TriState::Val)
|
||||||
|
}
|
||||||
|
fn visit_u64<E: serde::de::Error>(self, v: u64) -> Result<TriState<T>, E> {
|
||||||
|
T::deserialize(v.into_deserializer()).map(TriState::Val)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
deserializer.deserialize_any(Visitor)
|
deserializer.deserialize_any(Visitor(std::marker::PhantomData))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -86,11 +121,21 @@ impl<'de> Deserialize<'de> for TriState {
|
|||||||
#[derive(Deserialize, Debug, Default)]
|
#[derive(Deserialize, Debug, Default)]
|
||||||
pub struct UpdatePrefsRequest {
|
pub struct UpdatePrefsRequest {
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub voice: TriState,
|
pub voice: TriState<String>,
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub library: TriState,
|
pub library: TriState<String>,
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
pub display_tz: TriState,
|
pub display_tz: TriState<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub default_model: TriState<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub default_backend: TriState<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub default_persona_id: TriState<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub reel_span: TriState<String>,
|
||||||
|
#[serde(default)]
|
||||||
|
pub reel_max_segments: TriState<i32>,
|
||||||
}
|
}
|
||||||
|
|
||||||
#[derive(Serialize, Debug)]
|
#[derive(Serialize, Debug)]
|
||||||
@@ -109,6 +154,18 @@ fn is_valid_display_tz(name: &str) -> bool {
|
|||||||
Tz::from_str(name.trim()).is_ok()
|
Tz::from_str(name.trim()).is_ok()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A stored `default_backend` must be consumable downstream, i.e. accepted
|
||||||
|
/// by the canonical `BackendKind::parse` ("local"/"hybrid", case-insensitive).
|
||||||
|
/// Delegating to that parser keeps the write-time validator and the read-time
|
||||||
|
/// consumer from drifting apart; anything else stored here would 500 on every
|
||||||
|
/// insight/chat request instead of failing once at write time. The blank
|
||||||
|
/// string the parser tolerates (empty request field = app default) is
|
||||||
|
/// rejected here — a stored default is an explicit choice, and NULL already
|
||||||
|
/// means "no default".
|
||||||
|
fn is_valid_backend(name: &str) -> bool {
|
||||||
|
!name.trim().is_empty() && crate::ai::backend::BackendKind::parse(name).is_ok()
|
||||||
|
}
|
||||||
|
|
||||||
/// The full set of zone names `PUT /prefs` will accept — `chrono_tz::TZ_VARIANTS`
|
/// The full set of zone names `PUT /prefs` will accept — `chrono_tz::TZ_VARIANTS`
|
||||||
/// is exactly the domain of `Tz::from_str`, so the picker list and the
|
/// is exactly the domain of `Tz::from_str`, so the picker list and the
|
||||||
/// validation check can never drift apart.
|
/// validation check can never drift apart.
|
||||||
@@ -119,14 +176,13 @@ pub fn all_timezones() -> Vec<String> {
|
|||||||
.collect()
|
.collect()
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Merge a partial update onto the existing row. Returns
|
/// Merge a partial update onto the existing row. An omitted field keeps
|
||||||
/// `(voice, library, display_tz, changed)`. An omitted field keeps the
|
/// the stored value (or `None` when there is no row); a present field wins.
|
||||||
/// stored value (or `None` when there is no row); a present field wins.
|
fn tri_value<T: Clone>(
|
||||||
fn tri_value(
|
field: &TriState<T>,
|
||||||
field: &TriState,
|
|
||||||
existing: Option<&UserAiPrefs>,
|
existing: Option<&UserAiPrefs>,
|
||||||
keep: impl FnOnce(&UserAiPrefs) -> Option<String>,
|
keep: impl FnOnce(&UserAiPrefs) -> Option<T>,
|
||||||
) -> Option<String> {
|
) -> Option<T> {
|
||||||
match field {
|
match field {
|
||||||
TriState::Absent => existing.and_then(keep),
|
TriState::Absent => existing.and_then(keep),
|
||||||
TriState::Null => None,
|
TriState::Null => None,
|
||||||
@@ -134,17 +190,43 @@ fn tri_value(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn merge_patch(
|
/// The eight partial-update fields after a merge (everything except the
|
||||||
existing: Option<&UserAiPrefs>,
|
/// frozen `tz_offset_minutes`, which the handler copies through verbatim).
|
||||||
patch: &UpdatePrefsRequest,
|
struct MergedPrefs {
|
||||||
) -> (Option<String>, Option<String>, Option<String>, bool) {
|
voice: Option<String>,
|
||||||
let voice = tri_value(&patch.voice, existing, |e| e.voice.clone());
|
library: Option<String>,
|
||||||
let library = tri_value(&patch.library, existing, |e| e.library.clone());
|
display_tz: Option<String>,
|
||||||
let display_tz = tri_value(&patch.display_tz, existing, |e| e.display_tz.clone());
|
default_model: Option<String>,
|
||||||
|
default_backend: Option<String>,
|
||||||
|
default_persona_id: Option<String>,
|
||||||
|
reel_span: Option<String>,
|
||||||
|
reel_max_segments: Option<i32>,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn merge_patch(existing: Option<&UserAiPrefs>, patch: &UpdatePrefsRequest) -> (MergedPrefs, bool) {
|
||||||
|
let merged = MergedPrefs {
|
||||||
|
voice: tri_value(&patch.voice, existing, |e| e.voice.clone()),
|
||||||
|
library: tri_value(&patch.library, existing, |e| e.library.clone()),
|
||||||
|
display_tz: tri_value(&patch.display_tz, existing, |e| e.display_tz.clone()),
|
||||||
|
default_model: tri_value(&patch.default_model, existing, |e| e.default_model.clone()),
|
||||||
|
default_backend: tri_value(&patch.default_backend, existing, |e| {
|
||||||
|
e.default_backend.clone()
|
||||||
|
}),
|
||||||
|
default_persona_id: tri_value(&patch.default_persona_id, existing, |e| {
|
||||||
|
e.default_persona_id.clone()
|
||||||
|
}),
|
||||||
|
reel_span: tri_value(&patch.reel_span, existing, |e| e.reel_span.clone()),
|
||||||
|
reel_max_segments: tri_value(&patch.reel_max_segments, existing, |e| e.reel_max_segments),
|
||||||
|
};
|
||||||
let changed = !matches!(patch.voice, TriState::Absent)
|
let changed = !matches!(patch.voice, TriState::Absent)
|
||||||
|| !matches!(patch.library, TriState::Absent)
|
|| !matches!(patch.library, TriState::Absent)
|
||||||
|| !matches!(patch.display_tz, TriState::Absent);
|
|| !matches!(patch.display_tz, TriState::Absent)
|
||||||
(voice, library, display_tz, changed)
|
|| !matches!(patch.default_model, TriState::Absent)
|
||||||
|
|| !matches!(patch.default_backend, TriState::Absent)
|
||||||
|
|| !matches!(patch.default_persona_id, TriState::Absent)
|
||||||
|
|| !matches!(patch.reel_span, TriState::Absent)
|
||||||
|
|| !matches!(patch.reel_max_segments, TriState::Absent);
|
||||||
|
(merged, changed)
|
||||||
}
|
}
|
||||||
|
|
||||||
fn row_to_response(row: Option<&UserAiPrefs>) -> PrefsResponse {
|
fn row_to_response(row: Option<&UserAiPrefs>) -> PrefsResponse {
|
||||||
@@ -153,12 +235,22 @@ fn row_to_response(row: Option<&UserAiPrefs>) -> PrefsResponse {
|
|||||||
voice: row.voice.clone(),
|
voice: row.voice.clone(),
|
||||||
library: row.library.clone(),
|
library: row.library.clone(),
|
||||||
display_tz: row.display_tz.clone(),
|
display_tz: row.display_tz.clone(),
|
||||||
|
default_model: row.default_model.clone(),
|
||||||
|
default_backend: row.default_backend.clone(),
|
||||||
|
default_persona_id: row.default_persona_id.clone(),
|
||||||
|
reel_span: row.reel_span.clone(),
|
||||||
|
reel_max_segments: row.reel_max_segments,
|
||||||
updated_at: Some(row.updated_at),
|
updated_at: Some(row.updated_at),
|
||||||
},
|
},
|
||||||
None => PrefsResponse {
|
None => PrefsResponse {
|
||||||
voice: None,
|
voice: None,
|
||||||
library: None,
|
library: None,
|
||||||
display_tz: None,
|
display_tz: None,
|
||||||
|
default_model: None,
|
||||||
|
default_backend: None,
|
||||||
|
default_persona_id: None,
|
||||||
|
reel_span: None,
|
||||||
|
reel_max_segments: None,
|
||||||
updated_at: None,
|
updated_at: None,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -192,7 +284,8 @@ pub async fn put_prefs(
|
|||||||
) -> HttpResponse {
|
) -> HttpResponse {
|
||||||
let patch = body.0;
|
let patch = body.0;
|
||||||
|
|
||||||
// Validate before any write: an invalid zone name → 400, nothing written.
|
// Validate before any write: an invalid value → 400, nothing written
|
||||||
|
// (not even the other fields in the same body).
|
||||||
if let TriState::Val(name) = &patch.display_tz
|
if let TriState::Val(name) = &patch.display_tz
|
||||||
&& !is_valid_display_tz(name)
|
&& !is_valid_display_tz(name)
|
||||||
{
|
{
|
||||||
@@ -200,6 +293,20 @@ pub async fn put_prefs(
|
|||||||
"error": format!("invalid display_tz: {name}")
|
"error": format!("invalid display_tz: {name}")
|
||||||
}));
|
}));
|
||||||
}
|
}
|
||||||
|
if let TriState::Val(name) = &patch.default_backend
|
||||||
|
&& !is_valid_backend(name)
|
||||||
|
{
|
||||||
|
return HttpResponse::BadRequest().json(json!({
|
||||||
|
"error": format!("invalid default_backend: {name} (expected local|hybrid)")
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
if let TriState::Val(name) = &patch.reel_span
|
||||||
|
&& MemoriesSpan::from_str(name).is_err()
|
||||||
|
{
|
||||||
|
return HttpResponse::BadRequest().json(json!({
|
||||||
|
"error": format!("invalid reel_span: {name} (expected day|week|month)")
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
let mut dao = match app_state.user_ai_prefs_dao.lock() {
|
let mut dao = match app_state.user_ai_prefs_dao.lock() {
|
||||||
Ok(dao) => dao,
|
Ok(dao) => dao,
|
||||||
@@ -219,7 +326,7 @@ pub async fn put_prefs(
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
let (voice, library, display_tz, changed) = merge_patch(existing.as_ref(), &patch);
|
let (merged, changed) = merge_patch(existing.as_ref(), &patch);
|
||||||
if !changed {
|
if !changed {
|
||||||
// Empty body: no-op, return the current row without bumping updated_at.
|
// Empty body: no-op, return the current row without bumping updated_at.
|
||||||
return HttpResponse::Ok().json(row_to_response(existing.as_ref()));
|
return HttpResponse::Ok().json(row_to_response(existing.as_ref()));
|
||||||
@@ -231,18 +338,28 @@ pub async fn put_prefs(
|
|||||||
.as_secs() as i64;
|
.as_secs() as i64;
|
||||||
|
|
||||||
let resp = PrefsResponse {
|
let resp = PrefsResponse {
|
||||||
voice: voice.clone(),
|
voice: merged.voice.clone(),
|
||||||
library: library.clone(),
|
library: merged.library.clone(),
|
||||||
display_tz: display_tz.clone(),
|
display_tz: merged.display_tz.clone(),
|
||||||
|
default_model: merged.default_model.clone(),
|
||||||
|
default_backend: merged.default_backend.clone(),
|
||||||
|
default_persona_id: merged.default_persona_id.clone(),
|
||||||
|
reel_span: merged.reel_span.clone(),
|
||||||
|
reel_max_segments: merged.reel_max_segments,
|
||||||
updated_at: Some(now),
|
updated_at: Some(now),
|
||||||
};
|
};
|
||||||
let upsert = UpsertUserAiPrefs {
|
let upsert = UpsertUserAiPrefs {
|
||||||
voice,
|
voice: merged.voice,
|
||||||
// Frozen legacy column: preserved verbatim, never written by this
|
// Frozen legacy column: preserved verbatim, never written by this
|
||||||
// surface.
|
// surface.
|
||||||
tz_offset_minutes: existing.as_ref().and_then(|e| e.tz_offset_minutes),
|
tz_offset_minutes: existing.as_ref().and_then(|e| e.tz_offset_minutes),
|
||||||
library,
|
library: merged.library,
|
||||||
display_tz,
|
display_tz: merged.display_tz,
|
||||||
|
default_model: merged.default_model,
|
||||||
|
default_backend: merged.default_backend,
|
||||||
|
default_persona_id: merged.default_persona_id,
|
||||||
|
reel_span: merged.reel_span,
|
||||||
|
reel_max_segments: merged.reel_max_segments,
|
||||||
updated_at: now,
|
updated_at: now,
|
||||||
};
|
};
|
||||||
match dao.upsert_prefs(&ctx, &upsert) {
|
match dao.upsert_prefs(&ctx, &upsert) {
|
||||||
@@ -311,6 +428,27 @@ mod tests {
|
|||||||
library: library.map(str::to_string),
|
library: library.map(str::to_string),
|
||||||
updated_at: 1_700_000_000,
|
updated_at: 1_700_000_000,
|
||||||
display_tz: display_tz.map(str::to_string),
|
display_tz: display_tz.map(str::to_string),
|
||||||
|
default_model: None,
|
||||||
|
default_backend: None,
|
||||||
|
default_persona_id: None,
|
||||||
|
reel_span: None,
|
||||||
|
reel_max_segments: None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A row with owner defaults populated, for the owner-default tests.
|
||||||
|
fn seeded_row_with_defaults() -> UpsertUserAiPrefs {
|
||||||
|
UpsertUserAiPrefs {
|
||||||
|
voice: Some("grandma".to_string()),
|
||||||
|
tz_offset_minutes: Some(-240),
|
||||||
|
library: Some("1".to_string()),
|
||||||
|
updated_at: 1_700_000_000,
|
||||||
|
display_tz: Some("America/New_York".to_string()),
|
||||||
|
default_model: Some("llama3.2:3b".to_string()),
|
||||||
|
default_backend: Some("hybrid".to_string()),
|
||||||
|
default_persona_id: Some("cabin".to_string()),
|
||||||
|
reel_span: Some("week".to_string()),
|
||||||
|
reel_max_segments: Some(24),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -363,6 +501,11 @@ mod tests {
|
|||||||
"voice": null,
|
"voice": null,
|
||||||
"library": null,
|
"library": null,
|
||||||
"display_tz": null,
|
"display_tz": null,
|
||||||
|
"default_model": null,
|
||||||
|
"default_backend": null,
|
||||||
|
"default_persona_id": null,
|
||||||
|
"reel_span": null,
|
||||||
|
"reel_max_segments": null,
|
||||||
"updated_at": null
|
"updated_at": null
|
||||||
})
|
})
|
||||||
);
|
);
|
||||||
@@ -682,43 +825,242 @@ mod tests {
|
|||||||
|
|
||||||
// --- merge_patch (pure) --------------------------------------------------
|
// --- merge_patch (pure) --------------------------------------------------
|
||||||
|
|
||||||
#[test]
|
fn row_with_defaults() -> UserAiPrefs {
|
||||||
fn merge_patch_omitted_keeps_existing() {
|
UserAiPrefs {
|
||||||
let row = UserAiPrefs {
|
|
||||||
id: 1,
|
id: 1,
|
||||||
voice: Some("a".into()),
|
voice: Some("a".into()),
|
||||||
tz_offset_minutes: None,
|
tz_offset_minutes: None,
|
||||||
library: Some("b".into()),
|
library: Some("b".into()),
|
||||||
updated_at: 0,
|
updated_at: 0,
|
||||||
display_tz: Some("c".into()),
|
display_tz: Some("c".into()),
|
||||||
};
|
default_model: Some("m1".into()),
|
||||||
let (voice, library, display_tz, changed) =
|
default_backend: Some("local".into()),
|
||||||
merge_patch(Some(&row), &UpdatePrefsRequest::default());
|
default_persona_id: Some("p1".into()),
|
||||||
assert_eq!(voice.as_deref(), Some("a"));
|
reel_span: Some("week".into()),
|
||||||
assert_eq!(library.as_deref(), Some("b"));
|
reel_max_segments: Some(24),
|
||||||
assert_eq!(display_tz.as_deref(), Some("c"));
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn merge_patch_omitted_keeps_existing() {
|
||||||
|
let row = row_with_defaults();
|
||||||
|
let (merged, changed) = merge_patch(Some(&row), &UpdatePrefsRequest::default());
|
||||||
|
assert_eq!(merged.voice.as_deref(), Some("a"));
|
||||||
|
assert_eq!(merged.library.as_deref(), Some("b"));
|
||||||
|
assert_eq!(merged.display_tz.as_deref(), Some("c"));
|
||||||
|
assert_eq!(merged.default_model.as_deref(), Some("m1"));
|
||||||
|
assert_eq!(merged.default_backend.as_deref(), Some("local"));
|
||||||
|
assert_eq!(merged.default_persona_id.as_deref(), Some("p1"));
|
||||||
|
assert_eq!(merged.reel_span.as_deref(), Some("week"));
|
||||||
|
assert_eq!(merged.reel_max_segments, Some(24));
|
||||||
assert!(!changed);
|
assert!(!changed);
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn merge_patch_present_field_wins_including_null() {
|
fn merge_patch_present_field_wins_including_null() {
|
||||||
let row = UserAiPrefs {
|
let row = row_with_defaults();
|
||||||
id: 1,
|
|
||||||
voice: Some("a".into()),
|
|
||||||
tz_offset_minutes: None,
|
|
||||||
library: None,
|
|
||||||
updated_at: 0,
|
|
||||||
display_tz: None,
|
|
||||||
};
|
|
||||||
let patch = UpdatePrefsRequest {
|
let patch = UpdatePrefsRequest {
|
||||||
voice: TriState::Null, // clear
|
voice: TriState::Null, // clear
|
||||||
library: TriState::Val(String::from("x")), // set
|
library: TriState::Val(String::from("x")), // set
|
||||||
display_tz: TriState::Absent,
|
display_tz: TriState::Absent,
|
||||||
|
default_backend: TriState::Null, // clear an owner default
|
||||||
|
reel_span: TriState::Val(String::from("month")),
|
||||||
|
reel_max_segments: TriState::Val(12),
|
||||||
|
..UpdatePrefsRequest::default()
|
||||||
};
|
};
|
||||||
let (voice, library, display_tz, changed) = merge_patch(Some(&row), &patch);
|
let (merged, changed) = merge_patch(Some(&row), &patch);
|
||||||
assert!(voice.is_none());
|
assert!(merged.voice.is_none());
|
||||||
assert_eq!(library.as_deref(), Some("x"));
|
assert_eq!(merged.library.as_deref(), Some("x"));
|
||||||
assert!(display_tz.is_none());
|
// Absent field keeps the stored value.
|
||||||
|
assert_eq!(merged.display_tz.as_deref(), Some("c"));
|
||||||
|
assert!(merged.default_backend.is_none());
|
||||||
|
assert_eq!(merged.reel_span.as_deref(), Some("month"));
|
||||||
|
assert_eq!(merged.reel_max_segments, Some(12));
|
||||||
|
// Untouched owner defaults survive.
|
||||||
|
assert_eq!(merged.default_model.as_deref(), Some("m1"));
|
||||||
|
assert_eq!(merged.default_persona_id.as_deref(), Some("p1"));
|
||||||
assert!(changed);
|
assert!(changed);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// --- PUT /prefs: owner defaults ------------------------------------------
|
||||||
|
|
||||||
|
#[actix_rt::test]
|
||||||
|
async fn put_prefs_owner_defaults_set_and_persist() {
|
||||||
|
let (state, mut seed_dao) = prefs_state();
|
||||||
|
seed(
|
||||||
|
&mut seed_dao,
|
||||||
|
seeded_row(Some("grandma"), Some("1"), Some("America/New_York")),
|
||||||
|
);
|
||||||
|
let app = init_service(
|
||||||
|
App::new()
|
||||||
|
.service(
|
||||||
|
web::resource("/prefs")
|
||||||
|
.route(web::get().to(get_prefs))
|
||||||
|
.route(web::put().to(put_prefs)),
|
||||||
|
)
|
||||||
|
.service(web::resource("/timezones").route(web::get().to(list_timezones)))
|
||||||
|
.app_data(state.clone()),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
let (header, value) = auth();
|
||||||
|
let req = TestRequest::put()
|
||||||
|
.uri("/prefs")
|
||||||
|
.insert_header((header, value))
|
||||||
|
.set_json(serde_json::json!({
|
||||||
|
"default_model": "llama3.2:3b",
|
||||||
|
"default_backend": "hybrid",
|
||||||
|
"default_persona_id": "cabin",
|
||||||
|
"reel_span": "week",
|
||||||
|
"reel_max_segments": 24
|
||||||
|
}))
|
||||||
|
.to_request();
|
||||||
|
let resp = call_service(&app, req).await;
|
||||||
|
assert_eq!(resp.status(), 200);
|
||||||
|
|
||||||
|
let mut dao = state.user_ai_prefs_dao.lock().unwrap();
|
||||||
|
let row = dao
|
||||||
|
.get_prefs(&opentelemetry::Context::new())
|
||||||
|
.unwrap()
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(row.default_model.as_deref(), Some("llama3.2:3b"));
|
||||||
|
assert_eq!(row.default_backend.as_deref(), Some("hybrid"));
|
||||||
|
assert_eq!(row.default_persona_id.as_deref(), Some("cabin"));
|
||||||
|
assert_eq!(row.reel_span.as_deref(), Some("week"));
|
||||||
|
assert_eq!(row.reel_max_segments, Some(24));
|
||||||
|
// Mirror fields untouched by the owner-default patch.
|
||||||
|
assert_eq!(row.voice.as_deref(), Some("grandma"));
|
||||||
|
assert_eq!(row.library.as_deref(), Some("1"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[actix_rt::test]
|
||||||
|
async fn put_prefs_owner_defaults_null_clears() {
|
||||||
|
let (state, mut seed_dao) = prefs_state();
|
||||||
|
seed(&mut seed_dao, seeded_row_with_defaults());
|
||||||
|
let app = init_service(
|
||||||
|
App::new()
|
||||||
|
.service(
|
||||||
|
web::resource("/prefs")
|
||||||
|
.route(web::get().to(get_prefs))
|
||||||
|
.route(web::put().to(put_prefs)),
|
||||||
|
)
|
||||||
|
.service(web::resource("/timezones").route(web::get().to(list_timezones)))
|
||||||
|
.app_data(state.clone()),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
let (header, value) = auth();
|
||||||
|
let req = TestRequest::put()
|
||||||
|
.uri("/prefs")
|
||||||
|
.insert_header((header, value))
|
||||||
|
.set_json(serde_json::json!({
|
||||||
|
"default_model": null,
|
||||||
|
"default_backend": null,
|
||||||
|
"default_persona_id": null,
|
||||||
|
"reel_span": null,
|
||||||
|
"reel_max_segments": null
|
||||||
|
}))
|
||||||
|
.to_request();
|
||||||
|
let resp = call_service(&app, req).await;
|
||||||
|
assert_eq!(resp.status(), 200);
|
||||||
|
|
||||||
|
let mut dao = state.user_ai_prefs_dao.lock().unwrap();
|
||||||
|
let row = dao
|
||||||
|
.get_prefs(&opentelemetry::Context::new())
|
||||||
|
.unwrap()
|
||||||
|
.unwrap();
|
||||||
|
assert!(row.default_model.is_none());
|
||||||
|
assert!(row.default_backend.is_none());
|
||||||
|
assert!(row.default_persona_id.is_none());
|
||||||
|
assert!(row.reel_span.is_none());
|
||||||
|
assert!(row.reel_max_segments.is_none());
|
||||||
|
// Sibling fields survive a clear patch.
|
||||||
|
assert_eq!(row.voice.as_deref(), Some("grandma"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[actix_rt::test]
|
||||||
|
async fn put_prefs_invalid_backend_returns_400_and_writes_nothing() {
|
||||||
|
let (state, mut seed_dao) = prefs_state();
|
||||||
|
seed(
|
||||||
|
&mut seed_dao,
|
||||||
|
seeded_row(Some("grandma"), Some("1"), Some("America/New_York")),
|
||||||
|
);
|
||||||
|
let app = init_service(
|
||||||
|
App::new()
|
||||||
|
.service(
|
||||||
|
web::resource("/prefs")
|
||||||
|
.route(web::get().to(get_prefs))
|
||||||
|
.route(web::put().to(put_prefs)),
|
||||||
|
)
|
||||||
|
.service(web::resource("/timezones").route(web::get().to(list_timezones)))
|
||||||
|
.app_data(state.clone()),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
let (header, value) = auth();
|
||||||
|
let req = TestRequest::put()
|
||||||
|
.uri("/prefs")
|
||||||
|
.insert_header((header, value))
|
||||||
|
.set_json(serde_json::json!({"default_backend": "quantum", "voice": "dad"}))
|
||||||
|
.to_request();
|
||||||
|
let resp = call_service(&app, req).await;
|
||||||
|
assert_eq!(resp.status(), 400);
|
||||||
|
|
||||||
|
let mut dao = state.user_ai_prefs_dao.lock().unwrap();
|
||||||
|
let row = dao
|
||||||
|
.get_prefs(&opentelemetry::Context::new())
|
||||||
|
.unwrap()
|
||||||
|
.unwrap();
|
||||||
|
// Nothing written — not even the valid field in the same body.
|
||||||
|
assert_eq!(row.voice.as_deref(), Some("grandma"));
|
||||||
|
assert!(row.default_backend.is_none());
|
||||||
|
assert_eq!(row.updated_at, 1_700_000_000);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn backend_validator_delegates_to_canonical_parser() {
|
||||||
|
// Case-insensitive like BackendKind::parse (a stored pref must always
|
||||||
|
// be consumable downstream), but the blank string the parser tolerates
|
||||||
|
// for request fields is rejected — NULL already means "no default".
|
||||||
|
assert!(is_valid_backend("local"));
|
||||||
|
assert!(is_valid_backend("hybrid"));
|
||||||
|
assert!(is_valid_backend("Hybrid"));
|
||||||
|
assert!(is_valid_backend(" LOCAL "));
|
||||||
|
assert!(!is_valid_backend("quantum"));
|
||||||
|
assert!(!is_valid_backend(""));
|
||||||
|
assert!(!is_valid_backend(" "));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[actix_rt::test]
|
||||||
|
async fn put_prefs_invalid_reel_span_returns_400_and_writes_nothing() {
|
||||||
|
let (state, mut seed_dao) = prefs_state();
|
||||||
|
seed(
|
||||||
|
&mut seed_dao,
|
||||||
|
seeded_row(Some("grandma"), Some("1"), Some("America/New_York")),
|
||||||
|
);
|
||||||
|
let app = init_service(
|
||||||
|
App::new()
|
||||||
|
.service(
|
||||||
|
web::resource("/prefs")
|
||||||
|
.route(web::get().to(get_prefs))
|
||||||
|
.route(web::put().to(put_prefs)),
|
||||||
|
)
|
||||||
|
.service(web::resource("/timezones").route(web::get().to(list_timezones)))
|
||||||
|
.app_data(state.clone()),
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
let (header, value) = auth();
|
||||||
|
let req = TestRequest::put()
|
||||||
|
.uri("/prefs")
|
||||||
|
.insert_header((header, value))
|
||||||
|
.set_json(serde_json::json!({"reel_span": "decade"}))
|
||||||
|
.to_request();
|
||||||
|
let resp = call_service(&app, req).await;
|
||||||
|
assert_eq!(resp.status(), 400);
|
||||||
|
|
||||||
|
let mut dao = state.user_ai_prefs_dao.lock().unwrap();
|
||||||
|
let row = dao
|
||||||
|
.get_prefs(&opentelemetry::Context::new())
|
||||||
|
.unwrap()
|
||||||
|
.unwrap();
|
||||||
|
assert!(row.reel_span.is_none());
|
||||||
|
assert_eq!(row.updated_at, 1_700_000_000);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
+174
-15
@@ -18,6 +18,7 @@ pub mod selector;
|
|||||||
|
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
use std::path::{Path, PathBuf};
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::str::FromStr;
|
||||||
use std::sync::{LazyLock, Mutex, Mutex as StdMutex};
|
use std::sync::{LazyLock, Mutex, Mutex as StdMutex};
|
||||||
use std::time::{Duration, Instant};
|
use std::time::{Duration, Instant};
|
||||||
|
|
||||||
@@ -71,13 +72,38 @@ pub fn normalize_library_key(libs: &[Library], param: Option<&str>) -> String {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Reel span / segment-cap resolution: explicit request param → stored
|
||||||
|
/// owner pref → hard default. A stored span that no longer parses (e.g.
|
||||||
|
/// hand-edited) is skipped rather than erroring — the hard default still
|
||||||
|
/// applies. Non-positive stored segment caps are ignored the same way.
|
||||||
|
fn resolve_reel_params(
|
||||||
|
req_span: Option<MemoriesSpan>,
|
||||||
|
req_max_segments: Option<usize>,
|
||||||
|
prefs: Option<&crate::database::models::UserAiPrefs>,
|
||||||
|
) -> (MemoriesSpan, usize) {
|
||||||
|
let pref_span = prefs
|
||||||
|
.and_then(|p| p.reel_span.as_deref())
|
||||||
|
.and_then(|s| MemoriesSpan::from_str(s).ok());
|
||||||
|
let pref_max_segments = prefs
|
||||||
|
.and_then(|p| p.reel_max_segments)
|
||||||
|
.filter(|v| *v > 0)
|
||||||
|
.map(|v| v as usize);
|
||||||
|
let span = req_span.or(pref_span).unwrap_or(MemoriesSpan::Day);
|
||||||
|
let max_segments = req_max_segments
|
||||||
|
.or(pref_max_segments)
|
||||||
|
.unwrap_or(selector::DEFAULT_MAX_SEGMENTS);
|
||||||
|
(span, max_segments)
|
||||||
|
}
|
||||||
|
|
||||||
/// Best-effort: mirror the latest client reel params into `user_ai_prefs`
|
/// Best-effort: mirror the latest client reel params into `user_ai_prefs`
|
||||||
/// so the nightly pre-gen scheduler can pick them up. Never fails the
|
/// so the nightly pre-gen scheduler can pick them up. Never fails the
|
||||||
/// caller regardless of DB errors.
|
/// caller regardless of DB errors.
|
||||||
fn capture_prefs(
|
fn capture_prefs(
|
||||||
app_state: &AppState,
|
app_state: &AppState,
|
||||||
req: &web::Json<CreateReelRequest>,
|
req: &CreateReelRequest,
|
||||||
library_param: Option<&str>,
|
library_param: Option<&str>,
|
||||||
|
effective_span: MemoriesSpan,
|
||||||
|
effective_max_segments: usize,
|
||||||
) -> Result<(), anyhow::Error> {
|
) -> Result<(), anyhow::Error> {
|
||||||
use crate::database::models::UpsertUserAiPrefs;
|
use crate::database::models::UpsertUserAiPrefs;
|
||||||
let now = std::time::SystemTime::now()
|
let now = std::time::SystemTime::now()
|
||||||
@@ -93,6 +119,21 @@ fn capture_prefs(
|
|||||||
};
|
};
|
||||||
let mut dao = app_state.user_ai_prefs_dao.lock().expect("lock");
|
let mut dao = app_state.user_ai_prefs_dao.lock().expect("lock");
|
||||||
let ctx = opentelemetry::Context::new();
|
let ctx = opentelemetry::Context::new();
|
||||||
|
// The row passively mirrors the latest request's EFFECTIVE reel params
|
||||||
|
// (after the request → pref → default resolution done by the caller), so
|
||||||
|
// the stored reel_span / reel_max_segments track what was last used —
|
||||||
|
// idempotent when the pref itself supplied the effective value.
|
||||||
|
//
|
||||||
|
// The owner-default fields (model/backend/persona) are owned by PUT /prefs,
|
||||||
|
// not by reel requests. The upsert overwrites every column, so we must
|
||||||
|
// carry the existing values through unchanged — writing None here would
|
||||||
|
// silently wipe the owner's stored defaults on every reel creation. If the
|
||||||
|
// read fails we bail without writing: this is best-effort, and clobbering
|
||||||
|
// owner defaults on a transient error is worse than skipping a mirror.
|
||||||
|
let existing = match dao.get_prefs(&ctx) {
|
||||||
|
Ok(row) => row,
|
||||||
|
Err(e) => return Err(anyhow::anyhow!("failed to read prefs before mirror: {e}")),
|
||||||
|
};
|
||||||
dao.upsert_prefs(
|
dao.upsert_prefs(
|
||||||
&ctx,
|
&ctx,
|
||||||
&UpsertUserAiPrefs {
|
&UpsertUserAiPrefs {
|
||||||
@@ -104,6 +145,11 @@ fn capture_prefs(
|
|||||||
library: Some(library),
|
library: Some(library),
|
||||||
updated_at: now as i64,
|
updated_at: now as i64,
|
||||||
display_tz: req.display_tz.clone().filter(|s| !s.is_empty()),
|
display_tz: req.display_tz.clone().filter(|s| !s.is_empty()),
|
||||||
|
default_model: existing.as_ref().and_then(|e| e.default_model.clone()),
|
||||||
|
default_backend: existing.as_ref().and_then(|e| e.default_backend.clone()),
|
||||||
|
default_persona_id: existing.as_ref().and_then(|e| e.default_persona_id.clone()),
|
||||||
|
reel_span: Some(effective_span.to_string()),
|
||||||
|
reel_max_segments: Some(effective_max_segments as i32),
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
.map_err(|e| anyhow::anyhow!("failed to upsert user_ai_prefs: {e}"))
|
.map_err(|e| anyhow::anyhow!("failed to upsert user_ai_prefs: {e}"))
|
||||||
@@ -415,20 +461,21 @@ pub async fn create_reel_handler(
|
|||||||
}));
|
}));
|
||||||
}
|
}
|
||||||
|
|
||||||
let span = req.span.unwrap_or(MemoriesSpan::Day);
|
// Owner defaults: explicit request param → stored pref → hard default.
|
||||||
let max_segments = req.max_segments.unwrap_or(selector::DEFAULT_MAX_SEGMENTS);
|
// The stored row is read once and reused for the span, the segment cap,
|
||||||
// Grouping zone: explicit request param → stored pref → app default
|
// and the display zone. The legacy device-offset param is still mirrored
|
||||||
// (ADR-0001). The legacy device-offset param is still mirrored into the
|
// into the frozen prefs column by `capture_prefs` for rollback but no
|
||||||
// frozen prefs column by `capture_prefs` for rollback but no longer
|
// longer drives selection.
|
||||||
// drives selection.
|
let stored_prefs = app_state
|
||||||
|
.user_ai_prefs_dao
|
||||||
|
.lock()
|
||||||
|
.ok()
|
||||||
|
.and_then(|mut dao| dao.get_prefs(&opentelemetry::Context::new()).ok().flatten());
|
||||||
|
let (span, max_segments) =
|
||||||
|
resolve_reel_params(req.span, req.max_segments, stored_prefs.as_ref());
|
||||||
let display_zone_name = {
|
let display_zone_name = {
|
||||||
let req_tz = req.display_tz.clone().filter(|s| !s.trim().is_empty());
|
let req_tz = req.display_tz.clone().filter(|s| !s.trim().is_empty());
|
||||||
let stored = app_state
|
let stored = stored_prefs.as_ref().and_then(|p| p.display_tz.clone());
|
||||||
.user_ai_prefs_dao
|
|
||||||
.lock()
|
|
||||||
.ok()
|
|
||||||
.and_then(|mut dao| dao.get_prefs(&opentelemetry::Context::new()).ok().flatten())
|
|
||||||
.and_then(|p| p.display_tz);
|
|
||||||
crate::camera_tz::display_zone_for(req_tz.as_deref().or(stored.as_deref()))
|
crate::camera_tz::display_zone_for(req_tz.as_deref().or(stored.as_deref()))
|
||||||
};
|
};
|
||||||
let selector = ReelSelector::Memories {
|
let selector = ReelSelector::Memories {
|
||||||
@@ -489,7 +536,7 @@ pub async fn create_reel_handler(
|
|||||||
},
|
},
|
||||||
);
|
);
|
||||||
// Capture params for passive prefs mirror (best-effort, never fails).
|
// Capture params for passive prefs mirror (best-effort, never fails).
|
||||||
let _ = capture_prefs(&app_state, &req, req.library.as_deref());
|
let _ = capture_prefs(&app_state, &req, req.library.as_deref(), span, max_segments);
|
||||||
return HttpResponse::Accepted().json(ReelJobCreatedResponse {
|
return HttpResponse::Accepted().json(ReelJobCreatedResponse {
|
||||||
job_id: job_id.to_string(),
|
job_id: job_id.to_string(),
|
||||||
status: ReelJobStatus::Done,
|
status: ReelJobStatus::Done,
|
||||||
@@ -549,7 +596,7 @@ pub async fn create_reel_handler(
|
|||||||
with_job(job_id, |job| job.abort = Some(handle.abort_handle()));
|
with_job(job_id, |job| job.abort = Some(handle.abort_handle()));
|
||||||
|
|
||||||
// Capture params for passive prefs mirror (best-effort, never fails).
|
// Capture params for passive prefs mirror (best-effort, never fails).
|
||||||
let _ = capture_prefs(&app_state, &req, req.library.as_deref());
|
let _ = capture_prefs(&app_state, &req, req.library.as_deref(), span, max_segments);
|
||||||
|
|
||||||
HttpResponse::Accepted().json(ReelJobCreatedResponse {
|
HttpResponse::Accepted().json(ReelJobCreatedResponse {
|
||||||
job_id: job_id.to_string(),
|
job_id: job_id.to_string(),
|
||||||
@@ -1333,9 +1380,121 @@ fn sweep_reel_cache(app_state: &AppState) -> usize {
|
|||||||
#[cfg(test)]
|
#[cfg(test)]
|
||||||
mod tests {
|
mod tests {
|
||||||
use super::*;
|
use super::*;
|
||||||
|
use crate::database::models::UserAiPrefs;
|
||||||
use crate::libraries::Library;
|
use crate::libraries::Library;
|
||||||
use chrono::TimeZone;
|
use chrono::TimeZone;
|
||||||
|
|
||||||
|
fn prefs_row(span: Option<&str>, max_segments: Option<i32>) -> UserAiPrefs {
|
||||||
|
UserAiPrefs {
|
||||||
|
id: 1,
|
||||||
|
voice: None,
|
||||||
|
tz_offset_minutes: None,
|
||||||
|
library: None,
|
||||||
|
updated_at: 0,
|
||||||
|
display_tz: None,
|
||||||
|
default_model: None,
|
||||||
|
default_backend: None,
|
||||||
|
default_persona_id: None,
|
||||||
|
reel_span: span.map(str::to_string),
|
||||||
|
reel_max_segments: max_segments,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_reel_params_request_wins_over_pref() {
|
||||||
|
let prefs = prefs_row(Some("month"), Some(10));
|
||||||
|
let (span, max) = resolve_reel_params(Some(MemoriesSpan::Day), Some(5), Some(&prefs));
|
||||||
|
assert_eq!(span, MemoriesSpan::Day);
|
||||||
|
assert_eq!(max, 5);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_reel_params_pref_fills_omitted_request() {
|
||||||
|
let prefs = prefs_row(Some("week"), Some(24));
|
||||||
|
let (span, max) = resolve_reel_params(None, None, Some(&prefs));
|
||||||
|
assert_eq!(span, MemoriesSpan::Week);
|
||||||
|
assert_eq!(max, 24);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_reel_params_no_pref_falls_to_hard_defaults() {
|
||||||
|
let (span, max) = resolve_reel_params(None, None, None);
|
||||||
|
assert_eq!(span, MemoriesSpan::Day);
|
||||||
|
assert_eq!(max, selector::DEFAULT_MAX_SEGMENTS);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resolve_reel_params_invalid_or_nonpositive_pref_skipped() {
|
||||||
|
// Hand-edited / corrupt stored values fall through to hard defaults
|
||||||
|
// instead of erroring the request.
|
||||||
|
let bad_span = prefs_row(Some("decade"), Some(0));
|
||||||
|
let (span, max) = resolve_reel_params(None, None, Some(&bad_span));
|
||||||
|
assert_eq!(span, MemoriesSpan::Day);
|
||||||
|
assert_eq!(max, selector::DEFAULT_MAX_SEGMENTS);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Regression: reel creation is a PASSIVE mirror. The upsert overwrites
|
||||||
|
/// every column, so `capture_prefs` must carry the PUT /prefs-owned
|
||||||
|
/// default_model / default_backend / default_persona_id through unchanged
|
||||||
|
/// — a reel request has no business wiping the owner's AI defaults.
|
||||||
|
#[actix_rt::test]
|
||||||
|
async fn reel_capture_preserves_owner_defaults() {
|
||||||
|
use crate::database::models::UpsertUserAiPrefs;
|
||||||
|
use crate::database::{SqliteUserAiPrefsDao, UserAiPrefsDao};
|
||||||
|
use crate::state::AppState;
|
||||||
|
use std::sync::{Arc, Mutex};
|
||||||
|
|
||||||
|
let conn = Arc::new(Mutex::new(crate::database::test::in_memory_db_connection()));
|
||||||
|
let mut state = AppState::test_state();
|
||||||
|
state.user_ai_prefs_dao = Arc::new(Mutex::new(Box::new(
|
||||||
|
SqliteUserAiPrefsDao::from_connection(conn.clone()),
|
||||||
|
)));
|
||||||
|
{
|
||||||
|
let mut dao = SqliteUserAiPrefsDao::from_connection(conn.clone());
|
||||||
|
dao.upsert_prefs(
|
||||||
|
&opentelemetry::Context::new(),
|
||||||
|
&UpsertUserAiPrefs {
|
||||||
|
voice: None,
|
||||||
|
tz_offset_minutes: None,
|
||||||
|
library: None,
|
||||||
|
updated_at: 0,
|
||||||
|
display_tz: None,
|
||||||
|
default_model: Some("llama3.2:3b".into()),
|
||||||
|
default_backend: Some("hybrid".into()),
|
||||||
|
default_persona_id: Some("cabin".into()),
|
||||||
|
reel_span: None,
|
||||||
|
reel_max_segments: None,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
.expect("seed owner defaults");
|
||||||
|
}
|
||||||
|
|
||||||
|
let req = CreateReelRequest {
|
||||||
|
span: Some(MemoriesSpan::Week),
|
||||||
|
timezone_offset_minutes: Some(-240),
|
||||||
|
display_tz: None,
|
||||||
|
library: None,
|
||||||
|
voice: Some("grandma".into()),
|
||||||
|
max_segments: Some(12),
|
||||||
|
};
|
||||||
|
capture_prefs(&state, &req, None, MemoriesSpan::Week, 12).expect("capture");
|
||||||
|
|
||||||
|
let row = state
|
||||||
|
.user_ai_prefs_dao
|
||||||
|
.lock()
|
||||||
|
.expect("lock")
|
||||||
|
.get_prefs(&opentelemetry::Context::new())
|
||||||
|
.expect("read")
|
||||||
|
.expect("row");
|
||||||
|
// The mirrored reel params landed...
|
||||||
|
assert_eq!(row.reel_span.as_deref(), Some("week"));
|
||||||
|
assert_eq!(row.reel_max_segments, Some(12));
|
||||||
|
// ...and the owner defaults survived untouched.
|
||||||
|
assert_eq!(row.default_model.as_deref(), Some("llama3.2:3b"));
|
||||||
|
assert_eq!(row.default_backend.as_deref(), Some("hybrid"));
|
||||||
|
assert_eq!(row.default_persona_id.as_deref(), Some("cabin"));
|
||||||
|
}
|
||||||
|
|
||||||
fn photo(p: &str, lib: i32) -> SegmentMedia {
|
fn photo(p: &str, lib: i32) -> SegmentMedia {
|
||||||
SegmentMedia::Photo {
|
SegmentMedia::Photo {
|
||||||
rel_path: p.to_string(),
|
rel_path: p.to_string(),
|
||||||
|
|||||||
+2
-1
@@ -281,8 +281,9 @@ pub async fn generate_script(
|
|||||||
beats: &[PlannedBeat],
|
beats: &[PlannedBeat],
|
||||||
) -> Result<ReelScript> {
|
) -> Result<ReelScript> {
|
||||||
let (system, user) = build_script_messages(meta, beats);
|
let (system, user) = build_script_messages(meta, beats);
|
||||||
|
let cx = opentelemetry::Context::current();
|
||||||
let raw = client
|
let raw = client
|
||||||
.generate(&user, Some(&system), None)
|
.generate(&cx, &user, Some(&system), None)
|
||||||
.await
|
.await
|
||||||
.context("LLM script generation failed")?;
|
.context("LLM script generation failed")?;
|
||||||
Ok(parse_script_response(&raw, beats.len()))
|
Ok(parse_script_response(&raw, beats.len()))
|
||||||
|
|||||||
@@ -211,7 +211,7 @@ pub async fn unified_search<TagD: TagDao>(
|
|||||||
log::info!("unified_search: translating with model={}", backend.model());
|
log::info!("unified_search: translating with model={}", backend.model());
|
||||||
|
|
||||||
let today = chrono::Utc::now().date_naive();
|
let today = chrono::Utc::now().date_naive();
|
||||||
let sq = match translate_nl_query(backend.chat(), &nl, &tag_vocab, today).await {
|
let sq = match translate_nl_query(backend.chat(), &nl, &tag_vocab, today, &ctx).await {
|
||||||
Ok(sq) => sq,
|
Ok(sq) => sq,
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
log::warn!("unified_search: translate_nl_query failed: {e:?}");
|
log::warn!("unified_search: translate_nl_query failed: {e:?}");
|
||||||
|
|||||||
Reference in new issue
Block a user