Otel: replace remaining ambient Context::current() with threaded context #132

Closed
opened 2026-09-29 01:52:10 +00:00 by cameron · 2 comments
Owner

Root cause

The codebase threads OTel contexts explicitly and never calls .attach(); there is no context-aware tokio layer. So Context::current() returns the empty root context.

Important — the DAO/DB layer already works. DAO trait methods take cx: &Context as their first parameter by convention, and HTTP handlers do:

let context = extract_context_from_request(&request);
let span = tracer.start_with_context("add_tag", &context);
let span_context = opentelemetry::Context::current_with_span(span);
tag_dao.get_all_tags(&span_context, None)   // threaded — NOT Context::current()

So DB spans on the hot request path nest correctly under the request + handler span. This issue is not about the DAO layer — it's about the remaining call sites that still build spans from Context::current() instead of threading.

PR #129 (f850c53) fixed the legacy SSE streaming chat path (src/ai/insight_chat.rs). The sites below are what's left.

Bucket 1 — request-driven, real gaps (thread the request context)

These have a request (or a threaded parent) available but build llm.request / work spans from Context::current(), so they're orphaned roots.

  • AI/LLM paths (create llm.request spans) — direct analog of the PR #129 fix:
    • src/ai/persona_chat.rs: 532, 549, 568, 631, 740, 990, 1101, 1197
    • src/ai/insight_generator.rs: 449, 600, 701, 1083, 4197, 4667
  • Request-driven non-AI handlers that have the HttpRequest but don't extract/thread it:
    • src/knowledge.rs: 37 (in resolve_persona_filter(req: &HttpRequest, …)), plus handlers at ~461 / ~530; many more sites in the file
    • src/personas.rs: 153 (list_personas — a Responder handler that never calls extract_context_from_request), and 184, 232, 291, 328

Bucket 2 — search / tool paths (thread if a request context is available at the call site)

  • src/clip_search.rs::score_photos (189, 238)
  • src/unified_search.rs::unified_search (160)

These are pub functions (not handlers) called from tools/handlers. Thread a context if the caller has one; otherwise treat as Bucket 3.

Bucket 3 — background / watch jobs (root by design; optional job-root span)

No HTTP request exists, so Context::current() = root is expected. Not a bug — but a job-root span would group the per-item spans instead of scattering them as roots.

  • src/ai/daily_summary_job.rs: 169 → job span e.g. ai.daily_summary
  • src/face_watch.rs: 159
  • src/clip_watch.rs: 138
  • src/video/actors.rs: 809, 830, 846
  • src/cleanup/database_updater.rs: 30, 105
  • src/reels/script.rs: 284 (if background)

Fix pattern

  • Bucket 1/2: extract the request context at the handler (otel::extract_context_from_request) and thread it down to each span, mirroring the DAO idiom (current_with_span) and the PR #129 streaming fix.
  • Bucket 3: start one job-root span per job run and thread it to the per-item spans.

Out of scope

  • Test code (files.rs, tags.rs, faces.rs tests).
  • src/bin/* importers — CLI tools have no request context by design.
## Root cause The codebase threads OTel contexts explicitly and never calls `.attach()`; there is no context-aware tokio layer. So `Context::current()` returns the **empty root context**. **Important — the DAO/DB layer already works.** DAO trait methods take `cx: &Context` as their first parameter by convention, and HTTP handlers do: ```rust let context = extract_context_from_request(&request); let span = tracer.start_with_context("add_tag", &context); let span_context = opentelemetry::Context::current_with_span(span); tag_dao.get_all_tags(&span_context, None) // threaded — NOT Context::current() ``` So DB spans on the hot request path nest correctly under the request + handler span. This issue is **not** about the DAO layer — it's about the remaining call sites that still build spans from `Context::current()` instead of threading. PR #129 (`f850c53`) fixed the legacy SSE streaming chat path (`src/ai/insight_chat.rs`). The sites below are what's left. ## Bucket 1 — request-driven, real gaps (thread the request context) These have a request (or a threaded parent) available but build `llm.request` / work spans from `Context::current()`, so they're orphaned roots. - **AI/LLM paths** (create `llm.request` spans) — direct analog of the PR #129 fix: - `src/ai/persona_chat.rs`: 532, 549, 568, 631, 740, 990, 1101, 1197 - `src/ai/insight_generator.rs`: 449, 600, 701, 1083, 4197, 4667 - **Request-driven non-AI handlers** that have the `HttpRequest` but don't extract/thread it: - `src/knowledge.rs`: 37 (in `resolve_persona_filter(req: &HttpRequest, …)`), plus handlers at ~461 / ~530; many more sites in the file - `src/personas.rs`: 153 (`list_personas` — a `Responder` handler that never calls `extract_context_from_request`), and 184, 232, 291, 328 ## Bucket 2 — search / tool paths (thread if a request context is available at the call site) - `src/clip_search.rs::score_photos` (189, 238) - `src/unified_search.rs::unified_search` (160) These are pub functions (not handlers) called from tools/handlers. Thread a context if the caller has one; otherwise treat as Bucket 3. ## Bucket 3 — background / watch jobs (root by design; optional job-root span) No HTTP request exists, so `Context::current()` = root is expected. Not a bug — but a job-root span would group the per-item spans instead of scattering them as roots. - `src/ai/daily_summary_job.rs`: 169 → job span e.g. `ai.daily_summary` - `src/face_watch.rs`: 159 - `src/clip_watch.rs`: 138 - `src/video/actors.rs`: 809, 830, 846 - `src/cleanup/database_updater.rs`: 30, 105 - `src/reels/script.rs`: 284 (if background) ## Fix pattern - Bucket 1/2: extract the request context at the handler (`otel::extract_context_from_request`) and thread it down to each span, mirroring the DAO idiom (`current_with_span`) and the PR #129 streaming fix. - Bucket 3: start one job-root span per job run and thread it to the per-item spans. ## Out of scope - Test code (`files.rs`, `tags.rs`, `faces.rs` tests). - `src/bin/*` importers — CLI tools have no request context by design.
Author
Owner

Superseded by the tracing migration (#134), which is gated on the spike (#133). Once #133 reports proceed, the Bucket 1/2 sites here become #[instrument] / drop the cx param as part of #134 rather than being hand-threaded — so don't start hand-threading these in isolation. Bucket 3 (background/watch job-root spans) is also folded into #134.

Superseded by the tracing migration (#134), which is gated on the spike (#133). Once #133 reports proceed, the Bucket 1/2 sites here become `#[instrument]` / drop the `cx` param as part of #134 rather than being hand-threaded — so don't start hand-threading these in isolation. Bucket 3 (background/watch job-root spans) is also folded into #134.
Author
Owner

Superseded by #134 (migrate spans from raw OTel SDK to tracing + tracing-opentelemetry). #134 retires the manual context threading this issue tracks instead of patching the remaining Context::current() sites: it installs process-global tracing→OTel propagation and drops the cx: &Context first param, which removes the entire orphan-span class at the root. The three buckets here are subsumed by #134's scope (request-driven sites → #instrument / drop param; background jobs → job-root span). Closing so it doesn't get worked in isolation. See #133 (spike, verdict ADJUST) for the validated mechanism.

Superseded by #134 (migrate spans from raw OTel SDK to tracing + tracing-opentelemetry). #134 retires the manual context threading this issue tracks instead of patching the remaining Context::current() sites: it installs process-global tracing→OTel propagation and drops the cx: &Context first param, which removes the entire orphan-span class at the root. The three buckets here are subsumed by #134's scope (request-driven sites → #instrument / drop param; background jobs → job-root span). Closing so it doesn't get worked in isolation. See #133 (spike, verdict ADJUST) for the validated mechanism.
Sign in to join this conversation.