Wire the tracing subscriber (unify dev console + release OTLP) #138

New Issue

2026-09-29T14:57:07Z

cameron commented

2026-09-29 14:57:07 +00:00

Part of #135

Work

Install a tracing subscriber in otel::init_tracing so spans are produced in BOTH builds, replacing today's release-only global::set_tracer_provider with no subscriber.

Add deps: tracing, tracing-subscriber, tracing-log, tracing-opentelemetry = "0.32" (no otel bump).
Debug: tracing-subscriber::fmt layer -> console, so spans AND logs are visible; the tracing-log bridge keeps existing log:: calls flowing to the console (replacing env_logger).
Release: OpenTelemetry layer -> OTLP, only when OTLP_OTLS_ENDPOINT is set.
Install with try_init() (process-global), NOT set_default() (thread-local) - validated by spike #133.
Bring tests/otel_propagation.rs from the spike #133 patch in-tree as the propagation regression test; confirm it passes in the real build.
Revisit the #[cfg(not(debug_assertions))] fork around init_logs / init_tracing in main.rs and decide what remains gated.

Resolution

The subscriber config that landed, and confirmation the regression test is green.

Part of #135 ## Work Install a `tracing` subscriber in `otel::init_tracing` so spans are produced in BOTH builds, replacing today's release-only `global::set_tracer_provider` with no subscriber. - Add deps: `tracing`, `tracing-subscriber`, `tracing-log`, `tracing-opentelemetry = "0.32"` (no otel bump). - Debug: `tracing-subscriber::fmt` layer -> console, so spans AND logs are visible; the `tracing-log` bridge keeps existing `log::` calls flowing to the console (replacing `env_logger`). - Release: OpenTelemetry layer -> OTLP, only when `OTLP_OTLS_ENDPOINT` is set. - Install with `try_init()` (process-global), NOT `set_default()` (thread-local) - validated by spike #133. - Bring `tests/otel_propagation.rs` from the spike #133 patch in-tree as the propagation regression test; confirm it passes in the real build. - Revisit the `#[cfg(not(debug_assertions))]` fork around `init_logs` / `init_tracing` in `main.rs` and decide what remains gated. ## Resolution The subscriber config that landed, and confirmation the regression test is green.

cameron added a new dependency 2026-09-29 14:57:54 +00:00

#140 Migrate the LLM client + streaming spans to tracing

cameron added a new dependency 2026-09-29 14:57:55 +00:00

#141 Migrate the DAO-layer context threading to tracing

cameron added a new dependency 2026-09-29 14:57:55 +00:00

#142 Give background jobs a root span

cameron referenced this issue

2026-09-29 15:00:11 +00:00

Confirm the per-request root-span middleware works on actix-web 4 + otel 0.31 #136

cameron added a new dependency 2026-09-29 15:00:56 +00:00

#139 Add the per-request root span (middleware) and drop per-handler boilerplate

cameron commented

2026-09-29 16:15:47 +00:00

Resolved — commit 48a3f03 on feature/tracing-migration.

Delivered (per spec):

One tracing subscriber installed in both build profiles via otel::init_tracing(): registry() + EnvFilter (honors RUST_LOG, defaults to info) + OpenTelemetryLayer + console fmt layer. Previously release-only → spans were a no-op in dev.
OTLP export is env-gated: OTLP_OTLS_ENDPOINT set → batch OTLP exporter (release/staging); unset → provider builds with valid span contexts (so outgoing traceparent injection still works) but exports nowhere, and the console fmt layer shows spans.
Global W3C TraceContextPropagator set at init — required by the tracing-actix-web middleware (#139) and the outgoing traceparent helpers (both read the global propagator).
Deps: + tracing, tracing-subscriber (env-filter/fmt/ansi/registry), tracing-opentelemetry 0.32; opentelemetry_sdk gains testing.
main.rs: init_tracing() called in both builds; env_logger stays in debug, init_logs() stays in release.
Regression test tests/otel_propagation.rs brought in from the spike — pins the spawn-carry invariant (S2/S4 bare tokio::spawn orphan; S6 .instrument() / S7 span.enter() propagate).

Two design calls, both conventional:

tracing-log bridge left off (default-features = false) so the console fmt layer prints tracing spans only; log records stay on their existing path (env_logger debug / OTel release). Avoids double-printing every log line.
Console fmt layer present in both builds (a release run locally also prints spans). If you'd rather have release export-only, that's a one-line #[cfg(debug_assertions)] gate — say the word.

Evidence: cargo clippy --all-targets 0 warnings; 690 lib + 775 integration + 1 regression + 4 doc-tests pass; smoke-ran the debug binary — init_tracing() + env_logger + migrations + watcher all start cleanly (no panic).

Note: this is the foundation/plumbing. Console span visibility materializes in #139–#142 as the request/LLM/DB spans are migrated to #[instrument]; #138 just wires the pipe + propagator so those spans flow and cross spawn boundaries correctly.

Resolved — commit `48a3f03` on `feature/tracing-migration`. **Delivered (per spec):** - One `tracing` subscriber installed in **both** build profiles via `otel::init_tracing()`: `registry()` + `EnvFilter` (honors `RUST_LOG`, defaults to `info`) + `OpenTelemetryLayer` + console `fmt` layer. Previously release-only → spans were a no-op in dev. - **OTLP export is env-gated**: `OTLP_OTLS_ENDPOINT` set → batch OTLP exporter (release/staging); unset → provider builds with valid span contexts (so outgoing `traceparent` injection still works) but exports nowhere, and the console `fmt` layer shows spans. - **Global W3C `TraceContextPropagator` set** at init — required by the `tracing-actix-web` middleware (#139) and the outgoing traceparent helpers (both read the *global* propagator). - Deps: `+ tracing`, `tracing-subscriber` (`env-filter`/`fmt`/`ansi`/`registry`), `tracing-opentelemetry 0.32`; `opentelemetry_sdk` gains `testing`. - `main.rs`: `init_tracing()` called in both builds; `env_logger` stays in debug, `init_logs()` stays in release. - **Regression test `tests/otel_propagation.rs`** brought in from the spike — pins the spawn-carry invariant (S2/S4 bare `tokio::spawn` orphan; S6 `.instrument()` / S7 `span.enter()` propagate). **Two design calls, both conventional:** 1. `tracing-log` bridge left **off** (`default-features = false`) so the console `fmt` layer prints *tracing spans only*; `log` records stay on their existing path (env_logger debug / OTel release). Avoids double-printing every log line. 2. Console `fmt` layer present in **both** builds (a release run locally also prints spans). If you'd rather have release export-only, that's a one-line `#[cfg(debug_assertions)]` gate — say the word. **Evidence:** `cargo clippy --all-targets` 0 warnings; 690 lib + 775 integration + 1 regression + 4 doc-tests pass; smoke-ran the debug binary — `init_tracing()` + `env_logger` + migrations + watcher all start cleanly (no panic). **Note:** this is the foundation/plumbing. Console span *visibility* materializes in #139–#142 as the request/LLM/DB spans are migrated to `#[instrument]`; #138 just wires the pipe + propagator so those spans flow and cross spawn boundaries correctly.

cameron closed this issue

2026-09-29 16:15:48 +00:00

cameron referenced this issue

2026-09-29 16:16:22 +00:00

Wayfinder: migrate spans from raw OTel SDK to tracing + tracing-opentelemetry #135

cameron referenced this issue

2026-09-29 17:14:06 +00:00

Add the per-request root span (middleware) and drop per-handler boilerplate #139

cameron referenced this issue from a commit

2026-09-29 21:41:49 +00:00

Otel: unify tracing subscriber across dev+release; env-gated OTLP export (#138)

cameron referenced this issue from a commit