hls: per-library readiness gauges + GET /hls/stats endpoint

The hash-keyed pipeline transcodes lazily, so a freshly mounted (or
freshly upgraded) library is "mostly pending" for the first hour
while the watcher works through the backlog. The operator wants a
live read on remaining work so they can tune `HLS_CONCURRENCY` and
know when to stop waiting.

Adds:

- `src/hls_stats.rs` — pure compute path (`stats_from_rows`) and an
  Arc<Mutex<dyn ExifDao>> wrapper (`compute_and_publish`). Per
  library: `total`, `with_playlist`, `pending`, `unsupported`,
  `hashless_videos`. Dedup is by content_hash so duplicate-bytes-at-
  N-paths counts once (same domain rule as `faces::stats`).
  `hashless_videos` is a separate counter so the operator can see
  the "hash backfill, then transcode" pipeline depth instead of
  having NULL-hash rows just hide.

- Prometheus gauges labeled by library name:
  `imageserver_hls_videos_total`, `..._with_playlist`, `..._pending`,
  `..._unsupported`. Updated by the watcher at the end of every full-
  scan tick *and* on every `/hls/stats` hit, so whichever surface the
  operator is watching stays fresh. Registered in `main` alongside
  the existing image/video gauges.

- `GET /hls/stats` — Claims-protected JSON snapshot of the same data
  plus a top-level cross-library aggregate. Runs on a blocking pool
  so it doesn't pin the actix worker; per-call cost is one
  `list_paths_and_hashes_for_library` SQL query per library plus a
  `stat()` per distinct video hash. Bounded — never invoked from
  middleware, only from the explicit endpoint and the full-scan
  tick. The watcher's end-of-tick `info!` summary line mirrors the
  endpoint output for operators tailing the log.

- New `ExifDao::list_paths_and_hashes_for_library` method:
  `SELECT rel_path, content_hash FROM image_exif WHERE library_id =
  ?`. Single round-trip; callers filter to video extensions
  client-side because the schema doesn't carry media-type. Mock
  impl in `files.rs` returns an empty vec.

Tests in `hls_stats::tests` exercise stats_from_rows directly (videos-
only filter, hash dedup, playlist vs sentinel decision, NULL-hash
hashless counting) plus a publish_gauges round-trip that reads the
gauge value back. Full suite (347 lib + 360 bin = 707) passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Cameron Cordes
2026-05-14 15:58:46 -04:00
parent 7c153596fe
commit 7cd1ea3cf8
5 changed files with 488 additions and 0 deletions

View File

@@ -424,6 +424,17 @@ pub trait ExifDao: Sync + Send {
context: &opentelemetry::Context,
) -> Result<Vec<String>, DbError>;
/// Every row in `image_exif` for `library_id`, as
/// `(rel_path, content_hash)`. The hash is Option because rows
/// mid-backfill carry NULL. Used by HLS readiness stats; callers
/// filter by extension client-side because the DB schema doesn't
/// carry media type.
fn list_paths_and_hashes_for_library(
&mut self,
context: &opentelemetry::Context,
library_id: i32,
) -> Result<Vec<(String, Option<String>)>, DbError>;
/// Return image_exif rows that need their `date_taken` resolved by the
/// canonical-date waterfall (see `crate::date_resolver`): `date_taken
/// IS NULL`. Returns `(library_id, rel_path)`. The caller filters to
@@ -1261,6 +1272,30 @@ impl ExifDao for SqliteExifDao {
.map_err(|_| DbError::new(DbErrorKind::QueryError))
}
fn list_paths_and_hashes_for_library(
&mut self,
context: &opentelemetry::Context,
lib_id: i32,
) -> Result<Vec<(String, Option<String>)>, DbError> {
trace_db_call(
context,
"query",
"list_paths_and_hashes_for_library",
|_span| {
use schema::image_exif::dsl::*;
let mut connection = self.connection.lock().expect("Unable to get ExifDao");
image_exif
.filter(library_id.eq(lib_id))
.select((rel_path, content_hash))
.load::<(String, Option<String>)>(connection.deref_mut())
.map_err(|_| anyhow::anyhow!("Query error"))
},
)
.map_err(|_| DbError::new(DbErrorKind::QueryError))
}
fn get_rows_needing_date_backfill(
&mut self,
context: &opentelemetry::Context,