fix(watcher): refresh planner stats on full scans to stop write-lock starvation

sqlite_stat1 was last gathered by a migration while image_exif had 402
rows and an all-NULL content_hash, so the planner treated the hash index
as non-selective and scanned it for every row in the back-ref refresh's
correlated subqueries. The face_detections pass took ~20s per watcher
tick while holding the write lock, so chat/persona-chat persistence blew
past busy_timeout and returned 500s. With fresh stats the three
refreshes drop from ~22s to ~0.1s.

- library_maintenance: add refresh_planner_stats (plain ANALYZE;
  PRAGMA optimize doesn't catch this on SQLite 3.45), run on full-scan
  ticks ahead of reconcile / back-ref refresh.
- utils: retry backoff used 1_u32.pow(..), a flat ~100ms; make it the
  documented exponential 100/200/400ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Opus 5.5 committed 2026-09-24 09:33:37 -04:00
1 parent 33fe7ce77c
commit 23371ad0bc
3 files changed
+24 -2

No files matched your search

+19
View File
@@ -156,6 +156,25 @@ pub fn detect_missing_files_for_library(
(deleted, next_offset)
}
/// Re-gather query-planner statistics (`sqlite_stat1`).
///
/// The migrations run `ANALYZE` once, but the stats never refresh after
/// that. A snapshot taken while `image_exif.content_hash` was all-NULL
/// told the planner every hash matches every row, so the correlated
/// `content_hash = ...` subqueries in [`refresh_back_refs`] scanned
/// the index instead of seeking it: ~20s per tick holding the write
/// lock, starving chat persistence past `busy_timeout`. With fresh
/// stats the same pass takes ~0.1s. `PRAGMA optimize` doesn't catch
/// this on SQLite 3.45 (verified), hence a plain `ANALYZE`. It costs
/// well under a second at current scale, so run it on full-scan ticks.
pub fn refresh_planner_stats(conn: &mut SqliteConnection) {
let started = std::time::Instant::now();
match sql_query("ANALYZE").execute(conn) {
Ok(_) => debug!("ANALYZE completed in {:?}", started.elapsed()),
Err(e) => warn!("ANALYZE failed: {:?}", e),
}
}
/// Refresh the `(library_id, rel_path)` back-refs on hash-keyed
/// tables. A back-ref is stale when:
/// - its `content_hash` is non-null,