fix(watcher): refresh planner stats on full scans to stop write-lock starvation
sqlite_stat1 was last gathered by a migration while image_exif had 402 rows and an all-NULL content_hash, so the planner treated the hash index as non-selective and scanned it for every row in the back-ref refresh's correlated subqueries. The face_detections pass took ~20s per watcher tick while holding the write lock, so chat/persona-chat persistence blew past busy_timeout and returned 500s. With fresh stats the three refreshes drop from ~22s to ~0.1s. - library_maintenance: add refresh_planner_stats (plain ANALYZE; PRAGMA optimize doesn't catch this on SQLite 3.45), run on full-scan ticks ahead of reconcile / back-ref refresh. - utils: retry backoff used 1_u32.pow(..), a flat ~100ms; make it the documented exponential 100/200/400ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
33fe7ce77c
commit
23371ad0bc
3 files changed
+24
-2
No files matched your search
@@ -156,6 +156,25 @@ pub fn detect_missing_files_for_library(
|
||||
(deleted, next_offset)
|
||||
}
|
||||
|
||||
/// Re-gather query-planner statistics (`sqlite_stat1`).
|
||||
///
|
||||
/// The migrations run `ANALYZE` once, but the stats never refresh after
|
||||
/// that. A snapshot taken while `image_exif.content_hash` was all-NULL
|
||||
/// told the planner every hash matches every row, so the correlated
|
||||
/// `content_hash = ...` subqueries in [`refresh_back_refs`] scanned
|
||||
/// the index instead of seeking it: ~20s per tick holding the write
|
||||
/// lock, starving chat persistence past `busy_timeout`. With fresh
|
||||
/// stats the same pass takes ~0.1s. `PRAGMA optimize` doesn't catch
|
||||
/// this on SQLite 3.45 (verified), hence a plain `ANALYZE`. It costs
|
||||
/// well under a second at current scale, so run it on full-scan ticks.
|
||||
pub fn refresh_planner_stats(conn: &mut SqliteConnection) {
|
||||
let started = std::time::Instant::now();
|
||||
match sql_query("ANALYZE").execute(conn) {
|
||||
Ok(_) => debug!("ANALYZE completed in {:?}", started.elapsed()),
|
||||
Err(e) => warn!("ANALYZE failed: {:?}", e),
|
||||
}
|
||||
}
|
||||
|
||||
/// Refresh the `(library_id, rel_path)` back-refs on hash-keyed
|
||||
/// tables. A back-ref is stale when:
|
||||
/// - its `content_hash` is non-null,
|
||||
|
||||
Reference in new issue
Block a user