Drop fs_time from date-backfill eligibility

The drain queried `date_taken IS NULL OR date_taken_source = 'fs_time'`
ORDER BY id ASC LIMIT 500 every watcher tick. The resolver is
deterministic on file bytes + filename + fs metadata, so any row that
landed on fs_time once landed there again on every retry — the drain
spun on the same lowest-id rows in perpetuity, never advancing to
rows 501+ while still logging more_remain=true.

Side effect: 500 auto-commit UPDATEs per tick sustained the SQLite
write lock long enough that other writers on separate DAO connections
hit the 5s busy_timeout. Manifested as intermittent 500s on
PATCH /image/faces/{id} that succeeded on retry.

Narrow the partial index and query predicate to `date_taken IS NULL`.
If exiftool installs or a new filename regex lands, an operator can
re-resolve fs_time rows out-of-band rather than re-introducing the
steady-state churn.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Cameron Cordes
2026-05-12 14:37:36 -04:00
parent 3427c2916c
commit 2d56047497
4 changed files with 59 additions and 23 deletions

View File

@@ -0,0 +1,5 @@
DROP INDEX IF EXISTS idx_image_exif_date_backfill;
CREATE INDEX idx_image_exif_date_backfill
ON image_exif (library_id, id)
WHERE date_taken IS NULL OR date_taken_source = 'fs_time';

View File

@@ -0,0 +1,18 @@
-- Narrow the date-backfill partial index to NULL-only rows.
--
-- The original index (2026-05-06-000000_add_date_taken_source) also matched
-- `date_taken_source = 'fs_time'` so the drain could "re-resolve weak
-- entries when better tools become available." In practice the resolver
-- is deterministic on file bytes + filename + fs metadata: a row that
-- landed on fs_time once will land on fs_time again on every subsequent
-- tick. With `ORDER BY id ASC LIMIT 500`, the drain spun on the same
-- lowest-id fs_time rows in perpetuity, never advancing, while hammering
-- the SQLite write lock once per row and starving other writers (face
-- PATCHes were hitting busy_timeout and returning 500). Drop fs_time
-- from the eligibility set; if exiftool / a new filename pattern ever
-- comes online, a one-shot operator command can re-resolve.
DROP INDEX IF EXISTS idx_image_exif_date_backfill;
CREATE INDEX idx_image_exif_date_backfill
ON image_exif (library_id, id)
WHERE date_taken IS NULL;