Drop fs_time from date-backfill eligibility
The drain queried `date_taken IS NULL OR date_taken_source = 'fs_time'`
ORDER BY id ASC LIMIT 500 every watcher tick. The resolver is
deterministic on file bytes + filename + fs metadata, so any row that
landed on fs_time once landed there again on every retry — the drain
spun on the same lowest-id rows in perpetuity, never advancing to
rows 501+ while still logging more_remain=true.
Side effect: 500 auto-commit UPDATEs per tick sustained the SQLite
write lock long enough that other writers on separate DAO connections
hit the 5s busy_timeout. Manifested as intermittent 500s on
PATCH /image/faces/{id} that succeeded on retry.
Narrow the partial index and query predicate to `date_taken IS NULL`.
If exiftool installs or a new filename regex lands, an operator can
re-resolve fs_time rows out-of-band rather than re-introducing the
steady-state churn.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
18
migrations/2026-05-12-000000_date_backfill_null_only/up.sql
Normal file
18
migrations/2026-05-12-000000_date_backfill_null_only/up.sql
Normal file
@@ -0,0 +1,18 @@
|
||||
-- Narrow the date-backfill partial index to NULL-only rows.
|
||||
--
|
||||
-- The original index (2026-05-06-000000_add_date_taken_source) also matched
|
||||
-- `date_taken_source = 'fs_time'` so the drain could "re-resolve weak
|
||||
-- entries when better tools become available." In practice the resolver
|
||||
-- is deterministic on file bytes + filename + fs metadata: a row that
|
||||
-- landed on fs_time once will land on fs_time again on every subsequent
|
||||
-- tick. With `ORDER BY id ASC LIMIT 500`, the drain spun on the same
|
||||
-- lowest-id fs_time rows in perpetuity, never advancing, while hammering
|
||||
-- the SQLite write lock once per row and starving other writers (face
|
||||
-- PATCHes were hitting busy_timeout and returning 500). Drop fs_time
|
||||
-- from the eligibility set; if exiftool / a new filename pattern ever
|
||||
-- comes online, a one-shot operator command can re-resolve.
|
||||
DROP INDEX IF EXISTS idx_image_exif_date_backfill;
|
||||
|
||||
CREATE INDEX idx_image_exif_date_backfill
|
||||
ON image_exif (library_id, id)
|
||||
WHERE date_taken IS NULL;
|
||||
Reference in New Issue
Block a user