match_filename_date only applied the year gate (1995..=now+1) to the unix-timestamp branch. The five wall-clock patterns accepted any year, so counters and IDs shaped like YYYYMMDD_HHMMSS became dates of record. Prod has 67 such rows (years 267 to 9985), all matched by screenshot_compact:
VID_<counter>_<HHMMSS> videos, e.g. VID_22290524_080551_818.mp4 → 2229
Facebook downloads, e.g. 10530519_10152700326093436_…_o.jpg → 1053
Fix
One gate for both branches.plausible_filename_year is now used by the timestamp branch and by every wall pattern. A wall miss falls through to the next pattern, then to the rest of the waterfall (EXIF → exiftool → fs time).
Audit:filename_audit uses the same matcher, so it no longer proposes these dates.
Existing rows: migration 2026-10-16-000000_reresolve_implausible_filename_dates clears date_taken, date_taken_source, date_utc, camera_offset_minutes and date_taken_tz_source on filename-sourced rows dated outside [1995-01-01, 2028-01-01) UTC. date_taken IS NULL puts them back in backfill_missing_date_taken, which rewrites all five columns through the fixed waterfall on the next ticks. That covers the issue's note that the legacy date_taken needs revisiting too, which the true-instant drain alone wouldn't do. Only filename rows are touched; down.sql is a no-op.
Tests
test_wall_patterns_reject_implausible_years: every wall pattern, too far out (2229) and too far back (1053).
test_wall_patterns_reject_prod_counter_and_id_names: the shapes from the prod list.
test_wall_pattern_miss_falls_through_to_next_pattern: a bogus screenshot_compact match still lets compact find 2014.
test_wall_patterns_keep_plausible_edges: 1995 and now+1 still match; now+2 doesn't.
migration_clears_implausible_filename_dates_for_re_resolution: migrates to just before the fix, seeds old-matcher rows, then applies it. Bogus rows are cleared; a real 2014 filename row and a non-filename row are untouched.
Full suite passes (743 lib + 836 bin), and clippy is clean.
After deploy
The migration runs at startup and the drain re-resolves the 67 rows over the next tick or two (they're within DATE_BACKFILL_MAX_PER_TICK). To check, re-run the issue's query for wall_clock years outside 1995–2027; it should return 0 rows, and the videos should land in their YYYY/MM/ folder's month.
Closes #160.
## Problem
`match_filename_date` only applied the year gate (1995..=now+1) to the unix-timestamp branch. The five wall-clock patterns accepted any year, so counters and IDs shaped like `YYYYMMDD_HHMMSS` became dates of record. Prod has 67 such rows (years 267 to 9985), all matched by `screenshot_compact`:
- `VID_<counter>_<HHMMSS>` videos, e.g. `VID_22290524_080551_818.mp4` → 2229
- Facebook downloads, e.g. `10530519_10152700326093436_…_o.jpg` → 1053
## Fix
- **One gate for both branches.** `plausible_filename_year` is now used by the timestamp branch and by every wall pattern. A wall miss falls through to the next pattern, then to the rest of the waterfall (EXIF → exiftool → fs time).
- **Audit:** `filename_audit` uses the same matcher, so it no longer proposes these dates.
- **Existing rows:** migration `2026-10-16-000000_reresolve_implausible_filename_dates` clears `date_taken`, `date_taken_source`, `date_utc`, `camera_offset_minutes` and `date_taken_tz_source` on `filename`-sourced rows dated outside [1995-01-01, 2028-01-01) UTC. `date_taken IS NULL` puts them back in `backfill_missing_date_taken`, which rewrites all five columns through the fixed waterfall on the next ticks. That covers the issue's note that the legacy `date_taken` needs revisiting too, which the true-instant drain alone wouldn't do. Only filename rows are touched; `down.sql` is a no-op.
## Tests
- `test_wall_patterns_reject_implausible_years`: every wall pattern, too far out (2229) and too far back (1053).
- `test_wall_patterns_reject_prod_counter_and_id_names`: the shapes from the prod list.
- `test_wall_pattern_miss_falls_through_to_next_pattern`: a bogus `screenshot_compact` match still lets `compact` find 2014.
- `test_wall_patterns_keep_plausible_edges`: 1995 and now+1 still match; now+2 doesn't.
- `audit_leaves_out_counter_and_id_names_with_implausible_years`
- `migration_clears_implausible_filename_dates_for_re_resolution`: migrates to just before the fix, seeds old-matcher rows, then applies it. Bogus rows are cleared; a real 2014 filename row and a non-filename row are untouched.
- Full suite passes (743 lib + 836 bin), and clippy is clean.
## After deploy
The migration runs at startup and the drain re-resolves the 67 rows over the next tick or two (they're within `DATE_BACKFILL_MAX_PER_TICK`). To check, re-run the issue's query for wall_clock years outside 1995–2027; it should return 0 rows, and the videos should land in their `YYYY/MM/` folder's month.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
match_filename_date only gated the unix-timestamp branch to years
1995..=now+1; the five wall patterns returned any NaiveDateTime. Counters
and IDs shaped like YYYYMMDD_HHMMSS (an app's VID_<counter>_<HHMMSS>,
Facebook's <8 digits>_<id>_o.jpg) became dates of record in years from
267 to 9985: 67 rows on prod, all via screenshot_compact.
Both branches now share plausible_filename_year; a wall miss falls
through to the next pattern and then the rest of the waterfall. The
filename audit no longer proposes these dates.
A data migration clears the five date columns on filename-sourced rows
dated outside [1995, 2028) UTC, so backfill_missing_date_taken
re-resolves them from EXIF / exiftool / fs time on the next ticks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cameron
merged commit a11d20bdcd into master2026-10-17 03:22:43 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Closes #160.
Problem
match_filename_dateonly applied the year gate (1995..=now+1) to the unix-timestamp branch. The five wall-clock patterns accepted any year, so counters and IDs shaped likeYYYYMMDD_HHMMSSbecame dates of record. Prod has 67 such rows (years 267 to 9985), all matched byscreenshot_compact:VID_<counter>_<HHMMSS>videos, e.g.VID_22290524_080551_818.mp4→ 222910530519_10152700326093436_…_o.jpg→ 1053Fix
plausible_filename_yearis now used by the timestamp branch and by every wall pattern. A wall miss falls through to the next pattern, then to the rest of the waterfall (EXIF → exiftool → fs time).filename_audituses the same matcher, so it no longer proposes these dates.2026-10-16-000000_reresolve_implausible_filename_datesclearsdate_taken,date_taken_source,date_utc,camera_offset_minutesanddate_taken_tz_sourceonfilename-sourced rows dated outside [1995-01-01, 2028-01-01) UTC.date_taken IS NULLputs them back inbackfill_missing_date_taken, which rewrites all five columns through the fixed waterfall on the next ticks. That covers the issue's note that the legacydate_takenneeds revisiting too, which the true-instant drain alone wouldn't do. Only filename rows are touched;down.sqlis a no-op.Tests
test_wall_patterns_reject_implausible_years: every wall pattern, too far out (2229) and too far back (1053).test_wall_patterns_reject_prod_counter_and_id_names: the shapes from the prod list.test_wall_pattern_miss_falls_through_to_next_pattern: a bogusscreenshot_compactmatch still letscompactfind 2014.test_wall_patterns_keep_plausible_edges: 1995 and now+1 still match; now+2 doesn't.audit_leaves_out_counter_and_id_names_with_implausible_yearsmigration_clears_implausible_filename_dates_for_re_resolution: migrates to just before the fix, seeds old-matcher rows, then applies it. Bogus rows are cleared; a real 2014 filename row and a non-filename row are untouched.After deploy
The migration runs at startup and the drain re-resolves the 67 rows over the next tick or two (they're within
DATE_BACKFILL_MAX_PER_TICK). To check, re-run the issue's query for wall_clock years outside 1995–2027; it should return 0 rows, and the videos should land in theirYYYY/MM/folder's month.🤖 Generated with Claude Code