Closes#118. New per-tick drain at the existing date-drain seam converts legacy mixed-convention rows to true instants (ADR-0001).
watch_files tick, per library
if !health.is_online() → skip library
backfill_missing_date_taken
+ backfill_utc_dates(display_tz, &mut cursor[lib])
+ page rows WHERE date_utc IS NULL AND date_taken IS NOT NULL, id > cursor
+ manual → camera_tz::manual_date_resolution (never re-resolved from file)
+ missing file → page past (no cap spent)
+ exif/exiftool w/o exiftool → page past
+ else → collect (until DATE_BACKFILL_MAX_PER_TICK)
+ resolve_dates_batch(paths, {}) # one exiftool subprocess, same shape as #117 audit
+ write date_utc / camera_offset_minutes / date_taken_tz_source only (guarded date_utc IS NULL)
+ if display-zone day(legacy) != day(date_utc)
+ delete + unlink day-span precomputed_reels (lib + "all") covering either month-day
+ refresh_utc_backfill_progress → imageserver_date_backfill_remaining
GET /libraries → libraries[i]
{ ...library, health,
+ date_backfill_remaining } # live COUNT(date_utc IS NULL), partial index
New migration: idx_image_exif_utc_backfill ON image_exif (library_id, id) WHERE date_utc IS NULL.
Notes on scope:
Daily conversation summaries are SMS-derived (daily_summary_job.rs groups SmsMessage by date), so invalidation covers pre-generated day reels only.
Legacy manual rows convert the way a manual edit does today. Re-resolving them from the file would discard the operator's pick.
Rows are never downgraded: a row whose legacy date came from EXIF or exiftool is not written if it would now resolve to fs_time. That happens when exiftool is missing or fails on the file.
Progress counts every date_utc IS NULL row, including rows with both columns NULL and rows whose file is missing. That is the set cutover (#120) waits on.
Evidence
cargo test: 723 bin tests and 639 lib tests pass. cargo clippy --all-targets reports zero warnings. cargo fmt --check is clean.
New tests in src/backfill.rs use in-memory SQLite and fixture files (a real ffmpeg-made MP4 with container creation_time, and a JPEG with DateTimeOriginal + OffsetTimeOriginal):
converts_one_row_per_source_and_is_idempotent
exif 22:30 +02:00 → 20:30Z, 120, offset_tag (legacy untouched)
video creation 15:58:23Z → 15:58:23Z, NULL, utc_direct (legacy 19:58:23Z untouched)
filename wall → assumed_server
fs_time → instant, utc_direct
manual 12:00 (NY) → 17:00Z, -300, manual
2nd pass → no-op, progress 0
respects_cap_and_missing_files_dont_spend_it cap=2: (converted, remaining) = (2,6) (2,4) (1,3) (0,3) → retire missing → 0
unresolvable_rows_cannot_stall_the_rows_behind_them
Before (cursor reset each tick): 0 converted forever
After (cursor carried): 2 converted
skips_unavailable_library_then_resumes unmounted → nothing converts (not even manual); remount → all
invalidates_exactly_the_moved_days_reels 06-11/06-12 day reels (lib 1 + all) deleted and unlinked; lib 2, 03-01, week reels kept
utc_midnight_straddle_marks_nothing UTC date flips, NY day doesn't → 0 reels touched
leaves_null_date_rows_and_other_libraries_alone
Merge Danger
Door: two-way
The drain writes only the new-contract columns, which nothing reads yet (#119). Reverting the code is enough: the converted values can stay, because they are exactly what the drain would write again. The legacy columns are byte-identical. Dropping the index is the migration's down.sql.
Blast Radius: background
On first deploy the drain converts up to 500 rows per library per tick, with one exiftool subprocess per batch, which is the same load as the existing date drain. Day reels whose day moves are deleted. Reads still use the legacy column until #119, so a regenerated reel picks the same media, and today's day reel can be missing until the next nightly pre-gen run. /libraries gains one COUNT per library.
## Summary
Closes #118. New per-tick drain at the existing date-drain seam converts legacy mixed-convention rows to true instants (ADR-0001).
```diff
watch_files tick, per library
if !health.is_online() → skip library
backfill_missing_date_taken
+ backfill_utc_dates(display_tz, &mut cursor[lib])
+ page rows WHERE date_utc IS NULL AND date_taken IS NOT NULL, id > cursor
+ manual → camera_tz::manual_date_resolution (never re-resolved from file)
+ missing file → page past (no cap spent)
+ exif/exiftool w/o exiftool → page past
+ else → collect (until DATE_BACKFILL_MAX_PER_TICK)
+ resolve_dates_batch(paths, {}) # one exiftool subprocess, same shape as #117 audit
+ write date_utc / camera_offset_minutes / date_taken_tz_source only (guarded date_utc IS NULL)
+ if display-zone day(legacy) != day(date_utc)
+ delete + unlink day-span precomputed_reels (lib + "all") covering either month-day
+ refresh_utc_backfill_progress → imageserver_date_backfill_remaining
```
```diff
GET /libraries → libraries[i]
{ ...library, health,
+ date_backfill_remaining } # live COUNT(date_utc IS NULL), partial index
```
New migration: `idx_image_exif_utc_backfill ON image_exif (library_id, id) WHERE date_utc IS NULL`.
Notes on scope:
- **Daily conversation summaries are SMS-derived** (`daily_summary_job.rs` groups `SmsMessage` by date), so invalidation covers pre-generated day reels only.
- **Legacy `manual` rows** convert the way a manual edit does today. Re-resolving them from the file would discard the operator's pick.
- **Rows are never downgraded:** a row whose legacy date came from EXIF or exiftool is not written if it would now resolve to `fs_time`. That happens when exiftool is missing or fails on the file.
- **Progress counts every `date_utc IS NULL` row**, including rows with both columns NULL and rows whose file is missing. That is the set cutover (#120) waits on.
## Evidence
`cargo test`: 723 bin tests and 639 lib tests pass. `cargo clippy --all-targets` reports zero warnings. `cargo fmt --check` is clean.
New tests in `src/backfill.rs` use in-memory SQLite and fixture files (a real ffmpeg-made MP4 with container `creation_time`, and a JPEG with `DateTimeOriginal` + `OffsetTimeOriginal`):
```text
converts_one_row_per_source_and_is_idempotent
exif 22:30 +02:00 → 20:30Z, 120, offset_tag (legacy untouched)
video creation 15:58:23Z → 15:58:23Z, NULL, utc_direct (legacy 19:58:23Z untouched)
filename wall → assumed_server
fs_time → instant, utc_direct
manual 12:00 (NY) → 17:00Z, -300, manual
2nd pass → no-op, progress 0
respects_cap_and_missing_files_dont_spend_it cap=2: (converted, remaining) = (2,6) (2,4) (1,3) (0,3) → retire missing → 0
unresolvable_rows_cannot_stall_the_rows_behind_them
Before (cursor reset each tick): 0 converted forever
After (cursor carried): 2 converted
skips_unavailable_library_then_resumes unmounted → nothing converts (not even manual); remount → all
invalidates_exactly_the_moved_days_reels 06-11/06-12 day reels (lib 1 + all) deleted and unlinked; lib 2, 03-01, week reels kept
utc_midnight_straddle_marks_nothing UTC date flips, NY day doesn't → 0 reels touched
leaves_null_date_rows_and_other_libraries_alone
```
## Merge Danger
**Door:** two-way
The drain writes only the new-contract columns, which nothing reads yet (#119). Reverting the code is enough: the converted values can stay, because they are exactly what the drain would write again. The legacy columns are byte-identical. Dropping the index is the migration's `down.sql`.
**Blast Radius:** background
On first deploy the drain converts up to 500 rows per library per tick, with one exiftool subprocess per batch, which is the same load as the existing date drain. Day reels whose day moves are deleted. Reads still use the legacy column until #119, so a regenerated reel picks the same media, and today's day reel can be missing until the next nightly pre-gen run. `/libraries` gains one COUNT per library.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Per-tick drain (backfill_utc_dates) at the existing date-drain seam,
inside the per-library online gate. Selects date_utc IS NULL AND
date_taken IS NOT NULL (new partial index idx_image_exif_utc_backfill),
re-runs resolve_dates_batch with empty priors (one exiftool subprocess
per batch, same call shape as the #117 audit), and writes only the
new-contract triple; the legacy pair stays frozen. Capped by
DATE_BACKFILL_MAX_PER_TICK; missing-file rows are paged past without
spending the cap. Legacy manual overrides convert via
manual_date_resolution rather than being re-resolved from the file.
Day-boundary invalidation deletes the day-span precomputed reels
covering a re-dated row's old/new display-zone month-day so the nightly
pre-gen regenerates them. Progress: imageserver_date_backfill_remaining
gauge + date_backfill_remaining on GET /libraries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-up: rows that exist but can't be faithfully resolved spent
the cap and were re-collected first every tick. Carry a per-library
cursor across ticks (wrapping at end of table), share the cap parser
between the two date drains, and name the conversion tuple.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Summary
Closes #118. New per-tick drain at the existing date-drain seam converts legacy mixed-convention rows to true instants (ADR-0001).
New migration:
idx_image_exif_utc_backfill ON image_exif (library_id, id) WHERE date_utc IS NULL.Notes on scope:
daily_summary_job.rsgroupsSmsMessageby date), so invalidation covers pre-generated day reels only.manualrows convert the way a manual edit does today. Re-resolving them from the file would discard the operator's pick.fs_time. That happens when exiftool is missing or fails on the file.date_utc IS NULLrow, including rows with both columns NULL and rows whose file is missing. That is the set cutover (#120) waits on.Evidence
cargo test: 723 bin tests and 639 lib tests pass.cargo clippy --all-targetsreports zero warnings.cargo fmt --checkis clean.New tests in
src/backfill.rsuse in-memory SQLite and fixture files (a real ffmpeg-made MP4 with containercreation_time, and a JPEG withDateTimeOriginal+OffsetTimeOriginal):Merge Danger
Door: two-way
The drain writes only the new-contract columns, which nothing reads yet (#119). Reverting the code is enough: the converted values can stay, because they are exactly what the drain would write again. The legacy columns are byte-identical. Dropping the index is the migration's
down.sql.Blast Radius: background
On first deploy the drain converts up to 500 rows per library per tick, with one exiftool subprocess per batch, which is the same load as the existing date drain. Day reels whose day moves are deleted. Reads still use the legacy column until #119, so a regenerated reel picks the same media, and today's day reel can be missing until the next nightly pre-gen run.
/librariesgains one COUNT per library.🤖 Generated with Claude Code
b7b59c1ba2tofcaf8487c2