Pre-resolver rows already had a populated `date_taken` from the old
kamadak-exif-only ingest path. The column-add migration left their
`date_taken_source` as NULL, and the drain's eligibility predicate
(`date_taken IS NULL OR date_taken_source = 'fs_time'`) skips them —
so they remain unlabelled forever and never benefit from the
resolver's exiftool fallback even if they're videos that should
upgrade.
Label them all `'exif'` in a one-shot UPDATE. Safe because every
write path that populated `date_taken` before the resolver landed was
a kamadak-exif read. Idempotent (the WHERE matches nothing on a
second run). Down.sql is a no-op — the labels stay correct under any
schema state, and the column-add migration is the right place to
revert if needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the EXIF-loop + WalkDir-fallback pipeline that powered
`/memories` with a single per-library SQL query
(`get_memories_in_window`) that uses `strftime('%m-%d' | '%W' | '%m',
date_taken, 'unixepoch', tz_offset)` for calendar matching in the
client's timezone, plus a `years_back` lower bound and a
no-future-dates upper bound. Returns only the matching rows; the
handler applies per-library `PathExcluder` post-query and sorts.
Drops:
- `collect_exif_memories` — replaced by the single SQL query.
- `collect_filesystem_memories` — the canonical-date pipeline now
populates `date_taken` for every row at ingest, so the WalkDir
fallback that scanned 14k+ files each request is no longer needed.
- `get_memory_date_with_priority` and friends — request-time waterfall
superseded by `date_resolver` running at ingest. The associated
three priority-tests are dropped; their replacement lives in
`date_resolver::tests`.
On a ~14k-file library this drops `/memories` from 10–15 s
(dominated by `fs::metadata` per row) to single-digit ms.
Bumps `DEFAULT_YEARS_BACK` from 15 → 20 to surface deeper archives
on matching anniversaries.
Note vs. ISO weeks: the original Rust used `chrono::iso_week().week()`
for week-span matching. SQLite's `%W` is Monday-anchored but uses week
0 for days before the first Monday, so it can disagree with ISO at
year boundaries by ±1. Acceptable for nostalgia browsing.
Adds 3 new DAO tests covering month-span filter, library scoping, and
the unknown-span-token guard. Also adds a CLAUDE.md section describing
the canonical-date pipeline end-to-end and the new
`DATE_BACKFILL_MAX_PER_TICK` env var.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds two ExifDao methods (`get_rows_needing_date_backfill` /
`backfill_date_taken`) and a `backfill_missing_date_taken` watcher pass
that runs on every tick alongside `backfill_unhashed_backlog`.
The drain queries the partial index for rows where `date_taken IS NULL`
or `date_taken_source = 'fs_time'`, batches up to
`DATE_BACKFILL_MAX_PER_TICK` paths (default 500), and feeds them through
`date_resolver::resolve_dates_batch` — a single exiftool subprocess
covers the whole tick. Rows that newly resolve to `exiftool` /
`filename` / `fs_time` get persisted via `backfill_date_taken` (touches
only `date_taken` + `date_taken_source` so EXIF / hash / perceptual
columns survive).
`filename`-sourced rows are intentionally not re-resolved — the regex
is authoritative when it matches and re-running exiftool wouldn't
change the answer. Files that have disappeared from disk are skipped
so a ghost row doesn't loop through the drain forever; the
missing-file scan in `library_maintenance` retires those separately.
Comes with two DAO unit tests (eligibility filter + column-isolation).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires `date_resolver::resolve_date_taken` into the three call sites
that build `InsertImageExif`:
- `process_new_files` (file watcher) — every newly-registered file gets
the resolver's verdict so videos and EXIF-stripped images land with a
real date instead of NULL.
- Upload handler — same waterfall on the post-multipart-write path.
- GPS-write handler — re-runs the waterfall after exiftool writes GPS
and re-reads the EXIF, in case a previously fs_time-sourced row now
has a real EXIF date to upgrade to.
This is a behavior change vs. the pre-rewrite `/memories` request-time
priority: EXIF now beats filename when both are present. A photo
named `Screenshot_2014-06-01.png` whose EXIF `DateTime` is 2021 now
appears under 2021. The reverse case (no EXIF, parseable filename) is
unchanged and continues to surface the filename date with
`date_taken_source = 'filename'`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New module that consolidates the four-step ingest waterfall:
kamadak-exif (already in process via the caller's prior result) →
exiftool fallback → filename regex → earliest_fs_time. Each step is
tagged with a `DateSource` so the caller can persist provenance.
The exiftool fallback is what makes videos and MakerNote-hosted dates
land at all — kamadak-exif can't read QuickTime/MP4 or Nikon-style
sub-IFDs. Single-file mode shells out per call; batch mode pipes paths
on stdin via `-@ -` and fans the result through one subprocess so the
upcoming per-tick drain doesn't pay startup cost per row. The
`exiftool` PATH check is cached in a `OnceLock` to keep the drain
short-circuited on deploys without exiftool installed.
`SubSecDateTimeOriginal` and `ContentCreateDate` are pulled alongside
the standard tags to capture iPhone's sub-second precision and Apple's
preferred capture-time tag respectively. `FileModifyDate` is
deliberately *not* in the tag list — it's a filesystem-derived value
the resolver already covers via the `fs_time` step, and pulling it
through exiftool would mask "no real EXIF date" with a misleading
`source = exiftool` row.
Module is registered in both `lib.rs` and `main.rs` (sibling-module
pattern the rest of the bin uses); no callers wired in yet — that
lands in the next commit. Comes with 9 unit tests covering JSON
parsing edge cases, source-priority short-circuiting, and the
fs_time-when-no-exif path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New nullable TEXT column tracks which step of the canonical-date
waterfall (kamadak-exif → exiftool → filename → fs_time) populated
`date_taken`. Lets a later per-tick drain re-resolve weak sources
(`fs_time`) once stronger ones become available, and gives the UI/debug
surface a way to answer "why does this photo show up under this date?".
Adds the column at all `InsertImageExif` construction sites with `None`
placeholders (the resolver wiring lands in a follow-up commit), and
extends the `update_exif` SET tuple so the column survives the GPS-write
re-read path. Partial index `idx_image_exif_date_backfill` is created
for the upcoming drain query.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
