Dates: container (video) dates, filesystem branch, manual edit — true instants #115

New Issue

2026-09-24T22:53:01Z

cameron commented

2026-09-24 22:53:01 +00:00

Parent

ImageApi #113 — SPEC: true-instant date contract (ADR-0001).

What to build

Dates that don't come from an EXIF tag follow the same true-instant contract. Container dates (videos, QuickTime/MP4) are treated as the camera's wall clock and converted through the offset cascade — replacing the current behavior of interpreting them in the server's local zone, which is the root of the video-date errors. Filesystem-time rows are true instants with no wall-clock meaning (offset null). Manual date edits are entered as a wall clock in the display zone, stored as the resulting true instant with offset and source manual, and the previous value is snapshotted for revert.

Acceptance criteria

A video whose container date is a wall clock is stored as the true instant (wall clock converted with the cascade — offset tag if present on the file, else GPS, else assumed server), source recorded accordingly; the server's local zone plays no role in the conversion.
The batch metadata-tool call per drain tick is preserved (one subprocess per batch); the per-tick path and the ingest path produce the same result for the same file.
A filesystem-time row is stored as the true instant with offset null and no offset-source (a wall clock is meaningless for it).
The manual-date endpoint accepts a wall clock in the display zone and stores the resulting true instant, offset = the display-zone offset for that instant, source = manual, and snapshots the prior instant + offset for revert.
Reverting a manual date restores the snapshotted values.
A photo and a video taken at the same instant by the same camera store the same true instant (the video bug — 4–5 h off — is gone at the storage level).
Tests at the waterfall seam: container-date conversion (no server-zone interpretation), filesystem branch, manual round-trip and snapshot.

Blocked by

ImageApi #114 (foundation: schema + EXIF branch + display zone)

## Parent ImageApi #113 — SPEC: true-instant date contract (ADR-0001). ## What to build Dates that don't come from an EXIF tag follow the same true-instant contract. Container dates (videos, QuickTime/MP4) are treated as the camera's wall clock and converted through the offset cascade — replacing the current behavior of interpreting them in the server's local zone, which is the root of the video-date errors. Filesystem-time rows are true instants with no wall-clock meaning (offset null). Manual date edits are entered as a wall clock in the display zone, stored as the resulting true instant with offset and source `manual`, and the previous value is snapshotted for revert. ## Acceptance criteria - [ ] A video whose container date is a wall clock is stored as the true instant (wall clock converted with the cascade — offset tag if present on the file, else GPS, else assumed server), source recorded accordingly; the server's local zone plays no role in the conversion. - [ ] The batch metadata-tool call per drain tick is preserved (one subprocess per batch); the per-tick path and the ingest path produce the same result for the same file. - [ ] A filesystem-time row is stored as the true instant with offset null and no offset-source (a wall clock is meaningless for it). - [ ] The manual-date endpoint accepts a wall clock in the display zone and stores the resulting true instant, offset = the display-zone offset for that instant, source = `manual`, and snapshots the prior instant + offset for revert. - [ ] Reverting a manual date restores the snapshotted values. - [ ] A photo and a video taken at the same instant by the same camera store the same true instant (the video bug — 4–5 h off — is gone at the storage level). - [ ] Tests at the waterfall seam: container-date conversion (no server-zone interpretation), filesystem branch, manual round-trip and snapshot. ## Blocked by - ImageApi #114 (foundation: schema + EXIF branch + display zone)

cameron referenced this issue

2026-09-24 22:56:07 +00:00

Dates: the backfill drain — convert mixed-convention rows, progress surface, day-boundary invalidation #118

cameron referenced this issue

2026-09-24 22:58:16 +00:00

Dates: reads + insight context on the new contract (mixed-row safe) #119

cameron self-assigned this 2026-09-25 02:10:42 +00:00

cameron commented

2026-09-25 19:27:12 +00:00

Done — d2263be on feature/true-utc-dates.

Deviation from the ticket premise (verified, @cameron confirmed)

The ticket says to treat container dates as wall clocks converted through the offset cascade. The production-data investigation showed that would reproduce the bug: QuickTime/MP4/MOV creation_time fields are seconds-since-1904 in UTC — true instants that exiftool renders as UTC-naive strings. Three cross-checks on the NAS confirmed it: a phone video's filename date matches its raw container instant; a mirrorless Yellowstone video lands inside the same-moment photo burst; and the delta between the stored legacy value and the raw instant is the uniform server EDT/EST offset across all sampled files (541 exiftool-sourced videos in the prod DB). The old bug was exactly -d %s re-interpreting that UTC-naive rendering as server-local wall time.

So the exiftool branch now classifies by tag group instead:

zoned strings (own offset / Z) → true instant, utc_direct;
naive + container group (QuickTime / Track* / Apple) → true instant, utc_direct, no cascade;
naive + photo-EXIF group → camera wall clock → offset cascade (OffsetTime* pairing mirroring the kamadak branch, then GPS, then assumed server).

This still satisfies "the server's local zone plays no role in the conversion" for the video rows that were actually wrong, and keeps the cascade for the photo rows where a naive string genuinely is a wall clock.

What shipped

One exiftool subprocess per batch preserved. The invocation changed from -d "%s" to -j -q -G1 (raw strings + group prefixes, plus the OffsetTime/GPS evidence tags), parsed in Rust. exiftool_dates_batch and the ingest single-file path share resolve_exiftool_payload, so per-tick drain and ingest produce identical results for the same file.
Legacy column byte-identical. date_taken for exiftool rows is reproduced in Rust as the old server-local, DST-aware interpretation of the naive rendering (the verified -d %s behavior — it ignores the file's own offset tags); the new-contract triple (date_utc / camera_offset_minutes / date_taken_tz_source) carries the true instant.
Filesystem branch: fs-time rows already store true instants; they now also record date_utc = date_taken, offset null, tz_source = utc_direct (was left unclassified).
Manual edit: POST /image/exif/date reinterprets the client's naive-as-UTC epoch as a wall clock in the user's display zone (new pure seam camera_tz::manual_date_resolution, DST-correct), stores the instant with the display-zone offset and tz_source = manual, and keeps the sent value verbatim in legacy date_taken.
Revert: the snapshot now covers the new-contract triple as well as the legacy pair — three new nullable columns (original_date_utc, original_camera_offset_minutes, original_date_taken_tz_source) added by migration 2026-09-25-000000_manual_date_triple_snapshot (verified against the 21,298-row prod DB). Snapshot is taken once on the first override; revert restores both pairs and nulls all five snapshot columns.

Acceptance criteria

Container-date videos store the true instant (utc_direct), no server-zone involvement — 4–5 h bug gone at the storage level; photo + video at the same instant store the same instant (covered by unit tests with the real Yellowstone photo/video pair's semantics).
One subprocess per batch; drain and ingest share one resolve path.
Fs-time rows: true instant, offset null, no offset-source ambiguity (now explicitly utc_direct).
Manual endpoint: display-zone wall clock → true instant, display-zone offset, source manual, prior values snapshotted.
Revert restores both the legacy pair and the new-contract triple.

712 tests pass, cargo fmt + cargo clippy --all-targets clean. ADR-0001 gained the container-date finding and the legacy-reproduction rule; CLAUDE.md's date-pipeline section updated.

Done — `d2263be` on `feature/true-utc-dates`. ## Deviation from the ticket premise (verified, @cameron confirmed) The ticket says to treat container dates as *wall clocks converted through the offset cascade*. The production-data investigation showed that would **reproduce the bug**: QuickTime/MP4/MOV `creation_time` fields are seconds-since-1904 **in UTC** — true instants that exiftool renders as UTC-naive strings. Three cross-checks on the NAS confirmed it: a phone video's filename date matches its raw container instant; a mirrorless Yellowstone video lands inside the same-moment photo burst; and the delta between the stored legacy value and the raw instant is the **uniform server EDT/EST offset** across all sampled files (541 exiftool-sourced videos in the prod DB). The old bug was exactly `-d %s` re-interpreting that UTC-naive rendering as server-local wall time. So the exiftool branch now **classifies by tag group** instead: - **zoned strings** (own offset / `Z`) → true instant, `utc_direct`; - **naive + container group** (QuickTime / Track\* / Apple) → true instant, `utc_direct`, no cascade; - **naive + photo-EXIF group** → camera wall clock → offset cascade (`OffsetTime*` pairing mirroring the kamadak branch, then GPS, then assumed server). This still satisfies "the server's local zone plays no role in the conversion" for the video rows that were actually wrong, and keeps the cascade for the photo rows where a naive string genuinely is a wall clock. ## What shipped - **One exiftool subprocess per batch preserved.** The invocation changed from `-d "%s"` to `-j -q -G1` (raw strings + group prefixes, plus the OffsetTime/GPS evidence tags), parsed in Rust. `exiftool_dates_batch` and the ingest single-file path share `resolve_exiftool_payload`, so per-tick drain and ingest produce identical results for the same file. - **Legacy column byte-identical.** `date_taken` for exiftool rows is reproduced in Rust as the old server-local, DST-aware interpretation of the naive rendering (the verified `-d %s` behavior — it ignores the file's own offset tags); the new-contract triple (`date_utc` / `camera_offset_minutes` / `date_taken_tz_source`) carries the true instant. - **Filesystem branch:** fs-time rows already store true instants; they now also record `date_utc = date_taken`, offset null, `tz_source = utc_direct` (was left unclassified). - **Manual edit:** `POST /image/exif/date` reinterprets the client's naive-as-UTC epoch as a wall clock in the user's display zone (new pure seam `camera_tz::manual_date_resolution`, DST-correct), stores the instant with the display-zone offset and `tz_source = manual`, and keeps the sent value verbatim in legacy `date_taken`. - **Revert:** the snapshot now covers the new-contract triple as well as the legacy pair — three new nullable columns (`original_date_utc`, `original_camera_offset_minutes`, `original_date_taken_tz_source`) added by migration `2026-09-25-000000_manual_date_triple_snapshot` (verified against the 21,298-row prod DB). Snapshot is taken once on the first override; revert restores both pairs and nulls all five snapshot columns. ## Acceptance criteria - [x] Container-date videos store the true instant (`utc_direct`), no server-zone involvement — 4–5 h bug gone at the storage level; photo + video at the same instant store the same instant (covered by unit tests with the real Yellowstone photo/video pair's semantics). - [x] One subprocess per batch; drain and ingest share one resolve path. - [x] Fs-time rows: true instant, offset null, no offset-source ambiguity (now explicitly `utc_direct`). - [x] Manual endpoint: display-zone wall clock → true instant, display-zone offset, source `manual`, prior values snapshotted. - [x] Revert restores both the legacy pair and the new-contract triple. 712 tests pass, `cargo fmt` + `cargo clippy --all-targets` clean. ADR-0001 gained the container-date finding and the legacy-reproduction rule; CLAUDE.md's date-pipeline section updated.

cameron closed this issue

2026-09-25 19:27:17 +00:00

cameron referenced this issue from a commit

2026-09-25 19:28:07 +00:00

feat(dates): container video dates, filesystem branch, manual edit (#115)

cameron referenced this issue

2026-09-25 19:30:41 +00:00

feat(dates): true-instant date contract — container dates, fs branch, manual edit (#114, #115) #121

cameron referenced this issue from a commit

2026-09-25 19:39:28 +00:00

refactor(dates): condense the two date-contract migrations into one

cameron referenced this issue from a commit