Cameron Cordes
d2263be6ac
The exiftool branch of the date waterfall now pulls raw strings with group prefixes (one `exiftool -j -q -G1` subprocess per batch, no `-d %s`) and classifies each candidate: - zoned strings (own offset / Z) are true instants; - naive strings from container groups (QuickTime/Track*/Apple) are UTC-rendered creation_time fields -> true instants (utc_direct). Running them through the camera wall-clock cascade was the "videos off by 4-5 hours" bug: it reinterpreted a UTC instant in the server zone; - naive strings from photo-EXIF groups are camera wall clocks and run the offset cascade (OffsetTime* pairing + GPS evidence pulled in the same exiftool pass). The legacy date_taken column stays byte-identical to the old server-local interpretation (reproduced in Rust, DST-aware); the new-contract date_utc / camera_offset_minutes / date_taken_tz_source columns carry the true instant. Manual date edit (POST /image/exif/date) reinterprets the naive-as-UTC epoch the client sent as a wall clock in the user's display zone (camera_tz::manual_date_resolution, tz_source = 'manual'), snapshots the pre-override new-contract triple next to the existing legacy snapshot pair (3 new columns, migration 2026-09-25-000000), and revert restores both pairs. ADR-0001 records the container-date finding; CLAUDE.md date-pipeline section updated.
83 lines
5.4 KiB
Markdown
83 lines
5.4 KiB
Markdown
# ADR-0001: Dates are true instants; wall clocks are a display concern
|
||
|
||
**Status**: accepted (2026-09-24)
|
||
|
||
## Context
|
||
|
||
`image_exif.date_taken` (Unix i64) accumulated four conventions in one column, indistinguishable
|
||
without joining on `date_taken_source`: EXIF rows were camera wall-clocks reinterpreted as UTC,
|
||
exiftool rows were naive dates interpreted in the server's local zone, filename rows were
|
||
deliberate wall-as-UTC, and filesystem-time rows were true instants. SMS-API and the Apollo
|
||
location track were already true UTC. Every consumer had to re-derive the missing shift
|
||
(Apollo's `_naive_to_real_utc`, the frontend `naiveAsUtcToRealUtc`, FileViewer's bare
|
||
device-local formatting, `/memories`'s per-client offset) and they disagreed: the same photo
|
||
showed different clocks in the two apps, exiftool-sourced videos (whose dates only come from
|
||
exiftool) were double-shifted, and the insight-context prompt formatted a value 4–5 hours
|
||
wrong for video rows, rolling across midnight near day edges.
|
||
|
||
## Decision
|
||
|
||
The date a media file was captured is stored as a **true instant** (Unix seconds of the
|
||
actual moment). The wall clock the camera showed is a **display value**, recovered from the
|
||
instant plus a per-row **camera offset** resolved at ingest by a cascade (explicit EXIF
|
||
offset tag → GPS position → server zone, the last recorded as an assumption). Day-level
|
||
grouping uses the owner's **display zone** (an IANA zone name). Manual edits enter as wall
|
||
clocks in the display zone (rendered naive-as-UTC by the client) and are
|
||
stored as instants; the legacy column keeps the sent value verbatim and
|
||
revert restores a snapshot of both the legacy pair and the new-contract
|
||
triple (`original_date_taken*` / `original_date_utc` /
|
||
`original_camera_offset_minutes` / `original_date_taken_tz_source`).
|
||
|
||
Consequences of the decision:
|
||
|
||
- Matching photo↔SMS, photo↔location, and photo↔calendar becomes instant-to-instant with no
|
||
per-consumer shift; the re-anchoring helpers in the consumers are deleted, not migrated.
|
||
- A photo's displayed time can differ from the owner's local time (a photo taken in Japan
|
||
shows the Japan wall clock); grouping still uses the owner's day. This is deliberate.
|
||
- The legacy `date_taken` column is frozen during migration (a new `date_utc` column is
|
||
populated by a per-tick backfill drain), so rollback is a client-side flip and the old
|
||
contract stays byte-identical until cutover. The legacy column is retained after cutover
|
||
for historical context.
|
||
|
||
## Considered options
|
||
|
||
- **Standardize on wall-clock-as-UTC** (fix only the exiftool branch): cheapest, but every
|
||
consumer would still carry a display-zone re-anchor as a proxy for the camera's zone, and
|
||
cross-zone photo↔SMS matching would stay approximate. Rejected.
|
||
- **Two parallel fields as a permanent contract** (wall-as-UTC `date_taken` + `date_utc`):
|
||
honest, but leaves the ambiguity in the interface forever. The dual-field shape is used
|
||
*temporarily* (migration window) only.
|
||
- **Server-side conversion per request** (keep the column as-is, convert on read): pushes
|
||
the mixed-convention join onto every query and every consumer; rejected.
|
||
|
||
## Non-obvious rules worth remembering
|
||
|
||
- **Filename date class outranks EXIF** — for files whose filename carries the capture date
|
||
(screenshots, timestamp exports), the filename is the original date of record, inverting
|
||
the global EXIF-wins rule. The class is a documented exception, not a bug.
|
||
- **Offset resolution is per-row and audited** (`date_taken_tz_source`), because a library
|
||
contains photos from many zones; a single global offset would be wrong for every trip.
|
||
- **The display zone is a zone name, not minutes** — a hardcoded offset silently breaks at
|
||
DST. (The legacy `tz_offset_minutes` default of -480 on a New York server is a surviving
|
||
symptom of this.)
|
||
- **Re-dating across a day boundary invalidates derived day data** (daily summaries, reel
|
||
pre-gen), which regenerates on its normal schedule; the backfill drain detects the day
|
||
change per row.
|
||
- **Container dates are true instants, not wall clocks.** QuickTime/MP4/MOV
|
||
`creation_time` fields are seconds-since-1904 in UTC; exiftool renders them as
|
||
UTC-naive strings. Running a naive container string through the camera wall-clock
|
||
cascade reinterprets a UTC instant in the server zone and shifts every video by the
|
||
server's UTC offset (the "videos off by 4–5 hours" bug, ticket #115). Verified against
|
||
production data: a mirrorless video lands inside the same-moment photo burst, and the
|
||
delta across all sampled videos is the uniform server EDT/EST offset. So the exiftool
|
||
branch classifies the *group* of each tag: zoned strings are instants outright,
|
||
container-group naives are instants (`utc_direct`), and only photo-EXIF-group naives
|
||
run the cascade. The legacy column for those rows stays byte-identical to the old
|
||
`-d %s` server-local interpretation; the new-contract columns carry the true instant.
|
||
- **The legacy column is reproduced in Rust, not by re-invoking exiftool.** The old
|
||
`-d %s` path interpreted exiftool's rendered naive string in the *server's* local zone
|
||
(DST-aware, ignoring the file's own `OffsetTime*` tags — verified). `date_taken`
|
||
for exiftool rows must keep producing exactly that value, so the resolver parses the
|
||
raw string and applies the server-zone interpretation itself (`utc_direct` rows skip
|
||
the cascade but still get the server-local legacy value).
|