From d2263be6acd3690bb0d6061a437cb90d1e171790 Mon Sep 17 00:00:00 2001 From: Cameron Cordes Date: Fri, 25 Sep 2026 15:26:26 -0400 Subject: [PATCH] feat(dates): container video dates, filesystem branch, manual edit (#115) The exiftool branch of the date waterfall now pulls raw strings with group prefixes (one `exiftool -j -q -G1` subprocess per batch, no `-d %s`) and classifies each candidate: - zoned strings (own offset / Z) are true instants; - naive strings from container groups (QuickTime/Track*/Apple) are UTC-rendered creation_time fields -> true instants (utc_direct). Running them through the camera wall-clock cascade was the "videos off by 4-5 hours" bug: it reinterpreted a UTC instant in the server zone; - naive strings from photo-EXIF groups are camera wall clocks and run the offset cascade (OffsetTime* pairing + GPS evidence pulled in the same exiftool pass). The legacy date_taken column stays byte-identical to the old server-local interpretation (reproduced in Rust, DST-aware); the new-contract date_utc / camera_offset_minutes / date_taken_tz_source columns carry the true instant. Manual date edit (POST /image/exif/date) reinterprets the naive-as-UTC epoch the client sent as a wall clock in the user's display zone (camera_tz::manual_date_resolution, tz_source = 'manual'), snapshots the pre-override new-contract triple next to the existing legacy snapshot pair (3 new columns, migration 2026-09-25-000000), and revert restores both pairs. ADR-0001 records the container-date finding; CLAUDE.md date-pipeline section updated. --- CLAUDE.md | 23 +- docs/adr/0001-true-utc-dates.md | 23 +- .../down.sql | 3 + .../up.sql | 13 + src/ai/insight_chat.rs | 3 + src/camera_tz.rs | 81 +- src/data/mod.rs | 14 + src/database/mod.rs | 93 ++- src/database/models.rs | 8 + src/database/schema.rs | 3 + src/date_resolver.rs | 736 ++++++++++++++---- src/files.rs | 10 + src/handlers/image.rs | 34 +- 13 files changed, 866 insertions(+), 178 deletions(-) create mode 100644 migrations/2026-09-25-000000_manual_date_triple_snapshot/down.sql create mode 100644 migrations/2026-09-25-000000_manual_date_triple_snapshot/up.sql diff --git a/CLAUDE.md b/CLAUDE.md index 87d6674..063e395 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -380,9 +380,21 @@ land on this date?". Order: QuickTime/MP4 (`MediaCreateDate`, `TrackCreateDate`, `CreateDate`), Apple's `ContentCreateDate`, MakerNote sub-IFDs. Required for videos to land a real date. Single-file at ingest; the per-tick - drain feeds the whole batch through one `exiftool -@ -` subprocess. - Degrades silently when `exiftool` isn't on PATH (resolver caches the - "available" check via `OnceLock`). + drain feeds the whole batch through one `exiftool -j -q -G1 -@ -` + subprocess (raw strings + group prefixes, parsed in Rust — no + `-d %s`). Degrades silently when `exiftool` isn't on PATH (resolver + caches the "available" check via `OnceLock`). + **Group classification (ADR-0001, ticket #115):** zoned strings + (own offset / `Z`) are true instants; naive strings from container + groups (QuickTime/Track*/Apple) are UTC-rendered `creation_time` + fields → true instants (`utc_direct`, no cascade — running them + through the wall-clock cascade was the "videos off by 4–5 hours" + bug); naive strings from photo-EXIF groups are camera wall clocks → + offset cascade (`OffsetTime*` + GPS evidence pulled in the same + pass). The legacy `date_taken` column stays byte-identical to the + old server-local `-d %s` interpretation; the new-contract + `date_utc` / `camera_offset_minutes` / `date_taken_tz_source` + columns carry the true instant. 3. **`filename`** — `extract_date_from_filename` in `memories.rs` matches screenshot, chat-export, and timestamp-named patterns. 4. **`fs_time`** — `earliest_fs_time(metadata)` (earlier of created / @@ -401,8 +413,9 @@ watcher tick alongside `backfill_unhashed_backlog` (also `src/backfill.rs`). It `date_taken IS NULL` (backed by the `idx_image_exif_date_backfill` partial index), runs the waterfall batch via `resolve_dates_batch`, and writes results via the `backfill_date_taken` DAO method (touches -only `date_taken` + `date_taken_source` so EXIF / hash / perceptual -columns are preserved). Resolved rows — including the ones the +`date_taken` + `date_taken_source` plus the new-contract +`date_utc` / `camera_offset_minutes` / `date_taken_tz_source` triple, +so EXIF / hash / perceptual columns are preserved). Resolved rows — including the ones the waterfall could only resolve via `fs_time` — are not re-eligible: the resolver is deterministic on file bytes + filename + fs metadata, so re-running on the same inputs lands on the same source every time. diff --git a/docs/adr/0001-true-utc-dates.md b/docs/adr/0001-true-utc-dates.md index a6a81aa..ff9f077 100644 --- a/docs/adr/0001-true-utc-dates.md +++ b/docs/adr/0001-true-utc-dates.md @@ -22,7 +22,11 @@ actual moment). The wall clock the camera showed is a **display value**, recover instant plus a per-row **camera offset** resolved at ingest by a cascade (explicit EXIF offset tag → GPS position → server zone, the last recorded as an assumption). Day-level grouping uses the owner's **display zone** (an IANA zone name). Manual edits enter as wall -clocks in the display zone and are stored as instants. +clocks in the display zone (rendered naive-as-UTC by the client) and are +stored as instants; the legacy column keeps the sent value verbatim and +revert restores a snapshot of both the legacy pair and the new-contract +triple (`original_date_taken*` / `original_date_utc` / +`original_camera_offset_minutes` / `original_date_taken_tz_source`). Consequences of the decision: @@ -59,3 +63,20 @@ Consequences of the decision: - **Re-dating across a day boundary invalidates derived day data** (daily summaries, reel pre-gen), which regenerates on its normal schedule; the backfill drain detects the day change per row. +- **Container dates are true instants, not wall clocks.** QuickTime/MP4/MOV + `creation_time` fields are seconds-since-1904 in UTC; exiftool renders them as + UTC-naive strings. Running a naive container string through the camera wall-clock + cascade reinterprets a UTC instant in the server zone and shifts every video by the + server's UTC offset (the "videos off by 4–5 hours" bug, ticket #115). Verified against + production data: a mirrorless video lands inside the same-moment photo burst, and the + delta across all sampled videos is the uniform server EDT/EST offset. So the exiftool + branch classifies the *group* of each tag: zoned strings are instants outright, + container-group naives are instants (`utc_direct`), and only photo-EXIF-group naives + run the cascade. The legacy column for those rows stays byte-identical to the old + `-d %s` server-local interpretation; the new-contract columns carry the true instant. +- **The legacy column is reproduced in Rust, not by re-invoking exiftool.** The old + `-d %s` path interpreted exiftool's rendered naive string in the *server's* local zone + (DST-aware, ignoring the file's own `OffsetTime*` tags — verified). `date_taken` + for exiftool rows must keep producing exactly that value, so the resolver parses the + raw string and applies the server-zone interpretation itself (`utc_direct` rows skip + the cascade but still get the server-local legacy value). diff --git a/migrations/2026-09-25-000000_manual_date_triple_snapshot/down.sql b/migrations/2026-09-25-000000_manual_date_triple_snapshot/down.sql new file mode 100644 index 0000000..13202cf --- /dev/null +++ b/migrations/2026-09-25-000000_manual_date_triple_snapshot/down.sql @@ -0,0 +1,3 @@ +ALTER TABLE image_exif DROP COLUMN original_date_utc; +ALTER TABLE image_exif DROP COLUMN original_camera_offset_minutes; +ALTER TABLE image_exif DROP COLUMN original_date_taken_tz_source; diff --git a/migrations/2026-09-25-000000_manual_date_triple_snapshot/up.sql b/migrations/2026-09-25-000000_manual_date_triple_snapshot/up.sql new file mode 100644 index 0000000..785393f --- /dev/null +++ b/migrations/2026-09-25-000000_manual_date_triple_snapshot/up.sql @@ -0,0 +1,13 @@ +-- Manual date-edit rework (ticket #115). +-- +-- POST /image/exif/date now writes the new-contract triple +-- (date_utc, camera_offset_minutes, date_taken_tz_source = 'manual') +-- alongside the legacy (date_taken, date_taken_source) pair. Reverting an +-- override must restore the resolver's new-contract triple as well as the +-- legacy pair, so image_exif snapshots the pre-override new-contract +-- columns next to the existing legacy snapshot pair +-- (original_date_taken / original_date_taken_source). + +ALTER TABLE image_exif ADD COLUMN original_date_utc BIGINT; +ALTER TABLE image_exif ADD COLUMN original_camera_offset_minutes INTEGER; +ALTER TABLE image_exif ADD COLUMN original_date_taken_tz_source TEXT; diff --git a/src/ai/insight_chat.rs b/src/ai/insight_chat.rs index b34fa8f..7f454a1 100644 --- a/src/ai/insight_chat.rs +++ b/src/ai/insight_chat.rs @@ -3381,6 +3381,9 @@ mod tests { date_utc: Some(1_629_028_800), camera_offset_minutes: None, date_taken_tz_source: Some("utc_direct".to_string()), + original_date_utc: None, + original_camera_offset_minutes: None, + original_date_taken_tz_source: None, }); let out = resolve_date_taken_for_context(&exif, "Screenshot_2014-06-01.png"); assert_eq!(out.as_deref(), Some("2021-08-15")); diff --git a/src/camera_tz.rs b/src/camera_tz.rs index 1a06672..9c4d46c 100644 --- a/src/camera_tz.rs +++ b/src/camera_tz.rs @@ -40,9 +40,8 @@ pub enum TzSource { /// The resolver branch already produced a true UTC instant (exiftool, /// filesystem time, unix-timestamp filename). No offset is involved. UtcDirect, - /// An operator set the date manually. - // Written by the manual-edit rework (#118); nothing constructs it yet. - #[allow(dead_code)] + /// An operator set the date manually (POST /image/exif/date, ticket + /// #115): the wall the user picked, interpreted in their display zone. Manual, } @@ -218,13 +217,39 @@ pub const DEFAULT_DISPLAY_TZ: &str = "America/New_York"; /// NULL / empty / unparseable prefs fall back to the app default, so /// read paths (tickets #119/#120) never have to special-case the /// unset state themselves. -pub fn display_zone_for(stored: Option<&str>) -> &str { +pub fn display_zone_for(stored: Option<&str>) -> String { match stored.map(str::trim).filter(|s| !s.is_empty()) { - Some(name) if Tz::from_str(name).is_ok() => name, - _ => DEFAULT_DISPLAY_TZ, + Some(name) if Tz::from_str(name).is_ok() => name.to_string(), + _ => DEFAULT_DISPLAY_TZ.to_string(), } } +/// Pure seam for the manual date-edit endpoints (ADR-0001, ticket #115): +/// reinterpret the naive-as-UTC epoch the UI sent as a wall clock in the +/// user's display zone, and produce the new-contract +/// `(date_utc, camera_offset_minutes)` pair. The caller writes +/// `date_taken_tz_source = 'manual'` itself and stores the epoch verbatim +/// in the legacy `date_taken` column (byte-identical to the pre-rework +/// behavior). +/// +/// `stored_zone` is `user_ai_prefs.display_tz` (NULL / unparseable → app +/// default). The offset follows the wall's own date through the zone's DST +/// rules — ambiguous fall-back walls take the earlier (pre-transition) +/// offset; nonexistent spring-forward walls take the zone's offset at the +/// wall-as-UTC reading, within an hour of truth (see +/// [`offset_minutes_for_zone`]). `None` only when the zone name is +/// unparseable, which [`display_zone_for`] already rules out — the caller +/// then leaves the new-contract triple untouched. +pub fn manual_date_resolution( + stored_zone: Option<&str>, + wall_naive_epoch: i64, +) -> Option<(i64, i32)> { + let zone = Tz::from_str(&display_zone_for(stored_zone)).ok()?; + let wall = chrono::DateTime::from_timestamp(wall_naive_epoch, 0).map(|dt| dt.naive_utc())?; + let minutes = offset_minutes_for_zone(zone, wall); + Some((wall_to_instant(wall, minutes), minutes)) +} + /// Display-zone offset, in minutes, for a true UTC instant — DST-aware. /// `None` when the zone name doesn't parse. #[allow(dead_code)] // consumed by the read-side cutover (#119/#120) @@ -240,10 +265,13 @@ mod tests { #[test] fn display_zone_for_falls_back_to_default() { - assert_eq!(display_zone_for(None), DEFAULT_DISPLAY_TZ); - assert_eq!(display_zone_for(Some("")), DEFAULT_DISPLAY_TZ); - assert_eq!(display_zone_for(Some(" ")), DEFAULT_DISPLAY_TZ); - assert_eq!(display_zone_for(Some("Not/A_Zone")), DEFAULT_DISPLAY_TZ); + assert_eq!(display_zone_for(None).as_str(), DEFAULT_DISPLAY_TZ); + assert_eq!(display_zone_for(Some("")).as_str(), DEFAULT_DISPLAY_TZ); + assert_eq!(display_zone_for(Some(" ")).as_str(), DEFAULT_DISPLAY_TZ); + assert_eq!( + display_zone_for(Some("Not/A_Zone")).as_str(), + DEFAULT_DISPLAY_TZ + ); } #[test] @@ -359,6 +387,39 @@ mod tests { ); } + #[test] + fn manual_date_resolution_display_zone_edt() { + // The UI sends the wall rendered in the display zone, as a + // naive-as-UTC epoch: 1720080000 = "2024-07-04 04:00:00". July in + // New York is EDT (-240): the instant is 08:00:00Z. + let (utc, off) = manual_date_resolution(Some("America/New_York"), 1_720_080_000).unwrap(); + assert_eq!(off, -240); + assert_eq!(utc, 1_720_080_000 + 4 * 3600); + } + + #[test] + fn manual_date_resolution_display_zone_est() { + // Same function in January: EST (-300). Wall 2024-01-15 04:00:00 + // (naive epoch 1705291200) → 09:00:00Z. + let (utc, off) = manual_date_resolution(Some("America/New_York"), 1_705_291_200).unwrap(); + assert_eq!(off, -300); + assert_eq!(utc, 1_705_291_200 + 5 * 3600); + } + + #[test] + fn manual_date_resolution_positive_zone_and_default() { + // Tokyo is +09:00 year-round: wall 04:00 → previous day 19:00Z. + let (utc, off) = manual_date_resolution(Some("Asia/Tokyo"), 1_720_080_000).unwrap(); + assert_eq!(off, 540); + assert_eq!(utc, 1_720_080_000 - 540 * 60); + // NULL stored zone → app default (America/New_York), July → EDT. + let (_, off) = manual_date_resolution(None, 1_720_080_000).unwrap(); + assert_eq!(off, -240); + // Unparseable stored zone also falls back to the default. + let (_, off) = manual_date_resolution(Some("Not/A_Zone"), 1_720_080_000).unwrap(); + assert_eq!(off, -240); + } + #[test] fn display_zone_offset_minutes_unknown_zone_is_none() { assert_eq!( diff --git a/src/data/mod.rs b/src/data/mod.rs index 44cc640..71105cd 100644 --- a/src/data/mod.rs +++ b/src/data/mod.rs @@ -307,6 +307,14 @@ pub struct ExifMetadata { pub original_date_taken: Option, /// When `date_taken_source = "manual"`, the prior source. pub original_date_taken_source: Option, + /// When `date_taken_source = "manual"`, the prior `date_utc` snapshot + /// (the new-contract triple the revert restores alongside the legacy + /// pair above). + pub original_date_utc: Option, + /// When `date_taken_source = "manual"`, the prior camera offset. + pub original_camera_offset_minutes: Option, + /// When `date_taken_source = "manual"`, the prior tz source. + pub original_date_taken_tz_source: Option, } #[derive(Debug, Serialize)] @@ -397,6 +405,9 @@ impl From for ExifMetadata { date_taken_tz_source: exif.date_taken_tz_source, original_date_taken: exif.original_date_taken, original_date_taken_source: exif.original_date_taken_source, + original_date_utc: exif.original_date_utc, + original_camera_offset_minutes: exif.original_camera_offset_minutes, + original_date_taken_tz_source: exif.original_date_taken_tz_source, } } } @@ -624,6 +635,9 @@ mod date_contract_tests { date_utc: Some(1_720_108_800), camera_offset_minutes: Some(-240), date_taken_tz_source: Some("offset_tag".to_string()), + original_date_utc: None, + original_camera_offset_minutes: None, + original_date_taken_tz_source: None, } } diff --git a/src/database/mod.rs b/src/database/mod.rs index 536c641..04be69a 100644 --- a/src/database/mod.rs +++ b/src/database/mod.rs @@ -568,25 +568,32 @@ pub trait ExifDao: Sync + Send { model_version: Option<&str>, ) -> Result)>, DbError>; - /// Operator-driven date_taken override (POST /image/exif/date). Snapshots - /// the prior `(date_taken, date_taken_source)` into the `original_*` - /// pair on first override, then writes the new value with - /// `date_taken_source = 'manual'`. Subsequent overrides keep the - /// original snapshot intact so a single revert restores the resolver - /// result, not whatever override was set just before. Returns the - /// post-update row. + /// Operator-driven date_taken override (POST /image/exif/date). + /// `date_taken` is the naive-as-UTC epoch the UI sent; the legacy + /// column stores it verbatim (byte-identical to the pre-rework + /// behavior). The new-contract triple (ADR-0001, ticket #115) is + /// computed from the same value as a wall clock in `display_tz` — + /// the user's display zone — with `date_taken_source = 'manual'`. + /// Snapshots the prior legacy pair AND prior new-contract triple into + /// the `original_*` columns on first override; subsequent overrides + /// keep the original snapshot intact so a single revert restores the + /// resolver result, not whatever override was set just before. + /// Returns the post-update row. fn set_manual_date_taken( &mut self, context: &opentelemetry::Context, library_id: i32, rel_path: &str, date_taken: i64, + display_tz: &str, ) -> Result; /// Revert a manual override (POST /image/exif/date/clear): restore - /// `date_taken` + `date_taken_source` from the `original_*` snapshot, - /// then null both originals. No-op (returns current row unchanged) when - /// no override is active. + /// `date_taken` + `date_taken_source` AND the new-contract triple + /// (`date_utc` / `camera_offset_minutes` / `date_taken_tz_source`, + /// ADR-0001 ticket #115) from the `original_*` snapshots, then null + /// all five. No-op (returns current row unchanged) when no override is + /// active. fn clear_manual_date_taken( &mut self, context: &opentelemetry::Context, @@ -1640,6 +1647,7 @@ impl ExifDao for SqliteExifDao { library_id_val: i32, rel_path_val: &str, date_taken_val: i64, + display_tz_val: &str, ) -> Result { trace_db_call(context, "update", "set_manual_date_taken", |_span| { use schema::image_exif::dsl::*; @@ -1652,16 +1660,49 @@ impl ExifDao for SqliteExifDao { let current = locate_image_exif_row(connection.deref_mut(), library_id_val, rel_path_val)?; - // Snapshot only on first override. Subsequent overrides keep - // the original snapshot intact so a single revert restores - // the resolver-derived value, not the prior override. - let (orig_dt, orig_src) = if current.original_date_taken.is_none() { - (current.date_taken, current.date_taken_source.clone()) - } else { - ( - current.original_date_taken, - current.original_date_taken_source.clone(), - ) + // Snapshot both pairs only on first override. Subsequent + // overrides keep the original snapshot intact so a single + // revert restores the resolver-derived value, not the prior + // override. + let (orig_dt, orig_src, orig_dt_utc, orig_offset, orig_tz) = + if current.original_date_taken.is_none() { + ( + current.date_taken, + current.date_taken_source.clone(), + current.date_utc, + current.camera_offset_minutes, + current.date_taken_tz_source.clone(), + ) + } else { + ( + current.original_date_taken, + current.original_date_taken_source.clone(), + current.original_date_utc, + current.original_camera_offset_minutes, + current.original_date_taken_tz_source.clone(), + ) + }; + + // The user's wall clock reinterpreted in their display zone + // (ADR-0001, ticket #115). `manual_date_resolution` is the + // pure seam; `None` (unreachable in practice — + // `display_zone_for` always yields a parseable zone) keeps + // the resolver's existing triple while the legacy columns + // still update. + let (new_utc, new_offset, new_tz) = match crate::camera_tz::manual_date_resolution( + Some(display_tz_val), + date_taken_val, + ) { + Some((utc, offset)) => ( + Some(utc), + Some(offset), + Some(crate::camera_tz::TzSource::Manual.as_str().to_string()), + ), + None => ( + current.date_utc, + current.camera_offset_minutes, + current.date_taken_tz_source.clone(), + ), }; // Update by primary key so we hit the exact row we read, @@ -1673,6 +1714,12 @@ impl ExifDao for SqliteExifDao { date_taken_source.eq(Some("manual".to_string())), original_date_taken.eq(orig_dt), original_date_taken_source.eq(orig_src), + original_date_utc.eq(orig_dt_utc), + original_camera_offset_minutes.eq(orig_offset), + original_date_taken_tz_source.eq(orig_tz), + date_utc.eq(new_utc), + camera_offset_minutes.eq(new_offset), + date_taken_tz_source.eq(new_tz), )) .execute(connection.deref_mut()) .map_err(|e| anyhow::Error::from(e).context("update_failed"))?; @@ -1718,8 +1765,14 @@ impl ExifDao for SqliteExifDao { .set(( date_taken.eq(current.original_date_taken), date_taken_source.eq(current.original_date_taken_source.clone()), + date_utc.eq(current.original_date_utc), + camera_offset_minutes.eq(current.original_camera_offset_minutes), + date_taken_tz_source.eq(current.original_date_taken_tz_source.clone()), original_date_taken.eq::>(None), original_date_taken_source.eq::>(None), + original_date_utc.eq::>(None), + original_camera_offset_minutes.eq::>(None), + original_date_taken_tz_source.eq::>(None), )) .execute(connection.deref_mut()) .map_err(|e| anyhow::Error::from(e).context("update_failed"))?; diff --git a/src/database/models.rs b/src/database/models.rs index 455ec5a..9c5502e 100644 --- a/src/database/models.rs +++ b/src/database/models.rs @@ -208,6 +208,14 @@ pub struct ImageExif { pub camera_offset_minutes: Option, /// How the offset was determined. See `camera_tz::TzSource`. pub date_taken_tz_source: Option, + /// Snapshot of the prior `date_utc` taken on first manual override. + /// NULL when no override is active. POST /image/exif/date/clear + /// restores the new-contract triple from these columns. + pub original_date_utc: Option, + /// Snapshot of the prior `camera_offset_minutes` (first override). + pub original_camera_offset_minutes: Option, + /// Snapshot of the prior `date_taken_tz_source` (first override). + pub original_date_taken_tz_source: Option, } #[derive(Insertable, Clone)] diff --git a/src/database/schema.rs b/src/database/schema.rs index 5518eef..30336b8 100644 --- a/src/database/schema.rs +++ b/src/database/schema.rs @@ -143,6 +143,9 @@ diesel::table! { date_utc -> Nullable, camera_offset_minutes -> Nullable, date_taken_tz_source -> Nullable, + original_date_utc -> Nullable, + original_camera_offset_minutes -> Nullable, + original_date_taken_tz_source -> Nullable, } } diff --git a/src/date_resolver.rs b/src/date_resolver.rs index 69c957d..4db9d85 100644 --- a/src/date_resolver.rs +++ b/src/date_resolver.rs @@ -10,7 +10,12 @@ //! can't: QuickTime/MP4 (`MediaCreateDate`, `TrackCreateDate`, //! `CreateDate`), Apple's `ContentCreateDate`, MakerNote sub-IFDs. //! Required for videos to land a real date; degrades silently when -//! `exiftool` isn't on PATH. +//! `exiftool` isn't on PATH. Unlike the kamadak branch, this branch +//! reads raw date *strings* (no `-d %s`) and interprets them itself: +//! QuickTime/MP4 container dates are true instants (UTC-rendered +//! `creation_time` fields) and skip the cascade; photo-EXIF wall +//! clocks run the cascade over the file's own `OffsetTime*` / GPS +//! evidence (ADR-0001, ticket #115). //! 3. **filename regex** — `memories::extract_date_from_filename` covers //! common screenshot / chat-export / timestamp-named patterns. //! 4. **earliest filesystem time** — `utils::earliest_fs_time` picks the @@ -32,9 +37,8 @@ use std::path::{Path, PathBuf}; use std::process::{Command, Stdio}; use std::sync::OnceLock; -use chrono::{DateTime, NaiveDateTime, Utc}; +use chrono::{DateTime, Datelike, Local, LocalResult, NaiveDateTime, TimeZone, Utc}; use log::{debug, trace, warn}; -use serde::Deserialize; use crate::camera_tz; use crate::utils::earliest_fs_time; @@ -148,16 +152,8 @@ pub fn resolve_date_taken(path: &Path, prior_exif: Option<&ExifPrior>) -> Option if let Some(prior) = prior_exif { return Some(exif_prior_resolved(prior)); } - if let Some(ts) = exiftool_date_single(path) { - return Some(ResolvedDate { - timestamp: ts, - source: DateSource::Exiftool, - // exiftool's `-d %s` conversion interprets the container date - // in the server's local zone — already a true UTC instant. - date_utc: Some(ts), - camera_offset_minutes: None, - tz_source: Some(camera_tz::TzSource::UtcDirect), - }); + if let Some(resolved) = exiftool_date_single(path) { + return Some(resolved); } if let Some(dt) = path .file_name() @@ -214,20 +210,10 @@ pub fn resolve_dates_batch( } if !needs_exiftool.is_empty() { - let exiftool_results = exiftool_dates_batch(&needs_exiftool); - for path in &needs_exiftool { - if let Some(&ts) = exiftool_results.get(*path) { - out.insert( - path.to_path_buf(), - ResolvedDate { - timestamp: ts, - source: DateSource::Exiftool, - date_utc: Some(ts), - camera_offset_minutes: None, - tz_source: Some(camera_tz::TzSource::UtcDirect), - }, - ); - } + // One subprocess for the whole batch; each entry is already fully + // resolved (legacy + new-contract fields). + for (path, resolved) in exiftool_dates_batch(&needs_exiftool) { + out.insert(path, resolved); } } @@ -272,7 +258,7 @@ pub fn resolve_dates_batch( out } -/// Tag priority for exiftool extraction. First non-zero value wins. +/// Tag priority for exiftool extraction. First parseable value wins. /// /// Photos: `DateTimeOriginal` (original capture) and `SubSecDateTimeOriginal` /// are most authoritative. `CreateDate` is a common alias and a sane fallback. @@ -294,6 +280,19 @@ const EXIFTOOL_DATE_TAGS: &[&str] = &[ "ContentCreateDate", ]; +/// Offset + GPS evidence tags requested in the same exiftool pass. They +/// feed the camera-offset cascade for naive wall-clock readings that come +/// from photo-EXIF groups (ADR-0001). +const EXIFTOOL_EVIDENCE_TAGS: &[&str] = &[ + "OffsetTime", + "OffsetTimeOriginal", + "OffsetTimeDigitized", + "GPS:GPSLatitude", + "GPS:GPSLatitudeRef", + "GPS:GPSLongitude", + "GPS:GPSLongitudeRef", +]; + /// Cache the "exiftool exists on PATH" check across the process lifetime so /// the per-tick backfill doesn't fork a doomed subprocess every iteration on /// deploys without exiftool installed. @@ -314,6 +313,20 @@ fn exiftool_available() -> bool { }) } +/// Append the exiftool arguments shared by the single-file and batch +/// invocations: raw date strings (no `-d %s` — the resolver interprets +/// them itself) with group prefixes for classification, plus the offset +/// / GPS evidence tags for the cascade. +fn exiftool_common_args(cmd: &mut Command) { + cmd.arg("-j").arg("-q").arg("-G1"); + for tag in EXIFTOOL_DATE_TAGS { + cmd.arg(format!("-{}", tag)); + } + for tag in EXIFTOOL_EVIDENCE_TAGS { + cmd.arg(format!("-{}", tag)); + } +} + /// One-file exiftool invocation. Used by the upload + GPS-write paths, /// which deal with one file at a time. The batch path uses /// `exiftool_dates_batch` so we don't pay subprocess startup per row. @@ -326,31 +339,30 @@ fn exiftool_available() -> bool { /// that actually have a real capture date. We pre-filter to files that /// kamadak-exif couldn't read, so the JPEG fast-path is already covered /// — paying full-scan cost on the residual is the right trade. -fn exiftool_date_single(path: &Path) -> Option { +fn exiftool_date_single(path: &Path) -> Option { if !exiftool_available() { return None; } let mut cmd = Command::new("exiftool"); - cmd.arg("-j").arg("-q").arg("-d").arg("%s"); - for tag in EXIFTOOL_DATE_TAGS { - cmd.arg(format!("-{}", tag)); - } + exiftool_common_args(&mut cmd); cmd.arg(path); let output = cmd.output().ok()?; if !output.status.success() { trace!("exiftool exited non-zero for {:?}", path); return None; } - parse_exiftool_json(&output.stdout) + parse_exiftool_payload(&output.stdout) .into_iter() .next() - .map(|(_, ts)| ts) + .and_then(|f| resolve_exiftool_payload(&f.date_tags, &f.evidence)) } /// Drain a batch via a single exiftool subprocess. Paths are fed on stdin /// via `-@ -`, so the argv stays short regardless of batch size — safe for -/// libraries with very long path components. -fn exiftool_dates_batch(paths: &[&Path]) -> HashMap { +/// libraries with very long path components. Resolution is the same pure +/// function as the single-file path, so the drain and ingest produce +/// identical results for the same file. +fn exiftool_dates_batch(paths: &[&Path]) -> HashMap { let mut out = HashMap::new(); if paths.is_empty() || !exiftool_available() { return out; @@ -360,10 +372,7 @@ fn exiftool_dates_batch(paths: &[&Path]) -> HashMap { // No `-fast2` — see exiftool_date_single for the rationale (QuickTime // moov-at-end files miss CreateDate / MediaCreateDate when the trailer // is skipped). - cmd.arg("-j").arg("-q").arg("-d").arg("%s"); - for tag in EXIFTOOL_DATE_TAGS { - cmd.arg(format!("-{}", tag)); - } + exiftool_common_args(&mut cmd); cmd.arg("-@").arg("-"); cmd.stdin(Stdio::piped()) .stdout(Stdio::piped()) @@ -403,36 +412,328 @@ fn exiftool_dates_batch(paths: &[&Path]) -> HashMap { output.status.code() ); } - for (source, ts) in parse_exiftool_json(&output.stdout) { - out.insert(PathBuf::from(source), ts); + for file in parse_exiftool_payload(&output.stdout) { + if let Some(resolved) = resolve_exiftool_payload(&file.date_tags, &file.evidence) { + out.insert(PathBuf::from(file.source_file), resolved); + } } out } -/// One row per input file. exiftool emits any tag we asked for that was -/// present, plus the `SourceFile` it was reading. Tags are JSON values -/// because `-d %s` returns the timestamp as a *string* of digits, not a -/// number, when the date parses; absent tags are simply missing keys. -#[derive(Debug, Deserialize)] -struct ExiftoolEntry { - #[serde(rename = "SourceFile")] - source_file: String, - #[serde(rename = "DateTimeOriginal")] - date_time_original: Option, - #[serde(rename = "SubSecDateTimeOriginal")] - sub_sec_date_time_original: Option, - #[serde(rename = "CreateDate")] - create_date: Option, - #[serde(rename = "MediaCreateDate")] - media_create_date: Option, - #[serde(rename = "TrackCreateDate")] - track_create_date: Option, - #[serde(rename = "ContentCreateDate")] - content_create_date: Option, +/// One raw date string from the exiftool payload, with the exiftool group +/// it came from (`-G1` prefixes every tag key as `"Group:Tag"`). +#[derive(Debug, Clone)] +struct RawDateTag { + group: String, + tag: String, + value: String, } -fn parse_exiftool_json(stdout: &[u8]) -> Vec<(String, i64)> { - let entries: Vec = match serde_json::from_slice(stdout) { +/// Per-file offset/GPS evidence from the same exiftool payload. +#[derive(Copy, Clone, Debug, Default)] +struct ExiftoolEvidence { + /// `OffsetTime` / `OffsetTimeDigitized`, whichever was present first. + offset_time: Option, + /// `OffsetTimeOriginal`. + offset_time_original: Option, + /// Unsigned decimal degrees; the Ref fields carry the sign. + gps_lat: Option, + gps_lon: Option, + gps_lat_south: bool, + gps_lon_west: bool, +} + +impl ExiftoolEvidence { + /// Signed `(latitude, longitude)`, when both coordinates are present. + fn gps(self) -> Option<(f64, f64)> { + let lat = self.gps_lat?; + let lon = self.gps_lon?; + Some(( + if self.gps_lat_south { -lat } else { lat }, + if self.gps_lon_west { -lon } else { lon }, + )) + } +} + +/// How one raw date string should be interpreted. +#[derive(Debug, Clone, Copy, PartialEq)] +enum DateReading { + /// The string carried its own UTC offset (zoned) — already a true + /// instant. + Zoned(i64), + /// Naive wall-clock reading; needs group classification and (for + /// photo-EXIF groups) the offset cascade. + Naive(NaiveDateTime), +} + +/// Parse one exiftool date string. `None` for empty values, the +/// `"0000:00:00 00:00:00"` zero placeholder exiftool emits for +/// missing-but-allocated date slots, and anything before 1900. +/// +/// Zoned strings (a trailing `Z`/`z` or a `±HH:MM[:SS]` offset) are true +/// instants — XMP dates, some MakerNotes, and exiftool's own +/// `Composite:SubSec*` rendering when an `OffsetTime*` tag is present — +/// and are converted to unix seconds directly. Naive strings are returned +/// as-is for group classification + the offset cascade. +fn parse_exiftool_date_string(value: &str) -> Option { + let v = value.trim(); + if v.is_empty() { + return None; + } + // Zoned form: split off the trailing offset, parse the naive part, and + // fold the offset into a true instant. + if let Some((naive_part, offset_minutes)) = split_zoned_suffix(v) { + let naive = parse_naive_exiftool(naive_part)?; + if naive.year() < 1900 { + return None; + } + let instant = naive.and_utc().timestamp() - (offset_minutes as i64) * 60; + return Some(DateReading::Zoned(instant)); + } + let naive = parse_naive_exiftool(v)?; + if naive.year() < 1900 { + return None; + } + Some(DateReading::Naive(naive)) +} + +/// The naive date-time formats exiftool renders, in preference order. EXIF +/// uses `:` between date fields; ISO uses `-`. The time separator is ` ` or +/// `T`; fractional seconds are optional. +const NAIVE_EXIFTOOL_FORMATS: &[&str] = &[ + "%Y:%m:%d %H:%M:%S%.f", + "%Y-%m-%d %H:%M:%S%.f", + "%Y-%m-%dT%H:%M:%S%.f", + "%Y:%m:%d %H:%M:%S", + "%Y-%m-%d %H:%M:%S", + "%Y-%m-%dT%H:%M:%S", +]; + +/// Parse a naive (offset-less) exiftool date string. +fn parse_naive_exiftool(v: &str) -> Option { + for fmt in NAIVE_EXIFTOOL_FORMATS { + if let Ok(naive) = NaiveDateTime::parse_from_str(v, fmt) { + return Some(naive); + } + } + None +} + +/// Split a zoned date string into its naive part and its UTC offset in +/// signed minutes (local − UTC). Recognises a trailing `Z`/`z` (UTC) or a +/// `±HH:MM[:SS]` offset. The date's own `-` separators are skipped because +/// only a `±` followed by a well-formed offset (digits and an optional +/// `:SS` tail) is accepted. +fn split_zoned_suffix(v: &str) -> Option<(&str, i32)> { + let b = v.as_bytes(); + // Trailing Z / z → UTC. + if matches!(b.last(), Some(b'Z') | Some(b'z')) { + return Some((&v[..v.len() - 1], 0)); + } + for (i, &c) in b.iter().enumerate() { + if (c == b'+' || c == b'-') + && i + 3 <= v.len() + && b[i + 1].is_ascii_digit() + && b[i + 2].is_ascii_digit() + { + let rest = v[i + 3..].strip_prefix(':').unwrap_or(&v[i + 3..]); + if rest.is_empty() || (rest.len() <= 2 && rest.bytes().all(|x| x.is_ascii_digit())) { + let sign = if c == b'+' { 1i32 } else { -1 }; + let hh: i32 = v[i + 1..i + 3].parse().ok()?; + let mm: i32 = if rest.is_empty() { + 0 + } else { + rest.parse().ok()? + }; + return Some((&v[..i], sign * (hh * 60 + mm))); + } + } + } + None +} + +/// Parse an exiftool `OffsetTime*` rendering (`"+05:30"`, `"-06:00"`, +/// `"Z"`) into signed minutes (local − UTC). +fn parse_offset_minutes(value: &str) -> Option { + let v = value.trim(); + if v == "Z" { + return Some(0); + } + let (sign, rest) = if let Some(r) = v.strip_prefix('-') { + (-1i32, r) + } else if let Some(r) = v.strip_prefix('+') { + (1, r) + } else { + return None; + }; + let (h, m) = rest.split_once(':')?; + let h: i32 = h.trim().parse().ok()?; + let m: i32 = m.trim().parse().ok()?; + Some(sign * (h * 60 + m)) +} + +/// Parse exiftool's DMS GPS rendering, e.g. `"40 deg 42' 36.00\""`, into +/// unsigned decimal degrees. The Ref field carries the sign. +fn parse_gps_dms(value: &str) -> Option { + let v = value.trim(); + let deg_end = v.find(" deg")?; + let deg: f64 = v[..deg_end].trim().parse().ok()?; + let rest = &v[deg_end + " deg".len()..]; + let min_end = rest.find('\'')?; + let min: f64 = rest[..min_end].trim().parse().ok()?; + let rest = &rest[min_end + 1..]; + let sec_end = rest.find('"')?; + let sec: f64 = rest[..sec_end].trim().parse().ok()?; + Some(deg + min / 60.0 + sec / 3600.0) +} + +/// QuickTime/MP4/MOV container groups. The `creation_time` field in those +/// containers is seconds-since-1904 in **UTC** — a true instant — and +/// exiftool renders it as a UTC-naive string. Classifying these as camera +/// wall clocks (running the cascade) reinterprets a UTC instant in the +/// server zone and shifts every video by the server's UTC offset — the +/// "videos off by 4-5 hours" bug, ticket #115. +fn is_container_group(group: &str) -> bool { + group == "QuickTime" || group.starts_with("Track") || group == "Apple" +} + +/// When the same tag name is present in multiple groups (hybrid files), +/// prefer the photo-EXIF groups over container / XMP renderings. +fn group_rank(group: &str) -> u8 { + if group.starts_with("ExifIFD") + || group == "IFD0" + || group == "IFD1" + || group.starts_with("MakerNote") + || group.starts_with("IFD") + { + 0 + } else if group == "Composite" { + 1 + } else if is_container_group(group) { + 2 + } else if group.starts_with("XMP") { + 3 + } else { + 4 + } +} + +/// The legacy column's convention for a naive wall reading: interpret it +/// in the *server's* local zone — exactly what exiftool `-d %s` did before +/// this rework (verified: it ignores the file's own `OffsetTime*` tags). +/// DST-aware: ambiguous fall-back walls take the earlier reading; +/// nonexistent spring-forward walls take the earliest local interpretation +/// — within an hour of what the old host produced. +fn legacy_server_local(wall: NaiveDateTime) -> i64 { + match Local.from_local_datetime(&wall) { + LocalResult::Single(dt) => dt.timestamp(), + LocalResult::Ambiguous(earlier, _) => earlier.timestamp(), + // Nonexistent spring-forward wall: interpret the reading as UTC + // and take the zone's offset at that instant — within an hour of + // what the old exiftool/glibc path produced. + LocalResult::None => Local.from_utc_datetime(&wall).timestamp(), + } +} + +/// Interpret one file's raw exiftool payload into a [`ResolvedDate`] +/// under the new contract (ADR-0001): +/// +/// - **zoned** strings carry their own UTC offset → true instant, +/// `utc_direct`. +/// - **container-group** naive strings are UTC-rendered `creation_time` +/// fields → true instant, `utc_direct` (no cascade — see +/// [`is_container_group`]). +/// - **photo-EXIF / other** naive strings are camera wall clocks → offset +/// cascade over the file's own `OffsetTime*` + GPS evidence. +/// +/// Tag priority is [`EXIFTOOL_DATE_TAGS`]; within a tag name the photo +/// group wins ([`group_rank`]). Unparseable candidates (zero dates, +/// junk) fall through to the next candidate. +fn resolve_exiftool_payload( + date_tags: &[RawDateTag], + evidence: &ExiftoolEvidence, +) -> Option { + let mut candidates: Vec<&RawDateTag> = date_tags.iter().collect(); + candidates.sort_by(|a, b| { + let pa = EXIFTOOL_DATE_TAGS + .iter() + .position(|t| *t == a.tag) + .unwrap_or(usize::MAX); + let pb = EXIFTOOL_DATE_TAGS + .iter() + .position(|t| *t == b.tag) + .unwrap_or(usize::MAX); + pa.cmp(&pb) + .then_with(|| group_rank(&a.group).cmp(&group_rank(&b.group))) + }); + + for cand in candidates { + let Some(reading) = parse_exiftool_date_string(&cand.value) else { + continue; + }; + match reading { + DateReading::Zoned(ts) => { + return Some(ResolvedDate { + timestamp: ts, + source: DateSource::Exiftool, + date_utc: Some(ts), + camera_offset_minutes: None, + tz_source: Some(camera_tz::TzSource::UtcDirect), + }); + } + DateReading::Naive(wall) => { + if is_container_group(&cand.group) { + return Some(ResolvedDate { + timestamp: legacy_server_local(wall), + source: DateSource::Exiftool, + date_utc: Some(wall.and_utc().timestamp()), + camera_offset_minutes: None, + tz_source: Some(camera_tz::TzSource::UtcDirect), + }); + } + // Camera wall clock: offset-tag pairing mirrors the + // kamadak branch (exif.rs) — DateTimeOriginal / SubSec* + // pair with OffsetTimeOriginal first, everything else with + // OffsetTime first. + let prefer_original = + cand.tag == "DateTimeOriginal" || cand.tag == "SubSecDateTimeOriginal"; + let offset_tag = if prefer_original { + evidence.offset_time_original.or(evidence.offset_time) + } else { + evidence.offset_time.or(evidence.offset_time_original) + }; + let ev = camera_tz::OffsetEvidence { + offset_tag_minutes: offset_tag, + gps: evidence.gps(), + }; + let resolution = + camera_tz::resolve_camera_offset(wall, &ev, server_zone_cached(), gps_lookup()); + return Some(ResolvedDate { + timestamp: legacy_server_local(wall), + source: DateSource::Exiftool, + date_utc: Some(camera_tz::wall_to_instant(wall, resolution.minutes)), + camera_offset_minutes: Some(resolution.minutes), + tz_source: Some(resolution.source), + }); + } + } + } + None +} + +/// One file's share of the exiftool JSON payload. +#[derive(Debug)] +struct ExiftoolFileData { + source_file: String, + date_tags: Vec, + evidence: ExiftoolEvidence, +} + +/// Split the exiftool `-j -G1` output into per-file date tags + evidence. +/// With `-G1` every tag key is `"Group:Tag"` (only `SourceFile` is +/// unprefixed); absent tags are simply missing keys. +fn parse_exiftool_payload(stdout: &[u8]) -> Vec { + let entries: Vec = match serde_json::from_slice(stdout) { Ok(v) => v, Err(e) => { // Empty stdout on total failure isn't a parse error worth @@ -447,103 +748,264 @@ fn parse_exiftool_json(stdout: &[u8]) -> Vec<(String, i64)> { let mut out = Vec::with_capacity(entries.len()); for entry in entries { - // Walk the priority list. exiftool sometimes returns the literal - // string "0000:00:00 00:00:00" for missing-but-allocated date - // slots; with `-d %s` that becomes the unix epoch (0). Reject - // anything <= 0 so we fall through to the next tag. - let tags = [ - entry.date_time_original.as_ref(), - entry.sub_sec_date_time_original.as_ref(), - entry.create_date.as_ref(), - entry.media_create_date.as_ref(), - entry.track_create_date.as_ref(), - entry.content_create_date.as_ref(), - ]; - let mut chosen: Option = None; - for tag in tags.iter().flatten() { - if let Some(ts) = coerce_to_unix_seconds(tag) - && ts > 0 - { - chosen = Some(ts); - break; + let Some(obj) = entry.as_object() else { + continue; + }; + let Some(source_file) = obj.get("SourceFile").and_then(|v| v.as_str()) else { + continue; + }; + let mut data = ExiftoolFileData { + source_file: source_file.to_string(), + date_tags: Vec::new(), + evidence: ExiftoolEvidence::default(), + }; + for (key, value) in obj { + let Some((group, tag)) = key.split_once(':') else { + continue; + }; + let Some(v) = value.as_str() else { + continue; + }; + if EXIFTOOL_DATE_TAGS.contains(&tag) { + data.date_tags.push(RawDateTag { + group: group.to_string(), + tag: tag.to_string(), + value: v.to_string(), + }); + } else { + match tag { + "OffsetTime" | "OffsetTimeDigitized" => { + data.evidence.offset_time = data + .evidence + .offset_time + .or_else(|| parse_offset_minutes(v)); + } + "OffsetTimeOriginal" => { + data.evidence.offset_time_original = parse_offset_minutes(v); + } + "GPSLatitude" => data.evidence.gps_lat = parse_gps_dms(v), + "GPSLatitudeRef" => { + data.evidence.gps_lat_south = v.trim().eq_ignore_ascii_case("S") + } + "GPSLongitude" => data.evidence.gps_lon = parse_gps_dms(v), + "GPSLongitudeRef" => { + data.evidence.gps_lon_west = v.trim().eq_ignore_ascii_case("W") + } + _ => {} + } } } - if let Some(ts) = chosen { - out.push((entry.source_file, ts)); - } + out.push(data); } out } -/// `-d %s` should hand us a numeric string, but exiftool's JSON encoder -/// will emit a number when the tag was defined as numeric in its lib — -/// accept both shapes. -fn coerce_to_unix_seconds(v: &serde_json::Value) -> Option { - match v { - serde_json::Value::String(s) => s.trim().parse::().ok(), - serde_json::Value::Number(n) => n.as_i64(), - _ => None, - } -} - #[cfg(test)] mod tests { use super::*; - #[test] - fn parse_exiftool_json_picks_first_priority_tag() { - let json = br#"[{ - "SourceFile": "/lib/IMG.jpg", - "DateTimeOriginal": "1500000000", - "CreateDate": "1400000000" - }]"#; - let parsed = parse_exiftool_json(json); - assert_eq!(parsed, vec![("/lib/IMG.jpg".to_string(), 1500000000)]); + fn tags(pairs: &[(&str, &str, &str)]) -> Vec { + pairs + .iter() + .map(|(g, t, v)| RawDateTag { + group: g.to_string(), + tag: t.to_string(), + value: v.to_string(), + }) + .collect() } #[test] - fn parse_exiftool_json_falls_through_zeros() { - // exiftool emits "0000:00:00 00:00:00" → unix epoch 0 with -d %s. - // The resolver should skip those and pick the next tag. - let json = br#"[{ - "SourceFile": "/lib/clip.mov", - "DateTimeOriginal": "0", - "MediaCreateDate": "1500000000" - }]"#; - let parsed = parse_exiftool_json(json); - assert_eq!(parsed, vec![("/lib/clip.mov".to_string(), 1500000000)]); + fn parse_date_string_zoned_forms_are_true_instants() { + // exiftool's Composite:SubSecDateTimeOriginal rendering when an + // OffsetTime tag is present: EXIF date separator, zoned time. + assert_eq!( + parse_exiftool_date_string("2024:09:17 07:33:59.414-06:00"), + Some(DateReading::Zoned(1_726_580_039)), // 13:33:59Z + ); + assert_eq!( + parse_exiftool_date_string("2024-09-17T07:33:59Z"), + Some(DateReading::Zoned(1_726_558_439)), // 07:33:59Z + ); } #[test] - fn parse_exiftool_json_accepts_numeric_values() { + fn parse_date_string_naive_and_rejects() { + assert_eq!( + parse_exiftool_date_string("2024:09:17 07:33:59"), + Some(DateReading::Naive( + NaiveDateTime::parse_from_str("2024-09-17 07:33:59", "%Y-%m-%d %H:%M:%S").unwrap() + )), + ); + // Zero placeholder, empty, junk, pre-1900. + assert_eq!(parse_exiftool_date_string("0000:00:00 00:00:00"), None); + assert_eq!(parse_exiftool_date_string(""), None); + assert_eq!(parse_exiftool_date_string("nonsense"), None); + assert_eq!(parse_exiftool_date_string("1899:12:31 23:59:59"), None); + } + + #[test] + fn parse_offset_minutes_forms() { + assert_eq!(parse_offset_minutes("+05:30"), Some(330)); + assert_eq!(parse_offset_minutes("-06:00"), Some(-360)); + assert_eq!(parse_offset_minutes("Z"), Some(0)); + assert_eq!(parse_offset_minutes("junk"), None); + } + + #[test] + fn parse_gps_dms_decimal() { + // 40 deg 42' 45.60" = 40.712666...; the Ref fields carry the sign. + let v = parse_gps_dms("40 deg 42' 45.60\"").unwrap(); + assert!((v - 40.7126666667).abs() < 1e-9); + assert_eq!(parse_gps_dms("nope"), None); + } + + #[test] + fn parse_payload_groups_dates_and_evidence() { let json = br#"[{ "SourceFile": "/lib/a.jpg", - "CreateDate": 1234567890 + "ExifIFD:DateTimeOriginal": "2024:07:04 12:00:00", + "ExifIFD:OffsetTimeOriginal": "-04:00", + "GPS:GPSLatitude": "40 deg 42' 45.60\"", + "GPS:GPSLatitudeRef": "N", + "GPS:GPSLongitude": "74 deg 0' 21.60\"", + "GPS:GPSLongitudeRef": "W" }]"#; - let parsed = parse_exiftool_json(json); - assert_eq!(parsed, vec![("/lib/a.jpg".to_string(), 1234567890)]); + let files = parse_exiftool_payload(json); + assert_eq!(files.len(), 1); + assert_eq!(files[0].source_file, "/lib/a.jpg"); + assert_eq!(files[0].date_tags.len(), 1); + assert_eq!(files[0].date_tags[0].group, "ExifIFD"); + assert_eq!(files[0].date_tags[0].tag, "DateTimeOriginal"); + assert_eq!(files[0].evidence.offset_time_original, Some(-240)); + let (lat, lon) = files[0].evidence.gps().unwrap(); + assert!((lat - 40.7126666667).abs() < 1e-9); + assert!((lon - (-74.006)).abs() < 1e-9); } #[test] - fn parse_exiftool_json_emits_nothing_when_no_tag_present() { - let json = br#"[{"SourceFile": "/lib/no_dates.bin"}]"#; - let parsed = parse_exiftool_json(json); - assert!(parsed.is_empty()); + fn container_date_is_true_instant_not_wall_clock() { + // QuickTime creation_time renders as a UTC-naive string: the + // 15:58:23 reading IS the instant. The pre-rework `-d %s` path + // reinterpreted it in the server zone — the "videos off by 4-5 + // hours" bug — and the legacy column keeps exactly that (frozen + // for rollback). + let wall = + NaiveDateTime::parse_from_str("2017-06-11 15:58:23", "%Y-%m-%d %H:%M:%S").unwrap(); + let resolved = resolve_exiftool_payload( + &tags(&[("QuickTime", "CreateDate", "2017:06:11 15:58:23")]), + &ExiftoolEvidence::default(), + ) + .unwrap(); + assert_eq!(resolved.date_utc, Some(wall.and_utc().timestamp())); + assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::UtcDirect)); + assert_eq!(resolved.camera_offset_minutes, None); + assert_eq!(resolved.timestamp, legacy_server_local(wall)); + assert_eq!(resolved.source, DateSource::Exiftool); } #[test] - fn parse_exiftool_json_handles_multiple_entries() { - let json = br#"[ - {"SourceFile": "/lib/a.jpg", "DateTimeOriginal": "100"}, - {"SourceFile": "/lib/b.jpg", "CreateDate": "200"} - ]"#; - let parsed = parse_exiftool_json(json); - assert_eq!( - parsed, - vec![ - ("/lib/a.jpg".to_string(), 100), - ("/lib/b.jpg".to_string(), 200) - ] + fn photo_and_video_at_same_instant_agree() { + // Photo: EXIF wall 2024-07-04 12:00:00 with an OffsetTime tag of + // -04:00. Video: QuickTime container date 2024:07:04 16:00:00 — + // the same moment, UTC-rendered. Both must land on 16:00:00Z. + let photo = resolve_exiftool_payload( + &tags(&[("ExifIFD", "DateTimeOriginal", "2024:07:04 12:00:00")]), + &ExiftoolEvidence { + offset_time_original: Some(-240), + ..Default::default() + }, + ) + .unwrap(); + let video = resolve_exiftool_payload( + &tags(&[("QuickTime", "CreateDate", "2024:07:04 16:00:00")]), + &ExiftoolEvidence::default(), + ) + .unwrap(); + assert_eq!(photo.date_utc, video.date_utc); + assert_eq!(photo.date_utc, Some(1_720_108_800)); + assert_eq!(photo.tz_source, Some(camera_tz::TzSource::OffsetTag)); + assert_eq!(photo.camera_offset_minutes, Some(-240)); + assert_eq!(video.tz_source, Some(camera_tz::TzSource::UtcDirect)); + } + + #[test] + fn zoned_string_short_circuits_to_utc_direct() { + let resolved = resolve_exiftool_payload( + &tags(&[("XMP-xmp", "CreateDate", "2024-09-17T07:33:59-06:00")]), + &ExiftoolEvidence::default(), + ) + .unwrap(); + assert_eq!(resolved.date_utc, Some(1_726_580_039)); + assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::UtcDirect)); + assert_eq!(resolved.camera_offset_minutes, None); + } + + #[test] + fn exif_naive_with_gps_uses_capture_zone() { + // No offset tag; GPS at Manhattan → America/New_York, EDT in July. + let resolved = resolve_exiftool_payload( + &tags(&[("ExifIFD", "CreateDate", "2024:07:04 12:00:00")]), + &ExiftoolEvidence { + gps_lat: Some(40.7128), + gps_lon: Some(74.006), + gps_lon_west: true, + ..Default::default() + }, + ) + .unwrap(); + assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::Gps)); + assert_eq!(resolved.camera_offset_minutes, Some(-240)); + assert_eq!(resolved.date_utc, Some(1_720_108_800)); + } + + #[test] + fn zero_date_falls_through_to_next_tag() { + // exiftool emits "0000:00:00 00:00:00" for missing-but-allocated + // date slots (iPhone QuickTime files); the resolver skips it and + // picks the next candidate. + let wall = + NaiveDateTime::parse_from_str("2020-01-02 03:04:05", "%Y-%m-%d %H:%M:%S").unwrap(); + let resolved = resolve_exiftool_payload( + &tags(&[ + ("QuickTime", "CreateDate", "0000:00:00 00:00:00"), + ("QuickTime", "MediaCreateDate", "2020:01:02 03:04:05"), + ]), + &ExiftoolEvidence::default(), + ) + .unwrap(); + assert_eq!(resolved.date_utc, Some(wall.and_utc().timestamp())); + } + + #[test] + fn photo_group_wins_over_container_for_same_tag() { + // Hybrid file: ExifIFD and QuickTime both carry CreateDate. The + // photo-EXIF reading (a wall clock) is preferred and runs the + // cascade over the file's own offset tag. + let resolved = resolve_exiftool_payload( + &tags(&[ + ("QuickTime", "CreateDate", "2024:07:04 16:00:00"), + ("ExifIFD", "CreateDate", "2024:07:04 12:00:00"), + ]), + &ExiftoolEvidence { + offset_time: Some(-240), + ..Default::default() + }, + ) + .unwrap(); + assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::OffsetTag)); + assert_eq!(resolved.camera_offset_minutes, Some(-240)); + assert_eq!(resolved.date_utc, Some(1_720_108_800)); + } + + #[test] + fn no_usable_dates_returns_none() { + assert!( + resolve_exiftool_payload( + &tags(&[("QuickTime", "CreateDate", "0000:00:00 00:00:00")]), + &ExiftoolEvidence::default() + ) + .is_none() ); } diff --git a/src/files.rs b/src/files.rs index dd74a79..ef00173 100644 --- a/src/files.rs +++ b/src/files.rs @@ -1519,6 +1519,9 @@ mod tests { date_utc: None, camera_offset_minutes: None, date_taken_tz_source: None, + original_date_utc: None, + original_camera_offset_minutes: None, + original_date_taken_tz_source: None, } } @@ -1563,6 +1566,9 @@ mod tests { date_utc: data.date_utc, camera_offset_minutes: data.camera_offset_minutes, date_taken_tz_source: data.date_taken_tz_source.clone(), + original_date_utc: data.date_utc, + original_camera_offset_minutes: data.camera_offset_minutes, + original_date_taken_tz_source: data.date_taken_tz_source.clone(), }) } @@ -1614,6 +1620,9 @@ mod tests { date_utc: data.date_utc, camera_offset_minutes: data.camera_offset_minutes, date_taken_tz_source: data.date_taken_tz_source.clone(), + original_date_utc: data.date_utc, + original_camera_offset_minutes: data.camera_offset_minutes, + original_date_taken_tz_source: data.date_taken_tz_source.clone(), }) } @@ -1751,6 +1760,7 @@ mod tests { library_id: i32, rel_path: &str, date_taken: i64, + _display_tz: &str, ) -> Result { // Mock — files.rs tests don't exercise the date-override endpoints. // Returning a synthetic row keeps the trait satisfied without diff --git a/src/handlers/image.rs b/src/handlers/image.rs index b8801f3..35bbd29 100644 --- a/src/handlers/image.rs +++ b/src/handlers/image.rs @@ -826,10 +826,14 @@ pub async fn get_full_exif( } } -/// Body for `POST /image/exif/date` — operator-driven date_taken override. -/// `date_taken` is unix seconds (matches `image_exif.date_taken`'s convention -/// — naive local reinterpreted as UTC, not real UTC; the Apollo client passes -/// through the same value the photo carousel rendered before edit). +/// Body for `POST /image/exif/date` — operator-driven date override. +/// `date_taken` is unix seconds: the wall the user picked rendered in their +/// display zone, encoded naive-as-UTC (the Apollo client passes through the +/// same value the photo carousel rendered before edit). The legacy +/// `date_taken` column stores it verbatim; the new-contract triple +/// (`date_utc` / `camera_offset_minutes` / `date_taken_tz_source = +/// 'manual'`, ADR-0001 ticket #115) is derived from it as a wall clock in +/// the user's display zone. #[derive(serde::Deserialize)] struct SetDateRequest { path: String, @@ -910,9 +914,29 @@ pub async fn set_image_date( // doesn't end up with a backslash variant that misses the row. let normalized_path = body.path.replace('\\', "/"); + // The user's display zone — the wall the UI sent is rendered in this + // zone (ADR-0001, ticket #115). NULL / unparseable prefs fall back to + // the app default inside display_zone_for. + let display_tz = { + let mut prefs_dao = app_state + .user_ai_prefs_dao + .lock() + .expect("Unable to lock UserAiPrefsDao"); + match prefs_dao.get_prefs(&span_context) { + Ok(Some(prefs)) => crate::camera_tz::display_zone_for(prefs.display_tz.as_deref()), + _ => crate::camera_tz::display_zone_for(None), + } + }; + let updated = { let mut dao = exif_dao.lock().expect("Unable to lock ExifDao"); - dao.set_manual_date_taken(&span_context, library.id, &normalized_path, body.date_taken) + dao.set_manual_date_taken( + &span_context, + library.id, + &normalized_path, + body.date_taken, + &display_tz, + ) }; match updated {