From 6f73a31e08f34d16a3a84740013e2c10a5e71ab4 Mon Sep 17 00:00:00 2001 From: Cameron Cordes Date: Fri, 25 Sep 2026 16:13:28 -0400 Subject: [PATCH 1/2] docs(tracker): correct the Gitea native-dependency API call The wayfinding doc pointed at a .../dependencies/blocked_by subpath that 404s, and the working call needs the blocker's owner+repo in the JSON body (omitting them 404s with 'repository does not exist'). Verified against Gitea 1.27: POST/DELETE .../issues//dependencies with {"index", "owner", "repo"}. --- docs/agents/issue-tracker.md | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/docs/agents/issue-tracker.md b/docs/agents/issue-tracker.md index 6e9919f..6a63ea0 100644 --- a/docs/agents/issue-tracker.md +++ b/docs/agents/issue-tracker.md @@ -68,12 +68,19 @@ as tickets. dev. - **Blocking**: Gitea's **native issue dependencies** (enabled on this server), the canonical, UI-visible representation. Add an edge with - `tea api --method POST repos/{owner}/{repo}/issues//dependencies/blocked_by -F issue_id=` - (the Gitea dependencies API takes the issue index, i.e. the visible - `#number`, not an internal database id). Where dependencies aren't - available, fall back to a `Blocked by: #, #` line at the top of - the child description. A ticket is unblocked when every blocker is - closed. + `tea api --method POST repos/{owner}/{repo}/issues//dependencies -d '{"index": , "owner": "", "repo": ""}'` + (201 on success). Gotchas, verified against Gitea 1.27: + - The route is `.../dependencies` — there is **no** `blocked_by` + subpath (that 404s). + - The JSON body **must** name the blocker's repo (`owner` + `repo`); + omitting them yields a misleading `404 repository does not exist` + even though the child's path is correct. + - `index` is the visible `#number`, not an internal database id. + Verify with `tea api repos/{owner}/{repo}/issues//dependencies` + (lists blockers); remove with `--method DELETE` and the same body. + Where dependencies aren't available, fall back to a + `Blocked by: #, #` line at the top of the child description. + A ticket is unblocked when every blocker is closed. - **Frontier query**: list the map's open children (issues whose description carries `Part of #`), drop any with an open blocker (dependency or `Blocked by` line) or an assignee; first in map order wins. From a97bf53603c064c4c8f5e8d99edc95d0383ada30 Mon Sep 17 00:00:00 2001 From: Cameron Cordes Date: Fri, 25 Sep 2026 16:54:46 -0400 Subject: [PATCH 2/2] =?UTF-8?q?feat(dates):=20filename=20date=20class=20?= =?UTF-8?q?=E2=80=94=20per-pattern=20conventions=20+=20filename-over-EXIF?= =?UTF-8?q?=20priority=20(#116)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The filename date class is now first-class (ADR-0001): - memories::extract_date_from_filename returns FilenameDate: Wall(NaiveDateTime) for the screenshot/dash/compact patterns, Utc(DateTime) for the unix-timestamp patterns. No patterns added or removed; every existing pattern carries its convention. - Waterfall inversion in both resolve_date_taken and resolve_dates_batch: an in-class name is the original date of record and resolves BEFORE EXIF / container dates are consulted (shared filename_class_resolved step; in-class files skip exiftool entirely). Wall patterns run the offset cascade over the kamadak prior's evidence (empty evidence -> server-zone assumption, so a class file always resolves); timestamp patterns are true instants (tz_source utc_direct, no offset). - Legacy date_taken stays byte-identical: wall-as-UTC for wall patterns, epoch for timestamp patterns, via FilenameDate::legacy_datetime / legacy_timestamp at the seven consumer call sites. - Out-of-class files keep the global EXIF-wins waterfall unchanged. - CLAUDE.md waterfall section rewritten (step 0 = filename class; the EXIF-beats-filename note becomes out-of-class-only). Tests: five new waterfall-seam tests (wall no-evidence -> assumed_server; wall outranks EXIF with offset-tag evidence; wall GPS evidence; timestamp utc_direct; in-class video name resolves without exiftool) plus enum-shape assertions in every extractor test. Code-reviewed on both axes (standards: no violations, 3 of 4 P2 notes applied; spec: all acceptance criteria verified, no scope creep). 716 integration + 639 lib tests pass; fmt + clippy clean. --- CLAUDE.md | 30 +++-- src/ai/insight_chat.rs | 2 +- src/ai/insight_generator.rs | 6 +- src/date_resolver.rs | 257 +++++++++++++++++++++++++++--------- src/files.rs | 6 +- src/handlers/image.rs | 12 +- src/memories.rs | 188 +++++++++++++++++--------- 7 files changed, 358 insertions(+), 143 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 063e395..b10d25e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -374,6 +374,18 @@ per-tick drain can re-resolve weak entries when better tools become available, and so the UI/debug surface can answer "why did this photo land on this date?". Order: +0. **`filename` date class (ADR-0001, ticket #116)** — + `extract_date_from_filename` in `memories.rs` returns a + `FilenameDate`: wall-clock patterns (screenshot, dash, compact) or + unix-timestamp patterns. A file whose name matches **any** pattern is + *in the class*: the filename is the original date of record and wins + *before* EXIF / container dates are consulted, inverting the global + EXIF-wins rule for exactly those files (a renamed screenshot or a + chat-export timestamp is the only honest date left). Wall patterns + run the offset cascade over the file's own `OffsetTime*` / GPS + evidence (empty evidence → server-zone assumption, so a class file + always resolves); timestamp patterns are true instants outright + (`utc_direct`, no offset). In-class files never hit exiftool. 1. **`exif`** — kamadak-exif `DateTime` / `DateTimeOriginal`. Fast, in-process, image-only. 2. **`exiftool`** — shell-out fallback for tags kamadak can't reach: @@ -395,17 +407,15 @@ land on this date?". Order: old server-local `-d %s` interpretation; the new-contract `date_utc` / `camera_offset_minutes` / `date_taken_tz_source` columns carry the true instant. -3. **`filename`** — `extract_date_from_filename` in `memories.rs` - matches screenshot, chat-export, and timestamp-named patterns. -4. **`fs_time`** — `earliest_fs_time(metadata)` (earlier of created / - modified). Last resort. +3. **`fs_time`** — `earliest_fs_time(metadata)` (earlier of created / + modified). Last resort. (The filename step sits at step 0 for + in-class names; an out-of-class name matches nothing here.) -Notable behavior change vs. the pre-2026-05 request-time logic: -**EXIF beats filename when both are present.** A photo named -`Screenshot_2014-06-01.png` whose EXIF `DateTime` is 2021 now appears -under 2021, not 2014 — on the theory that EXIF is more reliable than -import-named filenames. The reverse case (no EXIF, filename has a -date) is unchanged. +**Out-of-class files keep EXIF-wins.** A photo named `IMG_1234.jpg` +whose EXIF `DateTime` is 2021 appears under 2021 — EXIF +`DateTimeOriginal` is more reliable than an import-time filename. The +filename class (step 0) is the documented exception where the name is +the original date itself. The `backfill_missing_date_taken` drain (`src/backfill.rs`) runs every watcher tick alongside `backfill_unhashed_backlog` (also `src/backfill.rs`). It loads up to diff --git a/src/ai/insight_chat.rs b/src/ai/insight_chat.rs index 7f454a1..73347bb 100644 --- a/src/ai/insight_chat.rs +++ b/src/ai/insight_chat.rs @@ -2037,7 +2037,7 @@ fn resolve_date_taken_for_context( return from_exif; } crate::memories::extract_date_from_filename(file_path) - .map(|dt| dt.format("%Y-%m-%d").to_string()) + .map(|fd| fd.legacy_datetime().format("%Y-%m-%d").to_string()) } /// Validate a stored→effective backend transition for a chat continuation. diff --git a/src/ai/insight_generator.rs b/src/ai/insight_generator.rs index be2c246..b84d3f6 100644 --- a/src/ai/insight_generator.rs +++ b/src/ai/insight_generator.rs @@ -23,7 +23,7 @@ use crate::database::{ LocationHistoryDao, SearchHistoryDao, }; use crate::libraries::Library; -use crate::memories::extract_date_from_filename; +use crate::memories::{FilenameDate, extract_date_from_filename}; use crate::otel::global_tracer; use crate::tags::TagDao; use crate::utils::{earliest_fs_time, normalize_path, retry_with_backoff, retry_with_backoff_if}; @@ -1062,7 +1062,7 @@ impl InsightGenerator { log::warn!("No date_taken in EXIF for {}, trying filename", file_path); extract_date_from_filename(&file_path) - .map(|dt| dt.timestamp()) + .map(FilenameDate::legacy_timestamp) .or_else(|| { let full_path = self.resolve_full_path(&file_path)?; File::open(&full_path) @@ -4153,7 +4153,7 @@ Return ONLY the summary, nothing else."#, } else { log::warn!("No date_taken in EXIF for {}, trying filename", file_path); extract_date_from_filename(&file_path) - .map(|dt| dt.timestamp()) + .map(FilenameDate::legacy_timestamp) .or_else(|| { let full_path = self.resolve_full_path(&file_path)?; File::open(&full_path) diff --git a/src/date_resolver.rs b/src/date_resolver.rs index 4db9d85..430ffba 100644 --- a/src/date_resolver.rs +++ b/src/date_resolver.rs @@ -16,20 +16,37 @@ //! `creation_time` fields) and skip the cascade; photo-EXIF wall //! clocks run the cascade over the file's own `OffsetTime*` / GPS //! evidence (ADR-0001, ticket #115). -//! 3. **filename regex** — `memories::extract_date_from_filename` covers -//! common screenshot / chat-export / timestamp-named patterns. -//! 4. **earliest filesystem time** — `utils::earliest_fs_time` picks the +//! 0. **filename date class** (ticket #116) — `memories::extract_date_from_filename` +//! covers common screenshot / dash / compact / unix-timestamp patterns. +//! A file whose name matches any pattern is **in the class**: the +//! filename is the original date of record and wins *before* EXIF / container +//! dates are even consulted (wall patterns run the offset cascade over the +//! file's own evidence; timestamp patterns are true instants). +//! 1. **kamadak-exif** — fast in-process EXIF read. Already done by +//! `exif::extract_exif_from_path` for image-bearing formats; callers +//! pass that result in via `prior_exif_date` so we don't re-parse. +//! 2. **exiftool** — shell-out fallback that reaches places kamadak-exif +//! can't: QuickTime/MP4 (`MediaCreateDate`, `TrackCreateDate`, +//! `CreateDate`), Apple's `ContentCreateDate`, MakerNote sub-IFDs. +//! Required for videos to land a real date; degrades silently when +//! `exiftool` isn't on PATH. Unlike the kamadak branch, this branch +//! reads raw date *strings* (no `-d %s`) and interprets them itself: +//! QuickTime/MP4 container dates are true instants (UTC-rendered +//! `creation_time` fields) and skip the cascade; photo-EXIF wall +//! clocks run the cascade over the file's own `OffsetTime*` / GPS +//! evidence (ADR-0001, ticket #115). +//! 3. **earliest filesystem time** — `utils::earliest_fs_time` picks the //! earlier of created / modified, which on copied-from-backup files is //! a better proxy for content age than either alone. //! //! `DateSource` records which step won so the per-tick drain can re-resolve //! weak sources (`fs_time`) once exiftool becomes available, and so the //! UI/debug surface can answer "why does this photo show up under this -//! date." Note that the previous `/memories` request-time logic preferred -//! filename even when EXIF was present; this resolver inverts that — EXIF -//! is authoritative when it exists, on the theory that an EXIF -//! `DateTimeOriginal` is more reliable than a filename pattern that may -//! reflect import time rather than capture time. +//! date." For out-of-class files EXIF is authoritative when it exists, +//! on the theory that an EXIF `DateTimeOriginal` is more reliable than a +//! filename pattern that may reflect import time rather than capture time. +//! In-class files invert that rule deliberately (ADR-0001): a renamed +//! screenshot or a chat-export timestamp is the only honest date left. use std::collections::HashMap; use std::io::Write; @@ -49,7 +66,8 @@ pub enum DateSource { Exif, /// exiftool fallback caught a video / MakerNote / QuickTime tag. Exiftool, - /// `extract_date_from_filename` matched a known pattern. + /// `extract_date_from_filename` matched a known pattern — the file is + /// in the filename date class and the name outranked EXIF (#116). Filename, /// Fell through to `earliest_fs_time(metadata)`. FsTime, @@ -84,14 +102,16 @@ pub struct ResolvedDate { /// behavior of each branch so rollback stays trivial (ADR-0001). pub timestamp: i64, pub source: DateSource, - /// True UTC instant under the new contract (ADR-0001). `None` for - /// filename-sourced rows: their convention (wall-clock vs UTC) is a - /// per-pattern decision made by the filename date-class work (#116). + /// True UTC instant under the new contract (ADR-0001). Every branch + /// classifies (wall clocks via the cascade, instants directly); + /// `Option` is kept for schema compatibility with rows written before + /// the date-class work landed (#116). pub date_utc: Option, - /// Camera offset (minutes) that produced `date_utc`; `None` when the - /// branch is a direct true instant or unclassified. + /// Camera offset (minutes) that produced `date_utc`; `None` for direct + /// true instants (no offset is involved). pub camera_offset_minutes: Option, - /// Why `date_utc` has the offset it does. `None` when unclassified. + /// Why `date_utc` has the offset it does; `None` on rows written + /// before every branch classified (#116). pub tz_source: Option, } @@ -132,6 +152,57 @@ fn exif_prior_resolved(prior: &ExifPrior) -> ResolvedDate { } } +/// The filename-class step (ticket #116), shared by both waterfalls: the +/// in-class name is the original date of record and outranks EXIF / +/// container dates. `evidence` is the kamadak prior's offset evidence when +/// the file has one; `None` means empty evidence — the cascade then assumes +/// the server zone, so a class file always resolves. +fn filename_class_resolved( + path: &Path, + evidence: Option<&camera_tz::OffsetEvidence>, +) -> Option { + let filename_date = path + .file_name() + .and_then(|f| f.to_str()) + .and_then(crate::memories::extract_date_from_filename)?; + let ev = evidence.copied().unwrap_or_default(); + Some(filename_resolved(&filename_date, &ev)) +} + +/// Classify one filename date (ticket #116). Wall patterns run the offset +/// cascade over whatever evidence the file carries; timestamp patterns are +/// true instants outright. The legacy value is the pattern's pre-migration +/// interpretation: wall-as-UTC / epoch. +fn filename_resolved( + filename_date: &crate::memories::FilenameDate, + evidence: &camera_tz::OffsetEvidence, +) -> ResolvedDate { + match filename_date { + crate::memories::FilenameDate::Utc(instant) => ResolvedDate { + timestamp: instant.timestamp(), + source: DateSource::Filename, + date_utc: Some(instant.timestamp()), + camera_offset_minutes: None, + tz_source: Some(camera_tz::TzSource::UtcDirect), + }, + crate::memories::FilenameDate::Wall(wall) => { + let resolution = camera_tz::resolve_camera_offset( + *wall, + evidence, + server_zone_cached(), + gps_lookup(), + ); + ResolvedDate { + timestamp: wall.and_utc().timestamp(), + source: DateSource::Filename, + date_utc: Some(camera_tz::wall_to_instant(*wall, resolution.minutes)), + camera_offset_minutes: Some(resolution.minutes), + tz_source: Some(resolution.source), + } + } + } +} + /// Host zone, resolved once per process. fn server_zone_cached() -> Option { static ZONE: OnceLock> = OnceLock::new(); @@ -149,28 +220,17 @@ fn gps_lookup() -> &'static camera_tz::TzfGpsZoneLookup { /// the waterfall fails — for files that exist on disk this should be /// vanishingly rare (the fs-time fallback alone almost always succeeds). pub fn resolve_date_taken(path: &Path, prior_exif: Option<&ExifPrior>) -> Option { + // Filename date class first (ticket #116): an in-class name is the + // original date of record and outranks EXIF / container dates. + if let Some(resolved) = filename_class_resolved(path, prior_exif.map(|p| &p.evidence)) { + return Some(resolved); + } if let Some(prior) = prior_exif { return Some(exif_prior_resolved(prior)); } if let Some(resolved) = exiftool_date_single(path) { return Some(resolved); } - if let Some(dt) = path - .file_name() - .and_then(|f| f.to_str()) - .and_then(crate::memories::extract_date_from_filename) - { - return Some(ResolvedDate { - timestamp: dt.timestamp(), - source: DateSource::Filename, - // Filename conventions (wall-clock vs UTC) are decided - // per-pattern by the filename date-class work (#116); until - // then these rows stay unclassified. - date_utc: None, - camera_offset_minutes: None, - tz_source: None, - }); - } if let Ok(meta) = std::fs::metadata(path) && let Some(t) = earliest_fs_time(&meta) { @@ -186,11 +246,12 @@ pub fn resolve_date_taken(path: &Path, prior_exif: Option<&ExifPrior>) -> Option None } -/// Batch waterfall. exiftool runs once over the whole batch (single -/// subprocess); everything else is per-file and runs only on misses. -/// `prior_exif_dates` lets the caller pass in already-known kamadak dates -/// keyed by path; entries without a prior date fall through to exiftool -/// and the rest of the waterfall. +/// Batch waterfall. In-class filenames resolve first (ticket #116) and +/// never hit exiftool; exiftool then runs once over the remaining +/// no-prior files (single subprocess); everything else is per-file and +/// runs only on misses. `prior_exif_priors` lets the caller pass in +/// already-known kamadak readings keyed by path; entries without a prior +/// date fall through to exiftool and the rest of the waterfall. /// /// The per-tick backfill drain is the primary caller — it loads ~500 rows /// at a time and uses one exiftool subprocess to drain the lot. @@ -202,6 +263,14 @@ pub fn resolve_dates_batch( let mut needs_exiftool: Vec<&Path> = Vec::with_capacity(paths.len()); for path in paths { + // Filename date class first (ticket #116) — same rule as the + // single-file waterfall; in-class files skip exiftool entirely. + if let Some(resolved) = + filename_class_resolved(path, prior_exif_priors.get(path).map(|p| &p.evidence)) + { + out.insert(path.clone(), resolved); + continue; + } if let Some(prior) = prior_exif_priors.get(path) { out.insert(path.clone(), exif_prior_resolved(prior)); } else { @@ -217,27 +286,13 @@ pub fn resolve_dates_batch( } } + // Remaining misses are out-of-class files with no EXIF / container + // date — the filename step can't match them (any match would have + // resolved in the first loop), so they fall straight to fs-time. for path in paths { if out.contains_key(path) { continue; } - if let Some(dt) = path - .file_name() - .and_then(|f| f.to_str()) - .and_then(crate::memories::extract_date_from_filename) - { - out.insert( - path.clone(), - ResolvedDate { - timestamp: dt.timestamp(), - source: DateSource::Filename, - date_utc: None, - camera_offset_minutes: None, - tz_source: None, - }, - ); - continue; - } if let Ok(meta) = std::fs::metadata(path) && let Some(t) = earliest_fs_time(&meta) { @@ -1089,19 +1144,103 @@ mod tests { } #[test] - fn resolve_filename_when_no_exif_and_file_missing() { - // No prior EXIF, no exiftool match (file missing), but the filename - // pattern still matches so the resolver lands on Filename. The - // filename branch stays unclassified (#116): date_utc is NULL. + fn filename_class_wall_without_evidence_assumes_server_zone() { + // In-class screenshot name, no prior EXIF, file missing. The wall + // clock runs the cascade with empty evidence and lands on + // assumed_server — a class file always resolves (#116). Legacy + // stays the pre-migration wall-as-UTC interpretation. let resolved = resolve_date_taken( Path::new("/nonexistent/Screenshot_2014-06-01-20-44-50.png"), None, ) .unwrap(); + let wall = + NaiveDateTime::parse_from_str("2014-06-01 20:44:50", "%Y-%m-%d %H:%M:%S").unwrap(); assert_eq!(resolved.source, DateSource::Filename); - assert_eq!(resolved.date_utc, None); + assert_eq!(resolved.timestamp, wall.and_utc().timestamp()); + assert!(resolved.date_utc.is_some()); + assert!(resolved.camera_offset_minutes.is_some()); + assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::AssumedServer)); + assert_eq!( + resolved.date_utc.unwrap(), + resolved.timestamp - (resolved.camera_offset_minutes.unwrap() as i64) * 60 + ); + } + + #[test] + fn filename_class_wall_outranks_exif_and_uses_its_evidence() { + // In-class filename beats the EXIF date entirely (the priority + // inversion), and the wall cascade uses the prior's offset-tag + // evidence. The EXIF wall (2020) never reaches the result. + let prior = ExifPrior { + wall: NaiveDateTime::parse_from_str("2020-05-05 05:05:05", "%Y-%m-%d %H:%M:%S") + .unwrap(), + evidence: camera_tz::OffsetEvidence { + offset_tag_minutes: Some(-240), + gps: None, + }, + }; + let resolved = resolve_date_taken( + Path::new("/nonexistent/2015-01-09_02-15-15.jpg"), + Some(&prior), + ) + .unwrap(); + assert_eq!(resolved.source, DateSource::Filename); + assert_eq!(resolved.timestamp, 1_420_769_715); // wall-as-UTC legacy + assert_eq!(resolved.camera_offset_minutes, Some(-240)); + assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::OffsetTag)); + // The offset tag short-circuits the cascade and is used as-stamped: + // wall − (−240) → 2015-01-09 06:15:15Z. + assert_eq!(resolved.date_utc, Some(1_420_784_115)); + } + + #[test] + fn filename_class_wall_gps_evidence_uses_capture_zone() { + // No offset tag; the prior carries Manhattan GPS. The bundled TZF + // geometry resolves America/New_York, whose offset for the + // July wall is EDT (-240). + let prior = ExifPrior { + wall: NaiveDateTime::parse_from_str("2024-07-04 12:00:00", "%Y-%m-%d %H:%M:%S") + .unwrap(), + evidence: camera_tz::OffsetEvidence { + offset_tag_minutes: None, + gps: Some((40.7128, -74.0060)), + }, + }; + let resolved = resolve_date_taken( + Path::new("/nonexistent/Screenshot_2024-07-04-12-00-00.png"), + Some(&prior), + ) + .unwrap(); + assert_eq!(resolved.source, DateSource::Filename); + assert_eq!(resolved.camera_offset_minutes, Some(-240)); + assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::Gps)); + assert_eq!(resolved.date_utc, Some(1_720_108_800)); // 16:00:00Z + } + + #[test] + fn filename_class_timestamp_is_utc_direct() { + // A unix-timestamp name is a true instant: no cascade, no offset, + // and legacy equals the epoch (the pre-migration interpretation). + let resolved = + resolve_date_taken(Path::new("/nonexistent/xyz_1401638400.jpeg"), None).unwrap(); + assert_eq!(resolved.source, DateSource::Filename); + assert_eq!(resolved.timestamp, 1_401_638_400); + assert_eq!(resolved.date_utc, Some(1_401_638_400)); assert_eq!(resolved.camera_offset_minutes, None); - assert_eq!(resolved.tz_source, None); + assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::UtcDirect)); + } + + #[test] + fn in_class_video_name_resolves_without_exiftool() { + // A chat-export-style name on a video path: the class check runs + // before exiftool is consulted — the file need not even exist for + // the result to be class-resolved. + let resolved = + resolve_date_taken(Path::new("/nonexistent/att_1422489664680106.mp4"), None).unwrap(); + assert_eq!(resolved.source, DateSource::Filename); + assert_eq!(resolved.date_utc, Some(1_422_489_664)); + assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::UtcDirect)); } #[test] diff --git a/src/files.rs b/src/files.rs index ef00173..ed1d72c 100644 --- a/src/files.rs +++ b/src/files.rs @@ -18,7 +18,7 @@ use crate::data::{ use crate::database::ExifDao; use crate::file_types; use crate::geo::{gps_bounding_box, haversine_distance}; -use crate::memories::extract_date_from_filename; +use crate::memories::{FilenameDate, extract_date_from_filename}; use crate::thumbnails::create_thumbnails; use crate::utils::earliest_fs_time; use actix_web::web::Data; @@ -145,7 +145,9 @@ fn in_memory_date_sort( let date_taken = exif_map .get(&(f.file_name.clone(), lib_id)) .copied() - .or_else(|| extract_date_from_filename(&f.file_name).map(|dt| dt.timestamp())) + .or_else(|| { + extract_date_from_filename(&f.file_name).map(FilenameDate::legacy_timestamp) + }) .or_else(|| { lib_roots.get(&lib_id).and_then(|root| { let full_path = Path::new(root).join(&f.file_name); diff --git a/src/handlers/image.rs b/src/handlers/image.rs index 35bbd29..b6dd82b 100644 --- a/src/handlers/image.rs +++ b/src/handlers/image.rs @@ -526,8 +526,8 @@ pub async fn get_file_metadata( response.library_name = Some(resolved_library.name.clone()); // Extract date from filename if possible - response.filename_date = - memories::extract_date_from_filename(&path.path).map(|dt| dt.timestamp()); + response.filename_date = memories::extract_date_from_filename(&path.path) + .map(memories::FilenameDate::legacy_timestamp); // Query EXIF data if available if let Ok(mut dao) = exif_dao.lock() @@ -721,8 +721,8 @@ pub async fn set_image_gps( response.exif = Some(row.into()); response.library_id = Some(resolved_library.id); response.library_name = Some(resolved_library.name.clone()); - response.filename_date = - memories::extract_date_from_filename(&body.path).map(|dt| dt.timestamp()); + response.filename_date = memories::extract_date_from_filename(&body.path) + .map(memories::FilenameDate::legacy_timestamp); span.set_status(Status::Ok); HttpResponse::Ok().json(response) } @@ -881,8 +881,8 @@ fn build_metadata_response_for_date_mutation( response.exif = Some(exif.into()); response.library_id = Some(library.id); response.library_name = Some(library.name.clone()); - response.filename_date = - memories::extract_date_from_filename(rel_path).map(|dt| dt.timestamp()); + response.filename_date = memories::extract_date_from_filename(rel_path) + .map(memories::FilenameDate::legacy_timestamp); response } diff --git a/src/memories.rs b/src/memories.rs index 2b1f473..053f09b 100644 --- a/src/memories.rs +++ b/src/memories.rs @@ -1,6 +1,6 @@ use actix_web::web::Data; use actix_web::{HttpRequest, HttpResponse, Responder, get, web}; -use chrono::{DateTime, FixedOffset, Local, NaiveDate, TimeZone, Utc}; +use chrono::{DateTime, FixedOffset, Local, NaiveDate, NaiveDateTime, Utc}; use log::{debug, trace, warn}; use opentelemetry::KeyValue; use opentelemetry::trace::{Span, Status, TraceContextExt, Tracer}; @@ -132,72 +132,108 @@ pub struct MemoriesResponse { pub items: Vec, } -pub fn extract_date_from_filename(filename: &str) -> Option> { - // Filenames carry only digits — no timezone. We deliberately interpret - // them as UTC so `.timestamp()` returns the wall-clock-as-UTC unix - // seconds, matching the "naive local reinterpreted as UTC" convention - // image_exif.date_taken uses for kamadak-exif DateTimeOriginal (which - // is also naive). Anything else (Local::from_local_datetime, the - // previous behavior) shifted filename-sourced dates by the SERVER's - // TZ offset relative to UTC, making them disagree with EXIF-sourced - // dates by hours and double-shifting through Apollo's photo matcher - // (which re-anchors naive-as-UTC via the browser TZ). - let build_date_from_ymd_capture = - |captures: ®ex::Captures| -> Option> { - let year = captures.get(1)?.as_str().parse::().ok()?; - let month = captures.get(2)?.as_str().parse::().ok()?; - let day = captures.get(3)?.as_str().parse::().ok()?; - let hour = captures.get(4)?.as_str().parse::().ok()?; - let min = captures.get(5)?.as_str().parse::().ok()?; - let sec = captures.get(6)?.as_str().parse::().ok()?; +/// A date carried by a filename, with its convention (ticket #116). +/// +/// The two families of pattern make fundamentally different claims: +/// - **`Wall`** — the device's own clock, as displayed (screenshots, +/// dash / compact name formats). It is a wall clock and needs the +/// offset cascade (offset tag → GPS → server zone) to become a true +/// instant, exactly like a photo-EXIF wall clock. +/// - **`Utc`** — a unix-timestamp name (`1401638400`, `att_1422489664…`) +/// is already a true instant; no offset is involved. +/// +/// Files whose name matches any pattern are **in the filename date +/// class**: the filename is the original date of record and outranks +/// EXIF / container dates (ADR-0001). Files whose name matches nothing +/// are out of the class and keep the global EXIF-wins waterfall. +#[derive(Copy, Clone, Debug, PartialEq)] +pub enum FilenameDate { + /// Wall clock the device displayed (screenshot / dash / compact). + Wall(NaiveDateTime), + /// True instant the unix-timestamp pattern names directly. + Utc(DateTime), +} - let naive = NaiveDate::from_ymd_opt(year, month, day)?.and_hms_opt(hour, min, sec)?; - Some(Utc.from_utc_datetime(&naive).fixed_offset()) - }; +impl FilenameDate { + /// The exact value the legacy `date_taken` column stores for this + /// filename: wall patterns as wall-as-UTC (the pre-migration + /// convention), timestamp patterns as their epoch. Consumers that + /// only need the legacy value (metadata responses, insight prompt + /// fallbacks) use this so the legacy contract stays byte-identical. + pub fn legacy_datetime(self) -> DateTime { + match self { + FilenameDate::Wall(wall) => wall.and_utc(), + FilenameDate::Utc(dt) => dt, + } + } + + /// The exact legacy `date_taken` unix seconds (`legacy_datetime` + /// as an epoch). The one-liner most consumers need. + pub fn legacy_timestamp(self) -> i64 { + self.legacy_datetime().timestamp() + } +} + +pub fn extract_date_from_filename(filename: &str) -> Option { + // The wall-clock patterns (screenshot / dash / compact) carry the + // device's own clock; the unix-timestamp pattern carries a true + // instant. `legacy_datetime()` reproduces the pre-migration + // interpretation of each (naive-as-UTC / epoch) for the legacy + // column and its consumers. + let build_wall_from_ymd_capture = |captures: ®ex::Captures| -> Option { + let year = captures.get(1)?.as_str().parse::().ok()?; + let month = captures.get(2)?.as_str().parse::().ok()?; + let day = captures.get(3)?.as_str().parse::().ok()?; + let hour = captures.get(4)?.as_str().parse::().ok()?; + let min = captures.get(5)?.as_str().parse::().ok()?; + let sec = captures.get(6)?.as_str().parse::().ok()?; + + NaiveDate::from_ymd_opt(year, month, day)?.and_hms_opt(hour, min, sec) + }; // 1. Screenshot format: Screenshot_2014-06-01-20-44-50.png - if let Some(captures) = regex::Regex::new(r"(\d{4})-(\d{2})-(\d{2})-(\d{2})-(\d{2})-(\d{2})") + if let Some(naive) = regex::Regex::new(r"(\d{4})-(\d{2})-(\d{2})-(\d{2})-(\d{2})-(\d{2})") .ok()? .captures(filename) - .and_then(|c| build_date_from_ymd_capture(&c)) + .and_then(|c| build_wall_from_ymd_capture(&c)) { - return Some(captures); + return Some(FilenameDate::Wall(naive)); } // Screenshot format: Screenshot_20140601[_-]204450.png - if let Some(captures) = regex::Regex::new(r"(\d{4})(\d{2})(\d{2})[_-](\d{2})(\d{2})(\d{2})") + if let Some(naive) = regex::Regex::new(r"(\d{4})(\d{2})(\d{2})[_-](\d{2})(\d{2})(\d{2})") .ok()? .captures(filename) - .and_then(|c| build_date_from_ymd_capture(&c)) + .and_then(|c| build_wall_from_ymd_capture(&c)) { - return Some(captures); + return Some(FilenameDate::Wall(naive)); } // 2. Dash format: 2015-01-09_02-15-15.jpg - if let Some(captures) = regex::Regex::new(r"(\d{4})-(\d{2})-(\d{2})_(\d{2})-(\d{2})-(\d{2})") + if let Some(naive) = regex::Regex::new(r"(\d{4})-(\d{2})-(\d{2})_(\d{2})-(\d{2})-(\d{2})") .ok()? .captures(filename) - .and_then(|c| build_date_from_ymd_capture(&c)) + .and_then(|c| build_wall_from_ymd_capture(&c)) { - return Some(captures); + return Some(FilenameDate::Wall(naive)); } // Dash with compact time format: 2015-01-09-021515.jpg - if let Some(captures) = regex::Regex::new(r"(\d{4})-(\d{2})-(\d{2})-(\d{2})(\d{2})(\d{2})") + if let Some(naive) = regex::Regex::new(r"(\d{4})-(\d{2})-(\d{2})-(\d{2})(\d{2})(\d{2})") .ok()? .captures(filename) - .and_then(|c| build_date_from_ymd_capture(&c)) + .and_then(|c| build_wall_from_ymd_capture(&c)) { - return Some(captures); + return Some(FilenameDate::Wall(naive)); } // 3. Compact format: 20140927101712.jpg - if let Some(captures) = regex::Regex::new(r"(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})") + if let Some(naive) = regex::Regex::new(r"(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})") .ok()? .captures(filename) - .and_then(|c| build_date_from_ymd_capture(&c)) + .and_then(|c| build_wall_from_ymd_capture(&c)) { - return Some(captures); + return Some(FilenameDate::Wall(naive)); } // 4. Timestamp format: 1401638400.jpeg, att_1422489664680106.jpeg, att_142248967186928.jpeg @@ -242,11 +278,10 @@ pub fn extract_date_from_filename(filename: &str) -> Option() .ok() - .and_then(DateTime::from_timestamp_millis) - .map(|naive_dt| naive_dt.fixed_offset()) + .and_then(DateTime::::from_timestamp_millis) .and_then(plausible_filename_date) { - return Some(date_time); + return Some(FilenameDate::Utc(date_time)); } // For 14-16 digits, treat first 10 digits as seconds to avoid far future dates @@ -255,11 +290,10 @@ pub fn extract_date_from_filename(filename: &str) -> Option() .ok() - .and_then(|timestamp_secs| DateTime::from_timestamp(timestamp_secs, 0)) - .map(|naive_dt| naive_dt.fixed_offset()) + .and_then(|ts| DateTime::::from_timestamp(ts, 0)) .and_then(plausible_filename_date) { - return Some(date_time); + return Some(FilenameDate::Utc(date_time)); } // Exactly 10 digits - seconds since epoch @@ -267,8 +301,7 @@ pub fn extract_date_from_filename(filename: &str) -> Option() .ok() - .and_then(|timestamp_secs| DateTime::from_timestamp(timestamp_secs, 0)) - .map(|naive_dt| naive_dt.fixed_offset()) + .and_then(|ts| DateTime::::from_timestamp(ts, 0)) .and_then(plausible_filename_date) { // Snapchat launched 2011-09-23. A 10-digit Snapchat filename @@ -278,7 +311,7 @@ pub fn extract_date_from_filename(filename: &str) -> Option Option() .ok() - .and_then(DateTime::from_timestamp_millis) - .map(|naive_dt| naive_dt.fixed_offset()) + .and_then(DateTime::::from_timestamp_millis) .and_then(plausible_filename_date) { - return Some(date_time); + return Some(FilenameDate::Utc(date_time)); } } @@ -304,7 +336,7 @@ pub fn extract_date_from_filename(filename: &str) -> Option) -> Option> { +fn plausible_filename_date(dt: DateTime) -> Option> { use chrono::Datelike; let year = dt.year(); // 1995 predates digital photography for most users; allowing one year @@ -524,7 +556,10 @@ mod tests { #[test] fn test_extract_date_from_filename_screenshot_format() { let filename = "Screenshot_2014-06-01-20-44-50.png"; - let date_time = extract_date_from_filename(filename).unwrap(); + let date_time = match extract_date_from_filename(filename).unwrap() { + FilenameDate::Wall(naive) => naive, + other => panic!("expected a wall-clock pattern, got {:?}", other), + }; assert_eq!(date_time.year(), 2014); assert_eq!(date_time.month(), 6); @@ -537,7 +572,10 @@ mod tests { #[test] fn test_extract_date_from_filename_screenshot_less_dashes_format() { let filename = "Screenshot_20140601-204450.png"; - let date_time = extract_date_from_filename(filename).unwrap(); + let date_time = match extract_date_from_filename(filename).unwrap() { + FilenameDate::Wall(naive) => naive, + other => panic!("expected a wall-clock pattern, got {:?}", other), + }; assert_eq!(date_time.year(), 2014); assert_eq!(date_time.month(), 6); @@ -550,7 +588,10 @@ mod tests { #[test] fn test_extract_date_from_filename_screenshot_underscores_format() { let filename = "20140601_204450.png"; - let date_time = extract_date_from_filename(filename).unwrap(); + let date_time = match extract_date_from_filename(filename).unwrap() { + FilenameDate::Wall(naive) => naive, + other => panic!("expected a wall-clock pattern, got {:?}", other), + }; assert_eq!(date_time.year(), 2014); assert_eq!(date_time.month(), 6); @@ -563,7 +604,10 @@ mod tests { #[test] fn test_extract_date_from_filename_dash_format() { let filename = "2015-01-09_02-15-15.jpg"; - let date_time = extract_date_from_filename(filename).unwrap(); + let date_time = match extract_date_from_filename(filename).unwrap() { + FilenameDate::Wall(naive) => naive, + other => panic!("expected a wall-clock pattern, got {:?}", other), + }; assert_eq!(date_time.year(), 2015); assert_eq!(date_time.month(), 1); @@ -576,7 +620,10 @@ mod tests { #[test] fn test_extract_date_from_filename_dash_compact_time_format() { let filename = "2015-01-09-021515.jpg"; - let date_time = extract_date_from_filename(filename).unwrap(); + let date_time = match extract_date_from_filename(filename).unwrap() { + FilenameDate::Wall(naive) => naive, + other => panic!("expected a wall-clock pattern, got {:?}", other), + }; assert_eq!(date_time.year(), 2015); assert_eq!(date_time.month(), 1); @@ -589,7 +636,10 @@ mod tests { #[test] fn test_extract_date_from_filename_compact_format() { let filename = "20140927101712.jpg"; - let date_time = extract_date_from_filename(filename).unwrap(); + let date_time = match extract_date_from_filename(filename).unwrap() { + FilenameDate::Wall(naive) => naive, + other => panic!("expected a wall-clock pattern, got {:?}", other), + }; assert_eq!(date_time.year(), 2014); assert_eq!(date_time.month(), 9); @@ -602,8 +652,10 @@ mod tests { #[test] fn test_extract_date_from_filename_timestamp_format() { let filename = "xyz_1401638400.jpeg"; // Unix timestamp for 2014-06-01 16:00:00 UTC - // Timestamps are already in UTC, so timezone doesn't matter for this test - let date_time = extract_date_from_filename(filename).unwrap(); + let date_time = match extract_date_from_filename(filename).unwrap() { + FilenameDate::Utc(dt) => dt, + other => panic!("expected a unix-timestamp pattern, got {:?}", other), + }; assert_eq!(date_time.year(), 2014); assert_eq!(date_time.month(), 6); @@ -616,7 +668,10 @@ mod tests { #[test] fn test_extract_date_from_filename_timestamp_millis_format() { let filename = "xyz_1401638400000.jpeg"; // Unix timestamp in milliseconds - let date_time = extract_date_from_filename(filename).unwrap(); + let date_time = match extract_date_from_filename(filename).unwrap() { + FilenameDate::Utc(dt) => dt, + other => panic!("expected a unix-timestamp pattern, got {:?}", other), + }; assert_eq!(date_time.year(), 2014); assert_eq!(date_time.month(), 6); @@ -631,7 +686,10 @@ mod tests { // att_142248967186928.jpeg - 15 digits, should parse first 10 as seconds // 1422489671 = 2015-01-28 23:07:51 UTC (converts to local timezone) let filename = "att_142248967186928.jpeg"; - let date_time = extract_date_from_filename(filename).unwrap(); + let date_time = match extract_date_from_filename(filename).unwrap() { + FilenameDate::Utc(dt) => dt, + other => panic!("expected a unix-timestamp pattern, got {:?}", other), + }; // Verify year and month are correct (2015-01) assert_eq!(date_time.year(), 2015); @@ -649,7 +707,10 @@ mod tests { // att_1422489664680106.jpeg - 16 digits, should parse first 10 as seconds // 1422489664 = 2015-01-28 23:07:44 UTC (converts to local timezone) let filename = "att_1422489664680106.jpeg"; - let date_time = extract_date_from_filename(filename).unwrap(); + let date_time = match extract_date_from_filename(filename).unwrap() { + FilenameDate::Utc(dt) => dt, + other => panic!("expected a unix-timestamp pattern, got {:?}", other), + }; // Verify year and month are correct (2015-01) assert_eq!(date_time.year(), 2015); @@ -714,7 +775,10 @@ mod tests { // Snapchat-1383929602.jpg → 1383929602 = 2013-11-08 16:53:22 UTC. // The blanket-prefix denial introduced for sequential IDs broke // these — restore via a length=10 + post-launch sanity gate. - let date_time = extract_date_from_filename("Snapchat-1383929602.jpg").unwrap(); + let date_time = match extract_date_from_filename("Snapchat-1383929602.jpg").unwrap() { + FilenameDate::Utc(dt) => dt, + other => panic!("expected a unix-timestamp pattern, got {:?}", other), + }; assert_eq!(date_time.timestamp(), 1383929602); }