Merge pull request 'feat(dates): filename date class — per-pattern conventions + filename-over-EXIF priority (#116)' (#122) from feature/filename-date-class into master
Reviewed-on: #122
This commit was merged in pull request #122.
This commit is contained in:
commit
3ac011abd7
8 files changed
+371
-149
No files matched your search
@@ -374,6 +374,18 @@ per-tick drain can re-resolve weak entries when better tools become
|
|||||||
available, and so the UI/debug surface can answer "why did this photo
|
available, and so the UI/debug surface can answer "why did this photo
|
||||||
land on this date?". Order:
|
land on this date?". Order:
|
||||||
|
|
||||||
|
0. **`filename` date class (ADR-0001, ticket #116)** —
|
||||||
|
`extract_date_from_filename` in `memories.rs` returns a
|
||||||
|
`FilenameDate`: wall-clock patterns (screenshot, dash, compact) or
|
||||||
|
unix-timestamp patterns. A file whose name matches **any** pattern is
|
||||||
|
*in the class*: the filename is the original date of record and wins
|
||||||
|
*before* EXIF / container dates are consulted, inverting the global
|
||||||
|
EXIF-wins rule for exactly those files (a renamed screenshot or a
|
||||||
|
chat-export timestamp is the only honest date left). Wall patterns
|
||||||
|
run the offset cascade over the file's own `OffsetTime*` / GPS
|
||||||
|
evidence (empty evidence → server-zone assumption, so a class file
|
||||||
|
always resolves); timestamp patterns are true instants outright
|
||||||
|
(`utc_direct`, no offset). In-class files never hit exiftool.
|
||||||
1. **`exif`** — kamadak-exif `DateTime` / `DateTimeOriginal`. Fast,
|
1. **`exif`** — kamadak-exif `DateTime` / `DateTimeOriginal`. Fast,
|
||||||
in-process, image-only.
|
in-process, image-only.
|
||||||
2. **`exiftool`** — shell-out fallback for tags kamadak can't reach:
|
2. **`exiftool`** — shell-out fallback for tags kamadak can't reach:
|
||||||
@@ -395,17 +407,15 @@ land on this date?". Order:
|
|||||||
old server-local `-d %s` interpretation; the new-contract
|
old server-local `-d %s` interpretation; the new-contract
|
||||||
`date_utc` / `camera_offset_minutes` / `date_taken_tz_source`
|
`date_utc` / `camera_offset_minutes` / `date_taken_tz_source`
|
||||||
columns carry the true instant.
|
columns carry the true instant.
|
||||||
3. **`filename`** — `extract_date_from_filename` in `memories.rs`
|
3. **`fs_time`** — `earliest_fs_time(metadata)` (earlier of created /
|
||||||
matches screenshot, chat-export, and timestamp-named patterns.
|
modified). Last resort. (The filename step sits at step 0 for
|
||||||
4. **`fs_time`** — `earliest_fs_time(metadata)` (earlier of created /
|
in-class names; an out-of-class name matches nothing here.)
|
||||||
modified). Last resort.
|
|
||||||
|
|
||||||
Notable behavior change vs. the pre-2026-05 request-time logic:
|
**Out-of-class files keep EXIF-wins.** A photo named `IMG_1234.jpg`
|
||||||
**EXIF beats filename when both are present.** A photo named
|
whose EXIF `DateTime` is 2021 appears under 2021 — EXIF
|
||||||
`Screenshot_2014-06-01.png` whose EXIF `DateTime` is 2021 now appears
|
`DateTimeOriginal` is more reliable than an import-time filename. The
|
||||||
under 2021, not 2014 — on the theory that EXIF is more reliable than
|
filename class (step 0) is the documented exception where the name is
|
||||||
import-named filenames. The reverse case (no EXIF, filename has a
|
the original date itself.
|
||||||
date) is unchanged.
|
|
||||||
|
|
||||||
The `backfill_missing_date_taken` drain (`src/backfill.rs`) runs every
|
The `backfill_missing_date_taken` drain (`src/backfill.rs`) runs every
|
||||||
watcher tick alongside `backfill_unhashed_backlog` (also `src/backfill.rs`). It loads up to
|
watcher tick alongside `backfill_unhashed_backlog` (also `src/backfill.rs`). It loads up to
|
||||||
|
|||||||
@@ -68,12 +68,19 @@ as tickets.
|
|||||||
dev.
|
dev.
|
||||||
- **Blocking**: Gitea's **native issue dependencies** (enabled on this
|
- **Blocking**: Gitea's **native issue dependencies** (enabled on this
|
||||||
server), the canonical, UI-visible representation. Add an edge with
|
server), the canonical, UI-visible representation. Add an edge with
|
||||||
`tea api --method POST repos/{owner}/{repo}/issues/<child>/dependencies/blocked_by -F issue_id=<blocker-issue-index>`
|
`tea api --method POST repos/{owner}/{repo}/issues/<child>/dependencies -d '{"index": <blocker-index>, "owner": "<owner>", "repo": "<repo>"}'`
|
||||||
(the Gitea dependencies API takes the issue index, i.e. the visible
|
(201 on success). Gotchas, verified against Gitea 1.27:
|
||||||
`#number`, not an internal database id). Where dependencies aren't
|
- The route is `.../dependencies` — there is **no** `blocked_by`
|
||||||
available, fall back to a `Blocked by: #<n>, #<n>` line at the top of
|
subpath (that 404s).
|
||||||
the child description. A ticket is unblocked when every blocker is
|
- The JSON body **must** name the blocker's repo (`owner` + `repo`);
|
||||||
closed.
|
omitting them yields a misleading `404 repository does not exist`
|
||||||
|
even though the child's path is correct.
|
||||||
|
- `index` is the visible `#number`, not an internal database id.
|
||||||
|
Verify with `tea api repos/{owner}/{repo}/issues/<child>/dependencies`
|
||||||
|
(lists blockers); remove with `--method DELETE` and the same body.
|
||||||
|
Where dependencies aren't available, fall back to a
|
||||||
|
`Blocked by: #<n>, #<n>` line at the top of the child description.
|
||||||
|
A ticket is unblocked when every blocker is closed.
|
||||||
- **Frontier query**: list the map's open children (issues whose
|
- **Frontier query**: list the map's open children (issues whose
|
||||||
description carries `Part of #<map>`), drop any with an open blocker
|
description carries `Part of #<map>`), drop any with an open blocker
|
||||||
(dependency or `Blocked by` line) or an assignee; first in map order wins.
|
(dependency or `Blocked by` line) or an assignee; first in map order wins.
|
||||||
|
|||||||
@@ -2037,7 +2037,7 @@ fn resolve_date_taken_for_context(
|
|||||||
return from_exif;
|
return from_exif;
|
||||||
}
|
}
|
||||||
crate::memories::extract_date_from_filename(file_path)
|
crate::memories::extract_date_from_filename(file_path)
|
||||||
.map(|dt| dt.format("%Y-%m-%d").to_string())
|
.map(|fd| fd.legacy_datetime().format("%Y-%m-%d").to_string())
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Validate a stored→effective backend transition for a chat continuation.
|
/// Validate a stored→effective backend transition for a chat continuation.
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ use crate::database::{
|
|||||||
LocationHistoryDao, SearchHistoryDao,
|
LocationHistoryDao, SearchHistoryDao,
|
||||||
};
|
};
|
||||||
use crate::libraries::Library;
|
use crate::libraries::Library;
|
||||||
use crate::memories::extract_date_from_filename;
|
use crate::memories::{FilenameDate, extract_date_from_filename};
|
||||||
use crate::otel::global_tracer;
|
use crate::otel::global_tracer;
|
||||||
use crate::tags::TagDao;
|
use crate::tags::TagDao;
|
||||||
use crate::utils::{earliest_fs_time, normalize_path, retry_with_backoff, retry_with_backoff_if};
|
use crate::utils::{earliest_fs_time, normalize_path, retry_with_backoff, retry_with_backoff_if};
|
||||||
@@ -1062,7 +1062,7 @@ impl InsightGenerator {
|
|||||||
log::warn!("No date_taken in EXIF for {}, trying filename", file_path);
|
log::warn!("No date_taken in EXIF for {}, trying filename", file_path);
|
||||||
|
|
||||||
extract_date_from_filename(&file_path)
|
extract_date_from_filename(&file_path)
|
||||||
.map(|dt| dt.timestamp())
|
.map(FilenameDate::legacy_timestamp)
|
||||||
.or_else(|| {
|
.or_else(|| {
|
||||||
let full_path = self.resolve_full_path(&file_path)?;
|
let full_path = self.resolve_full_path(&file_path)?;
|
||||||
File::open(&full_path)
|
File::open(&full_path)
|
||||||
@@ -4153,7 +4153,7 @@ Return ONLY the summary, nothing else."#,
|
|||||||
} else {
|
} else {
|
||||||
log::warn!("No date_taken in EXIF for {}, trying filename", file_path);
|
log::warn!("No date_taken in EXIF for {}, trying filename", file_path);
|
||||||
extract_date_from_filename(&file_path)
|
extract_date_from_filename(&file_path)
|
||||||
.map(|dt| dt.timestamp())
|
.map(FilenameDate::legacy_timestamp)
|
||||||
.or_else(|| {
|
.or_else(|| {
|
||||||
let full_path = self.resolve_full_path(&file_path)?;
|
let full_path = self.resolve_full_path(&file_path)?;
|
||||||
File::open(&full_path)
|
File::open(&full_path)
|
||||||
|
|||||||
+198
-59
@@ -16,20 +16,37 @@
|
|||||||
//! `creation_time` fields) and skip the cascade; photo-EXIF wall
|
//! `creation_time` fields) and skip the cascade; photo-EXIF wall
|
||||||
//! clocks run the cascade over the file's own `OffsetTime*` / GPS
|
//! clocks run the cascade over the file's own `OffsetTime*` / GPS
|
||||||
//! evidence (ADR-0001, ticket #115).
|
//! evidence (ADR-0001, ticket #115).
|
||||||
//! 3. **filename regex** — `memories::extract_date_from_filename` covers
|
//! 0. **filename date class** (ticket #116) — `memories::extract_date_from_filename`
|
||||||
//! common screenshot / chat-export / timestamp-named patterns.
|
//! covers common screenshot / dash / compact / unix-timestamp patterns.
|
||||||
//! 4. **earliest filesystem time** — `utils::earliest_fs_time` picks the
|
//! A file whose name matches any pattern is **in the class**: the
|
||||||
|
//! filename is the original date of record and wins *before* EXIF / container
|
||||||
|
//! dates are even consulted (wall patterns run the offset cascade over the
|
||||||
|
//! file's own evidence; timestamp patterns are true instants).
|
||||||
|
//! 1. **kamadak-exif** — fast in-process EXIF read. Already done by
|
||||||
|
//! `exif::extract_exif_from_path` for image-bearing formats; callers
|
||||||
|
//! pass that result in via `prior_exif_date` so we don't re-parse.
|
||||||
|
//! 2. **exiftool** — shell-out fallback that reaches places kamadak-exif
|
||||||
|
//! can't: QuickTime/MP4 (`MediaCreateDate`, `TrackCreateDate`,
|
||||||
|
//! `CreateDate`), Apple's `ContentCreateDate`, MakerNote sub-IFDs.
|
||||||
|
//! Required for videos to land a real date; degrades silently when
|
||||||
|
//! `exiftool` isn't on PATH. Unlike the kamadak branch, this branch
|
||||||
|
//! reads raw date *strings* (no `-d %s`) and interprets them itself:
|
||||||
|
//! QuickTime/MP4 container dates are true instants (UTC-rendered
|
||||||
|
//! `creation_time` fields) and skip the cascade; photo-EXIF wall
|
||||||
|
//! clocks run the cascade over the file's own `OffsetTime*` / GPS
|
||||||
|
//! evidence (ADR-0001, ticket #115).
|
||||||
|
//! 3. **earliest filesystem time** — `utils::earliest_fs_time` picks the
|
||||||
//! earlier of created / modified, which on copied-from-backup files is
|
//! earlier of created / modified, which on copied-from-backup files is
|
||||||
//! a better proxy for content age than either alone.
|
//! a better proxy for content age than either alone.
|
||||||
//!
|
//!
|
||||||
//! `DateSource` records which step won so the per-tick drain can re-resolve
|
//! `DateSource` records which step won so the per-tick drain can re-resolve
|
||||||
//! weak sources (`fs_time`) once exiftool becomes available, and so the
|
//! weak sources (`fs_time`) once exiftool becomes available, and so the
|
||||||
//! UI/debug surface can answer "why does this photo show up under this
|
//! UI/debug surface can answer "why does this photo show up under this
|
||||||
//! date." Note that the previous `/memories` request-time logic preferred
|
//! date." For out-of-class files EXIF is authoritative when it exists,
|
||||||
//! filename even when EXIF was present; this resolver inverts that — EXIF
|
//! on the theory that an EXIF `DateTimeOriginal` is more reliable than a
|
||||||
//! is authoritative when it exists, on the theory that an EXIF
|
//! filename pattern that may reflect import time rather than capture time.
|
||||||
//! `DateTimeOriginal` is more reliable than a filename pattern that may
|
//! In-class files invert that rule deliberately (ADR-0001): a renamed
|
||||||
//! reflect import time rather than capture time.
|
//! screenshot or a chat-export timestamp is the only honest date left.
|
||||||
|
|
||||||
use std::collections::HashMap;
|
use std::collections::HashMap;
|
||||||
use std::io::Write;
|
use std::io::Write;
|
||||||
@@ -49,7 +66,8 @@ pub enum DateSource {
|
|||||||
Exif,
|
Exif,
|
||||||
/// exiftool fallback caught a video / MakerNote / QuickTime tag.
|
/// exiftool fallback caught a video / MakerNote / QuickTime tag.
|
||||||
Exiftool,
|
Exiftool,
|
||||||
/// `extract_date_from_filename` matched a known pattern.
|
/// `extract_date_from_filename` matched a known pattern — the file is
|
||||||
|
/// in the filename date class and the name outranked EXIF (#116).
|
||||||
Filename,
|
Filename,
|
||||||
/// Fell through to `earliest_fs_time(metadata)`.
|
/// Fell through to `earliest_fs_time(metadata)`.
|
||||||
FsTime,
|
FsTime,
|
||||||
@@ -84,14 +102,16 @@ pub struct ResolvedDate {
|
|||||||
/// behavior of each branch so rollback stays trivial (ADR-0001).
|
/// behavior of each branch so rollback stays trivial (ADR-0001).
|
||||||
pub timestamp: i64,
|
pub timestamp: i64,
|
||||||
pub source: DateSource,
|
pub source: DateSource,
|
||||||
/// True UTC instant under the new contract (ADR-0001). `None` for
|
/// True UTC instant under the new contract (ADR-0001). Every branch
|
||||||
/// filename-sourced rows: their convention (wall-clock vs UTC) is a
|
/// classifies (wall clocks via the cascade, instants directly);
|
||||||
/// per-pattern decision made by the filename date-class work (#116).
|
/// `Option` is kept for schema compatibility with rows written before
|
||||||
|
/// the date-class work landed (#116).
|
||||||
pub date_utc: Option<i64>,
|
pub date_utc: Option<i64>,
|
||||||
/// Camera offset (minutes) that produced `date_utc`; `None` when the
|
/// Camera offset (minutes) that produced `date_utc`; `None` for direct
|
||||||
/// branch is a direct true instant or unclassified.
|
/// true instants (no offset is involved).
|
||||||
pub camera_offset_minutes: Option<i32>,
|
pub camera_offset_minutes: Option<i32>,
|
||||||
/// Why `date_utc` has the offset it does. `None` when unclassified.
|
/// Why `date_utc` has the offset it does; `None` on rows written
|
||||||
|
/// before every branch classified (#116).
|
||||||
pub tz_source: Option<camera_tz::TzSource>,
|
pub tz_source: Option<camera_tz::TzSource>,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -132,6 +152,57 @@ fn exif_prior_resolved(prior: &ExifPrior) -> ResolvedDate {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The filename-class step (ticket #116), shared by both waterfalls: the
|
||||||
|
/// in-class name is the original date of record and outranks EXIF /
|
||||||
|
/// container dates. `evidence` is the kamadak prior's offset evidence when
|
||||||
|
/// the file has one; `None` means empty evidence — the cascade then assumes
|
||||||
|
/// the server zone, so a class file always resolves.
|
||||||
|
fn filename_class_resolved(
|
||||||
|
path: &Path,
|
||||||
|
evidence: Option<&camera_tz::OffsetEvidence>,
|
||||||
|
) -> Option<ResolvedDate> {
|
||||||
|
let filename_date = path
|
||||||
|
.file_name()
|
||||||
|
.and_then(|f| f.to_str())
|
||||||
|
.and_then(crate::memories::extract_date_from_filename)?;
|
||||||
|
let ev = evidence.copied().unwrap_or_default();
|
||||||
|
Some(filename_resolved(&filename_date, &ev))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Classify one filename date (ticket #116). Wall patterns run the offset
|
||||||
|
/// cascade over whatever evidence the file carries; timestamp patterns are
|
||||||
|
/// true instants outright. The legacy value is the pattern's pre-migration
|
||||||
|
/// interpretation: wall-as-UTC / epoch.
|
||||||
|
fn filename_resolved(
|
||||||
|
filename_date: &crate::memories::FilenameDate,
|
||||||
|
evidence: &camera_tz::OffsetEvidence,
|
||||||
|
) -> ResolvedDate {
|
||||||
|
match filename_date {
|
||||||
|
crate::memories::FilenameDate::Utc(instant) => ResolvedDate {
|
||||||
|
timestamp: instant.timestamp(),
|
||||||
|
source: DateSource::Filename,
|
||||||
|
date_utc: Some(instant.timestamp()),
|
||||||
|
camera_offset_minutes: None,
|
||||||
|
tz_source: Some(camera_tz::TzSource::UtcDirect),
|
||||||
|
},
|
||||||
|
crate::memories::FilenameDate::Wall(wall) => {
|
||||||
|
let resolution = camera_tz::resolve_camera_offset(
|
||||||
|
*wall,
|
||||||
|
evidence,
|
||||||
|
server_zone_cached(),
|
||||||
|
gps_lookup(),
|
||||||
|
);
|
||||||
|
ResolvedDate {
|
||||||
|
timestamp: wall.and_utc().timestamp(),
|
||||||
|
source: DateSource::Filename,
|
||||||
|
date_utc: Some(camera_tz::wall_to_instant(*wall, resolution.minutes)),
|
||||||
|
camera_offset_minutes: Some(resolution.minutes),
|
||||||
|
tz_source: Some(resolution.source),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Host zone, resolved once per process.
|
/// Host zone, resolved once per process.
|
||||||
fn server_zone_cached() -> Option<chrono_tz::Tz> {
|
fn server_zone_cached() -> Option<chrono_tz::Tz> {
|
||||||
static ZONE: OnceLock<Option<chrono_tz::Tz>> = OnceLock::new();
|
static ZONE: OnceLock<Option<chrono_tz::Tz>> = OnceLock::new();
|
||||||
@@ -149,28 +220,17 @@ fn gps_lookup() -> &'static camera_tz::TzfGpsZoneLookup {
|
|||||||
/// the waterfall fails — for files that exist on disk this should be
|
/// the waterfall fails — for files that exist on disk this should be
|
||||||
/// vanishingly rare (the fs-time fallback alone almost always succeeds).
|
/// vanishingly rare (the fs-time fallback alone almost always succeeds).
|
||||||
pub fn resolve_date_taken(path: &Path, prior_exif: Option<&ExifPrior>) -> Option<ResolvedDate> {
|
pub fn resolve_date_taken(path: &Path, prior_exif: Option<&ExifPrior>) -> Option<ResolvedDate> {
|
||||||
|
// Filename date class first (ticket #116): an in-class name is the
|
||||||
|
// original date of record and outranks EXIF / container dates.
|
||||||
|
if let Some(resolved) = filename_class_resolved(path, prior_exif.map(|p| &p.evidence)) {
|
||||||
|
return Some(resolved);
|
||||||
|
}
|
||||||
if let Some(prior) = prior_exif {
|
if let Some(prior) = prior_exif {
|
||||||
return Some(exif_prior_resolved(prior));
|
return Some(exif_prior_resolved(prior));
|
||||||
}
|
}
|
||||||
if let Some(resolved) = exiftool_date_single(path) {
|
if let Some(resolved) = exiftool_date_single(path) {
|
||||||
return Some(resolved);
|
return Some(resolved);
|
||||||
}
|
}
|
||||||
if let Some(dt) = path
|
|
||||||
.file_name()
|
|
||||||
.and_then(|f| f.to_str())
|
|
||||||
.and_then(crate::memories::extract_date_from_filename)
|
|
||||||
{
|
|
||||||
return Some(ResolvedDate {
|
|
||||||
timestamp: dt.timestamp(),
|
|
||||||
source: DateSource::Filename,
|
|
||||||
// Filename conventions (wall-clock vs UTC) are decided
|
|
||||||
// per-pattern by the filename date-class work (#116); until
|
|
||||||
// then these rows stay unclassified.
|
|
||||||
date_utc: None,
|
|
||||||
camera_offset_minutes: None,
|
|
||||||
tz_source: None,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
if let Ok(meta) = std::fs::metadata(path)
|
if let Ok(meta) = std::fs::metadata(path)
|
||||||
&& let Some(t) = earliest_fs_time(&meta)
|
&& let Some(t) = earliest_fs_time(&meta)
|
||||||
{
|
{
|
||||||
@@ -186,11 +246,12 @@ pub fn resolve_date_taken(path: &Path, prior_exif: Option<&ExifPrior>) -> Option
|
|||||||
None
|
None
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Batch waterfall. exiftool runs once over the whole batch (single
|
/// Batch waterfall. In-class filenames resolve first (ticket #116) and
|
||||||
/// subprocess); everything else is per-file and runs only on misses.
|
/// never hit exiftool; exiftool then runs once over the remaining
|
||||||
/// `prior_exif_dates` lets the caller pass in already-known kamadak dates
|
/// no-prior files (single subprocess); everything else is per-file and
|
||||||
/// keyed by path; entries without a prior date fall through to exiftool
|
/// runs only on misses. `prior_exif_priors` lets the caller pass in
|
||||||
/// and the rest of the waterfall.
|
/// already-known kamadak readings keyed by path; entries without a prior
|
||||||
|
/// date fall through to exiftool and the rest of the waterfall.
|
||||||
///
|
///
|
||||||
/// The per-tick backfill drain is the primary caller — it loads ~500 rows
|
/// The per-tick backfill drain is the primary caller — it loads ~500 rows
|
||||||
/// at a time and uses one exiftool subprocess to drain the lot.
|
/// at a time and uses one exiftool subprocess to drain the lot.
|
||||||
@@ -202,6 +263,14 @@ pub fn resolve_dates_batch(
|
|||||||
let mut needs_exiftool: Vec<&Path> = Vec::with_capacity(paths.len());
|
let mut needs_exiftool: Vec<&Path> = Vec::with_capacity(paths.len());
|
||||||
|
|
||||||
for path in paths {
|
for path in paths {
|
||||||
|
// Filename date class first (ticket #116) — same rule as the
|
||||||
|
// single-file waterfall; in-class files skip exiftool entirely.
|
||||||
|
if let Some(resolved) =
|
||||||
|
filename_class_resolved(path, prior_exif_priors.get(path).map(|p| &p.evidence))
|
||||||
|
{
|
||||||
|
out.insert(path.clone(), resolved);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
if let Some(prior) = prior_exif_priors.get(path) {
|
if let Some(prior) = prior_exif_priors.get(path) {
|
||||||
out.insert(path.clone(), exif_prior_resolved(prior));
|
out.insert(path.clone(), exif_prior_resolved(prior));
|
||||||
} else {
|
} else {
|
||||||
@@ -217,27 +286,13 @@ pub fn resolve_dates_batch(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Remaining misses are out-of-class files with no EXIF / container
|
||||||
|
// date — the filename step can't match them (any match would have
|
||||||
|
// resolved in the first loop), so they fall straight to fs-time.
|
||||||
for path in paths {
|
for path in paths {
|
||||||
if out.contains_key(path) {
|
if out.contains_key(path) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
if let Some(dt) = path
|
|
||||||
.file_name()
|
|
||||||
.and_then(|f| f.to_str())
|
|
||||||
.and_then(crate::memories::extract_date_from_filename)
|
|
||||||
{
|
|
||||||
out.insert(
|
|
||||||
path.clone(),
|
|
||||||
ResolvedDate {
|
|
||||||
timestamp: dt.timestamp(),
|
|
||||||
source: DateSource::Filename,
|
|
||||||
date_utc: None,
|
|
||||||
camera_offset_minutes: None,
|
|
||||||
tz_source: None,
|
|
||||||
},
|
|
||||||
);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
if let Ok(meta) = std::fs::metadata(path)
|
if let Ok(meta) = std::fs::metadata(path)
|
||||||
&& let Some(t) = earliest_fs_time(&meta)
|
&& let Some(t) = earliest_fs_time(&meta)
|
||||||
{
|
{
|
||||||
@@ -1089,19 +1144,103 @@ mod tests {
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn resolve_filename_when_no_exif_and_file_missing() {
|
fn filename_class_wall_without_evidence_assumes_server_zone() {
|
||||||
// No prior EXIF, no exiftool match (file missing), but the filename
|
// In-class screenshot name, no prior EXIF, file missing. The wall
|
||||||
// pattern still matches so the resolver lands on Filename. The
|
// clock runs the cascade with empty evidence and lands on
|
||||||
// filename branch stays unclassified (#116): date_utc is NULL.
|
// assumed_server — a class file always resolves (#116). Legacy
|
||||||
|
// stays the pre-migration wall-as-UTC interpretation.
|
||||||
let resolved = resolve_date_taken(
|
let resolved = resolve_date_taken(
|
||||||
Path::new("/nonexistent/Screenshot_2014-06-01-20-44-50.png"),
|
Path::new("/nonexistent/Screenshot_2014-06-01-20-44-50.png"),
|
||||||
None,
|
None,
|
||||||
)
|
)
|
||||||
.unwrap();
|
.unwrap();
|
||||||
|
let wall =
|
||||||
|
NaiveDateTime::parse_from_str("2014-06-01 20:44:50", "%Y-%m-%d %H:%M:%S").unwrap();
|
||||||
assert_eq!(resolved.source, DateSource::Filename);
|
assert_eq!(resolved.source, DateSource::Filename);
|
||||||
assert_eq!(resolved.date_utc, None);
|
assert_eq!(resolved.timestamp, wall.and_utc().timestamp());
|
||||||
|
assert!(resolved.date_utc.is_some());
|
||||||
|
assert!(resolved.camera_offset_minutes.is_some());
|
||||||
|
assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::AssumedServer));
|
||||||
|
assert_eq!(
|
||||||
|
resolved.date_utc.unwrap(),
|
||||||
|
resolved.timestamp - (resolved.camera_offset_minutes.unwrap() as i64) * 60
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn filename_class_wall_outranks_exif_and_uses_its_evidence() {
|
||||||
|
// In-class filename beats the EXIF date entirely (the priority
|
||||||
|
// inversion), and the wall cascade uses the prior's offset-tag
|
||||||
|
// evidence. The EXIF wall (2020) never reaches the result.
|
||||||
|
let prior = ExifPrior {
|
||||||
|
wall: NaiveDateTime::parse_from_str("2020-05-05 05:05:05", "%Y-%m-%d %H:%M:%S")
|
||||||
|
.unwrap(),
|
||||||
|
evidence: camera_tz::OffsetEvidence {
|
||||||
|
offset_tag_minutes: Some(-240),
|
||||||
|
gps: None,
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let resolved = resolve_date_taken(
|
||||||
|
Path::new("/nonexistent/2015-01-09_02-15-15.jpg"),
|
||||||
|
Some(&prior),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(resolved.source, DateSource::Filename);
|
||||||
|
assert_eq!(resolved.timestamp, 1_420_769_715); // wall-as-UTC legacy
|
||||||
|
assert_eq!(resolved.camera_offset_minutes, Some(-240));
|
||||||
|
assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::OffsetTag));
|
||||||
|
// The offset tag short-circuits the cascade and is used as-stamped:
|
||||||
|
// wall − (−240) → 2015-01-09 06:15:15Z.
|
||||||
|
assert_eq!(resolved.date_utc, Some(1_420_784_115));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn filename_class_wall_gps_evidence_uses_capture_zone() {
|
||||||
|
// No offset tag; the prior carries Manhattan GPS. The bundled TZF
|
||||||
|
// geometry resolves America/New_York, whose offset for the
|
||||||
|
// July wall is EDT (-240).
|
||||||
|
let prior = ExifPrior {
|
||||||
|
wall: NaiveDateTime::parse_from_str("2024-07-04 12:00:00", "%Y-%m-%d %H:%M:%S")
|
||||||
|
.unwrap(),
|
||||||
|
evidence: camera_tz::OffsetEvidence {
|
||||||
|
offset_tag_minutes: None,
|
||||||
|
gps: Some((40.7128, -74.0060)),
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let resolved = resolve_date_taken(
|
||||||
|
Path::new("/nonexistent/Screenshot_2024-07-04-12-00-00.png"),
|
||||||
|
Some(&prior),
|
||||||
|
)
|
||||||
|
.unwrap();
|
||||||
|
assert_eq!(resolved.source, DateSource::Filename);
|
||||||
|
assert_eq!(resolved.camera_offset_minutes, Some(-240));
|
||||||
|
assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::Gps));
|
||||||
|
assert_eq!(resolved.date_utc, Some(1_720_108_800)); // 16:00:00Z
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn filename_class_timestamp_is_utc_direct() {
|
||||||
|
// A unix-timestamp name is a true instant: no cascade, no offset,
|
||||||
|
// and legacy equals the epoch (the pre-migration interpretation).
|
||||||
|
let resolved =
|
||||||
|
resolve_date_taken(Path::new("/nonexistent/xyz_1401638400.jpeg"), None).unwrap();
|
||||||
|
assert_eq!(resolved.source, DateSource::Filename);
|
||||||
|
assert_eq!(resolved.timestamp, 1_401_638_400);
|
||||||
|
assert_eq!(resolved.date_utc, Some(1_401_638_400));
|
||||||
assert_eq!(resolved.camera_offset_minutes, None);
|
assert_eq!(resolved.camera_offset_minutes, None);
|
||||||
assert_eq!(resolved.tz_source, None);
|
assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::UtcDirect));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn in_class_video_name_resolves_without_exiftool() {
|
||||||
|
// A chat-export-style name on a video path: the class check runs
|
||||||
|
// before exiftool is consulted — the file need not even exist for
|
||||||
|
// the result to be class-resolved.
|
||||||
|
let resolved =
|
||||||
|
resolve_date_taken(Path::new("/nonexistent/att_1422489664680106.mp4"), None).unwrap();
|
||||||
|
assert_eq!(resolved.source, DateSource::Filename);
|
||||||
|
assert_eq!(resolved.date_utc, Some(1_422_489_664));
|
||||||
|
assert_eq!(resolved.tz_source, Some(camera_tz::TzSource::UtcDirect));
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|||||||
+4
-2
@@ -18,7 +18,7 @@ use crate::data::{
|
|||||||
use crate::database::ExifDao;
|
use crate::database::ExifDao;
|
||||||
use crate::file_types;
|
use crate::file_types;
|
||||||
use crate::geo::{gps_bounding_box, haversine_distance};
|
use crate::geo::{gps_bounding_box, haversine_distance};
|
||||||
use crate::memories::extract_date_from_filename;
|
use crate::memories::{FilenameDate, extract_date_from_filename};
|
||||||
use crate::thumbnails::create_thumbnails;
|
use crate::thumbnails::create_thumbnails;
|
||||||
use crate::utils::earliest_fs_time;
|
use crate::utils::earliest_fs_time;
|
||||||
use actix_web::web::Data;
|
use actix_web::web::Data;
|
||||||
@@ -145,7 +145,9 @@ fn in_memory_date_sort(
|
|||||||
let date_taken = exif_map
|
let date_taken = exif_map
|
||||||
.get(&(f.file_name.clone(), lib_id))
|
.get(&(f.file_name.clone(), lib_id))
|
||||||
.copied()
|
.copied()
|
||||||
.or_else(|| extract_date_from_filename(&f.file_name).map(|dt| dt.timestamp()))
|
.or_else(|| {
|
||||||
|
extract_date_from_filename(&f.file_name).map(FilenameDate::legacy_timestamp)
|
||||||
|
})
|
||||||
.or_else(|| {
|
.or_else(|| {
|
||||||
lib_roots.get(&lib_id).and_then(|root| {
|
lib_roots.get(&lib_id).and_then(|root| {
|
||||||
let full_path = Path::new(root).join(&f.file_name);
|
let full_path = Path::new(root).join(&f.file_name);
|
||||||
|
|||||||
@@ -526,8 +526,8 @@ pub async fn get_file_metadata(
|
|||||||
response.library_name = Some(resolved_library.name.clone());
|
response.library_name = Some(resolved_library.name.clone());
|
||||||
|
|
||||||
// Extract date from filename if possible
|
// Extract date from filename if possible
|
||||||
response.filename_date =
|
response.filename_date = memories::extract_date_from_filename(&path.path)
|
||||||
memories::extract_date_from_filename(&path.path).map(|dt| dt.timestamp());
|
.map(memories::FilenameDate::legacy_timestamp);
|
||||||
|
|
||||||
// Query EXIF data if available
|
// Query EXIF data if available
|
||||||
if let Ok(mut dao) = exif_dao.lock()
|
if let Ok(mut dao) = exif_dao.lock()
|
||||||
@@ -721,8 +721,8 @@ pub async fn set_image_gps(
|
|||||||
response.exif = Some(row.into());
|
response.exif = Some(row.into());
|
||||||
response.library_id = Some(resolved_library.id);
|
response.library_id = Some(resolved_library.id);
|
||||||
response.library_name = Some(resolved_library.name.clone());
|
response.library_name = Some(resolved_library.name.clone());
|
||||||
response.filename_date =
|
response.filename_date = memories::extract_date_from_filename(&body.path)
|
||||||
memories::extract_date_from_filename(&body.path).map(|dt| dt.timestamp());
|
.map(memories::FilenameDate::legacy_timestamp);
|
||||||
span.set_status(Status::Ok);
|
span.set_status(Status::Ok);
|
||||||
HttpResponse::Ok().json(response)
|
HttpResponse::Ok().json(response)
|
||||||
}
|
}
|
||||||
@@ -881,8 +881,8 @@ fn build_metadata_response_for_date_mutation(
|
|||||||
response.exif = Some(exif.into());
|
response.exif = Some(exif.into());
|
||||||
response.library_id = Some(library.id);
|
response.library_id = Some(library.id);
|
||||||
response.library_name = Some(library.name.clone());
|
response.library_name = Some(library.name.clone());
|
||||||
response.filename_date =
|
response.filename_date = memories::extract_date_from_filename(rel_path)
|
||||||
memories::extract_date_from_filename(rel_path).map(|dt| dt.timestamp());
|
.map(memories::FilenameDate::legacy_timestamp);
|
||||||
response
|
response
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+126
-62
@@ -1,6 +1,6 @@
|
|||||||
use actix_web::web::Data;
|
use actix_web::web::Data;
|
||||||
use actix_web::{HttpRequest, HttpResponse, Responder, get, web};
|
use actix_web::{HttpRequest, HttpResponse, Responder, get, web};
|
||||||
use chrono::{DateTime, FixedOffset, Local, NaiveDate, TimeZone, Utc};
|
use chrono::{DateTime, FixedOffset, Local, NaiveDate, NaiveDateTime, Utc};
|
||||||
use log::{debug, trace, warn};
|
use log::{debug, trace, warn};
|
||||||
use opentelemetry::KeyValue;
|
use opentelemetry::KeyValue;
|
||||||
use opentelemetry::trace::{Span, Status, TraceContextExt, Tracer};
|
use opentelemetry::trace::{Span, Status, TraceContextExt, Tracer};
|
||||||
@@ -132,72 +132,108 @@ pub struct MemoriesResponse {
|
|||||||
pub items: Vec<MemoryItem>,
|
pub items: Vec<MemoryItem>,
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset>> {
|
/// A date carried by a filename, with its convention (ticket #116).
|
||||||
// Filenames carry only digits — no timezone. We deliberately interpret
|
///
|
||||||
// them as UTC so `.timestamp()` returns the wall-clock-as-UTC unix
|
/// The two families of pattern make fundamentally different claims:
|
||||||
// seconds, matching the "naive local reinterpreted as UTC" convention
|
/// - **`Wall`** — the device's own clock, as displayed (screenshots,
|
||||||
// image_exif.date_taken uses for kamadak-exif DateTimeOriginal (which
|
/// dash / compact name formats). It is a wall clock and needs the
|
||||||
// is also naive). Anything else (Local::from_local_datetime, the
|
/// offset cascade (offset tag → GPS → server zone) to become a true
|
||||||
// previous behavior) shifted filename-sourced dates by the SERVER's
|
/// instant, exactly like a photo-EXIF wall clock.
|
||||||
// TZ offset relative to UTC, making them disagree with EXIF-sourced
|
/// - **`Utc`** — a unix-timestamp name (`1401638400`, `att_1422489664…`)
|
||||||
// dates by hours and double-shifting through Apollo's photo matcher
|
/// is already a true instant; no offset is involved.
|
||||||
// (which re-anchors naive-as-UTC via the browser TZ).
|
///
|
||||||
let build_date_from_ymd_capture =
|
/// Files whose name matches any pattern are **in the filename date
|
||||||
|captures: ®ex::Captures| -> Option<DateTime<FixedOffset>> {
|
/// class**: the filename is the original date of record and outranks
|
||||||
let year = captures.get(1)?.as_str().parse::<i32>().ok()?;
|
/// EXIF / container dates (ADR-0001). Files whose name matches nothing
|
||||||
let month = captures.get(2)?.as_str().parse::<u32>().ok()?;
|
/// are out of the class and keep the global EXIF-wins waterfall.
|
||||||
let day = captures.get(3)?.as_str().parse::<u32>().ok()?;
|
#[derive(Copy, Clone, Debug, PartialEq)]
|
||||||
let hour = captures.get(4)?.as_str().parse::<u32>().ok()?;
|
pub enum FilenameDate {
|
||||||
let min = captures.get(5)?.as_str().parse::<u32>().ok()?;
|
/// Wall clock the device displayed (screenshot / dash / compact).
|
||||||
let sec = captures.get(6)?.as_str().parse::<u32>().ok()?;
|
Wall(NaiveDateTime),
|
||||||
|
/// True instant the unix-timestamp pattern names directly.
|
||||||
|
Utc(DateTime<Utc>),
|
||||||
|
}
|
||||||
|
|
||||||
let naive = NaiveDate::from_ymd_opt(year, month, day)?.and_hms_opt(hour, min, sec)?;
|
impl FilenameDate {
|
||||||
Some(Utc.from_utc_datetime(&naive).fixed_offset())
|
/// The exact value the legacy `date_taken` column stores for this
|
||||||
};
|
/// filename: wall patterns as wall-as-UTC (the pre-migration
|
||||||
|
/// convention), timestamp patterns as their epoch. Consumers that
|
||||||
|
/// only need the legacy value (metadata responses, insight prompt
|
||||||
|
/// fallbacks) use this so the legacy contract stays byte-identical.
|
||||||
|
pub fn legacy_datetime(self) -> DateTime<Utc> {
|
||||||
|
match self {
|
||||||
|
FilenameDate::Wall(wall) => wall.and_utc(),
|
||||||
|
FilenameDate::Utc(dt) => dt,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The exact legacy `date_taken` unix seconds (`legacy_datetime`
|
||||||
|
/// as an epoch). The one-liner most consumers need.
|
||||||
|
pub fn legacy_timestamp(self) -> i64 {
|
||||||
|
self.legacy_datetime().timestamp()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn extract_date_from_filename(filename: &str) -> Option<FilenameDate> {
|
||||||
|
// The wall-clock patterns (screenshot / dash / compact) carry the
|
||||||
|
// device's own clock; the unix-timestamp pattern carries a true
|
||||||
|
// instant. `legacy_datetime()` reproduces the pre-migration
|
||||||
|
// interpretation of each (naive-as-UTC / epoch) for the legacy
|
||||||
|
// column and its consumers.
|
||||||
|
let build_wall_from_ymd_capture = |captures: ®ex::Captures| -> Option<NaiveDateTime> {
|
||||||
|
let year = captures.get(1)?.as_str().parse::<i32>().ok()?;
|
||||||
|
let month = captures.get(2)?.as_str().parse::<u32>().ok()?;
|
||||||
|
let day = captures.get(3)?.as_str().parse::<u32>().ok()?;
|
||||||
|
let hour = captures.get(4)?.as_str().parse::<u32>().ok()?;
|
||||||
|
let min = captures.get(5)?.as_str().parse::<u32>().ok()?;
|
||||||
|
let sec = captures.get(6)?.as_str().parse::<u32>().ok()?;
|
||||||
|
|
||||||
|
NaiveDate::from_ymd_opt(year, month, day)?.and_hms_opt(hour, min, sec)
|
||||||
|
};
|
||||||
|
|
||||||
// 1. Screenshot format: Screenshot_2014-06-01-20-44-50.png
|
// 1. Screenshot format: Screenshot_2014-06-01-20-44-50.png
|
||||||
if let Some(captures) = regex::Regex::new(r"(\d{4})-(\d{2})-(\d{2})-(\d{2})-(\d{2})-(\d{2})")
|
if let Some(naive) = regex::Regex::new(r"(\d{4})-(\d{2})-(\d{2})-(\d{2})-(\d{2})-(\d{2})")
|
||||||
.ok()?
|
.ok()?
|
||||||
.captures(filename)
|
.captures(filename)
|
||||||
.and_then(|c| build_date_from_ymd_capture(&c))
|
.and_then(|c| build_wall_from_ymd_capture(&c))
|
||||||
{
|
{
|
||||||
return Some(captures);
|
return Some(FilenameDate::Wall(naive));
|
||||||
}
|
}
|
||||||
|
|
||||||
// Screenshot format: Screenshot_20140601[_-]204450.png
|
// Screenshot format: Screenshot_20140601[_-]204450.png
|
||||||
if let Some(captures) = regex::Regex::new(r"(\d{4})(\d{2})(\d{2})[_-](\d{2})(\d{2})(\d{2})")
|
if let Some(naive) = regex::Regex::new(r"(\d{4})(\d{2})(\d{2})[_-](\d{2})(\d{2})(\d{2})")
|
||||||
.ok()?
|
.ok()?
|
||||||
.captures(filename)
|
.captures(filename)
|
||||||
.and_then(|c| build_date_from_ymd_capture(&c))
|
.and_then(|c| build_wall_from_ymd_capture(&c))
|
||||||
{
|
{
|
||||||
return Some(captures);
|
return Some(FilenameDate::Wall(naive));
|
||||||
}
|
}
|
||||||
|
|
||||||
// 2. Dash format: 2015-01-09_02-15-15.jpg
|
// 2. Dash format: 2015-01-09_02-15-15.jpg
|
||||||
if let Some(captures) = regex::Regex::new(r"(\d{4})-(\d{2})-(\d{2})_(\d{2})-(\d{2})-(\d{2})")
|
if let Some(naive) = regex::Regex::new(r"(\d{4})-(\d{2})-(\d{2})_(\d{2})-(\d{2})-(\d{2})")
|
||||||
.ok()?
|
.ok()?
|
||||||
.captures(filename)
|
.captures(filename)
|
||||||
.and_then(|c| build_date_from_ymd_capture(&c))
|
.and_then(|c| build_wall_from_ymd_capture(&c))
|
||||||
{
|
{
|
||||||
return Some(captures);
|
return Some(FilenameDate::Wall(naive));
|
||||||
}
|
}
|
||||||
|
|
||||||
// Dash with compact time format: 2015-01-09-021515.jpg
|
// Dash with compact time format: 2015-01-09-021515.jpg
|
||||||
if let Some(captures) = regex::Regex::new(r"(\d{4})-(\d{2})-(\d{2})-(\d{2})(\d{2})(\d{2})")
|
if let Some(naive) = regex::Regex::new(r"(\d{4})-(\d{2})-(\d{2})-(\d{2})(\d{2})(\d{2})")
|
||||||
.ok()?
|
.ok()?
|
||||||
.captures(filename)
|
.captures(filename)
|
||||||
.and_then(|c| build_date_from_ymd_capture(&c))
|
.and_then(|c| build_wall_from_ymd_capture(&c))
|
||||||
{
|
{
|
||||||
return Some(captures);
|
return Some(FilenameDate::Wall(naive));
|
||||||
}
|
}
|
||||||
|
|
||||||
// 3. Compact format: 20140927101712.jpg
|
// 3. Compact format: 20140927101712.jpg
|
||||||
if let Some(captures) = regex::Regex::new(r"(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})")
|
if let Some(naive) = regex::Regex::new(r"(\d{4})(\d{2})(\d{2})(\d{2})(\d{2})(\d{2})")
|
||||||
.ok()?
|
.ok()?
|
||||||
.captures(filename)
|
.captures(filename)
|
||||||
.and_then(|c| build_date_from_ymd_capture(&c))
|
.and_then(|c| build_wall_from_ymd_capture(&c))
|
||||||
{
|
{
|
||||||
return Some(captures);
|
return Some(FilenameDate::Wall(naive));
|
||||||
}
|
}
|
||||||
|
|
||||||
// 4. Timestamp format: 1401638400.jpeg, att_1422489664680106.jpeg, att_142248967186928.jpeg
|
// 4. Timestamp format: 1401638400.jpeg, att_1422489664680106.jpeg, att_142248967186928.jpeg
|
||||||
@@ -242,11 +278,10 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
|
|||||||
&& let Some(date_time) = timestamp_str
|
&& let Some(date_time) = timestamp_str
|
||||||
.parse::<i64>()
|
.parse::<i64>()
|
||||||
.ok()
|
.ok()
|
||||||
.and_then(DateTime::from_timestamp_millis)
|
.and_then(DateTime::<Utc>::from_timestamp_millis)
|
||||||
.map(|naive_dt| naive_dt.fixed_offset())
|
|
||||||
.and_then(plausible_filename_date)
|
.and_then(plausible_filename_date)
|
||||||
{
|
{
|
||||||
return Some(date_time);
|
return Some(FilenameDate::Utc(date_time));
|
||||||
}
|
}
|
||||||
|
|
||||||
// For 14-16 digits, treat first 10 digits as seconds to avoid far future dates
|
// For 14-16 digits, treat first 10 digits as seconds to avoid far future dates
|
||||||
@@ -255,11 +290,10 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
|
|||||||
&& let Some(date_time) = timestamp_str[0..10]
|
&& let Some(date_time) = timestamp_str[0..10]
|
||||||
.parse::<i64>()
|
.parse::<i64>()
|
||||||
.ok()
|
.ok()
|
||||||
.and_then(|timestamp_secs| DateTime::from_timestamp(timestamp_secs, 0))
|
.and_then(|ts| DateTime::<Utc>::from_timestamp(ts, 0))
|
||||||
.map(|naive_dt| naive_dt.fixed_offset())
|
|
||||||
.and_then(plausible_filename_date)
|
.and_then(plausible_filename_date)
|
||||||
{
|
{
|
||||||
return Some(date_time);
|
return Some(FilenameDate::Utc(date_time));
|
||||||
}
|
}
|
||||||
|
|
||||||
// Exactly 10 digits - seconds since epoch
|
// Exactly 10 digits - seconds since epoch
|
||||||
@@ -267,8 +301,7 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
|
|||||||
&& let Some(date_time) = timestamp_str
|
&& let Some(date_time) = timestamp_str
|
||||||
.parse::<i64>()
|
.parse::<i64>()
|
||||||
.ok()
|
.ok()
|
||||||
.and_then(|timestamp_secs| DateTime::from_timestamp(timestamp_secs, 0))
|
.and_then(|ts| DateTime::<Utc>::from_timestamp(ts, 0))
|
||||||
.map(|naive_dt| naive_dt.fixed_offset())
|
|
||||||
.and_then(plausible_filename_date)
|
.and_then(plausible_filename_date)
|
||||||
{
|
{
|
||||||
// Snapchat launched 2011-09-23. A 10-digit Snapchat filename
|
// Snapchat launched 2011-09-23. A 10-digit Snapchat filename
|
||||||
@@ -278,7 +311,7 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
|
|||||||
if is_snapchat && date_time.timestamp() < SNAPCHAT_LAUNCH_TS {
|
if is_snapchat && date_time.timestamp() < SNAPCHAT_LAUNCH_TS {
|
||||||
return None;
|
return None;
|
||||||
}
|
}
|
||||||
return Some(date_time);
|
return Some(FilenameDate::Utc(date_time));
|
||||||
}
|
}
|
||||||
|
|
||||||
// 11-12 digits: try as milliseconds (might be partial millisecond timestamp)
|
// 11-12 digits: try as milliseconds (might be partial millisecond timestamp)
|
||||||
@@ -286,11 +319,10 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
|
|||||||
&& let Some(date_time) = timestamp_str
|
&& let Some(date_time) = timestamp_str
|
||||||
.parse::<i64>()
|
.parse::<i64>()
|
||||||
.ok()
|
.ok()
|
||||||
.and_then(DateTime::from_timestamp_millis)
|
.and_then(DateTime::<Utc>::from_timestamp_millis)
|
||||||
.map(|naive_dt| naive_dt.fixed_offset())
|
|
||||||
.and_then(plausible_filename_date)
|
.and_then(plausible_filename_date)
|
||||||
{
|
{
|
||||||
return Some(date_time);
|
return Some(FilenameDate::Utc(date_time));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -304,7 +336,7 @@ pub fn extract_date_from_filename(filename: &str) -> Option<DateTime<FixedOffset
|
|||||||
/// Rejecting them lets the date_resolver waterfall fall through to
|
/// Rejecting them lets the date_resolver waterfall fall through to
|
||||||
/// `fs_time`, which is a much better proxy for content age than a fake
|
/// `fs_time`, which is a much better proxy for content age than a fake
|
||||||
/// epoch date.
|
/// epoch date.
|
||||||
fn plausible_filename_date(dt: DateTime<FixedOffset>) -> Option<DateTime<FixedOffset>> {
|
fn plausible_filename_date(dt: DateTime<Utc>) -> Option<DateTime<Utc>> {
|
||||||
use chrono::Datelike;
|
use chrono::Datelike;
|
||||||
let year = dt.year();
|
let year = dt.year();
|
||||||
// 1995 predates digital photography for most users; allowing one year
|
// 1995 predates digital photography for most users; allowing one year
|
||||||
@@ -524,7 +556,10 @@ mod tests {
|
|||||||
#[test]
|
#[test]
|
||||||
fn test_extract_date_from_filename_screenshot_format() {
|
fn test_extract_date_from_filename_screenshot_format() {
|
||||||
let filename = "Screenshot_2014-06-01-20-44-50.png";
|
let filename = "Screenshot_2014-06-01-20-44-50.png";
|
||||||
let date_time = extract_date_from_filename(filename).unwrap();
|
let date_time = match extract_date_from_filename(filename).unwrap() {
|
||||||
|
FilenameDate::Wall(naive) => naive,
|
||||||
|
other => panic!("expected a wall-clock pattern, got {:?}", other),
|
||||||
|
};
|
||||||
|
|
||||||
assert_eq!(date_time.year(), 2014);
|
assert_eq!(date_time.year(), 2014);
|
||||||
assert_eq!(date_time.month(), 6);
|
assert_eq!(date_time.month(), 6);
|
||||||
@@ -537,7 +572,10 @@ mod tests {
|
|||||||
#[test]
|
#[test]
|
||||||
fn test_extract_date_from_filename_screenshot_less_dashes_format() {
|
fn test_extract_date_from_filename_screenshot_less_dashes_format() {
|
||||||
let filename = "Screenshot_20140601-204450.png";
|
let filename = "Screenshot_20140601-204450.png";
|
||||||
let date_time = extract_date_from_filename(filename).unwrap();
|
let date_time = match extract_date_from_filename(filename).unwrap() {
|
||||||
|
FilenameDate::Wall(naive) => naive,
|
||||||
|
other => panic!("expected a wall-clock pattern, got {:?}", other),
|
||||||
|
};
|
||||||
|
|
||||||
assert_eq!(date_time.year(), 2014);
|
assert_eq!(date_time.year(), 2014);
|
||||||
assert_eq!(date_time.month(), 6);
|
assert_eq!(date_time.month(), 6);
|
||||||
@@ -550,7 +588,10 @@ mod tests {
|
|||||||
#[test]
|
#[test]
|
||||||
fn test_extract_date_from_filename_screenshot_underscores_format() {
|
fn test_extract_date_from_filename_screenshot_underscores_format() {
|
||||||
let filename = "20140601_204450.png";
|
let filename = "20140601_204450.png";
|
||||||
let date_time = extract_date_from_filename(filename).unwrap();
|
let date_time = match extract_date_from_filename(filename).unwrap() {
|
||||||
|
FilenameDate::Wall(naive) => naive,
|
||||||
|
other => panic!("expected a wall-clock pattern, got {:?}", other),
|
||||||
|
};
|
||||||
|
|
||||||
assert_eq!(date_time.year(), 2014);
|
assert_eq!(date_time.year(), 2014);
|
||||||
assert_eq!(date_time.month(), 6);
|
assert_eq!(date_time.month(), 6);
|
||||||
@@ -563,7 +604,10 @@ mod tests {
|
|||||||
#[test]
|
#[test]
|
||||||
fn test_extract_date_from_filename_dash_format() {
|
fn test_extract_date_from_filename_dash_format() {
|
||||||
let filename = "2015-01-09_02-15-15.jpg";
|
let filename = "2015-01-09_02-15-15.jpg";
|
||||||
let date_time = extract_date_from_filename(filename).unwrap();
|
let date_time = match extract_date_from_filename(filename).unwrap() {
|
||||||
|
FilenameDate::Wall(naive) => naive,
|
||||||
|
other => panic!("expected a wall-clock pattern, got {:?}", other),
|
||||||
|
};
|
||||||
|
|
||||||
assert_eq!(date_time.year(), 2015);
|
assert_eq!(date_time.year(), 2015);
|
||||||
assert_eq!(date_time.month(), 1);
|
assert_eq!(date_time.month(), 1);
|
||||||
@@ -576,7 +620,10 @@ mod tests {
|
|||||||
#[test]
|
#[test]
|
||||||
fn test_extract_date_from_filename_dash_compact_time_format() {
|
fn test_extract_date_from_filename_dash_compact_time_format() {
|
||||||
let filename = "2015-01-09-021515.jpg";
|
let filename = "2015-01-09-021515.jpg";
|
||||||
let date_time = extract_date_from_filename(filename).unwrap();
|
let date_time = match extract_date_from_filename(filename).unwrap() {
|
||||||
|
FilenameDate::Wall(naive) => naive,
|
||||||
|
other => panic!("expected a wall-clock pattern, got {:?}", other),
|
||||||
|
};
|
||||||
|
|
||||||
assert_eq!(date_time.year(), 2015);
|
assert_eq!(date_time.year(), 2015);
|
||||||
assert_eq!(date_time.month(), 1);
|
assert_eq!(date_time.month(), 1);
|
||||||
@@ -589,7 +636,10 @@ mod tests {
|
|||||||
#[test]
|
#[test]
|
||||||
fn test_extract_date_from_filename_compact_format() {
|
fn test_extract_date_from_filename_compact_format() {
|
||||||
let filename = "20140927101712.jpg";
|
let filename = "20140927101712.jpg";
|
||||||
let date_time = extract_date_from_filename(filename).unwrap();
|
let date_time = match extract_date_from_filename(filename).unwrap() {
|
||||||
|
FilenameDate::Wall(naive) => naive,
|
||||||
|
other => panic!("expected a wall-clock pattern, got {:?}", other),
|
||||||
|
};
|
||||||
|
|
||||||
assert_eq!(date_time.year(), 2014);
|
assert_eq!(date_time.year(), 2014);
|
||||||
assert_eq!(date_time.month(), 9);
|
assert_eq!(date_time.month(), 9);
|
||||||
@@ -602,8 +652,10 @@ mod tests {
|
|||||||
#[test]
|
#[test]
|
||||||
fn test_extract_date_from_filename_timestamp_format() {
|
fn test_extract_date_from_filename_timestamp_format() {
|
||||||
let filename = "xyz_1401638400.jpeg"; // Unix timestamp for 2014-06-01 16:00:00 UTC
|
let filename = "xyz_1401638400.jpeg"; // Unix timestamp for 2014-06-01 16:00:00 UTC
|
||||||
// Timestamps are already in UTC, so timezone doesn't matter for this test
|
let date_time = match extract_date_from_filename(filename).unwrap() {
|
||||||
let date_time = extract_date_from_filename(filename).unwrap();
|
FilenameDate::Utc(dt) => dt,
|
||||||
|
other => panic!("expected a unix-timestamp pattern, got {:?}", other),
|
||||||
|
};
|
||||||
|
|
||||||
assert_eq!(date_time.year(), 2014);
|
assert_eq!(date_time.year(), 2014);
|
||||||
assert_eq!(date_time.month(), 6);
|
assert_eq!(date_time.month(), 6);
|
||||||
@@ -616,7 +668,10 @@ mod tests {
|
|||||||
#[test]
|
#[test]
|
||||||
fn test_extract_date_from_filename_timestamp_millis_format() {
|
fn test_extract_date_from_filename_timestamp_millis_format() {
|
||||||
let filename = "xyz_1401638400000.jpeg"; // Unix timestamp in milliseconds
|
let filename = "xyz_1401638400000.jpeg"; // Unix timestamp in milliseconds
|
||||||
let date_time = extract_date_from_filename(filename).unwrap();
|
let date_time = match extract_date_from_filename(filename).unwrap() {
|
||||||
|
FilenameDate::Utc(dt) => dt,
|
||||||
|
other => panic!("expected a unix-timestamp pattern, got {:?}", other),
|
||||||
|
};
|
||||||
|
|
||||||
assert_eq!(date_time.year(), 2014);
|
assert_eq!(date_time.year(), 2014);
|
||||||
assert_eq!(date_time.month(), 6);
|
assert_eq!(date_time.month(), 6);
|
||||||
@@ -631,7 +686,10 @@ mod tests {
|
|||||||
// att_142248967186928.jpeg - 15 digits, should parse first 10 as seconds
|
// att_142248967186928.jpeg - 15 digits, should parse first 10 as seconds
|
||||||
// 1422489671 = 2015-01-28 23:07:51 UTC (converts to local timezone)
|
// 1422489671 = 2015-01-28 23:07:51 UTC (converts to local timezone)
|
||||||
let filename = "att_142248967186928.jpeg";
|
let filename = "att_142248967186928.jpeg";
|
||||||
let date_time = extract_date_from_filename(filename).unwrap();
|
let date_time = match extract_date_from_filename(filename).unwrap() {
|
||||||
|
FilenameDate::Utc(dt) => dt,
|
||||||
|
other => panic!("expected a unix-timestamp pattern, got {:?}", other),
|
||||||
|
};
|
||||||
|
|
||||||
// Verify year and month are correct (2015-01)
|
// Verify year and month are correct (2015-01)
|
||||||
assert_eq!(date_time.year(), 2015);
|
assert_eq!(date_time.year(), 2015);
|
||||||
@@ -649,7 +707,10 @@ mod tests {
|
|||||||
// att_1422489664680106.jpeg - 16 digits, should parse first 10 as seconds
|
// att_1422489664680106.jpeg - 16 digits, should parse first 10 as seconds
|
||||||
// 1422489664 = 2015-01-28 23:07:44 UTC (converts to local timezone)
|
// 1422489664 = 2015-01-28 23:07:44 UTC (converts to local timezone)
|
||||||
let filename = "att_1422489664680106.jpeg";
|
let filename = "att_1422489664680106.jpeg";
|
||||||
let date_time = extract_date_from_filename(filename).unwrap();
|
let date_time = match extract_date_from_filename(filename).unwrap() {
|
||||||
|
FilenameDate::Utc(dt) => dt,
|
||||||
|
other => panic!("expected a unix-timestamp pattern, got {:?}", other),
|
||||||
|
};
|
||||||
|
|
||||||
// Verify year and month are correct (2015-01)
|
// Verify year and month are correct (2015-01)
|
||||||
assert_eq!(date_time.year(), 2015);
|
assert_eq!(date_time.year(), 2015);
|
||||||
@@ -714,7 +775,10 @@ mod tests {
|
|||||||
// Snapchat-1383929602.jpg → 1383929602 = 2013-11-08 16:53:22 UTC.
|
// Snapchat-1383929602.jpg → 1383929602 = 2013-11-08 16:53:22 UTC.
|
||||||
// The blanket-prefix denial introduced for sequential IDs broke
|
// The blanket-prefix denial introduced for sequential IDs broke
|
||||||
// these — restore via a length=10 + post-launch sanity gate.
|
// these — restore via a length=10 + post-launch sanity gate.
|
||||||
let date_time = extract_date_from_filename("Snapchat-1383929602.jpg").unwrap();
|
let date_time = match extract_date_from_filename("Snapchat-1383929602.jpg").unwrap() {
|
||||||
|
FilenameDate::Utc(dt) => dt,
|
||||||
|
other => panic!("expected a unix-timestamp pattern, got {:?}", other),
|
||||||
|
};
|
||||||
assert_eq!(date_time.timestamp(), 1383929602);
|
assert_eq!(date_time.timestamp(), 1383929602);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user