backfill_utc_dates refused any fs_time re-resolution of a tag-dated row,
assuming exiftool had failed on the file. Files whose date tags are all
0000:00:00 (e.g. uv-prod-long.mp4) were read fine and simply have no
date, so they sat unconverted and re-ran through exiftool every tick.
resolve_dates_batch_report now also returns the paths exiftool read
without error (-Error surfaces ExifTool:Error in the JSON). The drain
lets fs_time stand for an exiftool-legacy row only when exiftool read
the file; read errors / missing entries and kamadak exif-legacy rows
keep refusing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main.rs re-declared nearly every module (`mod ai; mod database; …`)
that lib.rs also declared, so the ~100k-line crate compiled twice in
every build (lib for the src/bin tools, bin for the server) and the
test suite ran most tests twice. watcher.rs even reached into the lib
copy via `image_api::` from inside the bin copy.
The six bin-only modules (backfill, handlers, hls_stats, knowledge,
reels, watcher) move into lib.rs; main.rs keeps only startup wiring
over `use image_api::…`. A few items main calls become `pub`, and the
dead_code allowances that only existed because each copy saw half the
callers are gone. CLAUDE.md says not to reintroduce `mod` in main.rs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The local location_history table is a one-time Takeout import: it ends
in 2019, still has the ghost-device and coarse cell-fix samples Apollo
has since excluded, and was read loosely. The per-photo line took the
nearest sample within +/-3 h at any distance (17% of photo moments got a
position >2 km away), and get_location_history listed the first 20 rows
of a +/-14-day window sorted ascending (always ~14 days before the date).
With APOLLO_API_BASE_URL set:
- per-photo context comes from /api/location/at (30 min max gap; no
sample -> no line). A Place stay renders as "at Home (14:02-18:30)" in
the display zone; an unnamed sample is reverse-geocoded.
- get_location_history returns Apollo's /api/location/day timeline for
the date (+/-1 day at most), and is offered even with an empty local
table.
Both fall back to the local table when Apollo is unset or a call fails;
the local tool path now covers the requested local day(s) and samples
evenly instead of listing a window's first rows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`image::open` chooses the decoder by extension, so a PNG saved as .jpg
(Android screenshots, chat exports) failed thumbnails, previews,
perceptual hashes, insights and face crops with "Illegal start bytes:
8950". New `file_types::open_image` sniffs the header and falls back to
the extension when it's unknown; every decode site uses it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A 422 from Apollo's /encode_image (bytes it can't decode — corrupt
files, non-images with an image extension) left clip_embedding NULL,
so the drain re-sent the file and logged the same "permanent failure
(will retry next pass)" on every scan.
Permanent rejections now record a clip_failures marker scoped to the
row's content_hash; list_clip_unencoded_candidates skips the row while
the hash still matches, so a replaced file gets another try with no
cleanup. Only 422 (new ClipError::Rejected) marks: other 4xx are
deployment problems (auth, routing) and would otherwise blacklist the
whole library.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scans, edited exports and some downloads carry only EXIF DateTime
(exiftool: IFD0:ModifyDate). The kamadak ingest path already takes it
as the capture date, but the exiftool resolver didn't ask for it, so
the date_utc drain resolved those rows to fs_time and then (rightly)
refused to overwrite their tag-sourced legacy date. 147 prod rows sat
on the legacy convention forever, re-running exiftool every tick.
ModifyDate is now the lowest-priority tag, and only from IFD0 — XMP /
QuickTime ModifyDate is an edit or mux time, not capture.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first #160 migration cleared rows whose legacy date_taken_source
was 'filename'. But the true-instant drain wrote only the new-contract
triple, so rows with a correct legacy date (e.g. an exiftool container
date) could still carry a filename-derived implausible year in
date_utc. Prod has three: VID_<counter> videos dated 2095 / 2229 / 8580
whose containers say 2016 / 2017.
Clear just the triple wherever date_utc is outside [1995, 2028) UTC;
backfill_utc_dates re-resolves it with the gated matcher, and the
frozen legacy pair is left alone. On prod those three are the only
rows with date_utc out of range.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/photos, /photos/exif and unified search take date_from_day /
date_to_day (YYYY-MM-DD): the server turns them into the first day's
start and the last day's end in the owner's display zone, the same days
/memories groups by. Each replaces its instant counterpart. Clients stop
computing midnights in the device's zone, which drifted by the travel
offset whenever the phone's zone differed from display_tz.
e2e check: every fixture is included by its display-zone day and
excluded by the days either side.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
match_filename_date only gated the unix-timestamp branch to years
1995..=now+1; the five wall patterns returned any NaiveDateTime. Counters
and IDs shaped like YYYYMMDD_HHMMSS (an app's VID_<counter>_<HHMMSS>,
Facebook's <8 digits>_<id>_o.jpg) became dates of record in years from
267 to 9985: 67 rows on prod, all via screenshot_compact.
Both branches now share plausible_filename_year; a wall miss falls
through to the next pattern and then the rest of the waterfall. The
filename audit no longer proposes these dates.
A data migration clears the five date columns on filename-sourced rows
dated outside [1995, 2028) UTC, so backfill_missing_date_taken
re-resolves them from EXIF / exiftool / fs time on the next ticks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Grid and memories tiles requested `size=large` (up to 2048px) for tiles
that display at ~590px (two columns) to ~1180px (one full-width column)
on a 3× phone, downloading 10–15× the pixels they show. `medium` caps
the long edge at 1024: crisp at both, a quarter of `large`'s pixels.
Cached under `_medium/` like the other tiers.
The large and xlarge generators were copies differing only in their
cap; fold them into one `generate_preview(src, dest, max_dim)` so the
third tier doesn't make a third copy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Legacy thumbnail and preview paths mirror the source's relative path, so
a video's JPEG thumbnail lives at `…/clip.mp4` and a RAW's at
`…/shot.arw`. NamedFile guessed the content type from that name and
served JPEG bytes as `video/mp4`. Sniff JPEG/PNG magic bytes instead.
The circular-thumbnail path had the same root cause: `image::open`
picks the decoder from the extension, so it failed on those thumbs and
quietly fell back to the square one. Guess the format from the bytes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
e2e.sh and gen_fixtures.py assumed WSL driving the Windows toolchain.
They now detect WSL (a microsoft kernel with cargo.exe available) and
otherwise use native tools:
- e2e.sh: cargo/curl instead of cargo.exe/curl.exe, pgrep/pkill
instead of tasklist/taskkill, Linux paths in the generated .env, and
binaries without the .exe suffix.
- gen_fixtures.py: picks the first ffmpeg that has the drawtext filter
(a minimal build earlier on PATH may not; E2E_FFMPEG overrides), finds
a monospace font with fc-match, and only translates paths with wslpath
for .exe tools.
- Quote USER_NAME in the generated .env: the unquoted space made the
dotenv parser reject the line.
- Mark the scripts executable.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The app's new search screen keeps tags, type, date, camera and place as
chips and sends free text to the unified endpoint. For the AI to add to
those chips instead of replacing them, the endpoint now takes them:
- /photos/search/unified accepts the /photos filters explicitly (tag_ids
with All/Any, exclude_tag_ids, media_type, dates, camera make/model,
gps circle) plus a recursive `path`. Explicit tags are their own
predicate on top of the model's ANY-mode tags; single-valued filters
replace the model's guess. `interpreted` echoes only what the model
added, so the client doesn't show the user's own chips twice.
- interpret=false skips the LLM and uses q verbatim as the CLIP term,
which replaces the app's separate "Semantic" mode with a visual match
inside the user's filters.
- /image/tags/all also returns `cameras` (make + model with counts) so
the camera picker can offer e.g. "Nexus 6" directly.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Long insight/persona chats on llama-swap failed with "exceeds the
available context size" even after raising Context in the app:
- llama-server fixes its window at launch (-c) and ignores num_ctx, but
the history budget trusted the requested num_ctx, so raising it kept
MORE history than the server could hold. LlmClient::context_window()
now reports the served window (llama-swap: /upstream/<model>/props,
cached 10 min) and the budget uses min(requested, served).
- apply_context_budget could only drop tool-call pairs, so plain Q&A
chats never shrank. It now also drops whole earlier exchanges (keeping
user/assistant alternation) and, last, the current turn's tool results,
never the in-progress question.
- The budget only ran once, before the tool loop. ContextFitter now
fits every LLM call (on a copy; the stored transcript is untouched)
and, on a context-overflow error, shrinks the budget from the server's
reported token counts and retries once.
GET /insights/models reports each llama-swap model's context_window when
it's known without loading the model (cached, or currently running), for
the app's Context picker.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The access log and the tracing root span's http.target both recorded the
full /video/original/stream?token=… URL, i.e. a credential for a
possibly sensitive file, in the journal and OTel exports. An outermost
middleware now moves the token into request extensions and rewrites the
URI to token=redacted before either logger sees it; the handler reads
the token from extensions and fails closed (401) without the middleware.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST /video/original mints a URL for one video's original bytes, signed
with a key derived from SECRET_KEY (distinct from the login-token key)
and valid for 6 hours. GET /video/original/stream?token= serves it via
NamedFile, so Range/seek works, without an Authorization header — native
players can use it directly. HLS stays the default path.
Also extracts generate_video's library fallback into resolve_request_path
and fixes the stale /video/stream entry in CLAUDE.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When NVENC is available the HLS transcode encoded on the GPU but still
decoded on the CPU, since the command had no -hwaccel. Rebuilding the
4K homemade videos ran three ffmpegs at 210-240% CPU each with the
machine at 0% idle, while NVDEC sat at 0%.
Pass -hwaccel cuda when transcoding with NVENC, as generate_preview_clip
already does. Frames return to system memory, so autorotate and the
scale filter work unchanged; NVDEC-unsupported codecs fall back to
software decode.
10 s of 4K H.264 (2025-USAF-MuseumRace.mp4): 27.3 s -> 6.1 s CPU,
9.6 s -> 3.6 s wall, identical output (1920x1080, 245 frames, 3 s
segments). Also checked a 90-degree-rotated phone clip (comes out
1080x1440 upright, 233/233 frames) and ProRes (software fallback,
correct output).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The docs said tests need BASE_PATH in .env. The full suite (lib, bin and
integration tests) passes with no .env and no BASE_PATH; the only thing
it needs is DATABASE_URL, and ":memory:" works. Without it, DB-backed
tests such as reels::tests::reel_capture_preserves_owner_defaults panic
with "DATABASE_URL must be set".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_generate_playlist was a private, never-called predecessor of the HLS
path in video/actors.rs. Its leftover args (-hls_list_size 100, width
1080 scale, libx264 only) no longer described how playlists are built.
Removing it also drops the now-unused std::process::Output import.
The reels encoder comment said its NVENC settings matched the HLS
transcode; reels use preset p4, HLS uses p1. Only the quality target
(cq 23) is shared.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-force_key_frames asks for a keyframe every 3 s, but h264_nvenc inserts
those as plain I-frames unless -forced-idr is set. The HLS muxer only
splits on IDR frames, so NVENC transcodes fell back to the encoder's
default 250-frame GOP: 8.34 s segments at 29.97 fps, ~10 s at 25 fps.
On prod ~480 playlists have a target duration above 4 s. Long segments
mean more data before playback starts and coarser seeks.
Verified on a 10 s 29.97 fps clip: segments went from 8.34 + 1.67 s to
3.003 x 3 + 1.001 s, matching the libx264 path, which was already
correct.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Variable-frame-rate sources (phone clips that change capture rate,
screen recordings) report r_frame_rate as their 1/90000 timebase. The
HLS transcode used the mpegts muxer's default CFR mode, so ffmpeg
duplicated frames to fill 90000 fps: on prod, three 2.6-5 s phone clips
became 3-second "segments" of up to 446 MB / 250k frames. Players
download the whole thing before starting and can't seek within it.
-fps_mode vfr passes source timestamps through. Verified on
20220208_164356_01.mp4: 233 frames over 2.578 s in, 233 frames over
2.578 s out, 5.9 MB vs 221 MB. The copy path already keeps timestamps
and is unchanged. Requires ffmpeg >= 5.1 (prod runs 8.0.1).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
HLS generation stream-copied any h264 source with no rotation and a
GOP within one segment, which keeps the source's resolution and
bitrate. Mirrorless footage (4K at 100-160 Mbps) came through as-is,
giving ~50 MB 3-second segments; playback start and seeks stalled on
anything slower than wired gigabit. On prod, 16 playlists run above
50 Mbps, versus ~10 Mbps typical for phone clips.
The stream probe now also reads width, height and bit_rate (stream
level, falling back to format level; all header fields, so the probe
stays header-only). Copy additionally requires the short side to be
<= 1080 and the bitrate <= HLS_COPY_MAX_BITRATE (default 20 Mbps).
Unknown size or bitrate transcodes, matching the GOP probe's fallback.
The transcode scale filter capped width at 1080, so landscape sources
came out at 1080x608. It now caps the short side instead: 3840x2160
-> 1920x1080, 2160x3840 -> 1080x1920, smaller sources unchanged
(verified with ffmpeg).
Existing oversized playlists need regenerating after deploy.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The exiftool branch matched GPSLatitudeRef / GPSLongitudeRef with an
exact one-letter compare ("S" / "W"), but exiftool's print conversion
(which parse_gps_dms relies on) renders them as "South" / "West". The
flags never set, so every western longitude and southern latitude
reached the camera-offset cascade positive and the GPS step looked up
the mirrored point: northeast Ohio became China, offset UTC+8. On prod,
all 280 western-hemisphere `gps`-sourced rows carry +480, putting their
date_utc 12-13 h off.
gps_ref_is matches the first letter case-insensitively, the same rule
as exif::extract_gps_coordinate (whose kamadak path was already
correct; the stored gps_latitude / gps_longitude columns are unaffected).
Tests use exiftool's real output strings ("West" / "South", verified
against exiftool 12.76 with the resolver's -j -q -G1 flags), including
a southern-hemisphere case; the new payload test fails on the old
compare. The existing test fed "W", which exiftool doesn't emit by
default, so it couldn't catch this.
Existing rows need a re-resolve after deploy; see #163 for the reset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mermaid sources for the system context, data ownership, photo ingest,
watcher tick, Apollo photo matching and photo insight chat, traced from
the current code of all three repos. Allow-lists docs/architecture/
alongside docs/adr/ and docs/agents/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
date_from / date_to on /photos, /photos/exif and unified search ran on
the legacy mixed-convention date_taken, so a photo could display inside
a range yet be filtered out of it by up to its UTC offset (or by years,
for rows the filename class re-dated).
The filter lives in the shared ExifDao::query_by_exif, so it's fixed
once there:
- query_by_exif takes Option<date_view::InstantRange> (bounds + display
and server zones) instead of raw date_from / date_to.
- SQL pre-filter: converted rows on date_utc exactly; unconverted rows
(drain pending, permanent floor) on date_taken widened by
LEGACY_INSTANT_BAND_SECS (±15 h, now shared with /memories). Converted
rows are deliberately not bounded by date_taken: a re-dated row's
legacy value can sit years from its instant.
- Exact pass in Rust via InstantRange::contains → row_true_instant, so
every returned row's true instant is inside the range.
Calendar days now mean the owner's days: nl_query::iso_to_unix builds
day bounds in the display zone (it used UTC midnights, which only lined
up with the legacy wall-as-UTC column), and unified search's "today" is
the owner's today rather than UTC's. The midnight-with-DST-gap fallback
moves from memories_anchor into date_view::day_start_instant.
Compatibility: clients already sending real instants (Apollo's
/photos/exif windows) get correct edges with no change. Clients sending
UTC-midnight day bounds see ranges shift by up to the UTC offset until
they send display-zone bounds.
No index on date_utc; the OR clause scans, which is milliseconds at
personal-library scale (same as /memories today).
dev/e2e check gains /photos/exif wall_clock + true_instant assertions and
range-edge checks ([t, t] in, [.., t-1] and [t+1, ..] out). Not run here:
the harness needs the WSL + Windows toolchain.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/photos/exif is Apollo's photo-to-track matching feed, but unlike
/image/metadata and /photos/gps-summary it only carried the raw
new-contract triple. date_utc is NULL for unconverted rows and, by
design, forever for the drain's edited/exported floor (#120), so a
client reading date_utc alone would have to re-implement the
per-source legacy conventions to cover those rows: the re-anchoring
ADR-0001 says consumers delete, not migrate.
ExifSummary now carries:
- true_instant: date_view::row_true_instant (date_utc when converted,
else the per-source fallback), for instant-to-instant matching.
- wall_clock: date_view::row_wall_clock, rendered verbatim.
The row → wire mapping moves to ExifSummary::from_row so the tests
cover the real projection, including an unconverted row falling back
to the same instant and wall as its converted twin. Additive: legacy
fields are unchanged, so current clients see the same values.
The date_from/date_to filter still runs on the legacy column; that's
#152, which now explicitly covers /photos/exif too.
Unblocks Apollo #37.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cameron CordesandClaude Opus 5.5
cameron