feat(versions): e2e fixtures, check_versions, docs

Fixtures gain an XMP-linked version (renamed file), a name-linked version
and an edit with a reference photo; e2e.sh counts with include_versions and
runs check_versions.py. CLAUDE.md documents the feature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Opus 5.5 committed 2026-10-30 13:53:21 -04:00
1 parent fa3554c411
commit 6135034d09
5 files changed
+394 -6

No files matched your search

+78
View File
@@ -77,6 +77,11 @@ cargo run --bin cleanup_files -- --base-path /path/to/media --database-url ./dat
# Create a login user (no HTTP registration route; idempotent)
cargo run --bin create_user -- <username> <password>
# Link photo versions ingested before linking existed. Dry run by default
# (prints a TSV); --apply snapshots the DB first and writes the exact links,
# --reviewed <tsv> adds the heuristic rows a human kept. See "Photo versions".
cargo run --bin link_versions
```
## Architecture Overview
@@ -351,6 +356,69 @@ the new hash, and the two histories cleanly split. Apollo's
`derived.db` cache will need an invalidation hook for the changed
hash — design it alongside Branch D.
### Photo versions
A **version** is a non-destructive restoration, upscale or edit of a photo that
an external process (the ComfyUI node in `comfyui-imageapi-versions`) writes next
to the original, as `<dir>/aiedits/<original name>.<kind>[.<n>].png`. ImageApi
only detects it (library roots stay read-only) and shows original + versions as
**one photo**; a *global* choice (not per user) says which file is displayed.
Code: `src/database/version_dao.rs` (tables, DAO, SQL predicates),
`src/versions/` (linking, endpoints, `xmp.rs` PNG chunk reader, `backfill.rs`
for the `link_versions` binary).
- **The link is the original's `content_hash`** (`photo_versions.version_hash ->
original_hash`, plus `kind`, `source` and a `(library_id, rel_path)` back-ref),
so it survives renames, moves and the archive handoff. A file is a version of at
most one original; chains flatten to the root. Reference photos (extra context fed
to the model) go to `photo_version_references` as provenance only and are never
hidden or grouped. `photo_version_display(original_hash -> display_hash)` is the
global choice; no row = the original, and a display file with no live row reads as
the original.
- **Linking at ingest** (`watcher::process_new_files`, before any derived work):
(1) a PNG's XMP `xmpMM:DerivedFrom/stRef:documentID = blake3:<hex>` (iTXt chunk
`XML:com.adobe.xmp`, read by hand, no decode); (2) the file-name convention next
to an indexed original in the same library (same dir, or the parent of
`aiedits/`). Name links are retried every tick (`sweep_name_links`), so an
original that wasn't hashed yet, or arrives later, still links. Only *new* rows
are examined, so files ingested before this existed need the `link_versions`
binary once.
- **Date/GPS/camera copy**: a version row takes its original's date triple, GPS and
camera fields with `date_taken_source = 'version_of'`, re-copied each tick
(`sync_version_metadata`) when the original changes (manual override, re-resolve).
Only a *converted* original (`date_utc` set) is copied, so the legacy-convention
date drains never fight the copy. Memories therefore put a standalone version on
the original's day.
- **Hiding** uses the same reachability rule as `duplicate_of_hash`, with one
addition: a version row is hidden only while a row for its original exists *in
the current view* (`visible_version_clause`; unscoped = any library). A version
whose original was deleted, or lives outside the requested library, shows as a
normal photo, so nothing disappears silently. Applied in `/photos` (SQL for the
recursive listing, `hidden_version_rows` for the folder walk and tag branch),
`/memories` and reels, `/photos/exif`, `/photos/gps-summary`, unified search, the
camera counts and the media gauges; `include_versions=true` lists them.
- **Tags and favorites** written or read on a version path bind to the original
(`canonical_rel_path` in the tag/favorite DAOs, `get_rel_paths_sharing_content`
for reads). Insight generation on a version path runs against the original. A
*manual date* set on a version path is overwritten by the copy above: send the
original's path.
- **Excluded from derived work** (`NOT_A_VERSION_CLAUSE`): face backlog and ingest
candidates, face stats (numerator and denominator), CLIP backlog and index, and
exact/perceptual duplicate grouping. Thumbnails and previews *are* generated.
Linking invalidates the duplicate caches.
- **Lifecycle**: version file deleted -> the missing-file scan retires its row and
the link plus any display choice naming it become orphan-GC candidates (two-tick,
all-libraries-online), reads fall back to the original meanwhile. Original
deleted -> versions surface as standalone photos and the links are kept (they
revive if the original returns). The back-ref refresh repoints
`photo_versions.(library_id, rel_path)` after a move.
- **`link_versions`** (`src/bin/link_versions.rs`): dry run by default, prints a
TSV. `exact` rows (XMP, file name) are what ingest would have linked and apply
with `--apply`; `review` rows (ComfyUI prompt `LoadImage` input, Upscayl-style
names, nearest pHash) apply only from a reviewed TSV (`--reviewed`), each checked
against the current hash; `unresolved` rows are listed, never guessed. `--apply`
takes a `VACUUM INTO` snapshot first.
### File Processing Pipeline
**Thumbnail Generation:**
@@ -554,6 +622,16 @@ POST /image/tags (add tag to file)
DELETE /image/tags (remove tag from file)
POST /image/tags/batch (bulk tag updates)
// Photo versions (#191): AI restorations / upscales / edits shown as ONE photo
GET /image/versions?path=...&library=... (path of the original or any version ->
{ original:{path,library_id,hash}, versions:[{path,
library_id,hash,kind,source,created_at,references,
width,height}], display_hash })
PUT /image/versions/display (body { path, display_hash|null, library? }; global choice)
// /photos hides versions behind their original (include_versions=true lists them) and adds
// parallel arrays photo_display_paths / photo_version_counts; memories items, search hits and
// /photos/exif rows carry display_path / version_count. All skipped when there are none.
// Memories (week-based grouping)
GET /memories?path=...&recursive=true