feat(versions): e2e fixtures, check_versions, docs
Fixtures gain an XMP-linked version (renamed file), a name-linked version and an edit with a reference photo; e2e.sh counts with include_versions and runs check_versions.py. CLAUDE.md documents the feature. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
fa3554c411
commit
6135034d09
5 files changed
+394
-6
No files matched your search
@@ -77,6 +77,11 @@ cargo run --bin cleanup_files -- --base-path /path/to/media --database-url ./dat
|
||||
|
||||
# Create a login user (no HTTP registration route; idempotent)
|
||||
cargo run --bin create_user -- <username> <password>
|
||||
|
||||
# Link photo versions ingested before linking existed. Dry run by default
|
||||
# (prints a TSV); --apply snapshots the DB first and writes the exact links,
|
||||
# --reviewed <tsv> adds the heuristic rows a human kept. See "Photo versions".
|
||||
cargo run --bin link_versions
|
||||
```
|
||||
|
||||
## Architecture Overview
|
||||
@@ -351,6 +356,69 @@ the new hash, and the two histories cleanly split. Apollo's
|
||||
`derived.db` cache will need an invalidation hook for the changed
|
||||
hash — design it alongside Branch D.
|
||||
|
||||
### Photo versions
|
||||
|
||||
A **version** is a non-destructive restoration, upscale or edit of a photo that
|
||||
an external process (the ComfyUI node in `comfyui-imageapi-versions`) writes next
|
||||
to the original, as `<dir>/aiedits/<original name>.<kind>[.<n>].png`. ImageApi
|
||||
only detects it (library roots stay read-only) and shows original + versions as
|
||||
**one photo**; a *global* choice (not per user) says which file is displayed.
|
||||
Code: `src/database/version_dao.rs` (tables, DAO, SQL predicates),
|
||||
`src/versions/` (linking, endpoints, `xmp.rs` PNG chunk reader, `backfill.rs`
|
||||
for the `link_versions` binary).
|
||||
|
||||
- **The link is the original's `content_hash`** (`photo_versions.version_hash ->
|
||||
original_hash`, plus `kind`, `source` and a `(library_id, rel_path)` back-ref),
|
||||
so it survives renames, moves and the archive handoff. A file is a version of at
|
||||
most one original; chains flatten to the root. Reference photos (extra context fed
|
||||
to the model) go to `photo_version_references` as provenance only and are never
|
||||
hidden or grouped. `photo_version_display(original_hash -> display_hash)` is the
|
||||
global choice; no row = the original, and a display file with no live row reads as
|
||||
the original.
|
||||
- **Linking at ingest** (`watcher::process_new_files`, before any derived work):
|
||||
(1) a PNG's XMP `xmpMM:DerivedFrom/stRef:documentID = blake3:<hex>` (iTXt chunk
|
||||
`XML:com.adobe.xmp`, read by hand, no decode); (2) the file-name convention next
|
||||
to an indexed original in the same library (same dir, or the parent of
|
||||
`aiedits/`). Name links are retried every tick (`sweep_name_links`), so an
|
||||
original that wasn't hashed yet, or arrives later, still links. Only *new* rows
|
||||
are examined, so files ingested before this existed need the `link_versions`
|
||||
binary once.
|
||||
- **Date/GPS/camera copy**: a version row takes its original's date triple, GPS and
|
||||
camera fields with `date_taken_source = 'version_of'`, re-copied each tick
|
||||
(`sync_version_metadata`) when the original changes (manual override, re-resolve).
|
||||
Only a *converted* original (`date_utc` set) is copied, so the legacy-convention
|
||||
date drains never fight the copy. Memories therefore put a standalone version on
|
||||
the original's day.
|
||||
- **Hiding** uses the same reachability rule as `duplicate_of_hash`, with one
|
||||
addition: a version row is hidden only while a row for its original exists *in
|
||||
the current view* (`visible_version_clause`; unscoped = any library). A version
|
||||
whose original was deleted, or lives outside the requested library, shows as a
|
||||
normal photo, so nothing disappears silently. Applied in `/photos` (SQL for the
|
||||
recursive listing, `hidden_version_rows` for the folder walk and tag branch),
|
||||
`/memories` and reels, `/photos/exif`, `/photos/gps-summary`, unified search, the
|
||||
camera counts and the media gauges; `include_versions=true` lists them.
|
||||
- **Tags and favorites** written or read on a version path bind to the original
|
||||
(`canonical_rel_path` in the tag/favorite DAOs, `get_rel_paths_sharing_content`
|
||||
for reads). Insight generation on a version path runs against the original. A
|
||||
*manual date* set on a version path is overwritten by the copy above: send the
|
||||
original's path.
|
||||
- **Excluded from derived work** (`NOT_A_VERSION_CLAUSE`): face backlog and ingest
|
||||
candidates, face stats (numerator and denominator), CLIP backlog and index, and
|
||||
exact/perceptual duplicate grouping. Thumbnails and previews *are* generated.
|
||||
Linking invalidates the duplicate caches.
|
||||
- **Lifecycle**: version file deleted -> the missing-file scan retires its row and
|
||||
the link plus any display choice naming it become orphan-GC candidates (two-tick,
|
||||
all-libraries-online), reads fall back to the original meanwhile. Original
|
||||
deleted -> versions surface as standalone photos and the links are kept (they
|
||||
revive if the original returns). The back-ref refresh repoints
|
||||
`photo_versions.(library_id, rel_path)` after a move.
|
||||
- **`link_versions`** (`src/bin/link_versions.rs`): dry run by default, prints a
|
||||
TSV. `exact` rows (XMP, file name) are what ingest would have linked and apply
|
||||
with `--apply`; `review` rows (ComfyUI prompt `LoadImage` input, Upscayl-style
|
||||
names, nearest pHash) apply only from a reviewed TSV (`--reviewed`), each checked
|
||||
against the current hash; `unresolved` rows are listed, never guessed. `--apply`
|
||||
takes a `VACUUM INTO` snapshot first.
|
||||
|
||||
### File Processing Pipeline
|
||||
|
||||
**Thumbnail Generation:**
|
||||
@@ -554,6 +622,16 @@ POST /image/tags (add tag to file)
|
||||
DELETE /image/tags (remove tag from file)
|
||||
POST /image/tags/batch (bulk tag updates)
|
||||
|
||||
// Photo versions (#191): AI restorations / upscales / edits shown as ONE photo
|
||||
GET /image/versions?path=...&library=... (path of the original or any version ->
|
||||
{ original:{path,library_id,hash}, versions:[{path,
|
||||
library_id,hash,kind,source,created_at,references,
|
||||
width,height}], display_hash })
|
||||
PUT /image/versions/display (body { path, display_hash|null, library? }; global choice)
|
||||
// /photos hides versions behind their original (include_versions=true lists them) and adds
|
||||
// parallel arrays photo_display_paths / photo_version_counts; memories items, search hits and
|
||||
// /photos/exif rows carry display_path / version_count. All skipped when there are none.
|
||||
|
||||
// Memories (week-based grouping)
|
||||
GET /memories?path=...&recursive=true
|
||||
|
||||
|
||||
Reference in new issue
Block a user