Photo versions (#191): AI restorations / upscales / edits are linked to their original by content hash and shown as one photo, with a global display choice. No client UI.
What was built
Schema: photo_versions (hash link + back-ref), photo_version_references (provenance only), photo_version_display (global choice). VersionDao in src/database/version_dao.rs; chains flatten to the root.
Ingest: XMP DerivedFrom blake3 from the PNG iTXt chunk (hand-rolled chunk reader, no decode, survives renames/moves), then the <name>.<kind>[.<n>].<ext> convention (same dir or parent of aiedits/; RAW+JPEG pairs resolve by full original name). Name links retry every tick, so unhashed/late originals still link. The original's date/GPS/camera are copied with date_taken_source='version_of' and re-copied each tick when they change.
Hiding: same reachability rule as duplicate_of_hash, applied to /photos (recursive, folder walk, tag branch), /memories + reels, /photos/exif, gps-summary, unified search, camera counts and media gauges. include_versions=true lists them.
Responses: /photos gets parallel arrays photo_display_paths / photo_version_counts; memories items, search hits and /photos/exif rows get display_path / version_count (flattened). All skipped when there are no versions (byte-identical).
Endpoints: GET /image/versions, PUT /image/versions/display. library is optional on both (query or body); without it the earliest hashed row for the path is used and resolved to its original.
Tags/favorites on a version path bind to the original. Excluded for versions: face backlog/ingest/stats, CLIP backlog/index, exact+perceptual duplicates; insight generation redirects to the original.
Lifecycle: orphan GC and back-ref refresh cover the new tables; a deleted version falls back to the original, a deleted original surfaces the versions (links kept).
link_versions binary: dry run by default (TSV). XMP/name rows are exact and apply with --apply (WAL-safe VACUUM INTO snapshot first); heuristic rows (ComfyUI LoadImage input, Upscayl names, nearest pHash) apply only from a reviewed TSV via --reviewed, checked against current hashes; unresolved cases are listed, not guessed.
e2e: fixtures gain an XMP-linked version (renamed file), a name-linked version and an edit with a reference photo; e2e.sh check runs a new check_versions.py.
Deviations from the ticket
Unscoped views hide a version only if its original has a row somewhere (duplicates assume the survivor always exists), so a version whose original was deleted is never hidden.
Only a converted original (date_utc set) is copied onto versions, so the legacy date drains never fight the copy.
--apply <tsv> became --apply (exact links) plus --reviewed <tsv> (heuristic rows).
references in /image/versions are objects {hash, path, library_id} (path null if not ingested).
Optional POST/DELETE /image/versions/link not implemented (an unlink would be re-linked by the name sweep without a tombstone).
A manual date set on a version path is overwritten by the copy; clients should target the original's path (what /photos returns).
Display PUT invalidates nothing: there is no server-side listing cache; precomputed reels are not display-aware.
Files ingested before this ships need link_versions --apply once (the watcher only examines new rows; name-convention rows are picked up by the sweep anyway).
Testing
cargo fmt, cargo clippy --all-targets clean; DATABASE_URL=":memory:" cargo test 974 passed (new: DAO, XMP parser incl. a real producer packet, ingest linking, listings hide/unhide, endpoints, tag/favorite binding, GC/back-ref, backfill planner). dev/e2e/e2e.sh check passes against a build of this branch (104/104 wall clocks, 3/3 version fixtures). link_versions dry run and --apply --reviewed exercised on a copy of the production DB only.
Closes #191 is withheld only if you consider the optional link/unlink endpoints required; every acceptance criterion is otherwise met.
Photo versions (#191): AI restorations / upscales / edits are linked to their original by content hash and shown as one photo, with a global display choice. No client UI.
## What was built
- **Schema**: `photo_versions` (hash link + back-ref), `photo_version_references` (provenance only), `photo_version_display` (global choice). `VersionDao` in `src/database/version_dao.rs`; chains flatten to the root.
- **Ingest**: XMP `DerivedFrom` blake3 from the PNG iTXt chunk (hand-rolled chunk reader, no decode, survives renames/moves), then the `<name>.<kind>[.<n>].<ext>` convention (same dir or parent of `aiedits/`; RAW+JPEG pairs resolve by full original name). Name links retry every tick, so unhashed/late originals still link. The original's date/GPS/camera are copied with `date_taken_source='version_of'` and re-copied each tick when they change.
- **Hiding**: same reachability rule as `duplicate_of_hash`, applied to /photos (recursive, folder walk, tag branch), /memories + reels, /photos/exif, gps-summary, unified search, camera counts and media gauges. `include_versions=true` lists them.
- **Responses**: /photos gets parallel arrays `photo_display_paths` / `photo_version_counts`; memories items, search hits and `/photos/exif` rows get `display_path` / `version_count` (flattened). All skipped when there are no versions (byte-identical).
- **Endpoints**: `GET /image/versions`, `PUT /image/versions/display`. `library` is optional on both (query or body); without it the earliest hashed row for the path is used and resolved to its original.
- **Tags/favorites** on a version path bind to the original. **Excluded** for versions: face backlog/ingest/stats, CLIP backlog/index, exact+perceptual duplicates; insight generation redirects to the original.
- **Lifecycle**: orphan GC and back-ref refresh cover the new tables; a deleted version falls back to the original, a deleted original surfaces the versions (links kept).
- **`link_versions`** binary: dry run by default (TSV). XMP/name rows are exact and apply with `--apply` (WAL-safe `VACUUM INTO` snapshot first); heuristic rows (ComfyUI `LoadImage` input, Upscayl names, nearest pHash) apply only from a reviewed TSV via `--reviewed`, checked against current hashes; unresolved cases are listed, not guessed.
- **e2e**: fixtures gain an XMP-linked version (renamed file), a name-linked version and an edit with a reference photo; `e2e.sh check` runs a new `check_versions.py`.
## Deviations from the ticket
- Unscoped views hide a version only if its original has a row somewhere (duplicates assume the survivor always exists), so a version whose original was deleted is never hidden.
- Only a *converted* original (`date_utc` set) is copied onto versions, so the legacy date drains never fight the copy.
- `--apply <tsv>` became `--apply` (exact links) plus `--reviewed <tsv>` (heuristic rows).
- `references` in `/image/versions` are objects `{hash, path, library_id}` (path null if not ingested).
- Optional POST/DELETE `/image/versions/link` not implemented (an unlink would be re-linked by the name sweep without a tombstone).
- A manual date set on a version path is overwritten by the copy; clients should target the original's path (what /photos returns).
- Display PUT invalidates nothing: there is no server-side listing cache; precomputed reels are not display-aware.
- Files ingested before this ships need `link_versions --apply` once (the watcher only examines new rows; name-convention rows are picked up by the sweep anyway).
## Testing
`cargo fmt`, `cargo clippy --all-targets` clean; `DATABASE_URL=":memory:" cargo test` 974 passed (new: DAO, XMP parser incl. a real producer packet, ingest linking, listings hide/unhide, endpoints, tag/favorite binding, GC/back-ref, backfill planner). `dev/e2e/e2e.sh check` passes against a build of this branch (104/104 wall clocks, 3/3 version fixtures). `link_versions` dry run and `--apply --reviewed` exercised on a copy of the production DB only.
`Closes #191` is withheld only if you consider the optional link/unlink endpoints required; every acceptance criterion is otherwise met.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
photo_versions links a version's content_hash to its primary original's;
photo_version_references keeps provenance-only reference photos;
photo_version_display holds the global display choice. VersionDao covers
linking (chains flatten to the root), the display choice, per-original
summaries for /photos, the hidden-row set for non-SQL listings, and the
copy of the original's date/GPS/camera onto version rows. Also the SQL
predicate listings use to hide a version while its original is reachable.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
XMP (PNG iTXt DerivedFrom, native chunk reader) then file-name convention
links at ingest; unlinked name candidates are retried each tick. Versions
are hidden from /photos (SQL for the recursive listing, hidden-row set for
the walk and tag branches), /memories, reels, /photos/exif, the GPS summary,
unified search and the media gauges while their original is reachable in the
view; include_versions=true lists them. /photos gains the parallel arrays
photo_display_paths / photo_version_counts, memories / search hits /
/photos/exif rows gain display_path / version_count, all skipped when empty.
GET /image/versions and PUT /image/versions/display (library optional).
Tags and favorites on a version path bind to the original; faces, CLIP and
duplicate grouping skip version hashes; insight generation redirects to the
original. Orphan GC and the back-ref refresh cover the new tables.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dry run by default: prints a TSV of proposed links. XMP (PNG DerivedFrom,
incl. files already ingested) and file-name links are exact and apply
without review; ComfyUI prompt LoadImage inputs, Upscayl-style names and
nearest pHash are heuristic and apply only from a reviewed TSV, checked
against current hashes. Unresolvable cases are listed, not guessed. --apply
takes a WAL-safe VACUUM INTO snapshot first and aborts if it fails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixtures gain an XMP-linked version (renamed file), a name-linked version
and an edit with a reference photo; e2e.sh counts with include_versions and
runs check_versions.py. CLAUDE.md documents the feature.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fixes pushed: (1) versions surface standalone when their original is demoted by duplicates resolve (scoped survivor rule); (2) recording a link deletes the version's face_detections, clears clip_embedding/clip_model_version and clip_failures (DAO-level, so ingest and link_versions --apply both covered); (3) display file is chosen from the original's library only, else the original shows, and PUT /image/versions/display returns 400 if the hash has no row there (wire format unchanged); (4) link_new_files chunks get_exif_batch at 500 and warns on errors. Tests added for each.
Review fixes pushed: (1) versions surface standalone when their original is demoted by duplicates resolve (scoped survivor rule); (2) recording a link deletes the version's face_detections, clears clip_embedding/clip_model_version and clip_failures (DAO-level, so ingest and link_versions --apply both covered); (3) display file is chosen from the original's library only, else the original shows, and PUT /image/versions/display returns 400 if the hash has no row there (wire format unchanged); (4) link_new_files chunks get_exif_batch at 500 and warns on errors. Tests added for each.
- A version no longer hides behind an original that duplicates resolve
demoted (visible_version_clause, hidden_version_rows).
- Recording a link drops the version's faces, CLIP data and clip failures
(covers link-at-ingest and link_versions --apply).
- The display file must live in the original's library; otherwise the
original shows, and PUT /image/versions/display answers 400.
- link_new_files looks up hashes in chunks of 500 and logs errors.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cameron
merged commit 4105430681 into master2026-10-30 18:36:03 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Photo versions (#191): AI restorations / upscales / edits are linked to their original by content hash and shown as one photo, with a global display choice. No client UI.
What was built
photo_versions(hash link + back-ref),photo_version_references(provenance only),photo_version_display(global choice).VersionDaoinsrc/database/version_dao.rs; chains flatten to the root.DerivedFromblake3 from the PNG iTXt chunk (hand-rolled chunk reader, no decode, survives renames/moves), then the<name>.<kind>[.<n>].<ext>convention (same dir or parent ofaiedits/; RAW+JPEG pairs resolve by full original name). Name links retry every tick, so unhashed/late originals still link. The original's date/GPS/camera are copied withdate_taken_source='version_of'and re-copied each tick when they change.duplicate_of_hash, applied to /photos (recursive, folder walk, tag branch), /memories + reels, /photos/exif, gps-summary, unified search, camera counts and media gauges.include_versions=truelists them.photo_display_paths/photo_version_counts; memories items, search hits and/photos/exifrows getdisplay_path/version_count(flattened). All skipped when there are no versions (byte-identical).GET /image/versions,PUT /image/versions/display.libraryis optional on both (query or body); without it the earliest hashed row for the path is used and resolved to its original.link_versionsbinary: dry run by default (TSV). XMP/name rows are exact and apply with--apply(WAL-safeVACUUM INTOsnapshot first); heuristic rows (ComfyUILoadImageinput, Upscayl names, nearest pHash) apply only from a reviewed TSV via--reviewed, checked against current hashes; unresolved cases are listed, not guessed.e2e.sh checkruns a newcheck_versions.py.Deviations from the ticket
date_utcset) is copied onto versions, so the legacy date drains never fight the copy.--apply <tsv>became--apply(exact links) plus--reviewed <tsv>(heuristic rows).referencesin/image/versionsare objects{hash, path, library_id}(path null if not ingested)./image/versions/linknot implemented (an unlink would be re-linked by the name sweep without a tombstone).link_versions --applyonce (the watcher only examines new rows; name-convention rows are picked up by the sweep anyway).Testing
cargo fmt,cargo clippy --all-targetsclean;DATABASE_URL=":memory:" cargo test974 passed (new: DAO, XMP parser incl. a real producer packet, ingest linking, listings hide/unhide, endpoints, tag/favorite binding, GC/back-ref, backfill planner).dev/e2e/e2e.sh checkpasses against a build of this branch (104/104 wall clocks, 3/3 version fixtures).link_versionsdry run and--apply --reviewedexercised on a copy of the production DB only.Closes #191is withheld only if you consider the optional link/unlink endpoints required; every acceptance criterion is otherwise met.🤖 Generated with Claude Code
Review fixes pushed: (1) versions surface standalone when their original is demoted by duplicates resolve (scoped survivor rule); (2) recording a link deletes the version's face_detections, clears clip_embedding/clip_model_version and clip_failures (DAO-level, so ingest and link_versions --apply both covered); (3) display file is chosen from the original's library only, else the original shows, and PUT /image/versions/display returns 400 if the hash has no row there (wire format unchanged); (4) link_new_files chunks get_exif_batch at 500 and warns on errors. Tests added for each.