Photo versions: link AI restorations/edits to their original, one photo with a global display choice #191

Closed
opened 2026-10-30 13:15:11 +00:00 by cameron · 7 comments
Owner

Non-destructive versions of a photo: an AI restoration, upscale or edit sits on disk near the original, and ImageApi shows the two as one photo. A global choice sets which file is displayed. The original is never touched, and every version can still be downloaded on its own.

Think Lightroom virtual copies, except an external process (a ComfyUI workflow) writes the copy into the library. ImageApi only detects it, so the read-only rule for library roots holds.

Decisions so far

  • The link is the content hash. A version records the blake3 content_hash of its primary original. File name and location are hints only. ComfyUI writes through a Windows UNC share and ImageApi reads /media/nas/..., so a path-based link would break.
  • One primary original per version. Edits often feed the model a second photo as context or another view of the subject. That photo is a reference: stored as provenance, never treated as the version's original, never hidden or grouped because of the version.
  • The display choice is global, not per user.
  • Versions download on their own. A version keeps a normal image_exif row and path, so /image?path=<version> keeps working.
  • Model-derived data comes from the original only. Faces, CLIP and insights are not computed on versions, so invented detail can't pollute person clusters or search.
  • Not a #175 stack. Burst stacks group different photos, each with its own data. A version set is one photo. The models stay separate; clients may reuse the stacked-tile UI.

Producer contract (ComfyUI node, already written)

A custom node pair (comfyui-imageapi-versions) loads the original by path and saves the result as:

<dir>/aiedits/<original file name>.<kind>[.<n>].png
e.g. 2004/aiedits/scan_0042.jpg.restored.png, …restored.2.png

XMP is embedded in the PNG iTXt chunk XML:com.adobe.xmp:

Field Value
xmpMM:DerivedFrom/stRef:documentID blake3:<64 hex>: the original's content_hash (the link)
xmpMM:DerivedFrom/stRef:filePath the original's base name (hint)
xmpMM:Ingredients[]/stRef:documentID blake3:<hex> per reference photo
imageapi:versionKind (ns https://git.crkc.xyz/Apps/ImageApi/ns/versions/1.0/) restored / upscale / edit

The node's hash was checked against image_exif.content_hash for real archive-library files, and they match. ComfyUI's own prompt / workflow text chunks are kept too. They record the model and settings used, which could later feed a version label.

Schema (proposed)

photo_versions
  version_hash   TEXT PK          -- a file is a version of at most one original
  original_hash  TEXT NOT NULL, indexed
  kind           TEXT NOT NULL    -- restored | upscale | edit
  source         TEXT NOT NULL    -- xmp | name | backfill | api
  created_at     BIGINT NOT NULL
  -- (library_id, rel_path) back-ref, same pattern as face_detections

photo_version_references
  version_hash, reference_hash    -- provenance only

photo_version_display
  original_hash  TEXT PK
  display_hash   TEXT NOT NULL
  decided_at     BIGINT NOT NULL

Everything is keyed on hashes, following the multi-library model in CLAUDE.md, so links survive archive handoff, moves and renames.

Ingest (watcher.rs::process_new_files)

For each new image file, in order:

  1. XMP link. If the file has an XML:com.adobe.xmp iTXt chunk with xmpMM:DerivedFrom/stRef:documentID = blake3:…, link to that hash. Read the chunk natively (the png crate is already pulled in by image), with no exiftool call. Only PNGs for now; JPEG/WebP XMP can follow if a producer ever writes them.
  2. Name link. If the name matches <name>.<kind>[.<n>].<ext> with a known kind, look for <name> in the same directory or the parent of an aiedits/ directory, in the same library, and link to that row's hash. If that row has no hash yet (transitional state), retry on a later tick rather than linking by path.
  3. Otherwise it's an ordinary photo.

On link, copy the original's date_taken / date_utc / offset / GPS / camera fields onto the version row with a new date_taken_source = 'version_of'. Re-copy when the original's date or GPS changes (manual override, re-resolve). Without this, versions get fs_time dates; all the existing test edits in the library show this today.

Hiding and reads

  • Hide a version row from listings only while its original is reachable in the current view. This is the same rule as duplicate_of_hash in database/mod.rs (~2064–2096): unscoped view → hide; library-scoped view → hide only if the original has a row in a visible library. A version whose original was deleted, or lives in a library outside the view, shows up as a normal photo, so nothing disappears silently and no separate promotion step is needed.
  • Apply this in /photos, /memories, /photos/search (CLIP), unified search, the GPS summary, reels, and media-count gauges. Add include_versions=true, the counterpart of include_duplicates.
  • /photos response: two new parallel arrays next to photo_libraries, both skipped when empty so older clients are unaffected:
    • photo_display_paths: Vec<Option<String>> is set when the displayed file isn't the original.
    • photo_version_counts: Vec<u32>.
      photos[i] stays the original's path. It anchors tags, sort order and dates, and old clients show the original, which is the safe default.
  • Tags and favorites written or read against a version path bind to the original's hash.

New endpoints

  • GET /image/versions?path= (original or any version path) returns the original (path, library, hash), the versions (path, library, hash, kind, source, created_at, references, width/height), and display_hash.
  • PUT /image/versions/display with { path, display_hash | null } sets or clears the global display choice. Invalidate whatever caches listing output.
  • (Optional) POST /image/versions/link and DELETE /image/versions/link for manual linking and unlinking from clients.

Excluded from derived work

  • Face and CLIP backlogs (face_watch.rs, clip_watch.rs) skip rows whose hash is in photo_versions.version_hash.
  • Insight generation runs against the original.
  • Duplicate detection (duplicates.rs): version rows don't take part in exact or perceptual groups. Otherwise every restore shows up as a perceptual duplicate, and a resolve would soft-mark it. Invalidate the perceptual cache when a link is created.
  • Thumbnails and previews are generated for versions, because clients render them.

Lifecycle

  • Version file deleted. The missing-file scan retires its row. The photo_versions and display rows for that hash become orphan-GC candidates under the existing two-tick, all-libraries-online consensus. Until GC runs, reads fall back to the original if display_hash has no live row.
  • Original deleted. Versions become visible as standalone photos (see the hiding rule) and keep the copied date and GPS.
  • Moves, renames, archive handoff. No work needed, since the links are hash-keyed. The back-ref refresh pass in library_maintenance.rs should cover the photo_versions back-ref like it does for faces and tags.

Backfill: src/bin/link_versions

There are existing edits from testing: about 13 ComfyUI outputs under an aiedits/ folder (default <prefix>_NNNNN_.png names), one beside its original, plus a couple of older Upscayl outputs (<name>-4x.png, <name>_upscayl_…png). None have XMP.

  • Default is a dry run that prints a TSV of proposed links: version path, proposed original, method, pHash distance.
    • Methods, in order: the ComfyUI prompt text chunk's LoadImage / LoadImageWithName image input (a base name, resolved within the same library, preferring the parent of aiedits/); a filename prefix (Upscayl-style suffixes); the nearest pHash in the same or parent directory.
    • Outputs with two loader inputs (primary + reference) choose the primary by nearest pHash, record the other as a reference, and are flagged for review.
    • Unresolvable cases (e.g. a loader name that doesn't match any file) are listed, not guessed.
  • --apply <reviewed.tsv> writes the reviewed rows only, after a WAL-safe .backup, proven on a copy first.

Acceptance criteria

  • A PNG saved by the ComfyUI node is linked at ingest via XMP, and a renamed or moved copy of it is still linked.
  • A name-convention file without XMP is linked to its sibling original; a RAW+JPEG pair links to the right one.
  • Versions are hidden from /photos, /memories, search and reels while their original is visible, and visible once it isn't.
  • /photos returns photo_display_paths and photo_version_counts; responses with no versions are byte-identical to today.
  • /image/versions and PUT /image/versions/display work when called with either the original's or a version's path.
  • A version takes its date and GPS from the original; Memories puts it on the original's day.
  • Faces, CLIP, insights and duplicate grouping never see version rows.
  • /image?path=<version> serves the full file.
  • Deleting the version file falls back to the original; deleting the original surfaces the version.
  • The link_versions dry run lists the existing test edits with their proposed originals; --apply is gated as described.
  • The e2e fixture library (dev/e2e/gen_fixtures.py) gains an original with an XMP-linked version, a name-linked version and a reference-photo case.

Out of scope

  • Client UI (follow-up tickets in FileViewer-React and Apollo).
  • ImageApi starting restores itself (submitting ComfyUI jobs). That would need writable storage outside library roots.
  • Video versions. The schema allows them, but nothing produces them yet.
  • In-place edits (Branch D in CLAUDE.md), the destructive counterpart of this feature.
Non-destructive **versions** of a photo: an AI restoration, upscale or edit sits on disk near the original, and ImageApi shows the two as **one photo**. A global choice sets which file is displayed. The original is never touched, and every version can still be downloaded on its own. Think Lightroom virtual copies, except an external process (a ComfyUI workflow) writes the copy into the library. ImageApi only detects it, so the read-only rule for library roots holds. ## Decisions so far - **The link is the content hash.** A version records the blake3 `content_hash` of its *primary* original. File name and location are hints only. ComfyUI writes through a Windows UNC share and ImageApi reads `/media/nas/...`, so a path-based link would break. - **One primary original per version.** Edits often feed the model a second photo as context or another view of the subject. That photo is a **reference**: stored as provenance, never treated as the version's original, never hidden or grouped because of the version. - **The display choice is global**, not per user. - **Versions download on their own.** A version keeps a normal `image_exif` row and path, so `/image?path=<version>` keeps working. - **Model-derived data comes from the original only.** Faces, CLIP and insights are not computed on versions, so invented detail can't pollute person clusters or search. - **Not a #175 stack.** Burst stacks group *different* photos, each with its own data. A version set is *one* photo. The models stay separate; clients may reuse the stacked-tile UI. ## Producer contract (ComfyUI node, already written) A custom node pair (`comfyui-imageapi-versions`) loads the original by path and saves the result as: ``` <dir>/aiedits/<original file name>.<kind>[.<n>].png e.g. 2004/aiedits/scan_0042.jpg.restored.png, …restored.2.png ``` XMP is embedded in the PNG iTXt chunk `XML:com.adobe.xmp`: | Field | Value | |---|---| | `xmpMM:DerivedFrom/stRef:documentID` | `blake3:<64 hex>`: the original's `content_hash` (**the link**) | | `xmpMM:DerivedFrom/stRef:filePath` | the original's base name (hint) | | `xmpMM:Ingredients[]/stRef:documentID` | `blake3:<hex>` per reference photo | | `imageapi:versionKind` (ns `https://git.crkc.xyz/Apps/ImageApi/ns/versions/1.0/`) | `restored` / `upscale` / `edit` | The node's hash was checked against `image_exif.content_hash` for real archive-library files, and they match. ComfyUI's own `prompt` / `workflow` text chunks are kept too. They record the model and settings used, which could later feed a version label. ## Schema (proposed) ``` photo_versions version_hash TEXT PK -- a file is a version of at most one original original_hash TEXT NOT NULL, indexed kind TEXT NOT NULL -- restored | upscale | edit source TEXT NOT NULL -- xmp | name | backfill | api created_at BIGINT NOT NULL -- (library_id, rel_path) back-ref, same pattern as face_detections photo_version_references version_hash, reference_hash -- provenance only photo_version_display original_hash TEXT PK display_hash TEXT NOT NULL decided_at BIGINT NOT NULL ``` Everything is keyed on hashes, following the multi-library model in CLAUDE.md, so links survive archive handoff, moves and renames. ## Ingest (`watcher.rs::process_new_files`) For each new image file, in order: 1. **XMP link.** If the file has an `XML:com.adobe.xmp` iTXt chunk with `xmpMM:DerivedFrom/stRef:documentID = blake3:…`, link to that hash. Read the chunk natively (the `png` crate is already pulled in by `image`), with no exiftool call. Only PNGs for now; JPEG/WebP XMP can follow if a producer ever writes them. 2. **Name link.** If the name matches `<name>.<kind>[.<n>].<ext>` with a known kind, look for `<name>` in the same directory or the parent of an `aiedits/` directory, in the same library, and link to that row's hash. If that row has no hash yet (transitional state), retry on a later tick rather than linking by path. 3. Otherwise it's an ordinary photo. On link, copy the original's `date_taken` / `date_utc` / offset / GPS / camera fields onto the version row with a new `date_taken_source = 'version_of'`. Re-copy when the original's date or GPS changes (manual override, re-resolve). Without this, versions get `fs_time` dates; all the existing test edits in the library show this today. ## Hiding and reads - **Hide a version row from listings only while its original is reachable in the current view.** This is the same rule as `duplicate_of_hash` in `database/mod.rs` (~2064–2096): unscoped view → hide; library-scoped view → hide only if the original has a row in a visible library. A version whose original was deleted, or lives in a library outside the view, shows up as a normal photo, so nothing disappears silently and no separate promotion step is needed. - Apply this in `/photos`, `/memories`, `/photos/search` (CLIP), unified search, the GPS summary, reels, and media-count gauges. Add `include_versions=true`, the counterpart of `include_duplicates`. - **`/photos` response:** two new parallel arrays next to `photo_libraries`, both skipped when empty so older clients are unaffected: - `photo_display_paths: Vec<Option<String>>` is set when the displayed file isn't the original. - `photo_version_counts: Vec<u32>`. `photos[i]` stays the **original's** path. It anchors tags, sort order and dates, and old clients show the original, which is the safe default. - **Tags and favorites** written or read against a version path bind to the original's hash. ## New endpoints - `GET /image/versions?path=` (original or any version path) returns the original (path, library, hash), the versions (path, library, hash, kind, source, created_at, references, width/height), and `display_hash`. - `PUT /image/versions/display` with `{ path, display_hash | null }` sets or clears the global display choice. Invalidate whatever caches listing output. - (Optional) `POST /image/versions/link` and `DELETE /image/versions/link` for manual linking and unlinking from clients. ## Excluded from derived work - Face and CLIP backlogs (`face_watch.rs`, `clip_watch.rs`) skip rows whose hash is in `photo_versions.version_hash`. - Insight generation runs against the original. - Duplicate detection (`duplicates.rs`): version rows don't take part in exact or perceptual groups. Otherwise every restore shows up as a perceptual duplicate, and a resolve would soft-mark it. Invalidate the perceptual cache when a link is created. - Thumbnails and previews **are** generated for versions, because clients render them. ## Lifecycle - **Version file deleted.** The missing-file scan retires its row. The `photo_versions` and display rows for that hash become orphan-GC candidates under the existing two-tick, all-libraries-online consensus. Until GC runs, reads fall back to the original if `display_hash` has no live row. - **Original deleted.** Versions become visible as standalone photos (see the hiding rule) and keep the copied date and GPS. - **Moves, renames, archive handoff.** No work needed, since the links are hash-keyed. The back-ref refresh pass in `library_maintenance.rs` should cover the `photo_versions` back-ref like it does for faces and tags. ## Backfill: `src/bin/link_versions` There are existing edits from testing: about 13 ComfyUI outputs under an `aiedits/` folder (default `<prefix>_NNNNN_.png` names), one beside its original, plus a couple of older Upscayl outputs (`<name>-4x.png`, `<name>_upscayl_…png`). None have XMP. - **Default is a dry run** that prints a TSV of proposed links: version path, proposed original, method, pHash distance. - Methods, in order: the ComfyUI `prompt` text chunk's `LoadImage` / `LoadImageWithName` `image` input (a base name, resolved within the same library, preferring the parent of `aiedits/`); a filename prefix (Upscayl-style suffixes); the nearest pHash in the same or parent directory. - Outputs with **two loader inputs** (primary + reference) choose the primary by nearest pHash, record the other as a reference, and are flagged for review. - Unresolvable cases (e.g. a loader name that doesn't match any file) are listed, not guessed. - `--apply <reviewed.tsv>` writes the reviewed rows only, after a WAL-safe `.backup`, proven on a copy first. ## Acceptance criteria - [ ] A PNG saved by the ComfyUI node is linked at ingest via XMP, and a renamed or moved copy of it is still linked. - [ ] A name-convention file without XMP is linked to its sibling original; a RAW+JPEG pair links to the right one. - [ ] Versions are hidden from `/photos`, `/memories`, search and reels while their original is visible, and visible once it isn't. - [ ] `/photos` returns `photo_display_paths` and `photo_version_counts`; responses with no versions are byte-identical to today. - [ ] `/image/versions` and `PUT /image/versions/display` work when called with either the original's or a version's path. - [ ] A version takes its date and GPS from the original; Memories puts it on the original's day. - [ ] Faces, CLIP, insights and duplicate grouping never see version rows. - [ ] `/image?path=<version>` serves the full file. - [ ] Deleting the version file falls back to the original; deleting the original surfaces the version. - [ ] The `link_versions` dry run lists the existing test edits with their proposed originals; `--apply` is gated as described. - [ ] The e2e fixture library (`dev/e2e/gen_fixtures.py`) gains an original with an XMP-linked version, a name-linked version and a reference-photo case. ## Out of scope - Client UI (follow-up tickets in FileViewer-React and Apollo). - ImageApi starting restores itself (submitting ComfyUI jobs). That would need writable storage outside library roots. - Video versions. The schema allows them, but nothing produces them yet. - In-place edits (Branch D in CLAUDE.md), the destructive counterpart of this feature.
cameron added the needs-triage label 2026-10-30 13:15:11 +00:00
Author
Owner
Client follow-ups: Apps/FileViewer-React#52, Apps/Apollo#55
Author
Owner

Producer node lives in Apps/comfyui-imageapi-versions (main @ 1f994eb).

Producer node lives in Apps/comfyui-imageapi-versions (main @ 1f994eb).
Author
Owner

Follow-up: on-demand enhance from the apps is #192 (depends on this ticket's ingest).

Follow-up: on-demand enhance from the apps is #192 (depends on this ticket's ingest).
Author
Owner

Backfill must also cover files that already carry the XMP link. Approved versions are landing before this ships (first one: 2009_200/aiedits/019_19.JPG.restored.png in the archive library, approved 2026-10-30). The watcher ingests them as ordinary photos with an fs_time date, and the link step only runs for new files, so those rows would never be linked. link_versions (or a one-off pass at startup after the migration) should read the XMP DerivedFrom of existing PNG rows and link them, then hide them and copy the original's date the same way ingest does. XMP-linked rows can be applied without manual review, since the link is exact; only the heuristic (no-XMP) matches need the reviewed TSV.

**Backfill must also cover files that already carry the XMP link.** Approved versions are landing before this ships (first one: `2009_200/aiedits/019_19.JPG.restored.png` in the archive library, approved 2026-10-30). The watcher ingests them as ordinary photos with an `fs_time` date, and the link step only runs for *new* files, so those rows would never be linked. `link_versions` (or a one-off pass at startup after the migration) should read the XMP `DerivedFrom` of existing PNG rows and link them, then hide them and copy the original's date the same way ingest does. XMP-linked rows can be applied without manual review, since the link is exact; only the heuristic (no-XMP) matches need the reviewed TSV.
Author
Owner

Client contract from FileViewer-React#53 (built against this ticket): /memories items and the search results (CLIP and unified) are per-photo records, so they carry optional display_path (null means the original) and version_count per record, both skipped when there are no versions. /photos keeps the parallel arrays photo_display_paths / photo_version_counts.

Client contract from FileViewer-React#53 (built against this ticket): `/memories` items and the search results (CLIP and unified) are per-photo records, so they carry optional `display_path` (null means the original) and `version_count` per record, both skipped when there are no versions. `/photos` keeps the parallel arrays `photo_display_paths` / `photo_version_counts`.
Author
Owner

Client contract from Apollo#56: Apollo's grid reads /photos/exif, so ExifSummary rows also carry the optional per-record display_path / version_count. library is optional on GET /image/versions and PUT /image/versions/display: Apollo sends it (as a string), but FileViewer#53 sends only {path, display_hash}.

Client contract from Apollo#56: Apollo's grid reads `/photos/exif`, so `ExifSummary` rows also carry the optional per-record `display_path` / `version_count`. `library` is optional on `GET /image/versions` and `PUT /image/versions/display`: Apollo sends it (as a string), but FileViewer#53 sends only `{path, display_hash}`.
Author
Owner

All acceptance items are live: XMP + name-convention linking (and name-link sweep), hiding in /photos, /memories, search, reels; display_paths/version_counts; /image/versions + display choice; date/GPS copy; exclusion from faces/CLIP/insights/duplicates; lifecycle via the missing-file scan + orphan GC; link_versions; the e2e Versions fixtures. Client UI shipped in FileViewer #52. Closing.

All acceptance items are live: XMP + name-convention linking (and name-link sweep), hiding in /photos, /memories, search, reels; display_paths/version_counts; /image/versions + display choice; date/GPS copy; exclusion from faces/CLIP/insights/duplicates; lifecycle via the missing-file scan + orphan GC; link_versions; the e2e Versions fixtures. Client UI shipped in FileViewer #52. Closing.
Sign in to join this conversation.