Enhance: Animate a photo (video output), built to support video Enhance next #208

Closed
opened 2026-11-02 00:23:35 +00:00 by cameron · 1 comment
Owner

Add an Animate Enhance template (Wan image-to-video on the desktop 4090), and lay the plumbing so Enhance can produce videos. Sharpening existing videos with SeedVR2 comes later on the same pieces.

Decisions (2026-11-01)

  • Template: animate, label "Animate", prompt_mode = "replace" (the user's text is the prompt, like Edit).
  • Workflow: /media/nas/image_wan_animate_imageapi_API.json, at native 720p. Per the desktop benchmark, that's ~160 s warm and ~19 GB peak, vs 480p + SeedVR2 at ~213 s and 23.5 GB.
  • Output:
    • An mp4 beside the photo, e.g. aiedits/IMG_1234.JPG.animated.mp4, version kind animated.
    • Linked to the photo by the same XMP DerivedFrom hash as image versions, stored in the mp4's XMP uuid box.
  • A "derived clip", not a display version:
    • Hidden from the grid while the photo exists (same rule as versions), so the default view isn't cluttered.
    • Never selectable as the photo's display version or thumbnail.
    • Shown on the photo in the viewer as a "▶ N clips" entry point, separate from the version chips, with a "made from" link back.
    • A "show clips" grid filter can come later.

Work

  1. comfyui-imageapi-versions: ImageApiSaveVideoVersion (input: core VIDEO) writes the mp4 and its XMP link, then stages and approves it like the PNGs.
  2. ImageApi:
    • Enhance results carry a media type (read video outputs from /history, serve previews with the right content type).
    • Templates get output = "image" | "video", exposed in /enhance/templates.
    • Version linking reads XMP from mp4.
    • animated is treated as a derived clip: hidden like a version, excluded from display selection, listed separately for the viewer.
  3. Workflow: Wan i2v Animate in API format, with imageapi:source, imageapi:prompt and imageapi:save (the video save node).
  4. FileViewer: video playback in the Enhance approve preview, the "clips" entry point on photos, and progress for long jobs.
  5. Later: a video loader node, plus Enhance on videos (Sharpen video with SeedVR2), with a duration cap or trim. 1080p SeedVR2 costs ~35 s of 4090 time per second of video.
Add an **Animate** Enhance template (Wan image-to-video on the desktop 4090), and lay the plumbing so Enhance can produce videos. Sharpening existing videos with SeedVR2 comes later on the same pieces. ## Decisions (2026-11-01) - **Template:** `animate`, label "Animate", `prompt_mode = "replace"` (the user's text is the prompt, like Edit). - **Workflow:** `/media/nas/image_wan_animate_imageapi_API.json`, at native 720p. Per the desktop benchmark, that's ~160 s warm and ~19 GB peak, vs 480p + SeedVR2 at ~213 s and 23.5 GB. - **Output:** - An mp4 beside the photo, e.g. `aiedits/IMG_1234.JPG.animated.mp4`, version kind `animated`. - Linked to the photo by the same XMP `DerivedFrom` hash as image versions, stored in the mp4's XMP `uuid` box. - **A "derived clip", not a display version:** - Hidden from the grid while the photo exists (same rule as versions), so the default view isn't cluttered. - Never selectable as the photo's display version or thumbnail. - Shown on the photo in the viewer as a "▶ N clips" entry point, separate from the version chips, with a "made from" link back. - A "show clips" grid filter can come later. ## Work 1. **comfyui-imageapi-versions:** `ImageApiSaveVideoVersion` (input: core `VIDEO`) writes the mp4 and its XMP link, then stages and approves it like the PNGs. 2. **ImageApi:** - Enhance results carry a media type (read video outputs from `/history`, serve previews with the right content type). - Templates get `output = "image" | "video"`, exposed in `/enhance/templates`. - Version linking reads XMP from mp4. - `animated` is treated as a derived clip: hidden like a version, excluded from display selection, listed separately for the viewer. 3. **Workflow:** Wan i2v Animate in API format, with `imageapi:source`, `imageapi:prompt` and `imageapi:save` (the video save node). 4. **FileViewer:** video playback in the Enhance approve preview, the "clips" entry point on photos, and progress for long jobs. 5. **Later:** a video loader node, plus Enhance on videos (Sharpen video with SeedVR2), with a duration cap or trim. 1080p SeedVR2 costs ~35 s of 4090 time per second of video.
cameron added the needs-triage label 2026-11-02 00:23:35 +00:00
Author
Owner

Shipped: ImageApiSaveVideoVersion (node pack #5/#6), derived clips + clip counts (#209/#210), the animate template (Wan i2v, NAS workflow), and FileViewer #98 (Animate tab, clip preview, ▶ clips chip) + #101 (jobs manager for long jobs). Item 5 (video loader + Sharpen video) moved to its own issue.

Shipped: ImageApiSaveVideoVersion (node pack #5/#6), derived clips + clip counts (#209/#210), the animate template (Wan i2v, NAS workflow), and FileViewer #98 (Animate tab, clip preview, ▶ clips chip) + #101 (jobs manager for long jobs). Item 5 (video loader + Sharpen video) moved to its own issue.
Sign in to join this conversation.