Memories: event/burst stacks (time gap + pHash, maybe CLIP) #175

Open
opened 2026-10-17 02:31:54 +00:00 by cameron · 0 comments
Owner

Follow-up to the Memories condensing work in FileViewer-React: the app is getting year sections with time-spread sampling and "+K" expansion. That cuts down bursts, but it doesn't actually understand events. This ticket covers grouping memories into event and burst stacks.

Idea

/memories returns a group_id (plus maybe a cover pick) per item, so the client can:

  • sample one per event when condensing a year, instead of one per time slot
  • show bursts and near-duplicates as a stacked tile with a count that fans out on tap

Signals we already have (no new indexing)

  • Time gaps: split a day into events wherever the gap between shots is larger than X minutes (tune X; maybe adapt it to the day's density).
  • pHash/dHash (src/perceptual_hash.rs, already used by src/duplicates.rs): near-identical frames, i.e. bursts and re-saves. Hamming distance on the stored i64s.
  • CLIP embeddings (clip_watch.rs): the same scene even when frames aren't pixel-similar, e.g. several angles of the same cake. Probably only needed within a time-gap event.

Open questions

  • What should the cover be: sharpest, most-tagged, has faces, or the middle frame?
  • Should videos join stacks, or always stand alone?
  • Group on the server per request, or precompute and store?
  • Prototype against the real library first to see whether the groupings feel right before committing to a UI.

Client side (FileViewer-React)

  • The Memories year sections sample one item per group.
  • A stack tile shows a count badge; tapping it expands in place or opens the viewer at the stack.
Follow-up to the Memories condensing work in FileViewer-React: the app is getting year sections with time-spread sampling and "+K" expansion. That cuts down bursts, but it doesn't actually understand events. This ticket covers grouping memories into event and burst stacks. ## Idea `/memories` returns a `group_id` (plus maybe a cover pick) per item, so the client can: - sample **one per event** when condensing a year, instead of one per time slot - show bursts and near-duplicates as a **stacked tile with a count** that fans out on tap ## Signals we already have (no new indexing) - **Time gaps**: split a day into events wherever the gap between shots is larger than X minutes (tune X; maybe adapt it to the day's density). - **pHash/dHash** (`src/perceptual_hash.rs`, already used by `src/duplicates.rs`): near-identical frames, i.e. bursts and re-saves. Hamming distance on the stored i64s. - **CLIP embeddings** (`clip_watch.rs`): the same scene even when frames aren't pixel-similar, e.g. several angles of the same cake. Probably only needed within a time-gap event. ## Open questions - What should the cover be: sharpest, most-tagged, has faces, or the middle frame? - Should videos join stacks, or always stand alone? - Group on the server per request, or precompute and store? - Prototype against the real library first to see whether the groupings feel right before committing to a UI. ## Client side (FileViewer-React) - The Memories year sections sample one item per group. - A stack tile shows a count badge; tapping it expands in place or opens the viewer at the stack.
Sign in to join this conversation.