Every scan logged this: clip_watch: permanent failure on 2014/10/h1a81hurcs00h-… - Copy.jpg (will retry next pass): clip 422 decode_failed
That file isn't an image (file reports it as data, i.e. random bytes). CLIP has no failure marker, so the drain sent it to Apollo again on every pass. The old code comment already suggested adding one if this got noisy.
Fix
New clip_failures table (migration 2026-10-18-000000):
Keyed on (library_id, rel_path).
Stores the content_hash that failed, plus the error and a timestamp.
mark_clip_failed: an upsert on ExifDao.
list_clip_unencoded_candidates: an anti-join skips a row while its current hash matches the failed one. Replacing the file makes it eligible again with no cleanup.
Retrying everything (e.g. after an Apollo decoder upgrade) is DELETE FROM clip_failures;.
New ClipError::Rejected for 422s, the only status that marks. Other 4xx stay Permanent and aren't marked. Those are deployment problems (auth, routing), and marking them would blacklist the whole library. For search, Rejected is treated the same as Permanent.
## Problem
Every scan logged this:
`clip_watch: permanent failure on 2014/10/h1a81hurcs00h-… - Copy.jpg (will retry next pass): clip 422 decode_failed`
That file isn't an image (`file` reports it as `data`, i.e. random bytes). CLIP has no failure marker, so the drain sent it to Apollo again on every pass. The old code comment already suggested adding one if this got noisy.
## Fix
- **New `clip_failures` table** (migration `2026-10-18-000000`):
- Keyed on `(library_id, rel_path)`.
- Stores the `content_hash` that failed, plus the error and a timestamp.
- **`mark_clip_failed`:** an upsert on `ExifDao`.
- **`list_clip_unencoded_candidates`:** an anti-join skips a row while its current hash matches the failed one. Replacing the file makes it eligible again with no cleanup.
- **Retrying everything** (e.g. after an Apollo decoder upgrade) is `DELETE FROM clip_failures;`.
- **New `ClipError::Rejected` for 422s, the only status that marks.** Other 4xx stay `Permanent` and aren't marked. Those are deployment problems (auth, routing), and marking them would blacklist the whole library. For search, `Rejected` is treated the same as `Permanent`.
## Tests
- `clip_failure_marker_skips_row_until_its_bytes_change`:
- a marked row is skipped
- the same path in another library isn't affected
- re-marking upserts
- a new hash brings the row back
- Classifier tests now expect `Rejected` for 422s.
- Full suite and clippy are clean.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
A 422 from Apollo's /encode_image (bytes it can't decode — corrupt
files, non-images with an image extension) left clip_embedding NULL,
so the drain re-sent the file and logged the same "permanent failure
(will retry next pass)" on every scan.
Permanent rejections now record a clip_failures marker scoped to the
row's content_hash; list_clip_unencoded_candidates skips the row while
the hash still matches, so a replaced file gets another try with no
cleanup. Only 422 (new ClipError::Rejected) marks: other 4xx are
deployment problems (auth, routing) and would otherwise blacklist the
whole library.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cameron
merged commit bc27aea81a into master2026-10-17 04:21:42 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Problem
Every scan logged this:
clip_watch: permanent failure on 2014/10/h1a81hurcs00h-… - Copy.jpg (will retry next pass): clip 422 decode_failedThat file isn't an image (
filereports it asdata, i.e. random bytes). CLIP has no failure marker, so the drain sent it to Apollo again on every pass. The old code comment already suggested adding one if this got noisy.Fix
clip_failurestable (migration2026-10-18-000000):(library_id, rel_path).content_hashthat failed, plus the error and a timestamp.mark_clip_failed: an upsert onExifDao.list_clip_unencoded_candidates: an anti-join skips a row while its current hash matches the failed one. Replacing the file makes it eligible again with no cleanup.DELETE FROM clip_failures;.ClipError::Rejectedfor 422s, the only status that marks. Other 4xx stayPermanentand aren't marked. Those are deployment problems (auth, routing), and marking them would blacklist the whole library. For search,Rejectedis treated the same asPermanent.Tests
clip_failure_marker_skips_row_until_its_bytes_change:Rejectedfor 422s.🤖 Generated with Claude Code