fix(clip): stop re-sending undecodable files every pass
A 422 from Apollo's /encode_image (bytes it can't decode — corrupt files, non-images with an image extension) left clip_embedding NULL, so the drain re-sent the file and logged the same "permanent failure (will retry next pass)" on every scan. Permanent rejections now record a clip_failures marker scoped to the row's content_hash; list_clip_unencoded_candidates skips the row while the hash still matches, so a replaced file gets another try with no cleanup. Only 422 (new ClipError::Rejected) marks: other 4xx are deployment problems (auth, routing) and would otherwise blacklist the whole library. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
a605ccd6f8
commit
e75e60786d
9 files changed
+179
-25
No files matched your search
@@ -0,0 +1,17 @@
|
||||
-- "Don't retry" markers for the CLIP drain. A permanent encode failure
|
||||
-- (bytes Apollo can't decode — corrupt or non-image files with an image
|
||||
-- extension) used to leave `clip_embedding` NULL, so the drain re-sent
|
||||
-- the file and logged the same 422 every pass.
|
||||
--
|
||||
-- Keyed on the path, scoped to the bytes that failed: the drain skips a
|
||||
-- row only while its `content_hash` still matches, so replacing the file
|
||||
-- makes it eligible again with no cleanup. To retry everything (e.g.
|
||||
-- after an Apollo decoder upgrade): `DELETE FROM clip_failures;`
|
||||
CREATE TABLE clip_failures (
|
||||
library_id INTEGER NOT NULL REFERENCES libraries(id) ON DELETE CASCADE,
|
||||
rel_path TEXT NOT NULL,
|
||||
content_hash TEXT NOT NULL,
|
||||
error TEXT NOT NULL,
|
||||
failed_at BIGINT NOT NULL,
|
||||
PRIMARY KEY (library_id, rel_path)
|
||||
);
|
||||
Reference in new issue
Block a user