Problem. When llama-swap has a model on the 3090, ffmpeg can't create a CUDA context (CUDA_ERROR_OUT_OF_MEMORY, exit 187). The playlist generator treated that like a bad file and wrote playlist.unsupported, so the video was skipped forever. Preview clips were marked failed the same way.
When an NVENC run fails that way, the playlist and the preview clip rerun once on libx264. The playlist retry keeps its semaphore slot and clears the partial output first.
A file that also fails on the CPU still gets the sentinel, so genuinely bad files stop being retried.
The HLS command and the timed run are moved out of the handler into hls_command and run_ffmpeg, so both attempts share them. The flags are unchanged.
Not done: option 3 from the issue (expire old sentinels). The fallback covers the GPU case. Prod has 4 sentinels, all from 2026-05, so there's nothing GPU-related to clean up.
Tested
Unit tests: the classifier against the real OOM stderr and bad-input stderr; the CPU command never asks for CUDA. DATABASE_URL=":memory:" cargo test: 1069 passing. Clippy clean.
End to end: e2e server built from this branch and started with CUDA_VISIBLE_DEVICES= (ffmpeg fails with CUDA_ERROR_NO_DEVICE, the same exit 187). Both fixture videos logged "GPU unavailable … retrying on the CPU encoder" and produced real playlists with no sentinels. Their preview clips also completed on the CPU.
Fixes #215.
**Problem.** When llama-swap has a model on the 3090, ffmpeg can't create a CUDA context (`CUDA_ERROR_OUT_OF_MEMORY`, exit 187). The playlist generator treated that like a bad file and wrote `playlist.unsupported`, so the video was skipped forever. Preview clips were marked `failed` the same way.
**Fix.**
- `ffmpeg::is_gpu_init_failure` matches CUDA/NVENC setup errors in stderr: `CUDA_ERROR_`, `cuCtxCreate`, `OpenEncodeSessionEx failed`, `No capable devices found`, `Cannot load libcuda`, `Device creation failed`.
- When an NVENC run fails that way, the playlist and the preview clip rerun once on libx264. The playlist retry keeps its semaphore slot and clears the partial output first.
- A file that also fails on the CPU still gets the sentinel, so genuinely bad files stop being retried.
- The HLS command and the timed run are moved out of the handler into `hls_command` and `run_ffmpeg`, so both attempts share them. The flags are unchanged.
**Not done:** option 3 from the issue (expire old sentinels). The fallback covers the GPU case. Prod has 4 sentinels, all from 2026-05, so there's nothing GPU-related to clean up.
**Tested**
- Unit tests: the classifier against the real OOM stderr and bad-input stderr; the CPU command never asks for CUDA. `DATABASE_URL=":memory:" cargo test`: 1069 passing. Clippy clean.
- End to end: e2e server built from this branch and started with `CUDA_VISIBLE_DEVICES=` (ffmpeg fails with `CUDA_ERROR_NO_DEVICE`, the same exit 187). Both fixture videos logged "GPU unavailable … retrying on the CPU encoder" and produced real playlists with no sentinels. Their preview clips also completed on the CPU.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
A busy GPU (llama-swap holding the 3090) made ffmpeg's CUDA setup fail
with CUDA_ERROR_OUT_OF_MEMORY, exit 187. The playlist generator took
that for a bad file and wrote playlist.unsupported, so the video never
played; preview clips were marked failed the same way.
is_gpu_init_failure() recognises CUDA/NVENC setup errors in ffmpeg's
stderr. When an NVENC run fails that way, the playlist and the preview
clip rerun once on libx264. A file that also fails on the CPU still gets
the sentinel. The HLS command and the timed run are pulled out of the
handler (hls_command, run_ffmpeg) so both attempts share them.
Fixes#215
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cameron
merged commit 3ccce38505 into master2026-11-02 21:22:33 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Fixes #215.
Problem. When llama-swap has a model on the 3090, ffmpeg can't create a CUDA context (
CUDA_ERROR_OUT_OF_MEMORY, exit 187). The playlist generator treated that like a bad file and wroteplaylist.unsupported, so the video was skipped forever. Preview clips were markedfailedthe same way.Fix.
ffmpeg::is_gpu_init_failurematches CUDA/NVENC setup errors in stderr:CUDA_ERROR_,cuCtxCreate,OpenEncodeSessionEx failed,No capable devices found,Cannot load libcuda,Device creation failed.hls_commandandrun_ffmpeg, so both attempts share them. The flags are unchanged.Not done: option 3 from the issue (expire old sentinels). The fallback covers the GPU case. Prod has 4 sentinels, all from 2026-05, so there's nothing GPU-related to clean up.
Tested
DATABASE_URL=":memory:" cargo test: 1069 passing. Clippy clean.CUDA_VISIBLE_DEVICES=(ffmpeg fails withCUDA_ERROR_NO_DEVICE, the same exit 187). Both fixture videos logged "GPU unavailable … retrying on the CPU encoder" and produced real playlists with no sentinels. Their preview clips also completed on the CPU.🤖 Generated with Claude Code