Goal: all AI on the Linux server (3090), with ComfyUI swapping in and out of the one GPU alongside the llama-swap models. The desktop 4090 stays as a fallback / redundancy.
Idea: register ComfyUI as a llama-swap model and point Enhance's url at http://localhost:9292/upstream/comfyui, so an Enhance request loads ComfyUI and evicts the LLMs, and llama-swap's ttl unloads it again.
What breaks today if Enhance points at /upstream/comfyui
Every request through that route loads ComfyUI. Enhance only uses plain HTTP (/system_stats, /prompt, /queue, /history, /view, /free; no websocket), so the proxying itself works. But:
available() probe: GET /system_stats with a 2 s timeout runs whenever the Enhance sheet opens. It would evict the LLMs just from opening the sheet, then time out on the cold start and report "unavailable".
Idle check every 30 s: polls /queue and /history, which would keep reloading ComfyUI.
probe_now() before submit: has the same 2 s timeout, so the first job after a cold start fails.
Proposed
An enhance.toml mode where llama-swap manages ComfyUI:
Availability comes from llama-swap (/v1/models lists the entry, or /running), not from pinging ComfyUI.
No idle /free loop; llama-swap's ttl handles unloading.
Submit waits out the cold start (ComfyUI boot of ~10–20 s plus model load).
llama-swap side:
ComfyUI runs as a subprocess inside the llama-swap container, like chatterbox. Try bind-mounting ~/ComfyUI and the conda env at the same paths instead of rebuilding the image.
Mount the NAS for path_map.
Add a matrix set so it never shares the GPU with the big LLMs. SeedVR2 Sharpen peaks at ~19–21.6 GB.
Models: Fibo2 weights need to move here. Disk is ~94% full, so free space first. The SeedVR2 models are already in ~/ComfyUI/models/SEEDVR2 (7B fp8-mixed, 3B fp16, VAE).
Notes
SeedVR2 on the 3090 (16 MP output): Sharpen took 66 s cold and 49 s warm. Rescue took ~27 s on a 2 MP screenshot.
SeedVR2's cache_model keeps about 9 GB of system RAM, and ComfyUI's /free doesn't release it. That matters if llama-swap is expected to free everything on unload.
Goal: all AI on the Linux server (3090), with ComfyUI swapping in and out of the one GPU alongside the llama-swap models. The desktop 4090 stays as a fallback / redundancy.
Idea: register ComfyUI as a llama-swap model and point Enhance's `url` at `http://localhost:9292/upstream/comfyui`, so an Enhance request loads ComfyUI and evicts the LLMs, and llama-swap's `ttl` unloads it again.
## What breaks today if Enhance points at `/upstream/comfyui`
Every request through that route loads ComfyUI. Enhance only uses plain HTTP (`/system_stats`, `/prompt`, `/queue`, `/history`, `/view`, `/free`; no websocket), so the proxying itself works. But:
1. **`available()` probe**: `GET /system_stats` with a 2 s timeout runs whenever the Enhance sheet opens. It would evict the LLMs just from opening the sheet, then time out on the cold start and report "unavailable".
2. **Idle check every 30 s**: polls `/queue` and `/history`, which would keep reloading ComfyUI.
3. **`probe_now()` before submit**: has the same 2 s timeout, so the first job after a cold start fails.
## Proposed
- An `enhance.toml` mode where llama-swap manages ComfyUI:
- Availability comes from llama-swap (`/v1/models` lists the entry, or `/running`), not from pinging ComfyUI.
- No idle `/free` loop; llama-swap's `ttl` handles unloading.
- Submit waits out the cold start (ComfyUI boot of ~10–20 s plus model load).
- llama-swap side:
- ComfyUI runs as a subprocess inside the llama-swap container, like chatterbox. Try bind-mounting `~/ComfyUI` and the conda env at the same paths instead of rebuilding the image.
- Mount the NAS for `path_map`.
- Add a matrix set so it never shares the GPU with the big LLMs. SeedVR2 Sharpen peaks at ~19–21.6 GB.
- Models: Fibo2 weights need to move here. Disk is ~94% full, so free space first. The SeedVR2 models are already in `~/ComfyUI/models/SEEDVR2` (7B fp8-mixed, 3B fp16, VAE).
## Notes
- **SeedVR2 on the 3090** (16 MP output): Sharpen took 66 s cold and 49 s warm. Rescue took ~27 s on a 2 MP screenshot.
- **SeedVR2's `cache_model`** keeps about 9 GB of system RAM, and ComfyUI's `/free` doesn't release it. That matters if llama-swap is expected to free everything on unload.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Goal: all AI on the Linux server (3090), with ComfyUI swapping in and out of the one GPU alongside the llama-swap models. The desktop 4090 stays as a fallback / redundancy.
Idea: register ComfyUI as a llama-swap model and point Enhance's
urlathttp://localhost:9292/upstream/comfyui, so an Enhance request loads ComfyUI and evicts the LLMs, and llama-swap'sttlunloads it again.What breaks today if Enhance points at
/upstream/comfyuiEvery request through that route loads ComfyUI. Enhance only uses plain HTTP (
/system_stats,/prompt,/queue,/history,/view,/free; no websocket), so the proxying itself works. But:available()probe:GET /system_statswith a 2 s timeout runs whenever the Enhance sheet opens. It would evict the LLMs just from opening the sheet, then time out on the cold start and report "unavailable"./queueand/history, which would keep reloading ComfyUI.probe_now()before submit: has the same 2 s timeout, so the first job after a cold start fails.Proposed
enhance.tomlmode where llama-swap manages ComfyUI:/v1/modelslists the entry, or/running), not from pinging ComfyUI./freeloop; llama-swap'sttlhandles unloading.~/ComfyUIand the conda env at the same paths instead of rebuilding the image.path_map.~/ComfyUI/models/SEEDVR2(7B fp8-mixed, 3B fp16, VAE).Notes
cache_modelkeeps about 9 GB of system RAM, and ComfyUI's/freedoesn't release it. That matters if llama-swap is expected to free everything on unload.