TTS (/tts/*, read-aloud + voice cloning) is served by Chatterbox bundled into a derived llama-swap image (llama-swap-chatterbox:local, see llama-swap/chatterbox/Dockerfile), exposed as the chatterbox model id. It works, but it carries several workarounds:
Derived image: every llama-swap / llama.cpp upgrade means rebuilding our custom image, and the PyTorch stack rides along with llama.cpp's.
Readiness hack:/health returns 200 mid-load, so the Dockerfile adds a /ready route for checkEndpoint.
Multilingual disabled:USE_MULTILINGUAL_MODEL=false because the multilingual branch fails to init.
Matrix wiring: hand-tuned tts sets/evict costs so TTS can stay warm beside small chat models.
Question
Does current upstream llama-swap handle this better? Possible directions:
Run Chatterbox as its own container and register it with llama-swap as an external/peer upstream (proxy: to a separate service) instead of baking it into the image.
Check whether newer llama-swap features (peers, selectors/profiles, improved health checks) remove the need for the /ready route and the custom matrix terms.
Alternatively, take TTS out of llama-swap entirely and give ImageApi a direct [tts] endpoint in the upcoming llm.toml.
Related
The deployed llama-swap build is older than upstream (the nemotron3.5 entry is blocked on an old llama.cpp). An image rebuild would fix both.
LLM config extraction (env → llm.toml + llama-swap per-model metadata): chatterbox is tagged metadata: { kind: tts }, so whichever setup wins only needs to keep that tag.
## Context
TTS (`/tts/*`, read-aloud + voice cloning) is served by Chatterbox bundled into a derived llama-swap image (`llama-swap-chatterbox:local`, see `llama-swap/chatterbox/Dockerfile`), exposed as the `chatterbox` model id. It works, but it carries several workarounds:
- **Derived image:** every llama-swap / llama.cpp upgrade means rebuilding our custom image, and the PyTorch stack rides along with llama.cpp's.
- **Readiness hack:** `/health` returns 200 mid-load, so the Dockerfile adds a `/ready` route for `checkEndpoint`.
- **Multilingual disabled:** `USE_MULTILINGUAL_MODEL=false` because the multilingual branch fails to init.
- **Matrix wiring:** hand-tuned `tts` sets/evict costs so TTS can stay warm beside small chat models.
## Question
Does current upstream llama-swap handle this better? Possible directions:
- Run Chatterbox as its own container and register it with llama-swap as an external/peer upstream (`proxy:` to a separate service) instead of baking it into the image.
- Check whether newer llama-swap features (peers, selectors/profiles, improved health checks) remove the need for the `/ready` route and the custom matrix terms.
- Alternatively, take TTS out of llama-swap entirely and give ImageApi a direct `[tts]` endpoint in the upcoming `llm.toml`.
## Related
- The deployed llama-swap build is older than upstream (the nemotron3.5 entry is blocked on an old llama.cpp). An image rebuild would fix both.
- LLM config extraction (env → `llm.toml` + llama-swap per-model `metadata`): `chatterbox` is tagged `metadata: { kind: tts }`, so whichever setup wins only needs to keep that tag.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Context
TTS (
/tts/*, read-aloud + voice cloning) is served by Chatterbox bundled into a derived llama-swap image (llama-swap-chatterbox:local, seellama-swap/chatterbox/Dockerfile), exposed as thechatterboxmodel id. It works, but it carries several workarounds:/healthreturns 200 mid-load, so the Dockerfile adds a/readyroute forcheckEndpoint.USE_MULTILINGUAL_MODEL=falsebecause the multilingual branch fails to init.ttssets/evict costs so TTS can stay warm beside small chat models.Question
Does current upstream llama-swap handle this better? Possible directions:
proxy:to a separate service) instead of baking it into the image./readyroute and the custom matrix terms.[tts]endpoint in the upcomingllm.toml.Related
llm.toml+ llama-swap per-modelmetadata):chatterboxis taggedmetadata: { kind: tts }, so whichever setup wins only needs to keep that tag.