Add TTS voice deletion, async speech jobs, voice-list cache, ref-seconds name tags

- DELETE /tts/voices/{name}: remove a cloned voice via the llama-swap
  passthrough (upstream chatterbox-tts-api exposes DELETE /voices/{name}).
- POST/GET/DELETE /tts/speech/jobs: durable job flow for long syntheses —
  dispatch returns 202 + job id, the synth queues on the GPU permit instead
  of fast-failing 429, and clients poll for the result (kept ~10 min).
- GET /tts/voices now serves an in-memory cache so listing voices doesn't
  make llama-swap spin up the TTS model (evicting the resident LLM);
  invalidated on create/delete, ?refresh=1 forces an upstream re-query.
- Created voice names are tagged with LLAMA_SWAP_TTS_REF_SECONDS (e.g.
  grandma-30s) so the library shows which ref length produced each clone.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Fable 5 committed 2026-06-10 17:36:15 -04:00
1 parent c78e751743
commit 03699f7413
5 files changed
+588 -16

No files matched your search

+28
View File
@@ -253,6 +253,34 @@ impl LlamaCppClient {
resp.json().await.context("parsing create_voice response")
}
/// Delete a cloned voice from the Chatterbox voice library
/// (`DELETE /voices/{name}` on the upstream, via llama-swap passthrough).
pub async fn delete_voice(&self, voice_name: &str) -> Result<Value> {
let url = format!(
"{}/upstream/{}/voices/{}",
self.swap_root(),
self.tts_model,
voice_name
);
let resp = self
.client
.delete(&url)
.send()
.await
.with_context(|| format!("DELETE {} failed", url))?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
bail!("llama-swap delete_voice failed: {} — {}", status, text);
}
// Some upstreams reply with an empty body on delete.
Ok(resp
.json()
.await
.unwrap_or_else(|_| json!({ "status": "deleted" })))
}
/// Translate canonical messages to the OpenAI-compatible wire shape.
/// Behaviorally identical to `OpenRouterClient::messages_to_openai` —
/// stringify tool-call arguments, rewrite images into content-parts, attach