feat(openrouter): cap max_tokens per request (OPENROUTER_MAX_TOKENS)

OpenRouter reserves credit against the requested max_tokens, falling
back to the model's full output window when omitted, so expensive
models failed with credit-limit errors even for short replies. Default
the cap to 16384; OPENROUTER_MAX_TOKENS overrides it and 0 disables it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Opus 5.5 committed 2026-09-22 21:32:56 -04:00
1 parent 4e3ee1973c
commit 33fe7ce77c
4 files changed
+30

No files matched your search

+7
View File
@@ -402,6 +402,13 @@ fn build_openrouter_from_env() -> Option<Arc<OpenRouterClient>> {
if let Ok(model) = env::var("OPENROUTER_EMBEDDING_MODEL") {
client.set_embedding_model(model);
}
// OPENROUTER_MAX_TOKENS=0 omits the cap (provider default).
if let Some(n) = env::var("OPENROUTER_MAX_TOKENS")
.ok()
.and_then(|v| v.trim().parse::<u32>().ok())
{
client.set_max_tokens((n > 0).then_some(n));
}
Some(Arc::new(client))
}