feat(openrouter): cap max_tokens per request (OPENROUTER_MAX_TOKENS)

OpenRouter reserves credit against the requested max_tokens, falling
back to the model's full output window when omitted, so expensive
models failed with credit-limit errors even for short replies. Default
the cap to 16384; OPENROUTER_MAX_TOKENS overrides it and 0 disables it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Opus 5.5 committed 2026-09-22 21:32:56 -04:00
1 parent 4e3ee1973c
commit 33fe7ce77c
4 files changed
+30

No files matched your search

+3
View File
@@ -636,6 +636,9 @@ OPENROUTER_BASE_URL=https://openrouter.ai/api/v1 # Override base URL (option
OPENROUTER_EMBEDDING_MODEL=openai/text-embedding-3-small # Optional, embeddings stay local today
OPENROUTER_HTTP_REFERER=https://your-site.example # Optional attribution header
OPENROUTER_APP_TITLE=ImageApi # Optional attribution header
OPENROUTER_MAX_TOKENS=16384 # Per-call max_tokens cap (default 16384; 0 = uncapped).
# OpenRouter reserves credit against this, so an uncapped
# request to a pricey model can fail on credit limits.
# Local LLM backend switch. `ollama` (default) keeps the OLLAMA_* settings
# above; `llamacpp` swaps the entire local stack (chat + vision describe +