feat(ai): LlmConfig loaded from llm.toml with legacy env overrides

Adds ai::config: one serde struct for every LLM setting (backend, llama.cpp,
Ollama, OpenRouter, embedding, TTS, agentic caps), resolved as defaults ->
llm.toml (LLM_CONFIG_PATH, default ./llm.toml) -> the old env vars. Env
overrides are kept for one release and listed in a single startup
deprecation warning; LLAMA_SWAP_ALLOWED_MODELS maps to picker.include so
current deploys keep their picker. Unknown keys, zero timeouts and
backend=llamacpp without a URL fail startup. OPENROUTER_API_KEY stays
env-only.

llm.example.toml documents the schema and is parsed by a test. main loads
the config right after tracing init; readers switch over next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Opus 5.5 committed 2026-10-26 17:26:32 -04:00
1 parent 3e3276b8a3
commit 5b8a99a2b2
6 files changed
+719

No files matched your search

Generated
+9
View File
@@ -2122,6 +2122,7 @@ dependencies = [
"tempfile",
"tokio",
"tokio-util",
"toml",
"tracing",
"tracing-actix-web",
"tracing-log",
@@ -4236,10 +4237,12 @@ version = "0.9.12+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf92845e79fc2e2def6a5d828f0801e29a2f8acc037becc5ab08595c7d5e9863"
dependencies = [
"indexmap",
"serde_core",
"serde_spanned",
"toml_datetime",
"toml_parser",
"toml_writer",
"winnow 0.7.15",
]
@@ -4261,6 +4264,12 @@ dependencies = [
"winnow 1.0.2",
]
[[package]]
name = "toml_writer"
version = "1.1.2+spec-1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7d56353a2a665ad0f41a421187180aab746c8c325620617ad883a99a1cbe66d2"
[[package]]
name = "tonic"
version = "0.14.5"