LLM settings move from .env into llm.toml. llama-swap's catalog (which models exist, vision, ctx) is discovered from /v1/models metadata instead of being configured twice. See ADR-0003.
llm_config() # OnceLock, loaded once at startup
defaults
← llm.toml # LLM_CONFIG_PATH, default ./llm.toml; unknown keys rejected
← legacy env vars # still override, with a deprecation warning
validate() # zero timeouts / llamacpp without url → startup fails
GET /insights/models (llamacpp)
- LLAMA_SWAP_ALLOWED_MODELS (hand-kept list, drifted)
+ discover_models() # /v1/models + metadata {kind, vision, ctx, dim}
+ picker_models() # kind: chat, aliases (:sub/:fast) hidden, exclude / include
resolve_backend (llamacpp)
- images_inline = true
+ images_inline = model_capabilities(model).has_vision # text-only → describe-then-inline
startup
+ log_catalog_warnings() # role models, picker ids, embed dim vs what llama-swap serves
OPENROUTER_API_KEY stays in env. llm.example.toml documents every key with its default; llm.toml is gitignored.
Before: prod's LLAMA_SWAP_ALLOWED_MODELS listed qwen3.6-27b-abilterated, qwen3.6-27b, gemma4-12b-coder (no longer served) and was missing code-vision, swift, e4b. After: startup check against live llama-swap with the prod env flags exactly those three stale ids; discovery returns 14 chat models with correct vision/ctx.
Prod .env resolves to identical values through LlmConfig (env-override layer).
Merge Danger
Door: two-way
The legacy env vars still override the file, so prod runs unchanged without an llm.toml. Revert = redeploy the old binary. No DB changes (the embedding fingerprint is deferred to #189).
Blast Radius: AI-config
A typo'd key in llm.toml now stops the server at startup instead of being ignored.
Text-only llama-swap models (e.g. qwen3.8-27b-code) now receive a vision-model description instead of raw images. That relies on correct vision: metadata in llama-swap's config.yaml.
Picker ids that llama-swap no longer serves drop out of /insights/models, even under the legacy allowlist.
## Summary
LLM settings move from `.env` into `llm.toml`. llama-swap's catalog (which models exist, vision, ctx) is discovered from `/v1/models` metadata instead of being configured twice. See ADR-0003.
```text
llm_config() # OnceLock, loaded once at startup
defaults
← llm.toml # LLM_CONFIG_PATH, default ./llm.toml; unknown keys rejected
← legacy env vars # still override, with a deprecation warning
validate() # zero timeouts / llamacpp without url → startup fails
```
```diff
GET /insights/models (llamacpp)
- LLAMA_SWAP_ALLOWED_MODELS (hand-kept list, drifted)
+ discover_models() # /v1/models + metadata {kind, vision, ctx, dim}
+ picker_models() # kind: chat, aliases (:sub/:fast) hidden, exclude / include
resolve_backend (llamacpp)
- images_inline = true
+ images_inline = model_capabilities(model).has_vision # text-only → describe-then-inline
startup
+ log_catalog_warnings() # role models, picker ids, embed dim vs what llama-swap serves
```
`OPENROUTER_API_KEY` stays in env. `llm.example.toml` documents every key with its default; `llm.toml` is gitignored.
## Evidence
- `cargo test`: 927 passing; `cargo clippy`: 0 warnings.
- **Before:** prod's `LLAMA_SWAP_ALLOWED_MODELS` listed `qwen3.6-27b-abilterated`, `qwen3.6-27b`, `gemma4-12b-coder` (no longer served) and was missing `code-vision`, `swift`, `e4b`.
**After:** startup check against live llama-swap with the prod env flags exactly those three stale ids; discovery returns 14 chat models with correct vision/ctx.
- Prod `.env` resolves to identical values through `LlmConfig` (env-override layer).
## Merge Danger
**Door:** two-way
The legacy env vars still override the file, so prod runs unchanged without an `llm.toml`. Revert = redeploy the old binary. No DB changes (the embedding fingerprint is deferred to #189).
**Blast Radius:** AI-config
- A typo'd key in `llm.toml` now stops the server at startup instead of being ignored.
- Text-only llama-swap models (e.g. `qwen3.8-27b-code`) now receive a vision-model description instead of raw images. That relies on correct `vision:` metadata in llama-swap's `config.yaml`.
- Picker ids that llama-swap no longer serves drop out of `/insights/models`, even under the legacy allowlist.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Adds ai::config: one serde struct for every LLM setting (backend, llama.cpp,
Ollama, OpenRouter, embedding, TTS, agentic caps), resolved as defaults ->
llm.toml (LLM_CONFIG_PATH, default ./llm.toml) -> the old env vars. Env
overrides are kept for one release and listed in a single startup
deprecation warning; LLAMA_SWAP_ALLOWED_MODELS maps to picker.include so
current deploys keep their picker. Unknown keys, zero timeouts and
backend=llamacpp without a URL fail startup. OPENROUTER_API_KEY stays
env-only.
llm.example.toml documents the schema and is parsed by a test. main loads
the config right after tracing init; readers switch over next.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every LLM env reader now goes through ai::config::llm_config():
build_ollama / build_llamacpp / build_openrouter take config sections
(OPENROUTER_API_KEY is still read from env), and the backend switch,
embedding dim + prefixes, Ollama / llama.cpp / TTS timeouts, TTS voice and
ref length, agentic caps, chat turn timeout and the unified-search model
read their sections. LocalLlm::from_env -> from_config; the
test_daily_summary and populate_knowledge bins take Ollama defaults from
the config. env_max_iterations / env_default_num_ctx are renamed to
max_chat_iterations / default_num_ctx.
The picker still uses picker.include (= LLAMA_SWAP_ALLOWED_MODELS) until
discovery lands. Legacy env vars keep working through the config's
override layer; checked against the deployed .env, which resolves to the
same values. The LLM_BACKEND parsing tests move to ai::config.
llm.example.toml now leaves the embed prefixes empty: copying the
nomic prefixes onto a corpus embedded without them breaks search.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/insights/models now lists llama-swap's models live from /v1/models
instead of LLAMA_SWAP_ALLOWED_MODELS: every kind=chat model, with
setParamsByID aliases (:sub / :fast) hidden, filtered by
[llamacpp.picker] exclude and the optional include allowlist. Ids the
include list names but llama-swap no longer serves drop out instead of
showing as dead picker entries.
has_vision and context_window come from each model's config.yaml metadata
(read from meta.llamaswap, where the deployed llama-swap merges it, or from
meta as upstream documents). A model without metadata is text-only chat.
A live served window still wins over the configured ctx.
Agentic insights on llama-swap now ask the chat model's capabilities:
text-only models (e.g. qwen3.8-27b-code, --no-mmproj) get the local vision
model's description inlined, like hybrid, instead of an image they can't
read. If llama-swap can't be asked, images are sent as before.
Drops AppState.llamacpp_allowed_models.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After AppState is built, main checks the config against llama-swap's
/v1/models in the background and logs one warning per problem: a role
model (chat, vision, embedding, unified_search, tts) that isn't served or
lacks what the role needs (vision, kind: embed / tts), an embed model
whose metadata dim differs from embedding.dim, picker include/exclude ids
llama-swap doesn't serve, and models with no metadata. Warn-only: llama-swap
may still be starting. Chat-side roles are only checked under
backend = "llamacpp"; TTS always is.
Against the deployed .env this reports exactly the three stale
LLAMA_SWAP_ALLOWED_MODELS ids.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
README, CLAUDE.md and .env.example now point at llm.toml for LLM settings
(OPENROUTER_API_KEY stays in env) and describe the legacy env override
window, the discovered picker and the startup cross-check. ADR-0003 records
the move and amends ADR-0002's deployment-config home; CONTEXT.md's
Deployment config / Hard default entries follow. Source comments and the
"set LLAMA_SWAP_URL" error strings name the llm.toml keys.
Also fixes README's AGENTIC_MAX_ITERATIONS default (10 -> 12, what the
code uses).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Summary
LLM settings move from
.envintollm.toml. llama-swap's catalog (which models exist, vision, ctx) is discovered from/v1/modelsmetadata instead of being configured twice. See ADR-0003.OPENROUTER_API_KEYstays in env.llm.example.tomldocuments every key with its default;llm.tomlis gitignored.Evidence
cargo test: 927 passing;cargo clippy: 0 warnings.LLAMA_SWAP_ALLOWED_MODELSlistedqwen3.6-27b-abilterated,qwen3.6-27b,gemma4-12b-coder(no longer served) and was missingcode-vision,swift,e4b.After: startup check against live llama-swap with the prod env flags exactly those three stale ids; discovery returns 14 chat models with correct vision/ctx.
.envresolves to identical values throughLlmConfig(env-override layer).Merge Danger
Door: two-way
The legacy env vars still override the file, so prod runs unchanged without an
llm.toml. Revert = redeploy the old binary. No DB changes (the embedding fingerprint is deferred to #189).Blast Radius: AI-config
llm.tomlnow stops the server at startup instead of being ignored.qwen3.8-27b-code) now receive a vision-model description instead of raw images. That relies on correctvision:metadata in llama-swap'sconfig.yaml./insights/models, even under the legacy allowlist.🤖 Generated with Claude Code