# ADR-0003: LLM deployment config moves to llm.toml; llama-swap owns model capabilities

**Status**: accepted (2026-10-26). Amends ADR-0002's *deployment config* home for LLM settings.

## Context

ADR-0002 put LLM deployment config in `.env`: ~40 vars across `OLLAMA_*`, `LLAMA_SWAP_*`,
`OPENROUTER_*`, `LLM_BACKEND`, `EMBEDDING_DIM`, `EMBED_*_PREFIX`, `AGENTIC_*`. Flat env
vars can't express nested or list-shaped settings, and they duplicated facts that
llama-swap's `config.yaml` already owns. The deployed `LLAMA_SWAP_ALLOWED_MODELS` had
drifted from llama-swap: it listed three model ids that no longer existed (one a typo),
was missing three new ones, and every entry reported `has_vision: true`, including a
`--no-mmproj` text-only model.

## Decision

- **`llm.toml`** (TOML, `LLM_CONFIG_PATH`, default `./llm.toml`) holds every LLM
  setting that ImageApi decides: the local `backend`, which model fills each role
  (chat, vision, embedding, TTS, unified search), the embedding contract, the Ollama /
  OpenRouter / TTS settings, agentic caps and the picker filters. Unknown keys fail startup.
- **Secrets stay in `.env`** (`OPENROUTER_API_KEY`), per ADR-0002.
- **llama-swap's `config.yaml` owns model capabilities.** Each model carries
  `metadata: { kind, vision, ctx, dim }`, which `/v1/models` passes through. ImageApi
  discovers the picker from it (every `kind: chat` model, param aliases hidden, minus
  `picker.exclude`, optionally restricted to `picker.include`) instead of keeping a
  hand-written allowlist. Text-only models get the vision model's description instead
  of the image.
- **Startup cross-checks, warn-only.** Role models that aren't served or lack the needed
  capability, and an embed `dim` that differs from `embedding.dim`, are logged. They
  don't fail startup, because llama-swap may still be starting.
- **One release of env compatibility.** The legacy vars override the file, with one
  deprecation warning listing the ones in use. Then they're removed.

The resolution chain from ADR-0002 is unchanged: request param → owner default
(`user_ai_prefs`) → hard default. `llm.toml` replaces `.env` as the hard-default layer
for LLM fields.

## Considered options

- **A standalone ImageApi model list** (roles plus per-model vision/ctx in `llm.toml`):
  easier to read than env, but still a second copy of llama-swap's facts, so it drifts
  the same way. Rejected.
- **Reading llama-swap's `config.yaml` directly**: couples ImageApi to a file on another
  host and to llama-swap's schema. `/v1/models` is the published interface. Rejected.
- **Owner defaults in the DB** (`user_ai_prefs`): right for per-request tuning, which
  stays there, but URLs and the embedding contract are needed before serving.
  Rejected for these fields, as in ADR-0002.

## Non-obvious rules worth remembering

- The deployed llama-swap merges `metadata` into `meta.llamaswap`, while upstream
  documents it at `meta`. `DiscoveredModel::from_json` reads both.
- A model with no metadata is treated as a text-only chat model, and the startup check
  lists it.
- `embedding.query_prefix` / `document_prefix` default to empty because the existing
  corpus was embedded without them. Set them only together with `reembed_embeddings`.