refactor(ai): read LLM settings from LlmConfig instead of env
Every LLM env reader now goes through ai::config::llm_config(): build_ollama / build_llamacpp / build_openrouter take config sections (OPENROUTER_API_KEY is still read from env), and the backend switch, embedding dim + prefixes, Ollama / llama.cpp / TTS timeouts, TTS voice and ref length, agentic caps, chat turn timeout and the unified-search model read their sections. LocalLlm::from_env -> from_config; the test_daily_summary and populate_knowledge bins take Ollama defaults from the config. env_max_iterations / env_default_num_ctx are renamed to max_chat_iterations / default_num_ctx. The picker still uses picker.include (= LLAMA_SWAP_ALLOWED_MODELS) until discovery lands. Legacy env vars keep working through the config's override layer; checked against the deployed .env, which resolves to the same values. The LLM_BACKEND parsing tests move to ai::config. llm.example.toml now leaves the embed prefixes empty: copying the nomic prefixes onto a corpus embedded without them breaks search. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
5b8a99a2b2
commit
aa3536afee
17 files changed
+144
-292
No files matched your search
@@ -639,15 +639,15 @@ async fn translate<TagD: TagDao>(
|
||||
// llama-swap); local only, no hybrid, per the feature's design.
|
||||
//
|
||||
// Translation-model precedence:
|
||||
// 1. UNIFIED_SEARCH_MODEL env — pin a small, fast model that can stay
|
||||
// 1. `unified_search_model` (llm.toml) — pin a small, fast model that can stay
|
||||
// co-resident with CLIP (and the chat model) so translation never
|
||||
// evicts them. This is the recommended setup on a tight VRAM budget.
|
||||
// 2. the client-selected model — routes translation to whatever the user
|
||||
// already has loaded (no swap) when no dedicated model is pinned.
|
||||
// 3. None → resolve_backend uses the configured default local model.
|
||||
let translation_model = std::env::var("UNIFIED_SEARCH_MODEL")
|
||||
.ok()
|
||||
.filter(|m| !m.trim().is_empty())
|
||||
let translation_model = crate::ai::config::llm_config()
|
||||
.unified_search_model
|
||||
.clone()
|
||||
.or_else(|| query.model.clone())
|
||||
.filter(|m| !m.trim().is_empty());
|
||||
let overrides = SamplingOverrides {
|
||||
|
||||
Reference in new issue
Block a user