refactor(ai): read LLM settings from LlmConfig instead of env
Every LLM env reader now goes through ai::config::llm_config(): build_ollama / build_llamacpp / build_openrouter take config sections (OPENROUTER_API_KEY is still read from env), and the backend switch, embedding dim + prefixes, Ollama / llama.cpp / TTS timeouts, TTS voice and ref length, agentic caps, chat turn timeout and the unified-search model read their sections. LocalLlm::from_env -> from_config; the test_daily_summary and populate_knowledge bins take Ollama defaults from the config. env_max_iterations / env_default_num_ctx are renamed to max_chat_iterations / default_num_ctx. The picker still uses picker.include (= LLAMA_SWAP_ALLOWED_MODELS) until discovery lands. Legacy env vars keep working through the config's override layer; checked against the deployed .env, which resolves to the same values. The LLM_BACKEND parsing tests move to ai::config. llm.example.toml now leaves the embed prefixes empty: copying the nomic prefixes onto a corpus embedded without them breaks search. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
5b8a99a2b2
commit
aa3536afee
17 files changed
+144
-292
No files matched your search
+4
-2
@@ -39,8 +39,10 @@ exclude = ["nemotron3.5-lightning-30b-A3B"]
|
||||
# of them means `cargo run --bin reembed_embeddings`.
|
||||
model = "embed" # llama-swap id (Ollama always uses nomic-embed-text:v1.5)
|
||||
dim = 768 # 768 nomic v1.5 | 1024 Qwen3-Embedding-0.6B
|
||||
query_prefix = "search_query: " # Qwen3: "Instruct: <task>\nQuery: "
|
||||
document_prefix = "search_document: " # Qwen3: ""
|
||||
# Prefixes default to empty. Set them only together with a re-embed; a
|
||||
# corpus embedded without them won't match queries embedded with them.
|
||||
query_prefix = "" # nomic v1.5: "search_query: " | Qwen3: "Instruct: <task>\nQuery: "
|
||||
document_prefix = "" # nomic v1.5: "search_document: " | Qwen3: ""
|
||||
|
||||
[tts]
|
||||
# Chatterbox behind llama-swap; needs llamacpp.url but not backend = "llamacpp".
|
||||
|
||||
Reference in new issue
Block a user