feat(ai): discover the llama-swap picker and vision from model metadata

/insights/models now lists llama-swap's models live from /v1/models
instead of LLAMA_SWAP_ALLOWED_MODELS: every kind=chat model, with
setParamsByID aliases (:sub / :fast) hidden, filtered by
[llamacpp.picker] exclude and the optional include allowlist. Ids the
include list names but llama-swap no longer serves drop out instead of
showing as dead picker entries.

has_vision and context_window come from each model's config.yaml metadata
(read from meta.llamaswap, where the deployed llama-swap merges it, or from
meta as upstream documents). A model without metadata is text-only chat.
A live served window still wins over the configured ctx.

Agentic insights on llama-swap now ask the chat model's capabilities:
text-only models (e.g. qwen3.8-27b-code, --no-mmproj) get the local vision
model's description inlined, like hybrid, instead of an image they can't
read. If llama-swap can't be asked, images are sent as before.

Drops AppState.llamacpp_allowed_models.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Opus 5.5 committed 2026-10-26 17:34:28 -04:00
1 parent aa3536afee
commit e10fc26ca7
4 files changed
+219 -92

No files matched your search

-9
View File
@@ -78,10 +78,6 @@ pub struct AppState {
/// threading through the generator.
#[allow(dead_code)]
pub llamacpp: Option<Arc<LlamaCppClient>>,
/// Curated list of llama-swap model ids exposed to clients. Sourced from
/// `LLAMA_SWAP_ALLOWED_MODELS` (comma-separated). Empty when unset; the
/// server then falls back to `LLAMA_SWAP_PRIMARY_MODEL`.
pub llamacpp_allowed_models: Vec<String>,
pub sms_client: SmsApiClient,
pub insight_generator: InsightGenerator,
/// Chat continuation service. Hold an Arc so handlers can clone cheaply.
@@ -134,7 +130,6 @@ impl AppState {
openrouter: Option<Arc<OpenRouterClient>>,
openrouter_allowed_models: Vec<String>,
llamacpp: Option<Arc<LlamaCppClient>>,
llamacpp_allowed_models: Vec<String>,
sms_client: SmsApiClient,
insight_generator: InsightGenerator,
insight_chat: Arc<InsightChatService>,
@@ -196,7 +191,6 @@ impl AppState {
openrouter,
openrouter_allowed_models,
llamacpp,
llamacpp_allowed_models,
sms_client,
insight_generator,
insight_chat,
@@ -232,7 +226,6 @@ impl Default for AppState {
let openrouter_allowed_models = llm.openrouter.allowed.clone();
let llamacpp = build_llamacpp(llm);
let llamacpp_allowed_models = llm.llamacpp.picker.include.clone().unwrap_or_default();
let sms_api_url =
env::var("SMS_API_URL").unwrap_or_else(|_| "http://localhost:8000".to_string());
@@ -369,7 +362,6 @@ impl Default for AppState {
openrouter,
openrouter_allowed_models,
llamacpp,
llamacpp_allowed_models,
sms_client,
insight_generator,
insight_chat,
@@ -548,7 +540,6 @@ impl AppState {
None,
Vec::new(),
None,
Vec::new(),
sms_client,
insight_generator,
insight_chat,