feat(ai): discover the llama-swap picker and vision from model metadata
/insights/models now lists llama-swap's models live from /v1/models instead of LLAMA_SWAP_ALLOWED_MODELS: every kind=chat model, with setParamsByID aliases (:sub / :fast) hidden, filtered by [llamacpp.picker] exclude and the optional include allowlist. Ids the include list names but llama-swap no longer serves drop out instead of showing as dead picker entries. has_vision and context_window come from each model's config.yaml metadata (read from meta.llamaswap, where the deployed llama-swap merges it, or from meta as upstream documents). A model without metadata is text-only chat. A live served window still wins over the configured ctx. Agentic insights on llama-swap now ask the chat model's capabilities: text-only models (e.g. qwen3.8-27b-code, --no-mmproj) get the local vision model's description inlined, like hybrid, instead of an image they can't read. If llama-swap can't be asked, images are sent as before. Drops AppState.llamacpp_allowed_models. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
aa3536afee
commit
e10fc26ca7
4 files changed
+219
-92
No files matched your search
@@ -78,10 +78,6 @@ pub struct AppState {
|
||||
/// threading through the generator.
|
||||
#[allow(dead_code)]
|
||||
pub llamacpp: Option<Arc<LlamaCppClient>>,
|
||||
/// Curated list of llama-swap model ids exposed to clients. Sourced from
|
||||
/// `LLAMA_SWAP_ALLOWED_MODELS` (comma-separated). Empty when unset; the
|
||||
/// server then falls back to `LLAMA_SWAP_PRIMARY_MODEL`.
|
||||
pub llamacpp_allowed_models: Vec<String>,
|
||||
pub sms_client: SmsApiClient,
|
||||
pub insight_generator: InsightGenerator,
|
||||
/// Chat continuation service. Hold an Arc so handlers can clone cheaply.
|
||||
@@ -134,7 +130,6 @@ impl AppState {
|
||||
openrouter: Option<Arc<OpenRouterClient>>,
|
||||
openrouter_allowed_models: Vec<String>,
|
||||
llamacpp: Option<Arc<LlamaCppClient>>,
|
||||
llamacpp_allowed_models: Vec<String>,
|
||||
sms_client: SmsApiClient,
|
||||
insight_generator: InsightGenerator,
|
||||
insight_chat: Arc<InsightChatService>,
|
||||
@@ -196,7 +191,6 @@ impl AppState {
|
||||
openrouter,
|
||||
openrouter_allowed_models,
|
||||
llamacpp,
|
||||
llamacpp_allowed_models,
|
||||
sms_client,
|
||||
insight_generator,
|
||||
insight_chat,
|
||||
@@ -232,7 +226,6 @@ impl Default for AppState {
|
||||
let openrouter_allowed_models = llm.openrouter.allowed.clone();
|
||||
|
||||
let llamacpp = build_llamacpp(llm);
|
||||
let llamacpp_allowed_models = llm.llamacpp.picker.include.clone().unwrap_or_default();
|
||||
|
||||
let sms_api_url =
|
||||
env::var("SMS_API_URL").unwrap_or_else(|_| "http://localhost:8000".to_string());
|
||||
@@ -369,7 +362,6 @@ impl Default for AppState {
|
||||
openrouter,
|
||||
openrouter_allowed_models,
|
||||
llamacpp,
|
||||
llamacpp_allowed_models,
|
||||
sms_client,
|
||||
insight_generator,
|
||||
insight_chat,
|
||||
@@ -548,7 +540,6 @@ impl AppState {
|
||||
None,
|
||||
Vec::new(),
|
||||
None,
|
||||
Vec::new(),
|
||||
sms_client,
|
||||
insight_generator,
|
||||
insight_chat,
|
||||
|
||||
Reference in new issue
Block a user