Make the embedding model swappable via env for A/B testing
Trialing Qwen3-Embedding-0.6B (1024-dim, instruct-prefixed queries) against nomic required code changes at every hardcoded seam; now it's a config flip plus a reembed_embeddings run. - EMBEDDING_DIM env (default 768) replaces every hardcoded dim check: daily summary / calendar / search / location DAOs, Ollama batch validation, reembed_embeddings - entities gains the dim guard it never had — a wrong-dim vector silently kills dedup/recall (cosine over mismatched lengths is 0), so store None and warn instead - embed_query / embed_document split with EMBED_QUERY_PREFIX / EMBED_DOCUMENT_PREFIX (literal \n expanded): retrieval models treat the two sides differently — nomic wants search_query:/search_document:, Qwen3 wants Instruct:...\nQuery: on queries only. All query-side call sites and all corpus writers now declare their side. - document the contract in CLAUDE.md: change the model or any of these vars → re-run reembed_embeddings or search is garbage Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b1493f5aca
commit
efd05db523
12 files changed
+159
-67
No files matched your search
@@ -75,6 +75,57 @@ pub fn local_backend_is_llamacpp() -> bool {
|
||||
)
|
||||
}
|
||||
|
||||
/// Expected embedding dimensionality, env-overridable via `EMBEDDING_DIM`
|
||||
/// (default 768, nomic-embed-text). Every store/query dim check reads this —
|
||||
/// swapping to a different-dim model (e.g. Qwen3-Embedding-0.6B at 1024) is
|
||||
/// then a config flip plus a `reembed_embeddings` run, not a code change.
|
||||
/// Cached for the process lifetime; a flip requires a restart anyway since
|
||||
/// the corpus must be re-embedded with it.
|
||||
pub fn embedding_dim() -> usize {
|
||||
static DIM: std::sync::OnceLock<usize> = std::sync::OnceLock::new();
|
||||
*DIM.get_or_init(|| {
|
||||
std::env::var("EMBEDDING_DIM")
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(768)
|
||||
})
|
||||
}
|
||||
|
||||
/// Read an embedding prefix from the environment. `.env` values can't hold
|
||||
/// real newlines, so a literal `\n` in the value is expanded — Qwen3-style
|
||||
/// query instructions need one ("Instruct: ...\nQuery: ").
|
||||
fn embed_prefix(key: &str) -> String {
|
||||
std::env::var(key)
|
||||
.map(|v| v.replace("\\n", "\n"))
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
/// Embed a search query. Applies `EMBED_QUERY_PREFIX` (default empty) —
|
||||
/// retrieval models distinguish query-side from document-side text:
|
||||
/// nomic v1.5 wants `search_query: `, Qwen3-Embedding wants
|
||||
/// `Instruct: <task>\nQuery: `. Must pair with the document prefix the
|
||||
/// corpus was embedded with or similarity degrades.
|
||||
pub async fn embed_query(
|
||||
ollama: &OllamaClient,
|
||||
llamacpp: Option<&LlamaCppClient>,
|
||||
text: &str,
|
||||
) -> anyhow::Result<Vec<f32>> {
|
||||
let prefixed = format!("{}{}", embed_prefix("EMBED_QUERY_PREFIX"), text);
|
||||
embed_one(ollama, llamacpp, &prefixed).await
|
||||
}
|
||||
|
||||
/// Embed corpus text (the stored side of retrieval). Applies
|
||||
/// `EMBED_DOCUMENT_PREFIX` (default empty; nomic v1.5 wants
|
||||
/// `search_document: `, Qwen3-Embedding wants none).
|
||||
pub async fn embed_document(
|
||||
ollama: &OllamaClient,
|
||||
llamacpp: Option<&LlamaCppClient>,
|
||||
text: &str,
|
||||
) -> anyhow::Result<Vec<f32>> {
|
||||
let prefixed = format!("{}{}", embed_prefix("EMBED_DOCUMENT_PREFIX"), text);
|
||||
embed_one(ollama, llamacpp, &prefixed).await
|
||||
}
|
||||
|
||||
/// Embed a batch of strings via the configured local backend. Routes
|
||||
/// through llama-swap when `LLM_BACKEND=llamacpp` (and a client is
|
||||
/// configured), else Ollama. See [`local_backend_is_llamacpp`] for the
|
||||
|
||||
Reference in new issue
Block a user