Make the embedding model swappable via env for A/B testing

Trialing Qwen3-Embedding-0.6B (1024-dim, instruct-prefixed queries)
against nomic required code changes at every hardcoded seam; now it's a
config flip plus a reembed_embeddings run.

- EMBEDDING_DIM env (default 768) replaces every hardcoded dim check:
  daily summary / calendar / search / location DAOs, Ollama batch
  validation, reembed_embeddings
- entities gains the dim guard it never had — a wrong-dim vector
  silently kills dedup/recall (cosine over mismatched lengths is 0),
  so store None and warn instead
- embed_query / embed_document split with EMBED_QUERY_PREFIX /
  EMBED_DOCUMENT_PREFIX (literal \n expanded): retrieval models treat
  the two sides differently — nomic wants search_query:/search_document:,
  Qwen3 wants Instruct:...\nQuery: on queries only. All query-side
  call sites and all corpus writers now declare their side.
- document the contract in CLAUDE.md: change the model or any of these
  vars → re-run reembed_embeddings or search is garbage

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Fable 5 committed 2026-06-11 21:40:40 -04:00
1 parent b1493f5aca
commit efd05db523
12 files changed
+159 -67

No files matched your search

+51
View File
@@ -75,6 +75,57 @@ pub fn local_backend_is_llamacpp() -> bool {
)
}
/// Expected embedding dimensionality, env-overridable via `EMBEDDING_DIM`
/// (default 768, nomic-embed-text). Every store/query dim check reads this —
/// swapping to a different-dim model (e.g. Qwen3-Embedding-0.6B at 1024) is
/// then a config flip plus a `reembed_embeddings` run, not a code change.
/// Cached for the process lifetime; a flip requires a restart anyway since
/// the corpus must be re-embedded with it.
pub fn embedding_dim() -> usize {
static DIM: std::sync::OnceLock<usize> = std::sync::OnceLock::new();
*DIM.get_or_init(|| {
std::env::var("EMBEDDING_DIM")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(768)
})
}
/// Read an embedding prefix from the environment. `.env` values can't hold
/// real newlines, so a literal `\n` in the value is expanded — Qwen3-style
/// query instructions need one ("Instruct: ...\nQuery: ").
fn embed_prefix(key: &str) -> String {
std::env::var(key)
.map(|v| v.replace("\\n", "\n"))
.unwrap_or_default()
}
/// Embed a search query. Applies `EMBED_QUERY_PREFIX` (default empty) —
/// retrieval models distinguish query-side from document-side text:
/// nomic v1.5 wants `search_query: `, Qwen3-Embedding wants
/// `Instruct: <task>\nQuery: `. Must pair with the document prefix the
/// corpus was embedded with or similarity degrades.
pub async fn embed_query(
ollama: &OllamaClient,
llamacpp: Option<&LlamaCppClient>,
text: &str,
) -> anyhow::Result<Vec<f32>> {
let prefixed = format!("{}{}", embed_prefix("EMBED_QUERY_PREFIX"), text);
embed_one(ollama, llamacpp, &prefixed).await
}
/// Embed corpus text (the stored side of retrieval). Applies
/// `EMBED_DOCUMENT_PREFIX` (default empty; nomic v1.5 wants
/// `search_document: `, Qwen3-Embedding wants none).
pub async fn embed_document(
ollama: &OllamaClient,
llamacpp: Option<&LlamaCppClient>,
text: &str,
) -> anyhow::Result<Vec<f32>> {
let prefixed = format!("{}{}", embed_prefix("EMBED_DOCUMENT_PREFIX"), text);
embed_one(ollama, llamacpp, &prefixed).await
}
/// Embed a batch of strings via the configured local backend. Routes
/// through llama-swap when `LLM_BACKEND=llamacpp` (and a client is
/// configured), else Ollama. See [`local_backend_is_llamacpp`] for the