Make the embedding model swappable via env for A/B testing

Trialing Qwen3-Embedding-0.6B (1024-dim, instruct-prefixed queries)
against nomic required code changes at every hardcoded seam; now it's a
config flip plus a reembed_embeddings run.

- EMBEDDING_DIM env (default 768) replaces every hardcoded dim check:
  daily summary / calendar / search / location DAOs, Ollama batch
  validation, reembed_embeddings
- entities gains the dim guard it never had — a wrong-dim vector
  silently kills dedup/recall (cosine over mismatched lengths is 0),
  so store None and warn instead
- embed_query / embed_document split with EMBED_QUERY_PREFIX /
  EMBED_DOCUMENT_PREFIX (literal \n expanded): retrieval models treat
  the two sides differently — nomic wants search_query:/search_document:,
  Qwen3 wants Instruct:...\nQuery: on queries only. All query-side
  call sites and all corpus writers now declare their side.
- document the contract in CLAUDE.md: change the model or any of these
  vars → re-run reembed_embeddings or search is garbage

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Fable 5 committed 2026-06-11 21:40:40 -04:00
1 parent b1493f5aca
commit efd05db523
12 files changed
+159 -67

No files matched your search

+5 -4
View File
@@ -1046,13 +1046,14 @@ Analyze the image and use specific details from both the visual content and the
}
};
// Validate embedding dimensions (should be 768 for nomic-embed-text:v1.5)
// Validate embedding dimensions (EMBEDDING_DIM; 768 for nomic-embed-text:v1.5)
for (i, embedding) in embeddings.iter().enumerate() {
if embedding.len() != 768 {
if embedding.len() != crate::ai::embedding_dim() {
log::warn!(
"Unexpected embedding dimensions for item {}: {} (expected 768)",
"Unexpected embedding dimensions for item {}: {} (expected {})",
i,
embedding.len()
embedding.len(),
crate::ai::embedding_dim()
);
}
}