Make the embedding model swappable via env for A/B testing
Trialing Qwen3-Embedding-0.6B (1024-dim, instruct-prefixed queries) against nomic required code changes at every hardcoded seam; now it's a config flip plus a reembed_embeddings run. - EMBEDDING_DIM env (default 768) replaces every hardcoded dim check: daily summary / calendar / search / location DAOs, Ollama batch validation, reembed_embeddings - entities gains the dim guard it never had — a wrong-dim vector silently kills dedup/recall (cosine over mismatched lengths is 0), so store None and warn instead - embed_query / embed_document split with EMBED_QUERY_PREFIX / EMBED_DOCUMENT_PREFIX (literal \n expanded): retrieval models treat the two sides differently — nomic wants search_query:/search_document:, Qwen3 wants Instruct:...\nQuery: on queries only. All query-side call sites and all corpus writers now declare their side. - document the contract in CLAUDE.md: change the model or any of these vars → re-run reembed_embeddings or search is garbage Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b1493f5aca
commit
efd05db523
12 files changed
+159
-67
No files matched your search
+5
-4
@@ -1046,13 +1046,14 @@ Analyze the image and use specific details from both the visual content and the
|
||||
}
|
||||
};
|
||||
|
||||
// Validate embedding dimensions (should be 768 for nomic-embed-text:v1.5)
|
||||
// Validate embedding dimensions (EMBEDDING_DIM; 768 for nomic-embed-text:v1.5)
|
||||
for (i, embedding) in embeddings.iter().enumerate() {
|
||||
if embedding.len() != 768 {
|
||||
if embedding.len() != crate::ai::embedding_dim() {
|
||||
log::warn!(
|
||||
"Unexpected embedding dimensions for item {}: {} (expected 768)",
|
||||
"Unexpected embedding dimensions for item {}: {} (expected {})",
|
||||
i,
|
||||
embedding.len()
|
||||
embedding.len(),
|
||||
crate::ai::embedding_dim()
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user