Stored vectors are only searchable with the backend, model, width and prefixes that built them. Nothing enforced that, so changing [embedding] (or backend) without running reembed_embeddings silently broke similarity search. The per-row dim check only caught a change of width.
What changes
New table: migration embedding_contract, a single row holding backend, model, dim, query_prefix, document_prefix and recorded_at. The prefixes are stored verbatim rather than hashed, so the mismatch error can show exactly what changed.
New module:src/ai/embedding_contract.rs.
The server checks the contract once after migrations.
The src/bin tools check it lazily on their first embed.
With no row yet, it records the running config.
On a mismatch every embed is refused. The guard sits in embed_many, so it covers queries and documents. Startup logs an error naming each changed field, for example dim: 768 → 1024; document_prefix: "search_document: " → "", plus the fix.
I chose refuse over warn: mixing vector spaces corrupts the corpus for good, while refusing only pauses semantic search.
A DB error in the check itself leaves embedding allowed and logs a warning.
reembed_embeddings:
bypasses the guard, which is what it's for;
prints the stored-vs-current difference;
records the contract after a full run (all four tables, no --limit, not --dry-run).
The CLAUDE.md [embedding] notes are updated.
Testing
cargo test: 1074 pass, including 5 new contract tests: first record, a same-width model or prefix swap, the single-row replace, Ollama's fixed model, and the message. Clippy is clean.
Prod DB snapshot (a WAL-safe .backup): up.sql applies cleanly. A dry reembed_embeddings against it with the prod llm.toml bypasses the guard and writes no contract.
e2e server:
the first start logs Embedding contract recorded: backend=ollama model=nomic-embed-text:v1.5 dim=768;
after the stored row was changed to dim 1024 with a document prefix, a restart logs the mismatch error;
the stored row is left untouched.
Prod: the corpus already disagrees with the query side
A dry-run sample on the snapshot (200 rows per table, prod llm.toml) compared each stored vector with a fresh embedding of the same text:
Table
Rows
Median cosine
Min
daily_conversation_summaries
2,793
0.88
0.77
calendar_events
599
0.61
0.32
search_history
58,155
0.83
0.36
entities
121
1.00
0.48
The old summary rows are labelled nomic-embed-text:v1.5 (Ollama), but queries now go through llama-swap's embed. So semantic search over those tables has been degraded already; this PR doesn't cause it. On first start the guard records the current config. To bring the corpus in line, run a full reembed_embeddings on prod after deploying, with a DB backup first. That run rewrites the vectors and records the contract again.
Closes #189.
Stored vectors are only searchable with the backend, model, width and prefixes that built them. Nothing enforced that, so changing `[embedding]` (or `backend`) without running `reembed_embeddings` silently broke similarity search. The per-row dim check only caught a change of width.
## What changes
- **New table:** migration `embedding_contract`, a single row holding `backend`, `model`, `dim`, `query_prefix`, `document_prefix` and `recorded_at`. The prefixes are stored verbatim rather than hashed, so the mismatch error can show exactly what changed.
- **New module:** `src/ai/embedding_contract.rs`.
- The server checks the contract once after migrations.
- The `src/bin` tools check it lazily on their first embed.
- With no row yet, it records the running config.
- **On a mismatch every embed is refused.** The guard sits in `embed_many`, so it covers queries and documents. Startup logs an error naming each changed field, for example `dim: 768 → 1024; document_prefix: "search_document: " → ""`, plus the fix.
- I chose refuse over warn: mixing vector spaces corrupts the corpus for good, while refusing only pauses semantic search.
- A DB error in the check itself leaves embedding allowed and logs a warning.
- **`reembed_embeddings`:**
- bypasses the guard, which is what it's for;
- prints the stored-vs-current difference;
- records the contract after a full run (all four tables, no `--limit`, not `--dry-run`).
- The CLAUDE.md `[embedding]` notes are updated.
## Testing
- `cargo test`: 1074 pass, including 5 new contract tests: first record, a same-width model or prefix swap, the single-row replace, Ollama's fixed model, and the message. Clippy is clean.
- **Prod DB snapshot** (a WAL-safe `.backup`): `up.sql` applies cleanly. A dry `reembed_embeddings` against it with the prod llm.toml bypasses the guard and writes no contract.
- **e2e server:**
- the first start logs `Embedding contract recorded: backend=ollama model=nomic-embed-text:v1.5 dim=768`;
- after the stored row was changed to dim 1024 with a document prefix, a restart logs the mismatch error;
- the stored row is left untouched.
## Prod: the corpus already disagrees with the query side
A dry-run sample on the snapshot (200 rows per table, prod llm.toml) compared each stored vector with a fresh embedding of the same text:
| Table | Rows | Median cosine | Min |
|---|---|---|---|
| daily_conversation_summaries | 2,793 | 0.88 | 0.77 |
| calendar_events | 599 | 0.61 | 0.32 |
| search_history | 58,155 | 0.83 | 0.36 |
| entities | 121 | 1.00 | 0.48 |
The old summary rows are labelled `nomic-embed-text:v1.5` (Ollama), but queries now go through llama-swap's `embed`. So semantic search over those tables has been degraded already; this PR doesn't cause it. On first start the guard records the current config. To bring the corpus in line, run a full `reembed_embeddings` on prod after deploying, with a DB backup first. That run rewrites the vectors and records the contract again.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Stored vectors are only searchable with the backend, model, width and
prefixes that built them, and nothing enforced that: changing [embedding]
(or backend) without reembed_embeddings silently broke similarity search,
and the per-row dim check missed same-width model or prefix swaps.
- embedding_contract: one row (backend, model, dim, both prefixes). The
server checks it once after migrations; the src/bin tools check it on
their first embed. No row yet = record the running config.
- On a mismatch every embed (embed_many, so queries and documents) fails
with a message naming each changed field and the fix. Mixing vector
spaces corrupts the corpus for good; refusing only pauses semantic
search until the config is put back or the corpus re-embedded.
- reembed_embeddings bypasses the guard and records the contract after a
full run (all four tables, no --limit, not --dry-run).
- A DB error in the check leaves embedding allowed and is logged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cameron
merged commit e317bfee98 into master2026-11-03 18:44:49 +00:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Closes #189.
Stored vectors are only searchable with the backend, model, width and prefixes that built them. Nothing enforced that, so changing
[embedding](orbackend) without runningreembed_embeddingssilently broke similarity search. The per-row dim check only caught a change of width.What changes
embedding_contract, a single row holdingbackend,model,dim,query_prefix,document_prefixandrecorded_at. The prefixes are stored verbatim rather than hashed, so the mismatch error can show exactly what changed.src/ai/embedding_contract.rs.src/bintools check it lazily on their first embed.embed_many, so it covers queries and documents. Startup logs an error naming each changed field, for exampledim: 768 → 1024; document_prefix: "search_document: " → "", plus the fix.reembed_embeddings:--limit, not--dry-run).[embedding]notes are updated.Testing
cargo test: 1074 pass, including 5 new contract tests: first record, a same-width model or prefix swap, the single-row replace, Ollama's fixed model, and the message. Clippy is clean..backup):up.sqlapplies cleanly. A dryreembed_embeddingsagainst it with the prod llm.toml bypasses the guard and writes no contract.Embedding contract recorded: backend=ollama model=nomic-embed-text:v1.5 dim=768;Prod: the corpus already disagrees with the query side
A dry-run sample on the snapshot (200 rows per table, prod llm.toml) compared each stored vector with a fresh embedding of the same text:
The old summary rows are labelled
nomic-embed-text:v1.5(Ollama), but queries now go through llama-swap'sembed. So semantic search over those tables has been degraded already; this PR doesn't cause it. On first start the guard records the current config. To bring the corpus in line, run a fullreembed_embeddingson prod after deploying, with a DB backup first. That run rewrites the vectors and records the contract again.🤖 Generated with Claude Code