knowledge: consolidation proposals endpoint
Finds near-duplicate entities the upsert-time cosine guard didn't
catch — typically legacy data from before that guard landed, or
pairs whose embeddings sit between 0.85 (default proposal floor)
and 0.92 (auto-collapse threshold). Pure read-side feature; the
actual merging still goes through the existing
/knowledge/entities/merge action.
New DAO method `find_consolidation_proposals(threshold,
max_groups)`:
- Loads every non-rejected entity with an embedding.
- Partitions by entity_type so a person can't cluster with a
place.
- Pairwise cosine, edges above threshold feed a union-find for
transitive grouping (Sara → Sarah → Sarah J. all land in one
cluster).
- Tracks min/max cosine per component so the UI can show "how
tight" each cluster is before clicking in.
- Returns groups of >= 2 sorted by size desc then max cosine
desc; trimmed to `max_groups`.
New endpoint `GET /knowledge/consolidation-proposals?threshold=
&limit=` accepts the threshold (clamped 0.5–0.99 to prevent the
"every entity in one mega-cluster" case) and returns groups with
per-entity persona fact-count breakdowns baked in — saves the UI
a separate query per cluster member.
ConsolidationGroup is exported through database/mod.rs so the
handler can use it without depending on knowledge_dao internals.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -59,8 +59,8 @@ pub use calendar_dao::{CalendarEventDao, SqliteCalendarEventDao};
|
||||
pub use daily_summary_dao::{DailySummaryDao, InsertDailySummary, SqliteDailySummaryDao};
|
||||
pub use insights_dao::{InsightDao, SqliteInsightDao};
|
||||
pub use knowledge_dao::{
|
||||
EntityFilter, EntityPatch, EntitySort, FactFilter, FactPatch, KnowledgeDao, PersonaFilter,
|
||||
RecentActivity, SqliteKnowledgeDao,
|
||||
ConsolidationGroup, EntityFilter, EntityPatch, EntitySort, FactFilter, FactPatch, KnowledgeDao,
|
||||
PersonaFilter, RecentActivity, SqliteKnowledgeDao,
|
||||
};
|
||||
pub use location_dao::{LocationHistoryDao, SqliteLocationHistoryDao};
|
||||
pub use persona_dao::{ImportPersona, PersonaDao, PersonaPatch, SqlitePersonaDao};
|
||||
|
||||
Reference in New Issue
Block a user