Adds blake3 content hashing as the basis for derivative dedup
(thumbnails, HLS) across libraries. Computed inline by the watcher on
ingest and by a new `backfill_hashes` binary for historical rows.
Key changes:
- `content_hash` and `size_bytes` are now populated on new image_exif
rows; a new ExifDao surface (`get_rows_missing_hash`,
`backfill_content_hash`, `find_by_content_hash`) supports backfill and
future hash-keyed lookups.
- The watcher now registers every image/video in image_exif, not just
files with parseable EXIF. EXIF becomes optional enrichment; videos
and other non-EXIF files still get a hashed row. This also makes
DB-indexed sort/filter cover the full library.
- `/image` thumbnail serve dual-looks up hash-keyed path first, then
falls back to the legacy mirrored layout.
- Upload flow accepts `?library=` query param + hashes uploaded files.
- Store_exif logs the underlying Diesel error on insert failure so
constraint violations surface instead of hiding behind a generic
InsertError.
- New migration normalizes rel_path separators to forward slash across
all tables, deduplicating any rows that collide after normalization.
Fixes spurious UNIQUE violations from mixed backslash/forward-slash
paths on Windows ingest.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a `libraries` registry table and threads library_id through
per-instance metadata tables (image_exif, photo_insights,
entity_photo_links, video_preview_clips). File-path columns renamed to
rel_path to make the relative-to-root semantics explicit. Adds
content_hash + size_bytes on image_exif to support future hash-keyed
thumbnail/HLS dedup. Tags and favorites stay library-agnostic so they
share across libraries by rel_path.
Behavior is unchanged: a single primary library (id=1) is seeded from
BASE_PATH on first boot; all handlers and DAOs route through it as a
transitional shim until the API gains a library query param.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds normalize_entity_type() which lowercases and canonicalises synonyms
(location→place, human→person, etc.) before every upsert. The SQL lookup
now uses lower(entity_type) on both sides so existing dirty rows (Person,
Location) correctly deduplicate against normalised writes without a migration.
Adds a pre-flight similarity check in tool_store_entity: before upserting,
searches active entities of the same type using the first name token. Any
non-exact matches are appended to the tool response so the agentic loop
can choose to reuse an existing entity ID rather than create a duplicate.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Date sorting previously used a DB-level query that acted as an inner join,
silently dropping files with no image_exif row. Replace it with the existing
in-memory sort which already falls back to filename-extracted and filesystem
dates, so all files appear in sorted results.
Also removes the now-unused get_files_sorted_by_date trait method and its
SqliteExifDao implementation and test mock.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements persistent cross-photo knowledge memory so the agentic
insight loop can learn and recall facts about people, places, and
events across the photo collection.
Changes:
- photo_insights: drop UNIQUE(file_path) + INSERT OR REPLACE, replace
with append-only rows + is_current flag for insight history retention
- New tables: entities, entity_facts, entity_photo_links with FK
constraints and confidence scoring
- KnowledgeDao trait + SqliteKnowledgeDao with upsert, merge, and
corroboration (confidence +0.1 on duplicate fact detection)
- Four new agent tools: recall_entities, recall_facts_for_photo,
store_entity, store_fact (with object_entity_id FK support)
- Cameron entity auto-seeded with stable ID injected into system prompt
- Pre-run photo link clearing + post-loop source_insight_id backfill
- Audit REST API: GET/PATCH/DELETE /knowledge/entities/{id},
POST /knowledge/entities/merge, GET/PATCH/DELETE /knowledge/facts/{id},
GET /knowledge/recent
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- cargo fmt applied across all modified source files
- Collapse nested if let Some / if !is_empty into a single let-chain (clippy::collapsible_match)
- All other warnings are pre-existing dead-code lint on unused trait methods
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Implement unit tests for PreviewClipRequest/PreviewStatusRequest serialization and deserialization.
- Add tests for PreviewDao (insert, update, batch retrieval, and status-based queries).
- Extend Actix-web integration tests for `/video/preview/status` endpoint scenarios.
- Introduce in-memory TestPreviewDao for mock database interactions.
- Update README with new config parameters for preview clips.
Backend (Rust/Actix-web):
- Add video_preview_clips table and PreviewDao for tracking preview generation
- Add ffmpeg preview clip generator: 10 equally-spaced 1s segments at 480p with CUDA NVENC auto-detection
- Add PreviewClipGenerator actor with semaphore-limited concurrent processing
- Add GET /video/preview and POST /video/preview/status endpoints
- Extend file watcher to detect and queue previews for new videos
- Use relative paths consistently for DB storage (matching EXIF convention)
Frontend (React Native/Expo):
- Add VideoWall grid view with 2-3 column layout of looping preview clips
- Add VideoWallItem component with ActiveVideoPlayer sub-component for lifecycle management
- Add useVideoWall hook for batch status polling with 5s refresh
- Add navigation button in grid header (visible when videos exist)
- Use TextureView surface type to fix Android z-ordering issues
- Optimize memory: players only mount while visible via FlatList windowSize
- Configure ExoPlayer buffer options and caching for short clips
- Tap to toggle audio focus, long press to open in full viewer
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implement database-level sorting with composite indexes for efficient date and tag queries. Add pagination metadata support and optimize tag count queries using batch processing.
- Updated InsightGenerator struct with calendar, location, and search DAOs
- Implemented hybrid context gathering methods:
* gather_calendar_context(): ±7 days with semantic ranking
* gather_location_context(): ±30 min with GPS proximity check
* gather_search_context(): ±30 days semantic search
- Added haversine_distance() utility for GPS calculations
- Updated generate_insight_for_photo_with_model() to use multi-source context
- Combined all context sources (SMS + Calendar + Location + Search) with equal weight
- Initialized new DAOs in AppState (both default and test implementations)
- All contexts are optional (graceful degradation if data missing)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implements Phase 1 & 2 of Google Takeout RAG integration:
- Database migrations for calendar_events, location_history, search_history
- DAO implementations with hybrid time + semantic search
- Parsers for .ics, JSON, and HTML Google Takeout formats
- Import utilities with batch insert optimization
Features:
- CalendarEventDao: Hybrid time-range + semantic search for events
- LocationHistoryDao: GPS proximity with Haversine distance calculation
- SearchHistoryDao: Semantic-first search (queries are embedding-rich)
- Batch inserts for performance (1M+ records in minutes vs hours)
- OpenTelemetry tracing for all database operations
Import utilities:
- import_calendar: Parse .ics with optional embedding generation
- import_location_history: High-volume GPS data with batch inserts
- import_search_history: Always generates embeddings for semantic search
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implements foundation for EXIF-based photo search capabilities:
- Add geo.rs module with GPS distance calculations (Haversine + bounding box)
- Extend FilesRequest with EXIF search parameters (camera, GPS, date, media type)
- Add MediaType enum and DateTakenAsc/DateTakenDesc sort options
- Create date_taken index migration for efficient date queries
- Implement ExifDao methods: get_exif_batch, query_by_exif, get_camera_makes
- Add FileWithMetadata struct for date-aware sorting
- Implement date sorting with filename extraction fallback
- Make extract_date_from_filename public for reuse
Next: Integrate EXIF filtering into list_photos() and enhance get_all_tags()
Cameron
Cameron Cordes