The /video/generate and /image/metadata handlers assumed files live under
the resolved library only, which broke when a mobile client passed no
library (union mode) but the file lived in a non-primary library. Both
now fall back to scanning every configured library for an existing file.
InsightGenerator held a single base_path, so vision-model loads and
filename-date fallbacks failed for non-primary libraries. It now takes
Vec<Library> and probes each root in resolve_full_path.
/image/metadata responses now carry library_id/library_name so the
mobile viewer can surface which library a file belongs to.
Thumbnail generation at startup is now spawned on a background thread
so the HTTP server can accept traffic while large libraries backfill.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tags and insights now follow content across libraries via content_hash
lookups on the read path, so the same file indexed at different rel_paths
in multiple libraries shares its annotations. Recursive tag search scopes
hits to the selected library by checking each tagged rel_path against
the library's disk (with a content-hash sibling fallback so tags attached
under one library's rel_path still match a content-equivalent file in
another). The /image and /image/metadata handlers fall back across
libraries when the file isn't under the resolved one, so union-mode
search results (which carry no library attribution in the response)
still serve correctly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The Phase 3 plumbing accepted `library=` but didn't actually route
requests through the scoped library once it was resolved. Three
concrete bugs surfaced when testing against a second mounted library:
- `/image` always resolved paths against AppState.base_path (primary),
so thumbnails for non-primary libraries 400'd when their rel_paths
didn't exist under primary. Now resolves against the scoped library
and defaults to primary when the param is omitted.
- `/memories` walked the scoped library correctly but its helper
functions hardcoded `library_id: PRIMARY_LIBRARY_ID` on every
MemoryItem, causing clients to route thumbnails back to primary
regardless of which library the memory actually came from.
- `/photos` non-recursive listing delegated to a `RealFileSystem`
constructed from AppState.base_path at startup, so walks always
hit primary even when `library=2` was passed. The non-primary
path now uses list_files against the scoped library's root;
primary still goes through FileSystemAccess to preserve the
existing test mock plumbing.
Also adds `library` to ThumbnailRequest so the /image query param
is actually parsed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds blake3 content hashing as the basis for derivative dedup
(thumbnails, HLS) across libraries. Computed inline by the watcher on
ingest and by a new `backfill_hashes` binary for historical rows.
Key changes:
- `content_hash` and `size_bytes` are now populated on new image_exif
rows; a new ExifDao surface (`get_rows_missing_hash`,
`backfill_content_hash`, `find_by_content_hash`) supports backfill and
future hash-keyed lookups.
- The watcher now registers every image/video in image_exif, not just
files with parseable EXIF. EXIF becomes optional enrichment; videos
and other non-EXIF files still get a hashed row. This also makes
DB-indexed sort/filter cover the full library.
- `/image` thumbnail serve dual-looks up hash-keyed path first, then
falls back to the legacy mirrored layout.
- Upload flow accepts `?library=` query param + hashes uploaded files.
- Store_exif logs the underlying Diesel error on insert failure so
constraint violations surface instead of hiding behind a generic
InsertError.
- New migration normalizes rel_path separators to forward slash across
all tables, deduplicating any rows that collide after normalization.
Fixes spurious UNIQUE violations from mixed backslash/forward-slash
paths on Windows ingest.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
`watch_files` and `create_thumbnails` now iterate every configured
library, tagging rows with the correct `library_id`. `process_new_files`
takes a `&Library` so InsertImageExif no longer hardcodes the primary
library. Upload accepts an optional `library` query param to pick a
target library; omitted still defaults to primary for backwards
compatibility.
Hash-keyed thumbnail/HLS storage with dual-lookup fallback is deferred
to Phase 5, where it's bundled with the content hash backfill that
actually makes the hash-keyed paths meaningful. Until hashes are
populated, the legacy mirrored layout is a no-op to change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New `/libraries` endpoint returns configured libraries so clients can
discover them. `FilesRequest` and `MemoriesRequest` gain an optional
`library` param (accepts name or numeric id). Unknown values are
rejected with 400; absent values span all libraries. `/memories`
now scopes its filesystem walk + EXIF query to the resolved library.
`MemoryItem` carries `library_id` so union-mode clients can render a
per-item source badge.
Behavior is unchanged in single-library mode: omitting `library` still
returns results from the primary library, which is the only one
configured until a second row is added to the libraries table.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a `libraries` registry table and threads library_id through
per-instance metadata tables (image_exif, photo_insights,
entity_photo_links, video_preview_clips). File-path columns renamed to
rel_path to make the relative-to-root semantics explicit. Adds
content_hash + size_bytes on image_exif to support future hash-keyed
thumbnail/HLS dedup. Tags and favorites stay library-agnostic so they
share across libraries by rel_path.
Behavior is unchanged: a single primary library (id=1) is seeded from
BASE_PATH on first boot; all handlers and DAOs route through it as a
transitional shim until the API gains a library query param.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds --temperature, --top-p, --top-k, --min-p flags so batch runs can
tune the same sampling params now supported by the API endpoints.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Expose Ollama sampling params through the insight generation endpoints
so users can tune creativity/determinism per request. All four are
optional — omitted values fall through to the model's server-side
defaults.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds normalize_entity_type() which lowercases and canonicalises synonyms
(location→place, human→person, etc.) before every upsert. The SQL lookup
now uses lower(entity_type) on both sides so existing dirty rows (Person,
Location) correctly deduplicate against normalised writes without a migration.
Adds a pre-flight similarity check in tool_store_entity: before upserting,
searches active entities of the same type using the first name token. Any
non-exact matches are appended to the tool response so the agentic loop
can choose to reuse an existing entity ID rather than create a duplicate.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a standalone binary that walks a directory and runs the agentic
insight loop over every image/video, skipping files already processed.
Supports --path, --model, --max-iterations, --timeout-secs, --num-ctx,
and --reprocess flags for flexible overnight/VPS batch runs.
Also adds OllamaClient::with_request_timeout() builder method so slow
large models are not cut off by the default 120s limit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Date sorting previously used a DB-level query that acted as an inner join,
silently dropping files with no image_exif row. Replace it with the existing
in-memory sort which already falls back to filename-extracted and filesystem
dates, so all files appear in sorted results.
Also removes the now-unused get_files_sorted_by_date trait method and its
SqliteExifDao implementation and test mock.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements persistent cross-photo knowledge memory so the agentic
insight loop can learn and recall facts about people, places, and
events across the photo collection.
Changes:
- photo_insights: drop UNIQUE(file_path) + INSERT OR REPLACE, replace
with append-only rows + is_current flag for insight history retention
- New tables: entities, entity_facts, entity_photo_links with FK
constraints and confidence scoring
- KnowledgeDao trait + SqliteKnowledgeDao with upsert, merge, and
corroboration (confidence +0.1 on duplicate fact detection)
- Four new agent tools: recall_entities, recall_facts_for_photo,
store_entity, store_fact (with object_entity_id FK support)
- Cameron entity auto-seeded with stable ID injected into system prompt
- Pre-run photo link clearing + post-loop source_insight_id backfill
- Audit REST API: GET/PATCH/DELETE /knowledge/entities/{id},
POST /knowledge/entities/merge, GET/PATCH/DELETE /knowledge/facts/{id},
GET /knowledge/recent
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Captures prompt_eval_count and eval_count from Ollama /api/chat responses
during the agentic loop and returns them in POST /insights/generate/agentic
so the frontend can display context window usage to the user.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Sanitise tool call arguments before re-sending in conversation history: non-object values (bool, string, null) that some models produce are normalised to {} to prevent Ollama 500s
- Map 'error parsing tool call' Ollama 500 to HTTP 400 with a descriptive message listing compatible models (llama3.1, llama3.2, qwen2.5, mistral-nemo)
- Add reverse_geocode tool backed by existing Nominatim helper; description hints model can chain it after get_location_history results
- Make get_sms_messages contact parameter optional (was required, forcing the model to guess); executor now passes None to fall back to all-contacts search
- Log tool result outcomes at warn level for errors/empty results, info for successes; log SMS API errors with full detail; log full request body on Ollama 500
- Strengthen system prompt to require 3-4 tool calls before final answer
- Try fallback server when checking model capabilities (primary-only check caused 500 for models only on fallback)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- cargo fmt applied across all modified source files
- Collapse nested if let Some / if !is_empty into a single let-chain (clippy::collapsible_match)
- All other warnings are pre-existing dead-code lint on unused trait methods
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Verify custom model exists on at least one configured server before starting agentic loop; returns HTTP 400 with descriptive error if not found
- has_tool_calling field auto-serialised in GET /insights/models via existing ModelCapabilities Serialize derive
- model_version stored from OllamaClient.primary_model (already correct in T006 implementation)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Register the agentic insight endpoint that validates tool-calling capability,
runs the agentic loop, and returns the stored PhotoInsightResponse. Returns 400
for unsupported models, 500 for other errors. Max iterations configurable via
AGENTIC_MAX_ITERATIONS env var (default 10).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add 6 tool executor methods (search_rag, get_sms_messages, get_calendar_events,
get_location_history, get_file_tags, describe_photo) and the agentic loop that
uses Ollama's chat_with_tools API to let the model decide which context to gather
before writing the final photo insight.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Aligns SqliteTagDao with the pattern used by SqliteExifDao and SqliteInsightDao.
The unsafe impl Sync workaround is no longer needed since Arc<Mutex<>> provides
safe interior mutability and automatic Sync derivation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Threads SqliteTagDao through InsightGenerator and AppState (both default
and test_state). Adds Send+Sync bounds to TagDao trait with unsafe impls
for SqliteTagDao (always Mutex-protected) and TestTagDao (single-threaded).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Configures a global JsonConfig error handler that logs the method, URI,
and parse error details at WARN level before returning the 400 response.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add format=yuv420p to preview clip filter chains to convert 10-bit
sources to 8-bit before encoding, since NVENC doesn't support 10-bit
H.264.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Implement unit tests for PreviewClipRequest/PreviewStatusRequest serialization and deserialization.
- Add tests for PreviewDao (insert, update, batch retrieval, and status-based queries).
- Extend Actix-web integration tests for `/video/preview/status` endpoint scenarios.
- Introduce in-memory TestPreviewDao for mock database interactions.
- Update README with new config parameters for preview clips.
Backend (Rust/Actix-web):
- Add video_preview_clips table and PreviewDao for tracking preview generation
- Add ffmpeg preview clip generator: 10 equally-spaced 1s segments at 480p with CUDA NVENC auto-detection
- Add PreviewClipGenerator actor with semaphore-limited concurrent processing
- Add GET /video/preview and POST /video/preview/status endpoints
- Extend file watcher to detect and queue previews for new videos
- Use relative paths consistently for DB storage (matching EXIF convention)
Frontend (React Native/Expo):
- Add VideoWall grid view with 2-3 column layout of looping preview clips
- Add VideoWallItem component with ActiveVideoPlayer sub-component for lifecycle management
- Add useVideoWall hook for batch status polling with 5s refresh
- Add navigation button in grid header (visible when videos exist)
- Use TextureView surface type to fix Android z-ordering issues
- Optimize memory: players only mount while visible via FlatList windowSize
- Configure ExoPlayer buffer options and caching for short clips
- Tap to toggle audio focus, long press to open in full viewer
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
