Rewind: POST /insights/chat/rewind truncates training_messages at a
given rendered index, dropping the target message plus any preceding
tool-call scaffolding. The initial user prompt is protected.
Metrics: log prompt_eval_count/duration and eval_count/duration from
every Ollama chat response, rendered as tokens + ms + tok/s.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds POST /insights/chat and GET /insights/chat/history. Replays the
stored agentic conversation through the same backend the insight was
generated with (or a per-turn override), runs a short tool-calling
loop, and persists the extended history in append or amend mode.
Backend switching: same-backend or hybrid->local replay verbatim;
local->hybrid is rejected in v1 (would require on-the-fly vision
description rewrite).
Per-(library, file) async mutex serialises concurrent turns. Soft
context budget drops oldest tool_call+result pairs when the
serialized history exceeds num_ctx - 2048 tokens.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add OPENROUTER_ALLOWED_MODELS env var and GET /insights/openrouter/models
endpoint returning the curated list verbatim. Drop the live capability
precheck in hybrid mode — trust the operator's allowlist; bad ids surface
as a chat-call error.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a `backend` column to photo_insights (default 'local', migration
2026-04-20-000000) and a corresponding optional `backend` field on the
agentic request. When a request sets backend=hybrid:
- The local Ollama vision model is called once via describe_image to
produce a text description.
- The description is inlined into the first user message as text —
no base64 image is ever sent to the chat model.
- The agentic tool-calling loop and title generation route through an
OpenRouterClient (dispatched via &dyn LlmClient), letting the user
pick any tool-capable model from OpenRouter per request.
- describe_photo is removed from the offered tools since the description
is already present.
Embeddings and vision stay on local Ollama regardless of backend.
Hybrid mode requires OPENROUTER_API_KEY; handlers return a clear error
when hybrid is requested without it, and also when the selected
OpenRouter model lacks tool-calling support.
AppState gains an optional openrouter client built from
OPENROUTER_API_KEY / OPENROUTER_BASE_URL / OPENROUTER_DEFAULT_MODEL /
OPENROUTER_EMBEDDING_MODEL / attribution headers. Default model is
anthropic/claude-sonnet-4.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
OpenAI-compatible client for OpenRouter. Translates canonical wire shapes at
the boundary: tool-call arguments stringify on send / parse on receive
(accepting both string and native-object forms); images rewritten from the
base64 images field into content-parts with image_url entries; role=tool
messages inherit tool_call_id from the preceding assistant's tool calls.
/models parsed into ModelCapabilities via supported_parameters (tool use)
and architecture.input_modalities (vision). 15-minute capabilities cache.
Bearer auth; HTTP-Referer / X-Title attribution headers optional.
Not wired into request routing yet — first consumer arrives with hybrid
backend mode. 11 unit tests cover the translation helpers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Preparation for a second LLM backend (OpenRouter) and hybrid vision-local /
chat-remote mode. Shared wire types (ChatMessage, Tool, ToolCall, etc.) move
into a new src/ai/llm_client.rs and are re-exported from ollama.rs so
existing imports keep working. OllamaClient now implements LlmClient.
No behavior change; callers still hold the concrete OllamaClient. Caller
migration to Arc<dyn LlmClient> is deferred to the PR that wires hybrid
backend routing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Doubled default row caps for search_rag/get_sms_messages/get_calendar_events/recall_entities and exposed an optional `limit` parameter on each so the agent can tune per call. Render all LLM-facing timestamps as server-local time with explicit offset so smaller models stop misreading UTC as wall-clock time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Silence forward-looking dead_code on unused DAO modules, annotate
individual placeholder items, rewrite tautological assert!(true/false)
in token tests as panic! arms, and pick up fmt drift.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The /photos/gps-summary handler validated the incoming path against
the primary library's root with new_file=false, which requires the
path to exist on disk. For a viewer opened on a file from a
non-primary library, tapping the GPS link produced activePath =
<folder from lib 2>, the primary-only check failed, and the server
400'd — so the map came up empty.
Validation here is purely a traversal guard (the DAO does a prefix
LIKE against rel_path), so we now accept the path as long as any
configured library can resolve it without escaping its root.
Also applies cargo fmt drift on files touched this session.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The /video/generate and /image/metadata handlers assumed files live under
the resolved library only, which broke when a mobile client passed no
library (union mode) but the file lived in a non-primary library. Both
now fall back to scanning every configured library for an existing file.
InsightGenerator held a single base_path, so vision-model loads and
filename-date fallbacks failed for non-primary libraries. It now takes
Vec<Library> and probes each root in resolve_full_path.
/image/metadata responses now carry library_id/library_name so the
mobile viewer can surface which library a file belongs to.
Thumbnail generation at startup is now spawned on a background thread
so the HTTP server can accept traffic while large libraries backfill.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tags and insights now follow content across libraries via content_hash
lookups on the read path, so the same file indexed at different rel_paths
in multiple libraries shares its annotations. Recursive tag search scopes
hits to the selected library by checking each tagged rel_path against
the library's disk (with a content-hash sibling fallback so tags attached
under one library's rel_path still match a content-equivalent file in
another). The /image and /image/metadata handlers fall back across
libraries when the file isn't under the resolved one, so union-mode
search results (which carry no library attribution in the response)
still serve correctly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a `libraries` registry table and threads library_id through
per-instance metadata tables (image_exif, photo_insights,
entity_photo_links, video_preview_clips). File-path columns renamed to
rel_path to make the relative-to-root semantics explicit. Adds
content_hash + size_bytes on image_exif to support future hash-keyed
thumbnail/HLS dedup. Tags and favorites stay library-agnostic so they
share across libraries by rel_path.
Behavior is unchanged: a single primary library (id=1) is seeded from
BASE_PATH on first boot; all handlers and DAOs route through it as a
transitional shim until the API gains a library query param.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Expose Ollama sampling params through the insight generation endpoints
so users can tune creativity/determinism per request. All four are
optional — omitted values fall through to the model's server-side
defaults.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds normalize_entity_type() which lowercases and canonicalises synonyms
(location→place, human→person, etc.) before every upsert. The SQL lookup
now uses lower(entity_type) on both sides so existing dirty rows (Person,
Location) correctly deduplicate against normalised writes without a migration.
Adds a pre-flight similarity check in tool_store_entity: before upserting,
searches active entities of the same type using the first name token. Any
non-exact matches are appended to the tool response so the agentic loop
can choose to reuse an existing entity ID rather than create a duplicate.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a standalone binary that walks a directory and runs the agentic
insight loop over every image/video, skipping files already processed.
Supports --path, --model, --max-iterations, --timeout-secs, --num-ctx,
and --reprocess flags for flexible overnight/VPS batch runs.
Also adds OllamaClient::with_request_timeout() builder method so slow
large models are not cut off by the default 120s limit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Captures prompt_eval_count and eval_count from Ollama /api/chat responses
during the agentic loop and returns them in POST /insights/generate/agentic
so the frontend can display context window usage to the user.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Sanitise tool call arguments before re-sending in conversation history: non-object values (bool, string, null) that some models produce are normalised to {} to prevent Ollama 500s
- Map 'error parsing tool call' Ollama 500 to HTTP 400 with a descriptive message listing compatible models (llama3.1, llama3.2, qwen2.5, mistral-nemo)
- Add reverse_geocode tool backed by existing Nominatim helper; description hints model can chain it after get_location_history results
- Make get_sms_messages contact parameter optional (was required, forcing the model to guess); executor now passes None to fall back to all-contacts search
- Log tool result outcomes at warn level for errors/empty results, info for successes; log SMS API errors with full detail; log full request body on Ollama 500
- Strengthen system prompt to require 3-4 tool calls before final answer
- Try fallback server when checking model capabilities (primary-only check caused 500 for models only on fallback)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- cargo fmt applied across all modified source files
- Collapse nested if let Some / if !is_empty into a single let-chain (clippy::collapsible_match)
- All other warnings are pre-existing dead-code lint on unused trait methods
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Verify custom model exists on at least one configured server before starting agentic loop; returns HTTP 400 with descriptive error if not found
- has_tool_calling field auto-serialised in GET /insights/models via existing ModelCapabilities Serialize derive
- model_version stored from OllamaClient.primary_model (already correct in T006 implementation)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Register the agentic insight endpoint that validates tool-calling capability,
runs the agentic loop, and returns the stored PhotoInsightResponse. Returns 400
for unsupported models, 500 for other errors. Max iterations configurable via
AGENTIC_MAX_ITERATIONS env var (default 10).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add 6 tool executor methods (search_rag, get_sms_messages, get_calendar_events,
get_location_history, get_file_tags, describe_photo) and the agentic loop that
uses Ollama's chat_with_tools API to let the model decide which context to gather
before writing the final photo insight.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Threads SqliteTagDao through InsightGenerator and AppState (both default
and test_state). Adds Send+Sync bounds to TagDao trait with unsafe impls
for SqliteTagDao (always Mutex-protected) and TestTagDao (single-threaded).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Updated InsightGenerator struct with calendar, location, and search DAOs
- Implemented hybrid context gathering methods:
* gather_calendar_context(): ±7 days with semantic ranking
* gather_location_context(): ±30 min with GPS proximity check
* gather_search_context(): ±30 days semantic search
- Added haversine_distance() utility for GPS calculations
- Updated generate_insight_for_photo_with_model() to use multi-source context
- Combined all context sources (SMS + Calendar + Location + Search) with equal weight
- Initialized new DAOs in AppState (both default and test implementations)
- All contexts are optional (graceful degradation if data missing)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implements Phase 1 & 2 of Google Takeout RAG integration:
- Database migrations for calendar_events, location_history, search_history
- DAO implementations with hybrid time + semantic search
- Parsers for .ics, JSON, and HTML Google Takeout formats
- Import utilities with batch insert optimization
Features:
- CalendarEventDao: Hybrid time-range + semantic search for events
- LocationHistoryDao: GPS proximity with Haversine distance calculation
- SearchHistoryDao: Semantic-first search (queries are embedding-rich)
- Batch inserts for performance (1M+ records in minutes vs hours)
- OpenTelemetry tracing for all database operations
Import utilities:
- import_calendar: Parse .ics with optional embedding generation
- import_location_history: High-volume GPS data with batch inserts
- import_search_history: Always generates embeddings for semantic search
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
