002-agentic-insights #53

cameron · 2026-04-02T16:46:01Z

cameron commented

2026-04-02 16:46:01 +00:00

Adds agentic loop when generating insights

cameron added 13 commits 2026-04-02 16:46:02 +00:00

feat: add generate_photo_description() to OllamaClient for RAG enrichment dd0715c081

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

refactor: use &str for generate_photo_description image parameter b31b4b903c

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add tag_dao to InsightGenerator for tag-based context enrichment 387ce23afd

Threads SqliteTagDao through InsightGenerator and AppState (both default
and test_state). Adds Send+Sync bounds to TagDao trait with unsafe impls
for SqliteTagDao (always Mutex-protected) and TestTagDao (single-threaded).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

refactor: use Arc<Mutex<SqliteConnection>> in SqliteTagDao, remove unsafe impl Sync 8ecd3c6cf8

Aligns SqliteTagDao with the pattern used by SqliteExifDao and SqliteInsightDao.
The unsafe impl Sync workaround is no longer needed since Arc<Mutex<>> provides
safe interior mutability and automatic Sync derivation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add Tags section to combine_contexts() for insight context c0d27d0b9e

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add enrichment parameter to gather_search_context() replacing weak metadata query e58b8fe743

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: photo-first RAG enrichment — early vision description + tags in RAG and search context 8196ef94a0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add tool-calling types, chat_with_tools(), and has_tool_calling capability detection 5e5a2a3167

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add tool executors and generate_agentic_insight_for_photo() to InsightGenerator 7615b9c99b

Add 6 tool executor methods (search_rag, get_sms_messages, get_calendar_events,
get_location_history, get_file_tags, describe_photo) and the agentic loop that
uses Ollama's chat_with_tools API to let the model decide which context to gather
before writing the final photo insight.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add POST /insights/generate/agentic handler and route 091327e5d9

Register the agentic insight endpoint that validates tool-calling capability,
runs the agentic loop, and returns the stored PhotoInsightResponse. Returns 400
for unsupported models, 500 for other errors. Max iterations configurable via
AGENTIC_MAX_ITERATIONS env var (default 10).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add model-availability validation to agentic insight generation (T009-T011) 5c9f5c7d0b

- Verify custom model exists on at least one configured server before starting agentic loop; returns HTTP 400 with descriptive error if not found
- has_tool_calling field auto-serialised in GET /insights/models via existing ModelCapabilities Serialize derive
- model_version stored from OllamaClient.primary_model (already correct in T006 implementation)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chore: cargo fmt + clippy fix for collapsed if-let chain (T017) c1b6013412

- cargo fmt applied across all modified source files
- Collapse nested if let Some / if !is_empty into a single let-chain (clippy::collapsible_match)
- All other warnings are pre-existing dead-code lint on unused trait methods

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix: agentic loop robustness — tool arg sanitisation, geocoding, better errors 54a49a8562

- Sanitise tool call arguments before re-sending in conversation history: non-object values (bool, string, null) that some models produce are normalised to {} to prevent Ollama 500s
- Map 'error parsing tool call' Ollama 500 to HTTP 400 with a descriptive message listing compatible models (llama3.1, llama3.2, qwen2.5, mistral-nemo)
- Add reverse_geocode tool backed by existing Nominatim helper; description hints model can chain it after get_location_history results
- Make get_sms_messages contact parameter optional (was required, forcing the model to guess); executor now passes None to fall back to all-contacts search
- Log tool result outcomes at warn level for errors/empty results, info for successes; log SMS API errors with full detail; log full request body on Ollama 500
- Strengthen system prompt to require 3-4 tool calls before final answer
- Try fallback server when checking model capabilities (primary-only check caused 500 for models only on fallback)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

cameron merged commit 50cf526b46 into master

2026-04-02 16:46:07 +00:00

cameron deleted branch 002-agentic-insights

2026-04-02 16:46:08 +00:00

cameron referenced this issue from a commit

2026-04-02 16:46:09 +00:00

Merge pull request '002-agentic-insights' (#53) from 002-agentic-insights into master

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Apps/ImageApi#53