Apps/ImageApi
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Apollo Places: enrich insights with personal place name + notes

Browse Source 
Optional integration with the sibling Apollo project's user-defined
Places (name + lat/lon + radius_m + description + category). When
APOLLO_API_BASE_URL is set, the per-photo location resolver folds the
most-specific containing Place into the LLM prompt's location string —
"Home (My house in Cambridge) — near Cambridge, MA" rather than the
city name alone. Smallest-radius wins; Apollo sorts server-side via
/api/places/contains, so the carousel badge in Apollo and the prompt
string here always agree.

Adds an agentic tool `get_personal_place_at(latitude, longitude)` that
the LLM can call during chat continuation. Tool description tells the
model the call returns the user's free-text notes, not just a name.
Deliberately narrow — no enumerate-all variant, lat/lon required.

Unset APOLLO_API_BASE_URL = legacy Nominatim-only path, tool is not
registered. 5 s timeout; all errors degrade silently.

Tests: 5 unit tests for compose_location_string (Apollo only, Nominatim
only, both, both-with-description, neither).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Cameron Cordes
2026-04-28 19:11:12 +00:00
parent 9d58547ce3
commit 4ae7be35e9
7 changed files with 339 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
src/ai/insight_chat.rs
View File

@@ -359,7 +359,10 @@ impl InsightChatService {
.map(|imgs| !imgs.is_empty())
.unwrap_or(false);
let offer_describe_tool = !is_hybrid && local_first_user_has_image;
let tools = InsightGenerator::build_tool_definitions(offer_describe_tool);
let tools = InsightGenerator::build_tool_definitions(
offer_describe_tool,
self.generator.apollo_enabled(),
);
// Image base64 only needed when describe_photo is on the menu. Load
// lazily to avoid disk IO when the loop never invokes it.
@@ -786,7 +789,10 @@ impl InsightChatService {
.map(|imgs| !imgs.is_empty())
.unwrap_or(false);
let offer_describe_tool = !is_hybrid && local_first_user_has_image;
let tools = InsightGenerator::build_tool_definitions(offer_describe_tool);
let tools = InsightGenerator::build_tool_definitions(
offer_describe_tool,
self.generator.apollo_enabled(),
);
let image_base64: Option<String> = if offer_describe_tool {
self.generator.load_image_as_base64(&normalized).ok()