Apps/ImageApi
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

feat(ai): USER_NAME env + shared summary prompt + test-bin knobs

Browse Source 
Introduces USER_NAME (default "Me") as the single source for the message
sender label and the first-person persona across daily summaries, SMS
context, insight generation, and chat. Eliminates the "Me:" transcript /
"what I did" ambiguity that confused smaller models, and unhardcodes
"Cameron" from prompt text + the knowledge-graph owner entity. Set
USER_NAME=Cameron in .env to preserve the existing owner entity row
(keyed on UNIQUE(name, entity_type)) — otherwise the next run creates
a fresh owner entity and orphans the existing facts/photo-links.

Also:
- search_messages redirect: when the model calls it with date/contact
  but no query, return a hint pointing at get_sms_messages instead of
  a bare missing-parameter error (prevents same-turn retry loops)
- sharpen search_messages vs get_sms_messages tool descriptions so
  content-vs-time-based intent is unambiguous
- extract build_daily_summary_prompt (+ DAILY_SUMMARY_MESSAGE_LIMIT,
  DAILY_SUMMARY_SYSTEM_PROMPT) shared by daily_summary_job and
  test_daily_summary binary — prompt tweaks now land in both
- EMBEDDING_MODEL const; fixes both insert sites that stored
  "mxbai-embed-large:335m" while generate_embeddings actually runs
  "nomic-embed-text:v1.5"
- test_daily_summary: add --num-ctx / --temperature / --top-p /
  --top-k / --min-p flags wired into OllamaClient setters, and print
  the configured knobs at the top of each run
- OllamaClient::generate now logs prompt/gen token counts and tok/s
  via log_chat_metrics (symmetric with chat_with_tools)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Cameron
2026-04-22 23:39:37 -04:00
parent e4a3536f87
commit 6831f50993
6 changed files with 226 additions and 156 deletions
Download Patch File Download Diff File Expand all files Collapse all files

79
src/ai/insight_generator.rs
View File

@@ -13,6 +13,7 @@ use crate::ai::llm_client::LlmClient;
use crate::ai::ollama::{ChatMessage, OllamaClient, Tool};
use crate::ai::openrouter::OpenRouterClient;
use crate::ai::sms_client::SmsApiClient;
use crate::ai::user_display_name;
use crate::database::models::InsertPhotoInsight;
use crate::database::{
CalendarEventDao, DailySummaryDao, ExifDao, InsightDao, KnowledgeDao, LocationHistoryDao,
@@ -1260,10 +1261,14 @@ impl InsightGenerator {
// Format a sample of messages for topic extraction
let sample_size = messages.len().min(20);
let user_name = user_display_name();
let sample_text: Vec<String> = messages
.iter()
.take(sample_size)
.map(|m| format!("{}: {}", if m.is_sent { "Me" } else { &m.contact }, m.body))
.map(|m| {
let sender: &str = if m.is_sent { &user_name } else { &m.contact };
format!("{}: {}", sender, m.body)
})
.collect();
let prompt = format!(
@@ -1361,10 +1366,11 @@ Return ONLY the summary, nothing else."#,
}
// Format messages
let user_name = user_display_name();
let formatted: Vec<String> = messages
.iter()
.map(|m| {
let sender = if m.is_sent { "Me" } else { &m.contact };
let sender: &str = if m.is_sent { &user_name } else { &m.contact };
let timestamp = chrono::DateTime::from_timestamp(m.timestamp, 0)
.map(|dt| {
dt.with_timezone(&Local)
@@ -1624,7 +1630,21 @@ Return ONLY the summary, nothing else."#,
async fn tool_search_messages(&self, args: &serde_json::Value) -> String {
let query = match args.get("query").and_then(|v| v.as_str()) {
Some(q) if !q.trim().is_empty() => q.trim(),
_ => return "Error: missing required parameter 'query'".to_string(),
_ => {
// Redirect when the model reached for this tool with a
// date/contact-shaped intent — get_sms_messages is the right
// call. Without this hint, small models often just retry
// search_messages again with the same args.
let has_date = args.get("date").is_some();
let has_contact = args.get("contact").is_some();
if has_date || has_contact {
return "Error: search_messages needs a 'query' (keywords/phrase). \
To fetch messages around a date or from a contact, call \
get_sms_messages with { date, contact? } instead."
.to_string();
}
return "Error: missing required parameter 'query'".to_string();
}
};
if query.len() < 3 {
return "Error: query must be at least 3 characters".to_string();
@@ -1662,11 +1682,12 @@ Return ONLY the summary, nothing else."#,
hits.len(),
mode
));
let user_name = user_display_name();
for h in hits {
let date = chrono::DateTime::from_timestamp(h.date, 0)
.map(|dt| dt.format("%Y-%m-%d").to_string())
.unwrap_or_else(|| h.date.to_string());
let direction = if h.type_ == 2 { "Me" } else { &h.contact_name };
let direction: &str = if h.type_ == 2 { &user_name } else { &h.contact_name };
let score = h
.similarity_score
.map(|s| format!(" [score {:.2}]", s))
@@ -1726,11 +1747,12 @@ Return ONLY the summary, nothing else."#,
.await
{
Ok(messages) if !messages.is_empty() => {
let user_name = user_display_name();
let formatted: Vec<String> = messages
.iter()
.take(limit)
.map(|m| {
let sender = if m.is_sent { "Me" } else { &m.contact };
let sender: &str = if m.is_sent { &user_name } else { &m.contact };
let ts = DateTime::from_timestamp(m.timestamp, 0)
.map(|dt| {
dt.with_timezone(&Local)
@@ -2359,7 +2381,7 @@ Return ONLY the summary, nothing else."#,
),
Tool::function(
"search_messages",
"Keyword/semantic/hybrid search over ALL SMS message bodies (not just summaries) across all time. Prefer this for specific phrases, proper nouns, URLs, or when you don't know the date. Modes: 'fts5' (keyword, supports \"phrase\" / prefix* / AND / NEAR(w1 w2, 5)), 'semantic' (embedding similarity), 'hybrid' (recommended — merges both via reciprocal rank fusion).",
"CONTENT search over SMS message bodies by keywords/phrases/topics across all time. Use when you're looking for specific wording (phrases, proper nouns, URLs, topics) and DON'T have a date in mind. NOT for time-based queries — if you know the date or want messages around a date, call get_sms_messages instead. Modes: 'fts5' (keyword, supports \"phrase\" / prefix* / AND / NEAR(w1 w2, 5)), 'semantic' (embedding similarity), 'hybrid' (recommended — merges both via reciprocal rank fusion).",
serde_json::json!({
"type": "object",
"required": ["query"],
@@ -2382,7 +2404,7 @@ Return ONLY the summary, nothing else."#,
),
Tool::function(
"get_sms_messages",
"Fetch SMS/text messages near a specific date. Returns the actual message conversation. Omit contact to search across all conversations.",
"TIME-BASED fetch of SMS/text messages around a specific date (and optionally from a specific contact). Returns the actual message conversation for that window. Use this whenever you know the date or want the context around a photo's timestamp. Omit contact to search across all conversations. For keyword/topic search without a date, use search_messages instead.",
serde_json::json!({
"type": "object",
"required": ["date"],
@@ -2561,7 +2583,7 @@ Return ONLY the summary, nothing else."#,
},
"object_entity_id": {
"type": "integer",
"description": "Use when the object is a known entity (e.g. Cameron's entity ID for 'is_friend_of Cameron'). Takes precedence over object_value."
"description": "Use when the object is a known entity (e.g. another person's entity ID for 'is_friend_of <that person>'). Takes precedence over object_value."
},
"object_value": {
"type": "string",
@@ -2871,8 +2893,9 @@ Return ONLY the summary, nothing else."#,
};
// 6. Clear existing entity-photo links for this file so the run starts fresh,
// and ensure the owner entity (Cameron) exists so the agent can reference it.
let cameron_entity_id: Option<i32> = {
// and ensure the owner entity exists so the agent can reference it.
let owner_name = user_display_name();
let owner_entity_id: Option<i32> = {
let mut kdao = self
.knowledge_dao
.lock()
@@ -2888,9 +2911,12 @@ Return ONLY the summary, nothing else."#,
// Upsert the owner entity so the agent always has a stable entity ID to reference.
let owner = crate::database::models::InsertEntity {
name: "Cameron".to_string(),
name: owner_name.clone(),
entity_type: "person".to_string(),
description: "The owner of this photo collection. All memories are written from Cameron's perspective.".to_string(),
description: format!(
"The owner of this photo collection. All memories are written from {}'s perspective.",
owner_name
),
embedding: None,
confidence: 1.0,
status: "active".to_string(),
@@ -2899,11 +2925,11 @@ Return ONLY the summary, nothing else."#,
};
match kdao.upsert_entity(&insight_cx, owner) {
Ok(e) => {
log::info!("Cameron entity ID: {}", e.id);
log::info!("Owner entity '{}' ID: {}", owner_name, e.id);
Some(e.id)
}
Err(e) => {
log::warn!("Failed to upsert Cameron entity: {:?}", e);
log::warn!("Failed to upsert owner entity '{}': {:?}", owner_name, e);
None
}
}
@@ -2953,28 +2979,30 @@ Return ONLY the summary, nothing else."#,
};
// 8. Build system message
let cameron_id_note = match cameron_entity_id {
let owner_id_note = match owner_entity_id {
Some(id) => format!(
"\n\nYour identity in the knowledge store: Cameron (entity ID: {}). \
When storing facts where you (Cameron) are the object — for example, someone is your friend, \
"\n\nYour identity in the knowledge store: {name} (entity ID: {id}). \
When storing facts where you ({name}) are the object — for example, someone is your friend, \
sibling, or colleague — use subject_entity_id for the other person and set object_value to \
\"Cameron\" (or use store_fact with the other person as subject). When storing facts about \
Cameron directly, use {} as the subject_entity_id.",
id, id
\"{name}\" (or use store_fact with the other person as subject). When storing facts about \
{name} directly, use {id} as the subject_entity_id.",
name = owner_name,
id = id
),
None => String::new(),
};
let base_system = format!(
"You are a personal photo memory assistant helping to reconstruct a memory from a photo.{cameron_id_note}\n\n\
"You are a personal photo memory assistant helping to reconstruct a memory from a photo.{owner_id_note}\n\n\
IMPORTANT INSTRUCTIONS:\n\
1. You MUST call multiple tools to gather context BEFORE writing any final insight. Do not produce a final answer after only one or two tool calls.\n\
2. When calling get_sms_messages and search_rag, always make at least one call WITHOUT a contact filter to capture what else was happening in Cameron's life around this date — other conversations, events, and activities provide important wider context even when a specific contact is known.\n\
2. When calling get_sms_messages and search_rag, always make at least one call WITHOUT a contact filter to capture what else was happening in {owner_name}'s life around this date — other conversations, events, and activities provide important wider context even when a specific contact is known.\n\
3. Use recall_facts_for_photo to load any previously stored knowledge about subjects in this photo.\n\
4. Use recall_entities to look up known people, places, or things that appear in this photo.\n\
5. When you identify people, places, events, or notable things in this photo: use store_entity to record them and store_fact to record key facts (relationships, roles, attributes). This builds a persistent memory for future insights.\n\
6. Only produce your final insight AFTER you have gathered context from at least 5 tool calls.\n\
7. If a tool returns no results, that is useful information — continue calling the remaining tools anyway.",
cameron_id_note = cameron_id_note
owner_id_note = owner_id_note,
owner_name = owner_name
);
let system_content = if let Some(ref custom) = custom_system_prompt {
format!("{}\n\n{}", custom, base_system)
@@ -3125,7 +3153,10 @@ Return ONLY the summary, nothing else."#,
iterations_used
);
messages.push(ChatMessage::user(
"Based on the context gathered, please write the final photo insight: a title and a detailed personal summary. Write in first person as Cameron.",
&format!(
"Based on the context gathered, please write the final photo insight: a title and a detailed personal summary. Write in first person as {}.",
user_display_name()
),
));
let (final_response, prompt_tokens, eval_tokens) = chat_backend
.chat_with_tools(messages.clone(), vec![])