faces: phase 2 — schema + manual face/person CRUD
Land the persistence model and HTTP surface for local face recognition.
Inference still lives in Apollo (Phase 1); this side adds the data home
plus every endpoint Apollo's UI and FileViewer-React will consume.
Schema (new migration 2026-04-29-000000_add_faces):
- persons: visual identities. Optional entity_id bridges to the
existing knowledge-graph entities table; auto-bridging is left to
the management UI (we don't muddy LLM provenance from face rows).
UNIQUE(name COLLATE NOCASE) so 'alice' / 'Alice' fold to one row.
- face_detections: keyed on content_hash (cross-library dedup), with
status='detected' carrying bbox + 512-d embedding BLOB, and
'no_faces' / 'failed' marker rows that tell Phase 3's file watcher
not to re-scan. Marker invariant enforced via CHECK; partial UNIQUE
on content_hash WHERE status='no_faces' guards against double-marks.
Schema regenerated with `diesel print-schema` against a clean migration
run; joinables added for face_detections → libraries / persons and
persons → entities.
face_client.rs (sibling of apollo_client.rs):
- reqwest multipart, 60 s timeout (CPU inference on a backlog can be
slow; bounded threadpool on Apollo serializes calls anyway).
- FaceDetectError::{Permanent, Transient, Disabled} — Phase 3 keys
its marker-row decision on this. 422 → mark failed, 5xx → defer.
- APOLLO_FACE_API_BASE_URL falls back to APOLLO_API_BASE_URL when
unset; both unset = is_enabled() false, callers no-op.
faces.rs (DAO + handlers):
- SqliteFaceDao implements the full FaceDao trait; person face counts
go through sql_query because diesel's BoxedSelectStatement +
group_by trips trait-resolver recursion.
- merge_persons re-points face rows in a transaction, copies notes
when target's are empty, deletes src.
- manual POST /image/faces resolves content_hash through image_exif,
crops the user-drawn bbox with 10% padding (detector wants context
around ears/jaw), POSTs the crop to face_client.embed for a real
ArcFace vector, then inserts source='manual'.
- Cluster-suggest (Phase 6) gets its data from
GET /faces/embeddings — base64-encoded paged BLOBs so Apollo's
DBSCAN can stream them without ImageApi pre-aggregating.
Endpoints registered alongside add_*_services in main.rs:
GET /faces/stats?library=
GET /faces/embeddings?library=&unassigned=&limit=&offset=
GET /image/faces?path=&library=
POST /image/faces (manual create via embed)
PATCH /image/faces/{id}
DELETE /image/faces/{id}
GET /persons?library=
POST /persons
GET /persons/{id}
PATCH /persons/{id}
DELETE /persons/{id}?cascade=set_null|delete (set_null default)
POST /persons/{id}/merge
GET /persons/{id}/faces?library=
The file-watch hook (Phase 3) and the rerun-on-one-photo handler
(Phase 6) live behind the FaceDao methods marked dead_code today —
they're called only when those phases land. Same shape for the trait
methods that aren't reached by Phase 2 routes.
Tests: 3 DAO unit tests cover person CRUD + case-insensitive uniqueness,
marker-row idempotency (mark_status is a no-op when any row exists),
and merge re-pointing faces.
Cargo.toml: reqwest gains the `multipart` feature.
cargo build / cargo test --lib / cargo fmt / cargo clippy --all-targets
all clean for the new code; the two pre-existing test_path_excluder
failures and the pre-existing sort_by clippy warnings are unrelated and
present on master.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -286,6 +286,15 @@ SMS_API_TOKEN=your-api-token # SMS API authentication token (o
|
|||||||
# `get_personal_place_at` tool. Unset = legacy Nominatim-only path.
|
# `get_personal_place_at` tool. Unset = legacy Nominatim-only path.
|
||||||
APOLLO_API_BASE_URL=http://apollo.lan:8000 # Base URL of the sibling Apollo backend
|
APOLLO_API_BASE_URL=http://apollo.lan:8000 # Base URL of the sibling Apollo backend
|
||||||
|
|
||||||
|
# Face inference (optional). Apollo also hosts the insightface inference
|
||||||
|
# service; ImageApi calls it from the file-watch hook (Phase 3) and from
|
||||||
|
# the manual face-create endpoint. Falls back to APOLLO_API_BASE_URL when
|
||||||
|
# unset (typical single-Apollo deploy). Both unset = feature disabled.
|
||||||
|
APOLLO_FACE_API_BASE_URL=http://apollo.lan:8000 # Override if face service runs separately
|
||||||
|
FACE_AUTOBIND_MIN_COS=0.4 # Phase 3: cosine-sim floor for tag-name auto-bind
|
||||||
|
FACE_DETECT_CONCURRENCY=8 # Phase 3: per-scan-tick parallel detect calls
|
||||||
|
FACE_DETECT_TIMEOUT_SEC=60 # reqwest client timeout (CPU inference can be slow)
|
||||||
|
|
||||||
# OpenRouter (Hybrid Backend) - keeps embeddings + vision local, routes chat to OpenRouter
|
# OpenRouter (Hybrid Backend) - keeps embeddings + vision local, routes chat to OpenRouter
|
||||||
OPENROUTER_API_KEY=sk-or-... # Required to enable hybrid backend
|
OPENROUTER_API_KEY=sk-or-... # Required to enable hybrid backend
|
||||||
OPENROUTER_DEFAULT_MODEL=anthropic/claude-sonnet-4 # Used when client doesn't pick a model
|
OPENROUTER_DEFAULT_MODEL=anthropic/claude-sonnet-4 # Used when client doesn't pick a model
|
||||||
|
|||||||
1
Cargo.lock
generated
1
Cargo.lock
generated
@@ -3229,6 +3229,7 @@ dependencies = [
|
|||||||
"js-sys",
|
"js-sys",
|
||||||
"log",
|
"log",
|
||||||
"mime",
|
"mime",
|
||||||
|
"mime_guess",
|
||||||
"native-tls",
|
"native-tls",
|
||||||
"percent-encoding",
|
"percent-encoding",
|
||||||
"pin-project-lite",
|
"pin-project-lite",
|
||||||
|
|||||||
@@ -49,7 +49,7 @@ opentelemetry-appender-log = "0.31.0"
|
|||||||
tempfile = "3.20.0"
|
tempfile = "3.20.0"
|
||||||
regex = "1.11.1"
|
regex = "1.11.1"
|
||||||
exif = { package = "kamadak-exif", version = "0.6.1" }
|
exif = { package = "kamadak-exif", version = "0.6.1" }
|
||||||
reqwest = { version = "0.12", features = ["json", "stream"] }
|
reqwest = { version = "0.12", features = ["json", "stream", "multipart"] }
|
||||||
async-stream = "0.3"
|
async-stream = "0.3"
|
||||||
tokio-util = { version = "0.7", features = ["io"] }
|
tokio-util = { version = "0.7", features = ["io"] }
|
||||||
bytes = "1"
|
bytes = "1"
|
||||||
|
|||||||
23
README.md
23
README.md
@@ -159,3 +159,26 @@ Daily conversation summaries are generated automatically on server startup. Conf
|
|||||||
- Contacts to process
|
- Contacts to process
|
||||||
- Model version used for embeddings: `nomic-embed-text:v1.5`
|
- Model version used for embeddings: `nomic-embed-text:v1.5`
|
||||||
|
|
||||||
|
### Apollo + Face Recognition (Optional)
|
||||||
|
|
||||||
|
Apollo (sibling project) hosts both the Places API and the local insightface
|
||||||
|
inference service. Both integrations are optional and degrade gracefully when
|
||||||
|
unset.
|
||||||
|
|
||||||
|
- `APOLLO_API_BASE_URL` - Base URL of the sibling Apollo backend.
|
||||||
|
- When set, photo-insight enrichment folds the user's personal place name
|
||||||
|
(Home, Work, Cabin, ...) into the location string, and the agentic loop
|
||||||
|
gains a `get_personal_place_at` tool. Unset = legacy Nominatim-only path.
|
||||||
|
- `APOLLO_FACE_API_BASE_URL` - Base URL for the face-detection service.
|
||||||
|
- Falls back to `APOLLO_API_BASE_URL` when unset (typical single-Apollo
|
||||||
|
deploy). Both unset = face feature disabled (file-watch hook and
|
||||||
|
manual-face endpoints short-circuit silently).
|
||||||
|
- `FACE_AUTOBIND_MIN_COS` (Phase 3) - Cosine-sim floor for auto-binding a
|
||||||
|
detected face to an existing same-named person via people-tag bootstrap
|
||||||
|
[default: `0.4`].
|
||||||
|
- `FACE_DETECT_CONCURRENCY` (Phase 3) - Per-scan-tick concurrent detect
|
||||||
|
calls fired by the file watcher [default: `8`]. Apollo serializes them
|
||||||
|
via its single-worker GPU pool.
|
||||||
|
- `FACE_DETECT_TIMEOUT_SEC` - reqwest client timeout per detect call
|
||||||
|
[default: `60`]. CPU inference on a backlog can take many seconds.
|
||||||
|
|
||||||
|
|||||||
2
migrations/2026-04-29-000000_add_faces/down.sql
Normal file
2
migrations/2026-04-29-000000_add_faces/down.sql
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
DROP TABLE IF EXISTS face_detections;
|
||||||
|
DROP TABLE IF EXISTS persons;
|
||||||
67
migrations/2026-04-29-000000_add_faces/up.sql
Normal file
67
migrations/2026-04-29-000000_add_faces/up.sql
Normal file
@@ -0,0 +1,67 @@
|
|||||||
|
-- Local face recognition tables.
|
||||||
|
--
|
||||||
|
-- `persons` are visual identities (the "who" of a face). The optional
|
||||||
|
-- `entity_id` bridges to the existing knowledge graph `entities` table —
|
||||||
|
-- when set, this person is the visual side of an LLM-extracted entity.
|
||||||
|
-- Don't auto-create entities from persons; the entity table represents
|
||||||
|
-- LLM-extracted knowledge with its own confidence semantics, and silently
|
||||||
|
-- filling it from face detections muddies the provenance.
|
||||||
|
--
|
||||||
|
-- `face_detections` carries one row per detected face on a content_hash,
|
||||||
|
-- plus marker rows with `status='no_faces'` or `status='failed'` so the
|
||||||
|
-- file watcher knows not to re-scan a hash. Keying on `content_hash`
|
||||||
|
-- (cross-library dedup) rather than `(library_id, rel_path)` means the
|
||||||
|
-- same JPEG in two libraries is scanned once. The denormalized `rel_path`
|
||||||
|
-- carries the most-recently-seen path — useful for cluster-thumb URL
|
||||||
|
-- generation; canonical path lookup goes through image_exif.
|
||||||
|
|
||||||
|
CREATE TABLE persons (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
||||||
|
name TEXT NOT NULL,
|
||||||
|
cover_face_id INTEGER, -- backfilled when the first face binds
|
||||||
|
entity_id INTEGER, -- optional bridge to entities(id)
|
||||||
|
created_from_tag BOOLEAN NOT NULL DEFAULT 0,
|
||||||
|
notes TEXT,
|
||||||
|
created_at BIGINT NOT NULL,
|
||||||
|
updated_at BIGINT NOT NULL,
|
||||||
|
CONSTRAINT fk_persons_entity FOREIGN KEY (entity_id) REFERENCES entities(id) ON DELETE SET NULL,
|
||||||
|
UNIQUE(name COLLATE NOCASE)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX idx_persons_entity ON persons(entity_id);
|
||||||
|
|
||||||
|
CREATE TABLE face_detections (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
||||||
|
library_id INTEGER NOT NULL,
|
||||||
|
content_hash TEXT NOT NULL, -- canonical key (cross-library dedup)
|
||||||
|
rel_path TEXT NOT NULL, -- denormalized; most recently seen
|
||||||
|
bbox_x REAL, -- normalized 0..1; NULL on marker rows
|
||||||
|
bbox_y REAL,
|
||||||
|
bbox_w REAL,
|
||||||
|
bbox_h REAL,
|
||||||
|
embedding BLOB, -- 512×f32 = 2048 bytes; NULL on marker rows
|
||||||
|
confidence REAL, -- detector score
|
||||||
|
source TEXT NOT NULL, -- 'auto' | 'manual'
|
||||||
|
person_id INTEGER,
|
||||||
|
status TEXT NOT NULL DEFAULT 'detected', -- 'detected' | 'no_faces' | 'failed'
|
||||||
|
model_version TEXT NOT NULL, -- e.g. 'buffalo_l'; embedding lineage
|
||||||
|
created_at BIGINT NOT NULL,
|
||||||
|
CONSTRAINT fk_fd_library FOREIGN KEY (library_id) REFERENCES libraries(id),
|
||||||
|
CONSTRAINT fk_fd_person FOREIGN KEY (person_id) REFERENCES persons(id) ON DELETE SET NULL,
|
||||||
|
-- Detected rows carry geometry + embedding; marker rows ('no_faces',
|
||||||
|
-- 'failed') carry neither. CHECK enforces the invariant so manual
|
||||||
|
-- inserts can't slip through with half a row.
|
||||||
|
CONSTRAINT chk_marker CHECK (
|
||||||
|
(status = 'detected' AND bbox_x IS NOT NULL AND embedding IS NOT NULL)
|
||||||
|
OR (status IN ('no_faces','failed') AND bbox_x IS NULL AND embedding IS NULL)
|
||||||
|
)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX idx_face_detections_hash ON face_detections(content_hash);
|
||||||
|
CREATE INDEX idx_face_detections_lib_path ON face_detections(library_id, rel_path);
|
||||||
|
CREATE INDEX idx_face_detections_person ON face_detections(person_id);
|
||||||
|
CREATE INDEX idx_face_detections_status ON face_detections(status);
|
||||||
|
-- One marker row per (content_hash, status='no_faces') so the file watcher
|
||||||
|
-- doesn't double-mark when a hash is seen on multiple full-scan passes.
|
||||||
|
CREATE UNIQUE INDEX idx_face_detections_no_faces_unique
|
||||||
|
ON face_detections(content_hash) WHERE status = 'no_faces';
|
||||||
312
src/ai/face_client.rs
Normal file
312
src/ai/face_client.rs
Normal file
@@ -0,0 +1,312 @@
|
|||||||
|
//! Thin async HTTP client for Apollo's `/api/internal/faces/*` endpoints.
|
||||||
|
//!
|
||||||
|
//! Apollo (the personal location-history viewer at the sibling repo) hosts the
|
||||||
|
//! insightface inference service. This client is the ImageApi side of the
|
||||||
|
//! contract — it shoves image bytes through `/detect` and returns boxes +
|
||||||
|
//! 512-d ArcFace embeddings, plus a single-embedding `/embed` for the manual
|
||||||
|
//! face-create flow.
|
||||||
|
//!
|
||||||
|
//! Mirrors `apollo_client.rs` shape: optional base URL (None = disabled, the
|
||||||
|
//! file watcher and manual-create handlers no-op), reqwest client with a
|
||||||
|
//! generous timeout because CPU inference on a backlog can take many seconds
|
||||||
|
//! per photo.
|
||||||
|
//!
|
||||||
|
//! Configured via `APOLLO_FACE_API_BASE_URL`, falling back to
|
||||||
|
//! `APOLLO_API_BASE_URL` when the dedicated var is unset (single-Apollo
|
||||||
|
//! deploys are the common case). Both unset → `is_enabled()` returns false.
|
||||||
|
//!
|
||||||
|
//! Wire format: multipart/form-data with `file=<bytes>` and `meta=<json>`.
|
||||||
|
//! `meta` carries `{content_hash, library_id, rel_path, orientation?,
|
||||||
|
//! model_version?}` — useful for Apollo-side logging and idempotency, ignored
|
||||||
|
//! by Apollo today but part of the stable wire contract so future versions
|
||||||
|
//! can act on it without a client change.
|
||||||
|
//!
|
||||||
|
//! Error mapping (reflected in [`FaceDetectError`]):
|
||||||
|
//! - 422 `decode_failed` → permanent: ImageApi marks `status='failed'` and
|
||||||
|
//! doesn't retry until manual rerun.
|
||||||
|
//! - 200 with `faces:[]` → `status='no_faces'` marker row.
|
||||||
|
//! - 503 `cuda_oom` / `engine_unavailable` → defer-and-retry: no marker
|
||||||
|
//! written.
|
||||||
|
//! - Any other 5xx / network error → defer.
|
||||||
|
|
||||||
|
use anyhow::{Context, Result};
|
||||||
|
use base64::Engine;
|
||||||
|
use reqwest::Client;
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize)]
|
||||||
|
pub struct DetectMeta {
|
||||||
|
pub content_hash: String,
|
||||||
|
pub library_id: i32,
|
||||||
|
pub rel_path: String,
|
||||||
|
/// EXIF orientation int (1..8). Apollo applies `exif_transpose` on the
|
||||||
|
/// bytes before inference, so this is informational only — supply when
|
||||||
|
/// the bytes were extracted from a RAW preview that lost the tag.
|
||||||
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
|
pub orientation: Option<i32>,
|
||||||
|
/// Echoed back in the response. ImageApi stores it in
|
||||||
|
/// `face_detections.model_version`.
|
||||||
|
#[serde(skip_serializing_if = "Option::is_none")]
|
||||||
|
pub model_version: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wire shape for the bbox sub-object Apollo returns. Read by Phase 3's
|
||||||
|
// file-watch hook; silence the dead-code lint until then.
|
||||||
|
#[allow(dead_code)]
|
||||||
|
#[derive(Debug, Clone, Deserialize)]
|
||||||
|
pub struct DetectedBbox {
|
||||||
|
pub x: f32,
|
||||||
|
pub y: f32,
|
||||||
|
pub w: f32,
|
||||||
|
pub h: f32,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[allow(dead_code)] // bbox consumed by Phase 3 file-watch hook
|
||||||
|
#[derive(Debug, Clone, Deserialize)]
|
||||||
|
pub struct DetectedFace {
|
||||||
|
pub bbox: DetectedBbox,
|
||||||
|
pub confidence: f32,
|
||||||
|
/// base64 of 2048 bytes (512×f32 LE). ImageApi stores the raw bytes
|
||||||
|
/// verbatim as a BLOB — see `decode_embedding` for the unpack.
|
||||||
|
pub embedding: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl DetectedFace {
|
||||||
|
/// Decode the wire-format embedding back into raw bytes for storage.
|
||||||
|
/// Returns the 2048-byte little-endian f32 buffer or an error if the
|
||||||
|
/// base64 is malformed or the wrong length.
|
||||||
|
pub fn decode_embedding(&self) -> Result<Vec<u8>> {
|
||||||
|
let bytes = base64::engine::general_purpose::STANDARD
|
||||||
|
.decode(self.embedding.as_bytes())
|
||||||
|
.context("face embedding base64 decode")?;
|
||||||
|
if bytes.len() != 2048 {
|
||||||
|
anyhow::bail!(
|
||||||
|
"face embedding wrong size: got {} bytes, expected 2048",
|
||||||
|
bytes.len()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
Ok(bytes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[allow(dead_code)] // duration_ms logged by Phase 3 file-watch hook
|
||||||
|
#[derive(Debug, Clone, Deserialize)]
|
||||||
|
pub struct DetectResponse {
|
||||||
|
pub model_version: String,
|
||||||
|
pub duration_ms: i64,
|
||||||
|
pub faces: Vec<DetectedFace>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Deserialize)]
|
||||||
|
#[allow(dead_code)] // Reported by Apollo; useful for future health-driven backoff
|
||||||
|
pub struct FaceHealth {
|
||||||
|
pub loaded: bool,
|
||||||
|
pub providers: Vec<String>,
|
||||||
|
pub model_version: String,
|
||||||
|
pub det_size: i32,
|
||||||
|
#[serde(default)]
|
||||||
|
pub load_error: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Distinguishes permanent failures (don't retry) from transient ones
|
||||||
|
/// (defer and retry on next scan tick). The file-watch hook keys its
|
||||||
|
/// marker-row decision on this — a `Permanent` outcome writes
|
||||||
|
/// `status='failed'`, a `Transient` outcome writes nothing so the next
|
||||||
|
/// pass tries again.
|
||||||
|
#[derive(Debug)]
|
||||||
|
pub enum FaceDetectError {
|
||||||
|
/// Apollo refused the bytes for a reason that won't change on retry
|
||||||
|
/// (decode failure, zero-dim image). Mark `status='failed'`.
|
||||||
|
Permanent(anyhow::Error),
|
||||||
|
/// Apollo couldn't process this turn but might next time (CUDA OOM,
|
||||||
|
/// engine not loaded yet, network hiccup). Don't mark anything.
|
||||||
|
Transient(anyhow::Error),
|
||||||
|
/// Feature is disabled (no `APOLLO_FACE_API_BASE_URL`). Caller should
|
||||||
|
/// silently no-op — same shape as `apollo_client::is_enabled()` false.
|
||||||
|
Disabled,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl std::fmt::Display for FaceDetectError {
|
||||||
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||||
|
match self {
|
||||||
|
FaceDetectError::Permanent(e) => write!(f, "permanent: {e}"),
|
||||||
|
FaceDetectError::Transient(e) => write!(f, "transient: {e}"),
|
||||||
|
FaceDetectError::Disabled => write!(f, "face client disabled"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl std::error::Error for FaceDetectError {}
|
||||||
|
|
||||||
|
#[derive(Clone)]
|
||||||
|
pub struct FaceClient {
|
||||||
|
client: Client,
|
||||||
|
/// `None` → disabled. Trim trailing slash at construction so url
|
||||||
|
/// building doesn't double up.
|
||||||
|
base_url: Option<String>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl FaceClient {
|
||||||
|
pub fn new(base_url: Option<String>) -> Self {
|
||||||
|
// 60 s timeout: CPU inference on a backlog can take many seconds
|
||||||
|
// per photo, especially the first call into a cold GPU. Apollo's
|
||||||
|
// bounded threadpool (1 worker on CUDA) means concurrent calls
|
||||||
|
// queue server-side; 60 s is enough headroom for a few items in
|
||||||
|
// the queue without surfacing a false transient.
|
||||||
|
let timeout_secs = std::env::var("FACE_DETECT_TIMEOUT_SEC")
|
||||||
|
.ok()
|
||||||
|
.and_then(|s| s.parse::<u64>().ok())
|
||||||
|
.unwrap_or(60);
|
||||||
|
let client = Client::builder()
|
||||||
|
.timeout(Duration::from_secs(timeout_secs))
|
||||||
|
.build()
|
||||||
|
.expect("reqwest client build");
|
||||||
|
Self {
|
||||||
|
client,
|
||||||
|
base_url: base_url.map(|u| u.trim_end_matches('/').to_string()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn is_enabled(&self) -> bool {
|
||||||
|
self.base_url.is_some()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Detect every face in `bytes`. ImageApi calls this from the file-watch
|
||||||
|
/// hook (Phase 3) and from the manual rerun handler. Empty `faces[]` in
|
||||||
|
/// the response is the no-faces signal — caller writes a marker row.
|
||||||
|
#[allow(dead_code)] // Phase 3 file-watch hook + rerun handler
|
||||||
|
pub async fn detect(
|
||||||
|
&self,
|
||||||
|
bytes: Vec<u8>,
|
||||||
|
meta: DetectMeta,
|
||||||
|
) -> std::result::Result<DetectResponse, FaceDetectError> {
|
||||||
|
let Some(base) = self.base_url.as_deref() else {
|
||||||
|
return Err(FaceDetectError::Disabled);
|
||||||
|
};
|
||||||
|
let url = format!("{}/api/internal/faces/detect", base);
|
||||||
|
self.post_multipart(&url, bytes, &meta).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Single-embedding endpoint for the manual face-create flow. Caller
|
||||||
|
/// crops the image to the user-drawn bbox and passes those bytes; we
|
||||||
|
/// run detection inside the crop and return the highest-confidence
|
||||||
|
/// face's embedding. Apollo returns 422 `no_face_in_crop` when the
|
||||||
|
/// box missed — surfaced here as `Permanent`.
|
||||||
|
pub async fn embed(
|
||||||
|
&self,
|
||||||
|
bytes: Vec<u8>,
|
||||||
|
meta: DetectMeta,
|
||||||
|
) -> std::result::Result<DetectResponse, FaceDetectError> {
|
||||||
|
let Some(base) = self.base_url.as_deref() else {
|
||||||
|
return Err(FaceDetectError::Disabled);
|
||||||
|
};
|
||||||
|
let url = format!("{}/api/internal/faces/embed", base);
|
||||||
|
self.post_multipart(&url, bytes, &meta).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Engine reachability + provider/model report. Used by ImageApi for a
|
||||||
|
/// startup sanity check; not on the hot path.
|
||||||
|
#[allow(dead_code)] // Phase 3 startup probe
|
||||||
|
pub async fn health(&self) -> Result<FaceHealth> {
|
||||||
|
let base = self.base_url.as_deref().context("face client disabled")?;
|
||||||
|
let url = format!("{}/api/internal/faces/health", base);
|
||||||
|
let resp = self.client.get(&url).send().await?.error_for_status()?;
|
||||||
|
let body: FaceHealth = resp.json().await?;
|
||||||
|
Ok(body)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn post_multipart(
|
||||||
|
&self,
|
||||||
|
url: &str,
|
||||||
|
bytes: Vec<u8>,
|
||||||
|
meta: &DetectMeta,
|
||||||
|
) -> std::result::Result<DetectResponse, FaceDetectError> {
|
||||||
|
let meta_json = serde_json::to_string(meta)
|
||||||
|
.map_err(|e| FaceDetectError::Permanent(anyhow::anyhow!("meta serialize: {e}")))?;
|
||||||
|
let form = reqwest::multipart::Form::new()
|
||||||
|
.text("meta", meta_json)
|
||||||
|
.part(
|
||||||
|
"file",
|
||||||
|
reqwest::multipart::Part::bytes(bytes)
|
||||||
|
.file_name(meta.rel_path.clone())
|
||||||
|
.mime_str("application/octet-stream")
|
||||||
|
.unwrap_or_else(|_| reqwest::multipart::Part::bytes(Vec::new())),
|
||||||
|
);
|
||||||
|
|
||||||
|
let resp = match self.client.post(url).multipart(form).send().await {
|
||||||
|
Ok(r) => r,
|
||||||
|
Err(e) if e.is_timeout() || e.is_connect() => {
|
||||||
|
return Err(FaceDetectError::Transient(anyhow::anyhow!(
|
||||||
|
"face client network: {e}"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
return Err(FaceDetectError::Transient(anyhow::anyhow!(
|
||||||
|
"face client request: {e}"
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
let status = resp.status();
|
||||||
|
if status.is_success() {
|
||||||
|
let body: DetectResponse = resp.json().await.map_err(|e| {
|
||||||
|
FaceDetectError::Transient(anyhow::anyhow!("face response decode: {e}"))
|
||||||
|
})?;
|
||||||
|
return Ok(body);
|
||||||
|
}
|
||||||
|
|
||||||
|
let body_text = resp.text().await.unwrap_or_default();
|
||||||
|
// Apollo encodes its error class in the JSON body's `detail`. Try
|
||||||
|
// to parse it; fall back to status-only classification.
|
||||||
|
let detail_code = serde_json::from_str::<serde_json::Value>(&body_text)
|
||||||
|
.ok()
|
||||||
|
.and_then(|v| {
|
||||||
|
// detail can be a string ("decode_failed") or an object
|
||||||
|
// ({"code": "cuda_oom", ...}) depending on the endpoint
|
||||||
|
// and Apollo's response shape — handle both.
|
||||||
|
v.get("detail")
|
||||||
|
.and_then(|d| d.as_str().map(str::to_string))
|
||||||
|
.or_else(|| {
|
||||||
|
v.get("detail")
|
||||||
|
.and_then(|d| d.get("code"))
|
||||||
|
.and_then(|c| c.as_str())
|
||||||
|
.map(str::to_string)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
.unwrap_or_default();
|
||||||
|
|
||||||
|
if status == reqwest::StatusCode::UNPROCESSABLE_ENTITY {
|
||||||
|
return Err(FaceDetectError::Permanent(anyhow::anyhow!(
|
||||||
|
"face detect 422 {}: {}",
|
||||||
|
detail_code,
|
||||||
|
body_text
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
if status == reqwest::StatusCode::SERVICE_UNAVAILABLE {
|
||||||
|
return Err(FaceDetectError::Transient(anyhow::anyhow!(
|
||||||
|
"face detect 503 {}: {}",
|
||||||
|
detail_code,
|
||||||
|
body_text
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
// Any other 4xx: be conservative and treat as Permanent so we
|
||||||
|
// don't loop forever on a stable rejection. Any other 5xx:
|
||||||
|
// Transient — likely intermittent.
|
||||||
|
if status.is_client_error() {
|
||||||
|
Err(FaceDetectError::Permanent(anyhow::anyhow!(
|
||||||
|
"face detect {} {}: {}",
|
||||||
|
status.as_u16(),
|
||||||
|
detail_code,
|
||||||
|
body_text
|
||||||
|
)))
|
||||||
|
} else {
|
||||||
|
Err(FaceDetectError::Transient(anyhow::anyhow!(
|
||||||
|
"face detect {} {}: {}",
|
||||||
|
status.as_u16(),
|
||||||
|
detail_code,
|
||||||
|
body_text
|
||||||
|
)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,5 +1,6 @@
|
|||||||
pub mod apollo_client;
|
pub mod apollo_client;
|
||||||
pub mod daily_summary_job;
|
pub mod daily_summary_job;
|
||||||
|
pub mod face_client;
|
||||||
pub mod handlers;
|
pub mod handlers;
|
||||||
pub mod insight_chat;
|
pub mod insight_chat;
|
||||||
pub mod insight_generator;
|
pub mod insight_generator;
|
||||||
|
|||||||
@@ -70,6 +70,26 @@ diesel::table! {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
diesel::table! {
|
||||||
|
face_detections (id) {
|
||||||
|
id -> Integer,
|
||||||
|
library_id -> Integer,
|
||||||
|
content_hash -> Text,
|
||||||
|
rel_path -> Text,
|
||||||
|
bbox_x -> Nullable<Float>,
|
||||||
|
bbox_y -> Nullable<Float>,
|
||||||
|
bbox_w -> Nullable<Float>,
|
||||||
|
bbox_h -> Nullable<Float>,
|
||||||
|
embedding -> Nullable<Binary>,
|
||||||
|
confidence -> Nullable<Float>,
|
||||||
|
source -> Text,
|
||||||
|
person_id -> Nullable<Integer>,
|
||||||
|
status -> Text,
|
||||||
|
model_version -> Text,
|
||||||
|
created_at -> BigInt,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
diesel::table! {
|
diesel::table! {
|
||||||
favorites (id) {
|
favorites (id) {
|
||||||
id -> Integer,
|
id -> Integer,
|
||||||
@@ -130,6 +150,19 @@ diesel::table! {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
diesel::table! {
|
||||||
|
persons (id) {
|
||||||
|
id -> Integer,
|
||||||
|
name -> Text,
|
||||||
|
cover_face_id -> Nullable<Integer>,
|
||||||
|
entity_id -> Nullable<Integer>,
|
||||||
|
created_from_tag -> Bool,
|
||||||
|
notes -> Nullable<Text>,
|
||||||
|
created_at -> BigInt,
|
||||||
|
updated_at -> BigInt,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
diesel::table! {
|
diesel::table! {
|
||||||
photo_insights (id) {
|
photo_insights (id) {
|
||||||
id -> Integer,
|
id -> Integer,
|
||||||
@@ -201,7 +234,10 @@ diesel::table! {
|
|||||||
diesel::joinable!(entity_facts -> photo_insights (source_insight_id));
|
diesel::joinable!(entity_facts -> photo_insights (source_insight_id));
|
||||||
diesel::joinable!(entity_photo_links -> entities (entity_id));
|
diesel::joinable!(entity_photo_links -> entities (entity_id));
|
||||||
diesel::joinable!(entity_photo_links -> libraries (library_id));
|
diesel::joinable!(entity_photo_links -> libraries (library_id));
|
||||||
|
diesel::joinable!(face_detections -> libraries (library_id));
|
||||||
|
diesel::joinable!(face_detections -> persons (person_id));
|
||||||
diesel::joinable!(image_exif -> libraries (library_id));
|
diesel::joinable!(image_exif -> libraries (library_id));
|
||||||
|
diesel::joinable!(persons -> entities (entity_id));
|
||||||
diesel::joinable!(photo_insights -> libraries (library_id));
|
diesel::joinable!(photo_insights -> libraries (library_id));
|
||||||
diesel::joinable!(tagged_photo -> tags (tag_id));
|
diesel::joinable!(tagged_photo -> tags (tag_id));
|
||||||
diesel::joinable!(video_preview_clips -> libraries (library_id));
|
diesel::joinable!(video_preview_clips -> libraries (library_id));
|
||||||
@@ -212,10 +248,12 @@ diesel::allow_tables_to_appear_in_same_query!(
|
|||||||
entities,
|
entities,
|
||||||
entity_facts,
|
entity_facts,
|
||||||
entity_photo_links,
|
entity_photo_links,
|
||||||
|
face_detections,
|
||||||
favorites,
|
favorites,
|
||||||
image_exif,
|
image_exif,
|
||||||
libraries,
|
libraries,
|
||||||
location_history,
|
location_history,
|
||||||
|
persons,
|
||||||
photo_insights,
|
photo_insights,
|
||||||
search_history,
|
search_history,
|
||||||
tagged_photo,
|
tagged_photo,
|
||||||
|
|||||||
1863
src/faces.rs
Normal file
1863
src/faces.rs
Normal file
File diff suppressed because it is too large
Load Diff
@@ -12,6 +12,7 @@ pub mod data;
|
|||||||
pub mod database;
|
pub mod database;
|
||||||
pub mod error;
|
pub mod error;
|
||||||
pub mod exif;
|
pub mod exif;
|
||||||
|
pub mod faces;
|
||||||
pub mod file_types;
|
pub mod file_types;
|
||||||
pub mod files;
|
pub mod files;
|
||||||
pub mod geo;
|
pub mod geo;
|
||||||
|
|||||||
@@ -66,6 +66,7 @@ mod data;
|
|||||||
mod database;
|
mod database;
|
||||||
mod error;
|
mod error;
|
||||||
mod exif;
|
mod exif;
|
||||||
|
mod faces;
|
||||||
mod file_types;
|
mod file_types;
|
||||||
mod files;
|
mod files;
|
||||||
mod geo;
|
mod geo;
|
||||||
@@ -1518,6 +1519,7 @@ fn main() -> std::io::Result<()> {
|
|||||||
let exif_dao = SqliteExifDao::new();
|
let exif_dao = SqliteExifDao::new();
|
||||||
let insight_dao = SqliteInsightDao::new();
|
let insight_dao = SqliteInsightDao::new();
|
||||||
let preview_dao = SqlitePreviewDao::new();
|
let preview_dao = SqlitePreviewDao::new();
|
||||||
|
let face_dao = faces::SqliteFaceDao::new();
|
||||||
let cors = Cors::default()
|
let cors = Cors::default()
|
||||||
.allowed_origin_fn(|origin, _req_head| {
|
.allowed_origin_fn(|origin, _req_head| {
|
||||||
// Allow all origins in development, or check against CORS_ALLOWED_ORIGINS env var
|
// Allow all origins in development, or check against CORS_ALLOWED_ORIGINS env var
|
||||||
@@ -1595,6 +1597,7 @@ fn main() -> std::io::Result<()> {
|
|||||||
.service(libraries::list_libraries)
|
.service(libraries::list_libraries)
|
||||||
.add_feature(add_tag_services::<_, SqliteTagDao>)
|
.add_feature(add_tag_services::<_, SqliteTagDao>)
|
||||||
.add_feature(knowledge::add_knowledge_services::<_, SqliteKnowledgeDao>)
|
.add_feature(knowledge::add_knowledge_services::<_, SqliteKnowledgeDao>)
|
||||||
|
.add_feature(faces::add_face_services::<_, faces::SqliteFaceDao>)
|
||||||
.app_data(app_data.clone())
|
.app_data(app_data.clone())
|
||||||
.app_data::<Data<RealFileSystem>>(Data::new(RealFileSystem::new(
|
.app_data::<Data<RealFileSystem>>(Data::new(RealFileSystem::new(
|
||||||
app_data.base_path.clone(),
|
app_data.base_path.clone(),
|
||||||
@@ -1616,6 +1619,10 @@ fn main() -> std::io::Result<()> {
|
|||||||
.app_data::<Data<Mutex<SqliteKnowledgeDao>>>(Data::new(Mutex::new(
|
.app_data::<Data<Mutex<SqliteKnowledgeDao>>>(Data::new(Mutex::new(
|
||||||
SqliteKnowledgeDao::new(),
|
SqliteKnowledgeDao::new(),
|
||||||
)))
|
)))
|
||||||
|
.app_data::<Data<Mutex<faces::SqliteFaceDao>>>(Data::new(Mutex::new(face_dao)))
|
||||||
|
.app_data::<Data<crate::ai::face_client::FaceClient>>(Data::new(
|
||||||
|
app_data.face_client.clone(),
|
||||||
|
))
|
||||||
.app_data(mp::form::MultipartFormConfig::default().total_limit(1024 * 1024 * 1024)) // 1GB upload limit
|
.app_data(mp::form::MultipartFormConfig::default().total_limit(1024 * 1024 * 1024)) // 1GB upload limit
|
||||||
.app_data(web::JsonConfig::default().error_handler(|err, req| {
|
.app_data(web::JsonConfig::default().error_handler(|err, req| {
|
||||||
let detail = err.to_string();
|
let detail = err.to_string();
|
||||||
|
|||||||
19
src/state.rs
19
src/state.rs
@@ -1,4 +1,5 @@
|
|||||||
use crate::ai::apollo_client::ApolloClient;
|
use crate::ai::apollo_client::ApolloClient;
|
||||||
|
use crate::ai::face_client::FaceClient;
|
||||||
use crate::ai::insight_chat::{ChatLockMap, InsightChatService};
|
use crate::ai::insight_chat::{ChatLockMap, InsightChatService};
|
||||||
use crate::ai::openrouter::OpenRouterClient;
|
use crate::ai::openrouter::OpenRouterClient;
|
||||||
use crate::ai::{InsightGenerator, OllamaClient, SmsApiClient};
|
use crate::ai::{InsightGenerator, OllamaClient, SmsApiClient};
|
||||||
@@ -48,6 +49,11 @@ pub struct AppState {
|
|||||||
pub insight_generator: InsightGenerator,
|
pub insight_generator: InsightGenerator,
|
||||||
/// Chat continuation service. Hold an Arc so handlers can clone cheaply.
|
/// Chat continuation service. Hold an Arc so handlers can clone cheaply.
|
||||||
pub insight_chat: Arc<InsightChatService>,
|
pub insight_chat: Arc<InsightChatService>,
|
||||||
|
/// Face inference client (calls Apollo's `/api/internal/faces/*`).
|
||||||
|
/// Disabled (`is_enabled() == false`) when neither `APOLLO_FACE_API_BASE_URL`
|
||||||
|
/// nor `APOLLO_API_BASE_URL` is set; the file-watch hook (Phase 3) and
|
||||||
|
/// manual-face-create handler short-circuit in that case.
|
||||||
|
pub face_client: FaceClient,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl AppState {
|
impl AppState {
|
||||||
@@ -82,6 +88,7 @@ impl AppState {
|
|||||||
insight_generator: InsightGenerator,
|
insight_generator: InsightGenerator,
|
||||||
insight_chat: Arc<InsightChatService>,
|
insight_chat: Arc<InsightChatService>,
|
||||||
preview_dao: Arc<Mutex<Box<dyn PreviewDao>>>,
|
preview_dao: Arc<Mutex<Box<dyn PreviewDao>>>,
|
||||||
|
face_client: FaceClient,
|
||||||
) -> Self {
|
) -> Self {
|
||||||
assert!(
|
assert!(
|
||||||
!libraries_vec.is_empty(),
|
!libraries_vec.is_empty(),
|
||||||
@@ -115,6 +122,7 @@ impl AppState {
|
|||||||
sms_client,
|
sms_client,
|
||||||
insight_generator,
|
insight_generator,
|
||||||
insight_chat,
|
insight_chat,
|
||||||
|
face_client,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -161,6 +169,15 @@ impl Default for AppState {
|
|||||||
// generator silently falls through to the legacy Nominatim path.
|
// generator silently falls through to the legacy Nominatim path.
|
||||||
let apollo_client = ApolloClient::new(env::var("APOLLO_API_BASE_URL").ok());
|
let apollo_client = ApolloClient::new(env::var("APOLLO_API_BASE_URL").ok());
|
||||||
|
|
||||||
|
// Face inference client. Falls back to APOLLO_API_BASE_URL when
|
||||||
|
// APOLLO_FACE_API_BASE_URL is unset (single-Apollo deploys are the
|
||||||
|
// common case). Both unset = feature disabled, file-watch hook
|
||||||
|
// and manual-face handlers short-circuit silently.
|
||||||
|
let face_client_url = env::var("APOLLO_FACE_API_BASE_URL")
|
||||||
|
.ok()
|
||||||
|
.or_else(|| env::var("APOLLO_API_BASE_URL").ok());
|
||||||
|
let face_client = FaceClient::new(face_client_url);
|
||||||
|
|
||||||
// Initialize DAOs
|
// Initialize DAOs
|
||||||
let insight_dao: Arc<Mutex<Box<dyn InsightDao>>> =
|
let insight_dao: Arc<Mutex<Box<dyn InsightDao>>> =
|
||||||
Arc::new(Mutex::new(Box::new(SqliteInsightDao::new())));
|
Arc::new(Mutex::new(Box::new(SqliteInsightDao::new())));
|
||||||
@@ -244,6 +261,7 @@ impl Default for AppState {
|
|||||||
insight_generator,
|
insight_generator,
|
||||||
insight_chat,
|
insight_chat,
|
||||||
preview_dao,
|
preview_dao,
|
||||||
|
face_client,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -382,6 +400,7 @@ impl AppState {
|
|||||||
insight_generator,
|
insight_generator,
|
||||||
insight_chat,
|
insight_chat,
|
||||||
preview_dao,
|
preview_dao,
|
||||||
|
FaceClient::new(None), // disabled in test
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user