docs: stack architecture diagrams for Apollo / ImageApi / FVR
Mermaid sources for the system context, data ownership, photo ingest, watcher tick, Apollo photo matching and photo insight chat, traced from the current code of all three repos. Allow-lists docs/architecture/ alongside docs/adr/ and docs/agents/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Cameron CordesandClaude Opus 5.5 committed
1 parent
d50cad7421
commit
7077aee4ca
2 files changed
+245
No files matched your search
@@ -11,6 +11,7 @@ database/target
|
||||
/docs/*
|
||||
!/docs/agents/
|
||||
!/docs/adr/
|
||||
!/docs/architecture/
|
||||
/specs
|
||||
|
||||
# Default ignored files
|
||||
|
||||
@@ -0,0 +1,244 @@
|
||||
# Stack architecture: Apollo · ImageApi · FileViewer-React
|
||||
|
||||
How the three apps and their shared services talk to each other on the
|
||||
single deploy host (`192.168.10.50`). Traced from code as of 2026-10-11:
|
||||
ImageApi `master@d50cad7`, Apollo `2a8715d`, FileViewer-React `main@db4ec37`,
|
||||
plus the live `.env` URLs. When a route or env var named here changes,
|
||||
update the diagram in the same PR.
|
||||
|
||||
Diagrams are Mermaid; Gitea renders them inline.
|
||||
|
||||
## 1. System context
|
||||
|
||||
FileViewer-React (FVR) talks to ImageApi and SMS-API directly. The Apollo
|
||||
web UI only talks to its own backend, which proxies ImageApi and SMS-API so
|
||||
the browser stays single-origin. ImageApi is also a *client* of Apollo: it
|
||||
calls Apollo (through nginx on `:8900`, i.e. `APOLLO_API_BASE_URL`) for face
|
||||
detection, CLIP embeddings and personal Places.
|
||||
|
||||
Solid = request path; dashed = optional, batch or telemetry.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph clients["Clients"]
|
||||
FVR["FileViewer-React<br/>Expo / React Native"]
|
||||
BROWSER["Apollo web UI<br/>React + MapLibre"]
|
||||
end
|
||||
|
||||
subgraph host["Host 192.168.10.50"]
|
||||
subgraph apollo["Apollo · docker compose"]
|
||||
NGINX["apollo-frontend<br/>nginx :8900"]
|
||||
APB["apollo-backend<br/>FastAPI :8000<br/>insightface + CLIP on GPU"]
|
||||
ADB[("apollo.db<br/>location · place · chat_thread")]
|
||||
DDB[("derived.db<br/>point_geo geohashes")]
|
||||
end
|
||||
IMG["ImageApi<br/>Rust / Actix :8088<br/>systemd image-rs"]
|
||||
IDB[("ImageApi SQLite<br/>image_exif · tags · faces<br/>insights · reels · personas")]
|
||||
FS[["Photo libraries<br/>thumbs · HLS · previews · reels"]]
|
||||
SMS["SMS-API<br/>Django :9000"]
|
||||
LS["llama-swap :9292<br/>chat · vision · embed · TTS"]
|
||||
SIG["SigNoz<br/>OTLP :4317 / :4318"]
|
||||
end
|
||||
|
||||
HA["Home Assistant host"]
|
||||
OR["OpenRouter"]
|
||||
NOM["Nominatim"]
|
||||
|
||||
FVR -->|"JWT REST: files, tags, memories,<br/>insights, chat, TTS, reels"| IMG
|
||||
FVR -->|"conversations + MMS media"| SMS
|
||||
BROWSER --> NGINX
|
||||
NGINX -->|"/api/*"| APB
|
||||
APB -->|"service JWT: /photos/exif, /image,<br/>tags, faces, persons, duplicates,<br/>/insights/chat/*"| IMG
|
||||
APB -->|"/api/conversations, /api/messages"| SMS
|
||||
APB --- ADB
|
||||
APB --- DDB
|
||||
IMG -->|"/api/internal/faces/detect + embed<br/>/api/internal/clip/encode_*<br/>/api/places/contains"| NGINX
|
||||
IMG --- IDB
|
||||
IMG --- FS
|
||||
IMG -->|"/api/messages/by-date-range"| SMS
|
||||
IMG -->|"LLM_BACKEND=llamacpp"| LS
|
||||
APB -->|"free + visit chat"| LS
|
||||
IMG -.->|"backend=hybrid chat"| OR
|
||||
IMG -.->|"reverse geocode"| NOM
|
||||
HA -.->|"snapshot rsync, then app.ha_siphon"| ADB
|
||||
IMG -.-> SIG
|
||||
APB -.-> SIG
|
||||
FVR -.->|"OTLP/HTTP"| SIG
|
||||
```
|
||||
|
||||
## 2. Data ownership
|
||||
|
||||
Each fact has one owner. Other apps reach it over HTTP and never open
|
||||
another app's database.
|
||||
|
||||
| Data | Owner | Store | Readers |
|
||||
|---|---|---|---|
|
||||
| Photo / video bytes, thumbnails, HLS, preview clips | ImageApi | library mounts + `THUMBNAILS`, `VIDEO_PATH` | FVR direct; Apollo via `/api/photos/*` proxy (Range passthrough) |
|
||||
| EXIF, dates (`date_utc`, `wall_clock`), content hash | ImageApi | `image_exif` | FVR; Apollo via `/photos/exif` |
|
||||
| Tags, favorites | ImageApi | `tags`, `tagged_photo`, `favorites` | FVR; Apollo proxy (5 min TTL cache) |
|
||||
| Face rows, persons | ImageApi | `face_detections`, `persons` | Apollo FacesScreen via proxy |
|
||||
| Face + CLIP inference | Apollo | none (stateless, idle-evicted models) | ImageApi only, via `/api/internal/*` |
|
||||
| Photo insights, photo-chat transcripts, personas, entity facts | ImageApi | `photo_insights`, `personas`, `entity_facts` | FVR; Apollo (photo threads pass through) |
|
||||
| Location history (track) | Apollo | `apollo.db.location` → `derived.db` | Apollo UI. ImageApi has its own imported `location_history` for insight tools |
|
||||
| Places | Apollo | `apollo.db.place` | ImageApi insight enrichment (smallest radius wins) |
|
||||
| Free / visit chat threads | Apollo | `apollo.db.chat_thread` | Apollo UI |
|
||||
| SMS / MMS | SMS-API | SMS-API `db.sqlite3` | FVR direct; Apollo proxy; ImageApi insight tools + daily summaries |
|
||||
|
||||
## 3. Photo ingest: file to faces and embeddings
|
||||
|
||||
New files are found by the watcher, not an upload hook. Derived data is
|
||||
keyed on the blake3 `content_hash`, so a copy of the same bytes in a second
|
||||
library is never re-detected or re-embedded. Apollo's detect responses map
|
||||
to: 422 → `failed`, 200 with no faces → `no_faces`, 503 → transient (no
|
||||
marker, retried next tick).
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant FS as Library mount
|
||||
participant W as ImageApi watcher
|
||||
participant DB as ImageApi SQLite
|
||||
participant AP as Apollo /api/internal
|
||||
participant TH as Thumbnails / HLS actors
|
||||
|
||||
FS->>W: quick scan (mtime within 60 s) or hourly full scan
|
||||
W->>DB: batch lookup by (library_id, rel_path)
|
||||
W->>W: blake3 hash, EXIF, date waterfall (filename, exif, exiftool, fs_time)
|
||||
W->>DB: insert image_exif row with content_hash, date_utc
|
||||
W->>TH: queue thumbnail, HLS playlist, preview clip (videos)
|
||||
W->>DB: FaceDao.already_scanned(hash)?
|
||||
alt not scanned yet (photo)
|
||||
W->>AP: POST faces/detect (bytes, or embedded JPEG for RAW)
|
||||
AP-->>W: bboxes + 512-d embeddings, or faces = []
|
||||
W->>DB: face_detections rows, auto-bind if tag name matches a person
|
||||
else Apollo returns 503
|
||||
W->>W: defer, no marker row, retried next tick
|
||||
end
|
||||
W->>AP: POST clip/encode_image (CLIP backlog, capped per tick)
|
||||
AP-->>W: image embedding
|
||||
W->>DB: store embedding for /photos/search
|
||||
```
|
||||
|
||||
## 4. One watcher tick
|
||||
|
||||
Source: `src/watcher.rs::watch_files`, `src/backfill.rs`,
|
||||
`src/library_maintenance.rs`. The availability probe gates every write, so
|
||||
an unmounted share pauses work instead of looking like a mass delete. The
|
||||
face backlog only runs when a face API base URL is configured.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
START(["tick: quick, or full when 3600 s elapsed"]) --> LOOP{{"for each library"}}
|
||||
LOOP --> EN{"enabled?"}
|
||||
EN -- "no" --> LOOP
|
||||
EN -- "yes" --> PROBE["presence probe<br/>root exists, readable, non-empty"]
|
||||
PROBE --> ON{"online?"}
|
||||
ON -- "stale" --> LOOP
|
||||
ON -- "online" --> B1["backfill_unhashed_backlog<br/>process_face_backlog"]
|
||||
B1 --> B2["process_clip_backlog"]
|
||||
B2 --> B3["backfill_missing_date_taken<br/>backfill_utc_dates"]
|
||||
B3 --> PNF["process_new_files<br/>quick: recent mtime only"]
|
||||
PNF --> CNT["update_media_counts"]
|
||||
CNT --> MISS["missing-file scan<br/>page 500, delete cap 200, NotFound only"]
|
||||
MISS --> LOOP
|
||||
LOOP -- "done" --> REC["reconcile: collapse rows<br/>that now share a hash"]
|
||||
REC --> BR["refresh_back_refs<br/>repoint tags / faces / insights"]
|
||||
BR --> GC{"all libraries online<br/>two ticks in a row?"}
|
||||
GC -- "yes" --> DEL["orphan GC of hash-keyed rows"]
|
||||
GC -- "no" --> HOLD["keep pending set"]
|
||||
DEL --> FULL{"full scan?"}
|
||||
HOLD --> FULL
|
||||
FULL -- "yes" --> HLS["hls_stats gauges"]
|
||||
FULL -- "no" --> END(["sleep"])
|
||||
HLS --> END
|
||||
```
|
||||
|
||||
## 5. Apollo map: matching photos to the track
|
||||
|
||||
Photos and videos come from two ImageApi calls run in parallel, then go
|
||||
through one matcher against `apollo.db.location`. Tag CRUD and duplicate
|
||||
resolve in Apollo invalidate its caches; changes made from FVR are not
|
||||
signalled, so Apollo can lag by up to the 5 min TTL.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant UI as Apollo web UI
|
||||
participant BE as apollo-backend
|
||||
participant C as TTL caches
|
||||
participant IA as ImageApi
|
||||
participant L as apollo.db.location
|
||||
|
||||
UI->>BE: GET /api/photos/match?start&end&tz&library
|
||||
BE->>C: _PHOTO_MATCH_CACHE lookup
|
||||
alt cache hit
|
||||
C-->>BE: body + ETag
|
||||
else miss (single-flight)
|
||||
par photos
|
||||
BE->>IA: GET /photos/exif?date_from&date_to
|
||||
and videos
|
||||
BE->>IA: GET /photos?media_type=video&recursive=true
|
||||
end
|
||||
BE->>BE: naive date_taken re-anchored to real UTC using tz
|
||||
BE->>L: nearest samples within 2 h, GPS within 200 m
|
||||
BE->>BE: classify GPS-tagged, orphan, time-only, unlocated
|
||||
BE->>IA: GET /image/tags per file (concurrency 20, _TAG_CACHE)
|
||||
BE->>C: store body + ETag (5 min TTL)
|
||||
end
|
||||
BE-->>UI: 200, or 304 when ETag matches
|
||||
UI->>BE: GET /api/photos/thumbnail/path
|
||||
BE->>IA: GET /image?size=thumb (streamed, Range passthrough)
|
||||
```
|
||||
|
||||
## 6. Photo insight chat
|
||||
|
||||
Photo-anchored chat always runs in ImageApi. FVR starts a turn and follows
|
||||
it over SSE; Apollo's DISCUSS button proxies `/insights/chat/stream` byte
|
||||
for byte, so both clients parse one event shape. Apollo's own chat engine
|
||||
only handles free and visit threads. A per-`(library, file)` mutex
|
||||
serialises turns on one insight.
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant FVR as FileViewer-React
|
||||
participant AW as Apollo web + backend
|
||||
participant IA as ImageApi agentic loop
|
||||
participant LLM as llama-swap or OpenRouter
|
||||
participant AP as Apollo /api/places
|
||||
participant SMS as SMS-API
|
||||
participant DB as ImageApi SQLite
|
||||
|
||||
alt from the phone
|
||||
FVR->>IA: POST /insights/chat/turn, then SSE on GET /insights/chat/turn/{id}
|
||||
else from Apollo
|
||||
AW->>IA: POST /insights/chat/stream (passthrough)
|
||||
end
|
||||
IA->>DB: load training_messages, persona
|
||||
loop up to AGENTIC_CHAT_MAX_ITERATIONS
|
||||
IA->>LLM: chat with tool schema
|
||||
LLM-->>IA: tool_calls or final text
|
||||
opt tools
|
||||
IA->>AP: get_personal_place_at
|
||||
IA->>SMS: get_sms_messages, search_sms
|
||||
IA->>DB: recall_facts, search_rag, get_faces_in_photo, get_location_history
|
||||
end
|
||||
end
|
||||
IA->>DB: persist transcript (append) or new insight row (amend)
|
||||
IA-->>FVR: buffered turn events over SSE (resumable)
|
||||
IA-->>AW: SSE: iteration_start, text, tool_call, tool_result, done
|
||||
```
|
||||
|
||||
## Cross-app conventions that bite
|
||||
|
||||
- **Two time conventions.** ImageApi exposes true instants (`date_utc`) and a
|
||||
display `wall_clock`. Apollo still treats `date_taken` as naive-as-UTC and
|
||||
re-anchors it with the browser zone before matching. SMS-API timestamps
|
||||
are real UTC.
|
||||
- **Auth.** FVR logs in for a 5-day JWT. Apollo uses a long-lived service
|
||||
JWT minted with ImageApi's `SECRET_KEY`. Apollo's `/api/*`, its internal
|
||||
inference routes and SMS-API are unauthenticated (LAN trust).
|
||||
- **One model namespace.** ImageApi and Apollo share one llama-swap;
|
||||
Apollo's model picker is fed from ImageApi's `/insights/models`.
|
||||
- **Smallest-radius-wins.** The Place rule is implemented in Apollo's
|
||||
backend, Apollo's UI and ImageApi's caller; all three must agree.
|
||||
Reference in new issue
Block a user