ImageApi/docs/architecture/README.md

# Stack architecture: Apollo · ImageApi · FileViewer-React

How the three apps and their shared services talk to each other on the
single deploy host (`192.168.10.50`). Traced from code as of 2026-10-11:
ImageApi `master@d50cad7`, Apollo `2a8715d`, FileViewer-React `main@db4ec37`,
plus the live `.env` URLs. When a route or env var named here changes,
update the diagram in the same PR.

Diagrams are Mermaid; Gitea renders them inline.

## 1. System context

FileViewer-React (FVR) talks to ImageApi and SMS-API directly. The Apollo
web UI only talks to its own backend, which proxies ImageApi and SMS-API so
the browser stays single-origin. ImageApi is also a *client* of Apollo: it
calls Apollo (through nginx on `:8900`, i.e. `APOLLO_API_BASE_URL`) for face
detection, CLIP embeddings and personal Places.

Solid = request path; dashed = optional, batch or telemetry.

```mermaid
flowchart LR
  subgraph clients["Clients"]
    FVR["FileViewer-React<br/>Expo / React Native"]
    BROWSER["Apollo web UI<br/>React + MapLibre"]
  end

  subgraph host["Host 192.168.10.50"]
    subgraph apollo["Apollo · docker compose"]
      NGINX["apollo-frontend<br/>nginx :8900"]
      APB["apollo-backend<br/>FastAPI :8000<br/>insightface + CLIP on GPU"]
      ADB[("apollo.db<br/>location · place · chat_thread")]
      DDB[("derived.db<br/>point_geo geohashes")]
    end
    IMG["ImageApi<br/>Rust / Actix :8088<br/>systemd image-rs"]
    IDB[("ImageApi SQLite<br/>image_exif · tags · faces<br/>insights · reels · personas")]
    FS[["Photo libraries<br/>thumbs · HLS · previews · reels"]]
    SMS["SMS-API<br/>Django :9000"]
    LS["llama-swap :9292<br/>chat · vision · embed · TTS"]
    SIG["SigNoz<br/>OTLP :4317 / :4318"]
  end

  HA["Home Assistant host"]
  OR["OpenRouter"]
  NOM["Nominatim"]

  FVR -->|"JWT REST: files, tags, memories,<br/>insights, chat, TTS, reels"| IMG
  FVR -->|"conversations + MMS media"| SMS
  BROWSER --> NGINX
  NGINX -->|"/api/*"| APB
  APB -->|"service JWT: /photos/exif, /image,<br/>tags, faces, persons, duplicates,<br/>/insights/chat/*"| IMG
  APB -->|"/api/conversations, /api/messages"| SMS
  APB --- ADB
  APB --- DDB
  IMG -->|"/api/internal/faces/detect + embed<br/>/api/internal/clip/encode_*<br/>/api/places/contains"| NGINX
  IMG --- IDB
  IMG --- FS
  IMG -->|"/api/messages/by-date-range"| SMS
  IMG -->|"LLM_BACKEND=llamacpp"| LS
  APB -->|"free + visit chat"| LS
  IMG -.->|"backend=hybrid chat"| OR
  IMG -.->|"reverse geocode"| NOM
  HA -.->|"snapshot rsync, then app.ha_siphon"| ADB
  IMG -.-> SIG
  APB -.-> SIG
  FVR -.->|"OTLP/HTTP"| SIG
```

## 2. Data ownership

Each fact has one owner. Other apps reach it over HTTP and never open
another app's database.

| Data | Owner | Store | Readers |
|---|---|---|---|
| Photo / video bytes, thumbnails, HLS, preview clips | ImageApi | library mounts + `THUMBNAILS`, `VIDEO_PATH` | FVR direct; Apollo via `/api/photos/*` proxy (Range passthrough) |
| EXIF, dates (`date_utc`, `wall_clock`), content hash | ImageApi | `image_exif` | FVR; Apollo via `/photos/exif` |
| Tags, favorites | ImageApi | `tags`, `tagged_photo`, `favorites` | FVR; Apollo proxy (5 min TTL cache) |
| Face rows, persons | ImageApi | `face_detections`, `persons` | Apollo FacesScreen via proxy |
| Face + CLIP inference | Apollo | none (stateless, idle-evicted models) | ImageApi only, via `/api/internal/*` |
| Photo insights, photo-chat transcripts, personas, entity facts | ImageApi | `photo_insights`, `personas`, `entity_facts` | FVR; Apollo (photo threads pass through) |
| Location history (track) | Apollo | `apollo.db.location` → `derived.db` | Apollo UI. ImageApi has its own imported `location_history` for insight tools |
| Places | Apollo | `apollo.db.place` | ImageApi insight enrichment (smallest radius wins) |
| Free / visit chat threads | Apollo | `apollo.db.chat_thread` | Apollo UI |
| SMS / MMS | SMS-API | SMS-API `db.sqlite3` | FVR direct; Apollo proxy; ImageApi insight tools + daily summaries |

## 3. Photo ingest: file to faces and embeddings

New files are found by the watcher, not an upload hook. Derived data is
keyed on the blake3 `content_hash`, so a copy of the same bytes in a second
library is never re-detected or re-embedded. Apollo's detect responses map
to: 422 → `failed`, 200 with no faces → `no_faces`, 503 → transient (no
marker, retried next tick).

```mermaid
sequenceDiagram
  autonumber
  participant FS as Library mount
  participant W as ImageApi watcher
  participant DB as ImageApi SQLite
  participant AP as Apollo /api/internal
  participant TH as Thumbnails / HLS actors

  FS->>W: quick scan (mtime within 60 s) or hourly full scan
  W->>DB: batch lookup by (library_id, rel_path)
  W->>W: blake3 hash, EXIF, date waterfall (filename, exif, exiftool, fs_time)
  W->>DB: insert image_exif row with content_hash, date_utc
  W->>TH: queue thumbnail, HLS playlist, preview clip (videos)
  W->>DB: FaceDao.already_scanned(hash)?
  alt not scanned yet (photo)
    W->>AP: POST faces/detect (bytes, or embedded JPEG for RAW)
    AP-->>W: bboxes + 512-d embeddings, or faces = []
    W->>DB: face_detections rows, auto-bind if tag name matches a person
  else Apollo returns 503
    W->>W: defer, no marker row, retried next tick
  end
  W->>AP: POST clip/encode_image (CLIP backlog, capped per tick)
  AP-->>W: image embedding
  W->>DB: store embedding for /photos/search
```

## 4. One watcher tick

Source: `src/watcher.rs::watch_files`, `src/backfill.rs`,
`src/library_maintenance.rs`. The availability probe gates every write, so
an unmounted share pauses work instead of looking like a mass delete. The
face backlog only runs when a face API base URL is configured.

```mermaid
flowchart TD
  START(["tick: quick, or full when 3600 s elapsed"]) --> LOOP{{"for each library"}}
  LOOP --> EN{"enabled?"}
  EN -- "no" --> LOOP
  EN -- "yes" --> PROBE["presence probe<br/>root exists, readable, non-empty"]
  PROBE --> ON{"online?"}
  ON -- "stale" --> LOOP
  ON -- "online" --> B1["backfill_unhashed_backlog<br/>process_face_backlog"]
  B1 --> B2["process_clip_backlog"]
  B2 --> B3["backfill_missing_date_taken<br/>backfill_utc_dates"]
  B3 --> PNF["process_new_files<br/>quick: recent mtime only"]
  PNF --> CNT["update_media_counts"]
  CNT --> MISS["missing-file scan<br/>page 500, delete cap 200, NotFound only"]
  MISS --> LOOP
  LOOP -- "done" --> REC["reconcile: collapse rows<br/>that now share a hash"]
  REC --> BR["refresh_back_refs<br/>repoint tags / faces / insights"]
  BR --> GC{"all libraries online<br/>two ticks in a row?"}
  GC -- "yes" --> DEL["orphan GC of hash-keyed rows"]
  GC -- "no" --> HOLD["keep pending set"]
  DEL --> FULL{"full scan?"}
  HOLD --> FULL
  FULL -- "yes" --> HLS["hls_stats gauges"]
  FULL -- "no" --> END(["sleep"])
  HLS --> END
```

## 5. Apollo map: matching photos to the track

Photos and videos come from two ImageApi calls run in parallel, then go
through one matcher against `apollo.db.location`. Tag CRUD and duplicate
resolve in Apollo invalidate its caches; changes made from FVR are not
signalled, so Apollo can lag by up to the 5 min TTL.

```mermaid
sequenceDiagram
  autonumber
  participant UI as Apollo web UI
  participant BE as apollo-backend
  participant C as TTL caches
  participant IA as ImageApi
  participant L as apollo.db.location

  UI->>BE: GET /api/photos/match?start&end&tz&library
  BE->>C: _PHOTO_MATCH_CACHE lookup
  alt cache hit
    C-->>BE: body + ETag
  else miss (single-flight)
    par photos
      BE->>IA: GET /photos/exif?date_from&date_to
    and videos
      BE->>IA: GET /photos?media_type=video&recursive=true
    end
    BE->>BE: naive date_taken re-anchored to real UTC using tz
    BE->>L: nearest samples within 2 h, GPS within 200 m
    BE->>BE: classify GPS-tagged, orphan, time-only, unlocated
    BE->>IA: GET /image/tags per file (concurrency 20, _TAG_CACHE)
    BE->>C: store body + ETag (5 min TTL)
  end
  BE-->>UI: 200, or 304 when ETag matches
  UI->>BE: GET /api/photos/thumbnail/path
  BE->>IA: GET /image?size=thumb (streamed, Range passthrough)
```

## 6. Photo insight chat

Photo-anchored chat always runs in ImageApi. FVR starts a turn and follows
it over SSE; Apollo's DISCUSS button proxies `/insights/chat/stream` byte
for byte, so both clients parse one event shape. Apollo's own chat engine
only handles free and visit threads. A per-`(library, file)` mutex
serialises turns on one insight.

```mermaid
sequenceDiagram
  autonumber
  participant FVR as FileViewer-React
  participant AW as Apollo web + backend
  participant IA as ImageApi agentic loop
  participant LLM as llama-swap or OpenRouter
  participant AP as Apollo /api/places
  participant SMS as SMS-API
  participant DB as ImageApi SQLite

  alt from the phone
    FVR->>IA: POST /insights/chat/turn, then SSE on GET /insights/chat/turn/{id}
  else from Apollo
    AW->>IA: POST /insights/chat/stream (passthrough)
  end
  IA->>DB: load training_messages, persona
  loop up to AGENTIC_CHAT_MAX_ITERATIONS
    IA->>LLM: chat with tool schema
    LLM-->>IA: tool_calls or final text
    opt tools
      IA->>AP: get_personal_place_at
      IA->>SMS: get_sms_messages, search_sms
      IA->>DB: recall_facts, search_rag, get_faces_in_photo, get_location_history
    end
  end
  IA->>DB: persist transcript (append) or new insight row (amend)
  IA-->>FVR: buffered turn events over SSE (resumable)
  IA-->>AW: SSE: iteration_start, text, tool_call, tool_result, done
```

## Cross-app conventions that bite

- **Two time conventions.** ImageApi exposes true instants (`date_utc`) and a
  display `wall_clock`. Apollo still treats `date_taken` as naive-as-UTC and
  re-anchors it with the browser zone before matching. SMS-API timestamps
  are real UTC.
- **Auth.** FVR logs in for a 5-day JWT. Apollo uses a long-lived service
  JWT minted with ImageApi's `SECRET_KEY`. Apollo's `/api/*`, its internal
  inference routes and SMS-API are unauthenticated (LAN trust).
- **One model namespace.** ImageApi and Apollo share one llama-swap;
  Apollo's model picker is fed from ImageApi's `/insights/models`.
- **Smallest-radius-wins.** The Place rule is implemented in Apollo's
  backend, Apollo's UI and ImageApi's caller; all three must agree.