ImageApi/docs/architecture/README.md

Stack architecture: Apollo · ImageApi · FileViewer-React

How the three apps and their shared services talk to each other on the single deploy host (192.168.10.50). Traced from code as of 2026-10-11: ImageApi master@d50cad7, Apollo 2a8715d, FileViewer-React main@db4ec37, plus the live .env URLs. When a route or env var named here changes, update the diagram in the same PR.

Diagrams are Mermaid; Gitea renders them inline.

1. System context

FileViewer-React (FVR) talks to ImageApi and SMS-API directly. The Apollo web UI only talks to its own backend, which proxies ImageApi and SMS-API so the browser stays single-origin. ImageApi is also a client of Apollo: it calls Apollo (through nginx on :8900, i.e. APOLLO_API_BASE_URL) for face detection, CLIP embeddings and personal Places.

Solid = request path; dashed = optional, batch or telemetry.

flowchart LR
  subgraph clients["Clients"]
    FVR["FileViewer-React<br/>Expo / React Native"]
    BROWSER["Apollo web UI<br/>React + MapLibre"]
  end

  subgraph host["Host 192.168.10.50"]
    subgraph apollo["Apollo · docker compose"]
      NGINX["apollo-frontend<br/>nginx :8900"]
      APB["apollo-backend<br/>FastAPI :8000<br/>insightface + CLIP on GPU"]
      ADB[("apollo.db<br/>location · place · chat_thread")]
      DDB[("derived.db<br/>point_geo geohashes")]
    end
    IMG["ImageApi<br/>Rust / Actix :8088<br/>systemd image-rs"]
    IDB[("ImageApi SQLite<br/>image_exif · tags · faces<br/>insights · reels · personas")]
    FS[["Photo libraries<br/>thumbs · HLS · previews · reels"]]
    SMS["SMS-API<br/>Django :9000"]
    LS["llama-swap :9292<br/>chat · vision · embed · TTS"]
    SIG["SigNoz<br/>OTLP :4317 / :4318"]
  end

  HA["Home Assistant host"]
  OR["OpenRouter"]
  NOM["Nominatim"]

  FVR -->|"JWT REST: files, tags, memories,<br/>insights, chat, TTS, reels"| IMG
  FVR -->|"conversations + MMS media"| SMS
  BROWSER --> NGINX
  NGINX -->|"/api/*"| APB
  APB -->|"service JWT: /photos/exif, /image,<br/>tags, faces, persons, duplicates,<br/>/insights/chat/*"| IMG
  APB -->|"/api/conversations, /api/messages"| SMS
  APB --- ADB
  APB --- DDB
  IMG -->|"/api/internal/faces/detect + embed<br/>/api/internal/clip/encode_*<br/>/api/places/contains"| NGINX
  IMG --- IDB
  IMG --- FS
  IMG -->|"/api/messages/by-date-range"| SMS
  IMG -->|"LLM_BACKEND=llamacpp"| LS
  APB -->|"free + visit chat"| LS
  IMG -.->|"backend=hybrid chat"| OR
  IMG -.->|"reverse geocode"| NOM
  HA -.->|"snapshot rsync, then app.ha_siphon"| ADB
  IMG -.-> SIG
  APB -.-> SIG
  FVR -.->|"OTLP/HTTP"| SIG

2. Data ownership

Each fact has one owner. Other apps reach it over HTTP and never open another app's database.

Data	Owner	Store	Readers
Photo / video bytes, thumbnails, HLS, preview clips	ImageApi	library mounts + THUMBNAILS, VIDEO_PATH	FVR direct; Apollo via /api/photos/* proxy (Range passthrough)
EXIF, dates (date_utc, wall_clock), content hash	ImageApi	image_exif	FVR; Apollo via /photos/exif
Tags, favorites	ImageApi	tags, tagged_photo, favorites	FVR; Apollo proxy (5 min TTL cache)
Face rows, persons	ImageApi	face_detections, persons	Apollo FacesScreen via proxy
Face + CLIP inference	Apollo	none (stateless, idle-evicted models)	ImageApi only, via /api/internal/*
Photo insights, photo-chat transcripts, personas, entity facts	ImageApi	photo_insights, personas, entity_facts	FVR; Apollo (photo threads pass through)
Location history (track)	Apollo	apollo.db.location → derived.db	Apollo UI. ImageApi has its own imported location_history for insight tools
Places	Apollo	apollo.db.place	ImageApi insight enrichment (smallest radius wins)
Free / visit chat threads	Apollo	apollo.db.chat_thread	Apollo UI
SMS / MMS	SMS-API	SMS-API db.sqlite3	FVR direct; Apollo proxy; ImageApi insight tools + daily summaries

3. Photo ingest: file to faces and embeddings

New files are found by the watcher, not an upload hook. Derived data is keyed on the blake3 content_hash, so a copy of the same bytes in a second library is never re-detected or re-embedded. Apollo's detect responses map to: 422 → failed, 200 with no faces → no_faces, 503 → transient (no marker, retried next tick).

sequenceDiagram
  autonumber
  participant FS as Library mount
  participant W as ImageApi watcher
  participant DB as ImageApi SQLite
  participant AP as Apollo /api/internal
  participant TH as Thumbnails / HLS actors

  FS->>W: quick scan (mtime within 60 s) or hourly full scan
  W->>DB: batch lookup by (library_id, rel_path)
  W->>W: blake3 hash, EXIF, date waterfall (filename, exif, exiftool, fs_time)
  W->>DB: insert image_exif row with content_hash, date_utc
  W->>TH: queue thumbnail, HLS playlist, preview clip (videos)
  W->>DB: FaceDao.already_scanned(hash)?
  alt not scanned yet (photo)
    W->>AP: POST faces/detect (bytes, or embedded JPEG for RAW)
    AP-->>W: bboxes + 512-d embeddings, or faces = []
    W->>DB: face_detections rows, auto-bind if tag name matches a person
  else Apollo returns 503
    W->>W: defer, no marker row, retried next tick
  end
  W->>AP: POST clip/encode_image (CLIP backlog, capped per tick)
  AP-->>W: image embedding
  W->>DB: store embedding for /photos/search

4. One watcher tick

Source: src/watcher.rs::watch_files, src/backfill.rs, src/library_maintenance.rs. The availability probe gates every write, so an unmounted share pauses work instead of looking like a mass delete. The face backlog only runs when a face API base URL is configured.

flowchart TD
  START(["tick: quick, or full when 3600 s elapsed"]) --> LOOP{{"for each library"}}
  LOOP --> EN{"enabled?"}
  EN -- "no" --> LOOP
  EN -- "yes" --> PROBE["presence probe<br/>root exists, readable, non-empty"]
  PROBE --> ON{"online?"}
  ON -- "stale" --> LOOP
  ON -- "online" --> B1["backfill_unhashed_backlog<br/>process_face_backlog"]
  B1 --> B2["process_clip_backlog"]
  B2 --> B3["backfill_missing_date_taken<br/>backfill_utc_dates"]
  B3 --> PNF["process_new_files<br/>quick: recent mtime only"]
  PNF --> CNT["update_media_counts"]
  CNT --> MISS["missing-file scan<br/>page 500, delete cap 200, NotFound only"]
  MISS --> LOOP
  LOOP -- "done" --> REC["reconcile: collapse rows<br/>that now share a hash"]
  REC --> BR["refresh_back_refs<br/>repoint tags / faces / insights"]
  BR --> GC{"all libraries online<br/>two ticks in a row?"}
  GC -- "yes" --> DEL["orphan GC of hash-keyed rows"]
  GC -- "no" --> HOLD["keep pending set"]
  DEL --> FULL{"full scan?"}
  HOLD --> FULL
  FULL -- "yes" --> HLS["hls_stats gauges"]
  FULL -- "no" --> END(["sleep"])
  HLS --> END

5. Apollo map: matching photos to the track

Photos and videos come from two ImageApi calls run in parallel, then go through one matcher against apollo.db.location. Tag CRUD and duplicate resolve in Apollo invalidate its caches; changes made from FVR are not signalled, so Apollo can lag by up to the 5 min TTL.

sequenceDiagram
  autonumber
  participant UI as Apollo web UI
  participant BE as apollo-backend
  participant C as TTL caches
  participant IA as ImageApi
  participant L as apollo.db.location

  UI->>BE: GET /api/photos/match?start&end&tz&library
  BE->>C: _PHOTO_MATCH_CACHE lookup
  alt cache hit
    C-->>BE: body + ETag
  else miss (single-flight)
    par photos
      BE->>IA: GET /photos/exif?date_from&date_to
    and videos
      BE->>IA: GET /photos?media_type=video&recursive=true
    end
    BE->>BE: naive date_taken re-anchored to real UTC using tz
    BE->>L: nearest samples within 2 h, GPS within 200 m
    BE->>BE: classify GPS-tagged, orphan, time-only, unlocated
    BE->>IA: GET /image/tags per file (concurrency 20, _TAG_CACHE)
    BE->>C: store body + ETag (5 min TTL)
  end
  BE-->>UI: 200, or 304 when ETag matches
  UI->>BE: GET /api/photos/thumbnail/path
  BE->>IA: GET /image?size=thumb (streamed, Range passthrough)

6. Photo insight chat

Photo-anchored chat always runs in ImageApi. FVR starts a turn and follows it over SSE; Apollo's DISCUSS button proxies /insights/chat/stream byte for byte, so both clients parse one event shape. Apollo's own chat engine only handles free and visit threads. A per-(library, file) mutex serialises turns on one insight.

sequenceDiagram
  autonumber
  participant FVR as FileViewer-React
  participant AW as Apollo web + backend
  participant IA as ImageApi agentic loop
  participant LLM as llama-swap or OpenRouter
  participant AP as Apollo /api/places
  participant SMS as SMS-API
  participant DB as ImageApi SQLite

  alt from the phone
    FVR->>IA: POST /insights/chat/turn, then SSE on GET /insights/chat/turn/{id}
  else from Apollo
    AW->>IA: POST /insights/chat/stream (passthrough)
  end
  IA->>DB: load training_messages, persona
  loop up to AGENTIC_CHAT_MAX_ITERATIONS
    IA->>LLM: chat with tool schema
    LLM-->>IA: tool_calls or final text
    opt tools
      IA->>AP: get_personal_place_at
      IA->>SMS: get_sms_messages, search_sms
      IA->>DB: recall_facts, search_rag, get_faces_in_photo, get_location_history
    end
  end
  IA->>DB: persist transcript (append) or new insight row (amend)
  IA-->>FVR: buffered turn events over SSE (resumable)
  IA-->>AW: SSE: iteration_start, text, tool_call, tool_result, done

Cross-app conventions that bite

• Two time conventions. ImageApi exposes true instants (date_utc) and a display wall_clock. Apollo still treats date_taken as naive-as-UTC and re-anchors it with the browser zone before matching. SMS-API timestamps are real UTC.
• Auth. FVR logs in for a 5-day JWT. Apollo uses a long-lived service JWT minted with ImageApi's SECRET_KEY. Apollo's /api/*, its internal inference routes and SMS-API are unauthenticated (LAN trust).
• One model namespace. ImageApi and Apollo share one llama-swap; Apollo's model picker is fed from ImageApi's /insights/models.
• Smallest-radius-wins. The Place rule is implemented in Apollo's backend, Apollo's UI and ImageApi's caller; all three must agree.