From 6c543ffa68c64bbfe7d4fc77b3f65fae6a3fca41 Mon Sep 17 00:00:00 2001 From: Cameron Date: Wed, 24 Dec 2025 16:07:03 -0500 Subject: [PATCH] Add CLAUDE.md documentation for Claude Code MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Comprehensive guide covering build commands, architecture overview, database patterns, file processing pipeline, API structure, and development workflows. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 --- CLAUDE.md | 266 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 266 insertions(+) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..2e3b17f --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,266 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +An Actix-web REST API for serving images and videos from a filesystem with automatic thumbnail generation, EXIF extraction, tag organization, and a memories feature for browsing photos by date. Uses SQLite/Diesel ORM for data persistence and ffmpeg for video processing. + +## Development Commands + +### Building & Running +```bash +# Build for development +cargo build + +# Build for release (uses thin LTO optimization) +cargo build --release + +# Run the server (requires .env file with DATABASE_URL, BASE_PATH, THUMBNAILS, VIDEO_PATH, BIND_URL, SECRET_KEY) +cargo run + +# Run with specific log level +RUST_LOG=debug cargo run +``` + +### Testing +```bash +# Run all tests (requires BASE_PATH in .env) +cargo test + +# Run specific test +cargo test test_name + +# Run tests with output +cargo test -- --nocapture +``` + +### Database Migrations +```bash +# Install diesel CLI (one-time setup) +cargo install diesel_cli --no-default-features --features sqlite + +# Create new migration +diesel migration generate migration_name + +# Run migrations (also runs automatically on app startup) +diesel migration run + +# Revert last migration +diesel migration revert + +# Regenerate schema.rs after manual migration changes +diesel print-schema > src/database/schema.rs +``` + +### Code Quality +```bash +# Format code +cargo fmt + +# Run clippy linter +cargo clippy + +# Fix automatically fixable issues +cargo fix +``` + +### Utility Binaries +```bash +# Two-phase cleanup: resolve missing files and validate file types +cargo run --bin cleanup_files -- --base-path /path/to/media --database-url ./database.db + +# Batch extract EXIF for existing files +cargo run --bin migrate_exif +``` + +## Architecture Overview + +### Core Components + +**Layered Architecture:** +- **HTTP Layer** (`main.rs`): Route handlers for images, videos, metadata, tags, favorites, memories +- **Auth Layer** (`auth.rs`): JWT token validation, Claims extraction via FromRequest trait +- **Service Layer** (`files.rs`, `exif.rs`, `memories.rs`): Business logic for file operations and EXIF extraction +- **DAO Layer** (`database/mod.rs`): Trait-based data access (ExifDao, UserDao, FavoriteDao, TagDao) +- **Database Layer**: Diesel ORM with SQLite, schema in `database/schema.rs` + +**Async Actor System (Actix):** +- `StreamActor`: Manages ffmpeg video processing lifecycle +- `VideoPlaylistManager`: Scans directories and queues videos +- `PlaylistGenerator`: Creates HLS playlists for video streaming + +### Database Schema & Patterns + +**Tables:** +- `users`: Authentication (id, username, password_hash) +- `favorites`: User-specific favorites (userid, path) +- `tags`: Custom labels with timestamps +- `tagged_photo`: Many-to-many photo-tag relationships +- `image_exif`: Rich metadata (file_path + 16 EXIF fields: camera, GPS, dates, exposure settings) + +**DAO Pattern:** +All database access goes through trait-based DAOs (e.g., `ExifDao`, `SqliteExifDao`). Connection pooling uses `Arc>`. All DB operations are traced with OpenTelemetry in release builds. + +**Key DAO Methods:** +- `store_exif()`, `get_exif()`, `get_exif_batch()`: EXIF CRUD operations +- `query_by_exif()`: Complex filtering by camera, GPS bounds, date ranges +- Batch operations minimize DB hits during file watching + +### File Processing Pipeline + +**Thumbnail Generation:** +1. Startup scan: Rayon parallel walk of BASE_PATH +2. Creates 200x200 thumbnails in THUMBNAILS directory (mirrors source structure) +3. Videos: extracts frame at 3-second mark via ffmpeg +4. Images: uses `image` crate for JPEG/PNG processing + +**File Watching:** +Runs in background thread with two-tier strategy: +- **Quick scan** (default 60s): Recently modified files only +- **Full scan** (default 3600s): Comprehensive directory check +- Batch queries EXIF DB to detect new files +- Configurable via `WATCH_QUICK_INTERVAL_SECONDS` and `WATCH_FULL_INTERVAL_SECONDS` + +**EXIF Extraction:** +- Uses `kamadak-exif` crate +- Supports: JPEG, TIFF, RAW (NEF, CR2, CR3), HEIF/HEIC, PNG, WebP +- Extracts: camera make/model, lens, dimensions, GPS coordinates, focal length, aperture, shutter speed, ISO, date taken +- Triggered on upload and during file watching + +**File Upload Behavior:** +If file exists, appends timestamp to filename (`photo_1735124234.jpg`) to preserve history without overwrites. + +### Authentication Flow + +**Login:** +1. POST `/login` with username/password +2. Verify with `bcrypt::verify()` against password_hash +3. Generate JWT with claims: `{ sub: user_id, exp: 5_days_from_now }` +4. Sign with HS256 using `SECRET_KEY` environment variable + +**Authorization:** +All protected endpoints extract `Claims` via `FromRequest` trait implementation. Token passed as `Authorization: Bearer ` header. + +### API Structure + +**Key Endpoint Patterns:** + +```rust +// Image serving & upload +GET /image?path=...&size=...&format=... +POST /image (multipart file upload) + +// Metadata & EXIF +GET /image/metadata?path=... + +// Advanced search with filters +GET /photos?path=...&recursive=true&sort=DateTakenDesc&camera_make=Canon&gps_lat=...&gps_lon=...&gps_radius_km=10&date_from=...&date_to=...&tag_ids=1,2,3&media_type=Photo + +// Video streaming (HLS) +POST /video/generate (creates .m3u8 playlist + .ts segments) +GET /video/stream?path=... (serves playlist) + +// Tags +GET /image/tags/all +POST /image/tags (add tag to file) +DELETE /image/tags (remove tag from file) +POST /image/tags/batch (bulk tag updates) + +// Memories (week-based grouping) +GET /memories?path=...&recursive=true +``` + +**Request Types:** +- `FilesRequest`: Supports complex filtering (tags, EXIF fields, GPS radius, date ranges) +- `SortType`: Shuffle, NameAsc/Desc, TagCountAsc/Desc, DateTakenAsc/Desc + +### Important Patterns + +**Service Builder Pattern:** +Routes are registered via composable `ServiceBuilder` trait in `service.rs`. Allows modular feature addition. + +**Path Validation:** +Always use `is_valid_full_path(&base_path, &requested_path, check_exists)` to prevent directory traversal attacks. + +**File Type Detection:** +Centralized in `file_types.rs` with constants `IMAGE_EXTENSIONS` and `VIDEO_EXTENSIONS`. Provides both `Path` and `DirEntry` variants for performance. + +**OpenTelemetry Tracing:** +All database operations and HTTP handlers wrapped in spans. In release builds, exports to OTLP endpoint via `OTLP_OTLS_ENDPOINT`. Debug builds use basic logger. + +**Memory Exclusion:** +`PathExcluder` in `memories.rs` filters out directories from memories API via `EXCLUDED_DIRS` environment variable (comma-separated paths or substring patterns). + +### Startup Sequence + +1. Load `.env` file +2. Run embedded Diesel migrations +3. Spawn file watcher thread +4. Create initial thumbnails (parallel scan) +5. Generate video GIF thumbnails +6. Initialize AppState with Actix actors +7. Set up Prometheus metrics (`imageserver_image_total`, `imageserver_video_total`) +8. Scan directory for videos and queue HLS processing +9. Start HTTP server on `BIND_URL` + localhost:8088 + +## Testing Patterns + +Tests require `BASE_PATH` environment variable. Many integration tests create temporary directories and files. + +When testing database code: +- Use in-memory SQLite: `DATABASE_URL=":memory:"` +- Run migrations in test setup +- Clean up with `DROP TABLE` or use `#[serial]` from `serial_test` crate if parallel tests conflict + +## Common Gotchas + +**EXIF Date Parsing:** +Multiple formats supported (EXIF DateTime, ISO8601, Unix timestamp). Fallback chain attempts multiple parsers. + +**Video Processing:** +ffmpeg processes run asynchronously via actors. Use `StreamActor` to track completion. HLS segments written to `VIDEO_PATH`. + +**File Extensions:** +Extension detection is case-insensitive. Use `file_types.rs` helpers rather than manual string matching. + +**Migration Workflow:** +After creating a migration, manually edit the SQL, then regenerate `schema.rs` with `diesel print-schema`. Migrations auto-run on startup via `embedded_migrations!()` macro. + +**Path Absolutization:** +Use `path-absolutize` crate's `.absolutize()` method when converting user-provided paths to ensure they're within `BASE_PATH`. + +## Required Environment Variables + +```bash +DATABASE_URL=./database.db # SQLite database path +BASE_PATH=/path/to/media # Root media directory +THUMBNAILS=/path/to/thumbnails # Thumbnail storage +VIDEO_PATH=/path/to/video/hls # HLS playlist output +GIFS_DIRECTORY=/path/to/gifs # Video GIF thumbnails +BIND_URL=0.0.0.0:8080 # Server binding +CORS_ALLOWED_ORIGINS=http://localhost:3000 +SECRET_KEY=your-secret-key-here # JWT signing secret +RUST_LOG=info # Log level +EXCLUDED_DIRS=/private,/archive # Comma-separated paths to exclude from memories +``` + +Optional: +```bash +WATCH_QUICK_INTERVAL_SECONDS=60 # Quick scan interval +WATCH_FULL_INTERVAL_SECONDS=3600 # Full scan interval +OTLP_OTLS_ENDPOINT=http://... # OpenTelemetry collector (release builds) +``` + +## Dependencies of Note + +- **actix-web**: HTTP framework +- **diesel**: ORM for SQLite +- **jsonwebtoken**: JWT implementation +- **kamadak-exif**: EXIF parsing +- **image**: Thumbnail generation +- **walkdir**: Directory traversal +- **rayon**: Parallel processing +- **opentelemetry**: Distributed tracing +- **bcrypt**: Password hashing +- **infer**: Magic number file type detection