Apps/ImageApi
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

multi-library: per-library excluded_dirs

Browse Source 
Adds a nullable comma-separated TEXT column to the libraries table.
Effective excludes for a walk = (env-var globals) ∪
(library.excluded_dirs). Empty / NULL = no library-specific
extras; the global env var still applies.

Migration (2026-05-01-110000_libraries_excluded_dirs)

  ALTER TABLE libraries ADD COLUMN excluded_dirs TEXT. NULL on every
  existing row — no behavior change on upgrade.

Library struct + helpers (libraries.rs)

  - Library gains excluded_dirs: Vec<String>, parsed from the column
    by parse_excluded_dirs_column (drops empties / whitespace,
    matches the env-var parser).
  - Library::effective_excluded_dirs(globals) returns the union.
  - From<LibraryRow> hydrates the field on AppState construction so
    /libraries surfaces it.

Watcher / walkers / memories

  Every per-library walker now consults the effective set:
    - process_new_files (file-watch ingest, RAW/EXIF/face)
    - process_face_backlog (filter_excluded inherits)
    - create_thumbnails (startup + new-file branch)
    - update_media_counts (Prometheus gauge)
    - cleanup_orphaned_playlists (per-library source-existence check)
    - memories endpoint (PathExcluder)

  Effective set is computed once per per-library iteration in the
  watcher tick and threaded through; called functions retain their
  flat &[String] signature (no per-library awareness needed inside
  the walker primitives).

Use case: mount a parent directory while a sibling library covers
a child subtree, and exclude the child subtree from the parent so
the libraries don't double-walk / double-write image_exif. With
hash-keyed derived data (Branches B/C), the duplication-avoidance
is the only cost prevented — face / tag / insight sharing was
already correct via content_hash.

Tests: 228 pass (226 from previous + 2 new in libraries::tests:
parse_excluded_dirs_column edge cases,
effective_excluded_dirs_unions_global_and_per_library).

CLAUDE.md gains a "Per-library excludes" subsection of the
multi-library data model.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Cameron Cordes
2026-05-01 19:54:17 +00:00
parent 4f17af688e
commit 814066551e
11 changed files with 149 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
src/main.rs
View File

@@ -1335,10 +1335,14 @@ fn create_thumbnails(libs: &[libraries::Library], excluded_dirs: &[String]) {
lib.name, lib.root_path
);
let images = PathBuf::from(&lib.root_path);
// Effective excludes = global env-var excludes ∪ library row's
// excluded_dirs. Lets a parent-library mount skip the subtree
// already covered by a child library.
let effective_excludes = lib.effective_excluded_dirs(excluded_dirs);
// Prune EXCLUDED_DIRS so we don't generate thumbnails-of-thumbnails
// for Synology @eaDir trees. file_scan handles filter_entry pruning.
image_api::file_scan::walk_library_files(&images, excluded_dirs)
image_api::file_scan::walk_library_files(&images, &effective_excludes)
.into_par_iter()
.for_each(|entry| {
let src = entry.path();
@@ -1413,7 +1417,8 @@ fn create_thumbnails(libs: &[libraries::Library], excluded_dirs: &[String]) {
debug!("Finished making thumbnails");
for lib in libs {
update_media_counts(Path::new(&lib.root_path), excluded_dirs);
let effective_excludes = lib.effective_excluded_dirs(excluded_dirs);
update_media_counts(Path::new(&lib.root_path), &effective_excludes);
}
}
@@ -1801,9 +1806,10 @@ fn cleanup_orphaned_playlists(
// playlist isn't orphaned.
let mut video_exists = false;
'libs: for lib in &libs {
let effective = lib.effective_excluded_dirs(&excluded_dirs);
for entry in image_api::file_scan::walk_library_files(
Path::new(&lib.root_path),
&excluded_dirs,
&effective,
) {
if let Some(entry_stem) = entry.path().file_stem()
&& entry_stem == filename
@@ -2048,6 +2054,11 @@ fn watch_files(
// — without these standalone passes, backfill +
// detection only progressed during full scans
// (default once an hour).
// Effective excludes for this library: global env-var
// ∪ row's excluded_dirs. Compute once per tick — used
// by every walker below for this library.
let effective_excludes = lib.effective_excluded_dirs(&excluded_dirs);
if face_client.is_enabled() {
let context = opentelemetry::Context::new();
backfill_unhashed_backlog(&context, lib, &exif_dao);
@@ -2057,7 +2068,7 @@ fn watch_files(
&face_client,
&face_dao,
&watcher_tag_dao,
&excluded_dirs,
&effective_excludes,
);
}
@@ -2073,7 +2084,7 @@ fn watch_files(
Arc::clone(&face_dao),
Arc::clone(&watcher_tag_dao),
face_client.clone(),
&excluded_dirs,
&effective_excludes,
None,
playlist_manager.clone(),
preview_generator.clone(),
@@ -2094,7 +2105,7 @@ fn watch_files(
Arc::clone(&face_dao),
Arc::clone(&watcher_tag_dao),
face_client.clone(),
&excluded_dirs,
&effective_excludes,
Some(check_since),
playlist_manager.clone(),
preview_generator.clone(),
@@ -2102,7 +2113,7 @@ fn watch_files(
}
// Update media counts per library (metric aggregates across all)
update_media_counts(Path::new(&lib.root_path), &excluded_dirs);
update_media_counts(Path::new(&lib.root_path), &effective_excludes);
// Missing-file detection: prune image_exif rows whose
// source file is no longer on disk. Per-library, so we