Compare commits
6 Commits
master
...
39d284dbbb
| Author | SHA1 | Date | |
|---|---|---|---|
|
39d284dbbb | ||
|
|
125ba6192e | ||
|
|
50d557001b | ||
|
|
cf9dd826c1 | ||
|
|
4834cacfc3 | ||
|
|
51081d01c6 |
@@ -1,3 +0,0 @@
|
|||||||
[target.x86_64-unknown-linux-gnu]
|
|
||||||
linker = "/usr/bin/gcc"
|
|
||||||
rustflags = ["-C", "link-arg=-fuse-ld=mold"]
|
|
||||||
@@ -1,184 +0,0 @@
|
|||||||
---
|
|
||||||
description: Perform a non-destructive cross-artifact consistency and quality analysis across spec.md, plan.md, and tasks.md after task generation.
|
|
||||||
---
|
|
||||||
|
|
||||||
## User Input
|
|
||||||
|
|
||||||
```text
|
|
||||||
$ARGUMENTS
|
|
||||||
```
|
|
||||||
|
|
||||||
You **MUST** consider the user input before proceeding (if not empty).
|
|
||||||
|
|
||||||
## Goal
|
|
||||||
|
|
||||||
Identify inconsistencies, duplications, ambiguities, and underspecified items across the three core artifacts (`spec.md`, `plan.md`, `tasks.md`) before implementation. This command MUST run only after `/speckit.tasks` has successfully produced a complete `tasks.md`.
|
|
||||||
|
|
||||||
## Operating Constraints
|
|
||||||
|
|
||||||
**STRICTLY READ-ONLY**: Do **not** modify any files. Output a structured analysis report. Offer an optional remediation plan (user must explicitly approve before any follow-up editing commands would be invoked manually).
|
|
||||||
|
|
||||||
**Constitution Authority**: The project constitution (`.specify/memory/constitution.md`) is **non-negotiable** within this analysis scope. Constitution conflicts are automatically CRITICAL and require adjustment of the spec, plan, or tasks—not dilution, reinterpretation, or silent ignoring of the principle. If a principle itself needs to change, that must occur in a separate, explicit constitution update outside `/speckit.analyze`.
|
|
||||||
|
|
||||||
## Execution Steps
|
|
||||||
|
|
||||||
### 1. Initialize Analysis Context
|
|
||||||
|
|
||||||
Run `.specify/scripts/powershell/check-prerequisites.ps1 -Json -RequireTasks -IncludeTasks` once from repo root and parse JSON for FEATURE_DIR and AVAILABLE_DOCS. Derive absolute paths:
|
|
||||||
|
|
||||||
- SPEC = FEATURE_DIR/spec.md
|
|
||||||
- PLAN = FEATURE_DIR/plan.md
|
|
||||||
- TASKS = FEATURE_DIR/tasks.md
|
|
||||||
|
|
||||||
Abort with an error message if any required file is missing (instruct the user to run missing prerequisite command).
|
|
||||||
For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
|
||||||
|
|
||||||
### 2. Load Artifacts (Progressive Disclosure)
|
|
||||||
|
|
||||||
Load only the minimal necessary context from each artifact:
|
|
||||||
|
|
||||||
**From spec.md:**
|
|
||||||
|
|
||||||
- Overview/Context
|
|
||||||
- Functional Requirements
|
|
||||||
- Non-Functional Requirements
|
|
||||||
- User Stories
|
|
||||||
- Edge Cases (if present)
|
|
||||||
|
|
||||||
**From plan.md:**
|
|
||||||
|
|
||||||
- Architecture/stack choices
|
|
||||||
- Data Model references
|
|
||||||
- Phases
|
|
||||||
- Technical constraints
|
|
||||||
|
|
||||||
**From tasks.md:**
|
|
||||||
|
|
||||||
- Task IDs
|
|
||||||
- Descriptions
|
|
||||||
- Phase grouping
|
|
||||||
- Parallel markers [P]
|
|
||||||
- Referenced file paths
|
|
||||||
|
|
||||||
**From constitution:**
|
|
||||||
|
|
||||||
- Load `.specify/memory/constitution.md` for principle validation
|
|
||||||
|
|
||||||
### 3. Build Semantic Models
|
|
||||||
|
|
||||||
Create internal representations (do not include raw artifacts in output):
|
|
||||||
|
|
||||||
- **Requirements inventory**: Each functional + non-functional requirement with a stable key (derive slug based on imperative phrase; e.g., "User can upload file" → `user-can-upload-file`)
|
|
||||||
- **User story/action inventory**: Discrete user actions with acceptance criteria
|
|
||||||
- **Task coverage mapping**: Map each task to one or more requirements or stories (inference by keyword / explicit reference patterns like IDs or key phrases)
|
|
||||||
- **Constitution rule set**: Extract principle names and MUST/SHOULD normative statements
|
|
||||||
|
|
||||||
### 4. Detection Passes (Token-Efficient Analysis)
|
|
||||||
|
|
||||||
Focus on high-signal findings. Limit to 50 findings total; aggregate remainder in overflow summary.
|
|
||||||
|
|
||||||
#### A. Duplication Detection
|
|
||||||
|
|
||||||
- Identify near-duplicate requirements
|
|
||||||
- Mark lower-quality phrasing for consolidation
|
|
||||||
|
|
||||||
#### B. Ambiguity Detection
|
|
||||||
|
|
||||||
- Flag vague adjectives (fast, scalable, secure, intuitive, robust) lacking measurable criteria
|
|
||||||
- Flag unresolved placeholders (TODO, TKTK, ???, `<placeholder>`, etc.)
|
|
||||||
|
|
||||||
#### C. Underspecification
|
|
||||||
|
|
||||||
- Requirements with verbs but missing object or measurable outcome
|
|
||||||
- User stories missing acceptance criteria alignment
|
|
||||||
- Tasks referencing files or components not defined in spec/plan
|
|
||||||
|
|
||||||
#### D. Constitution Alignment
|
|
||||||
|
|
||||||
- Any requirement or plan element conflicting with a MUST principle
|
|
||||||
- Missing mandated sections or quality gates from constitution
|
|
||||||
|
|
||||||
#### E. Coverage Gaps
|
|
||||||
|
|
||||||
- Requirements with zero associated tasks
|
|
||||||
- Tasks with no mapped requirement/story
|
|
||||||
- Non-functional requirements not reflected in tasks (e.g., performance, security)
|
|
||||||
|
|
||||||
#### F. Inconsistency
|
|
||||||
|
|
||||||
- Terminology drift (same concept named differently across files)
|
|
||||||
- Data entities referenced in plan but absent in spec (or vice versa)
|
|
||||||
- Task ordering contradictions (e.g., integration tasks before foundational setup tasks without dependency note)
|
|
||||||
- Conflicting requirements (e.g., one requires Next.js while other specifies Vue)
|
|
||||||
|
|
||||||
### 5. Severity Assignment
|
|
||||||
|
|
||||||
Use this heuristic to prioritize findings:
|
|
||||||
|
|
||||||
- **CRITICAL**: Violates constitution MUST, missing core spec artifact, or requirement with zero coverage that blocks baseline functionality
|
|
||||||
- **HIGH**: Duplicate or conflicting requirement, ambiguous security/performance attribute, untestable acceptance criterion
|
|
||||||
- **MEDIUM**: Terminology drift, missing non-functional task coverage, underspecified edge case
|
|
||||||
- **LOW**: Style/wording improvements, minor redundancy not affecting execution order
|
|
||||||
|
|
||||||
### 6. Produce Compact Analysis Report
|
|
||||||
|
|
||||||
Output a Markdown report (no file writes) with the following structure:
|
|
||||||
|
|
||||||
## Specification Analysis Report
|
|
||||||
|
|
||||||
| ID | Category | Severity | Location(s) | Summary | Recommendation |
|
|
||||||
|----|----------|----------|-------------|---------|----------------|
|
|
||||||
| A1 | Duplication | HIGH | spec.md:L120-134 | Two similar requirements ... | Merge phrasing; keep clearer version |
|
|
||||||
|
|
||||||
(Add one row per finding; generate stable IDs prefixed by category initial.)
|
|
||||||
|
|
||||||
**Coverage Summary Table:**
|
|
||||||
|
|
||||||
| Requirement Key | Has Task? | Task IDs | Notes |
|
|
||||||
|-----------------|-----------|----------|-------|
|
|
||||||
|
|
||||||
**Constitution Alignment Issues:** (if any)
|
|
||||||
|
|
||||||
**Unmapped Tasks:** (if any)
|
|
||||||
|
|
||||||
**Metrics:**
|
|
||||||
|
|
||||||
- Total Requirements
|
|
||||||
- Total Tasks
|
|
||||||
- Coverage % (requirements with >=1 task)
|
|
||||||
- Ambiguity Count
|
|
||||||
- Duplication Count
|
|
||||||
- Critical Issues Count
|
|
||||||
|
|
||||||
### 7. Provide Next Actions
|
|
||||||
|
|
||||||
At end of report, output a concise Next Actions block:
|
|
||||||
|
|
||||||
- If CRITICAL issues exist: Recommend resolving before `/speckit.implement`
|
|
||||||
- If only LOW/MEDIUM: User may proceed, but provide improvement suggestions
|
|
||||||
- Provide explicit command suggestions: e.g., "Run /speckit.specify with refinement", "Run /speckit.plan to adjust architecture", "Manually edit tasks.md to add coverage for 'performance-metrics'"
|
|
||||||
|
|
||||||
### 8. Offer Remediation
|
|
||||||
|
|
||||||
Ask the user: "Would you like me to suggest concrete remediation edits for the top N issues?" (Do NOT apply them automatically.)
|
|
||||||
|
|
||||||
## Operating Principles
|
|
||||||
|
|
||||||
### Context Efficiency
|
|
||||||
|
|
||||||
- **Minimal high-signal tokens**: Focus on actionable findings, not exhaustive documentation
|
|
||||||
- **Progressive disclosure**: Load artifacts incrementally; don't dump all content into analysis
|
|
||||||
- **Token-efficient output**: Limit findings table to 50 rows; summarize overflow
|
|
||||||
- **Deterministic results**: Rerunning without changes should produce consistent IDs and counts
|
|
||||||
|
|
||||||
### Analysis Guidelines
|
|
||||||
|
|
||||||
- **NEVER modify files** (this is read-only analysis)
|
|
||||||
- **NEVER hallucinate missing sections** (if absent, report them accurately)
|
|
||||||
- **Prioritize constitution violations** (these are always CRITICAL)
|
|
||||||
- **Use examples over exhaustive rules** (cite specific instances, not generic patterns)
|
|
||||||
- **Report zero issues gracefully** (emit success report with coverage statistics)
|
|
||||||
|
|
||||||
## Context
|
|
||||||
|
|
||||||
$ARGUMENTS
|
|
||||||
@@ -1,294 +0,0 @@
|
|||||||
---
|
|
||||||
description: Generate a custom checklist for the current feature based on user requirements.
|
|
||||||
---
|
|
||||||
|
|
||||||
## Checklist Purpose: "Unit Tests for English"
|
|
||||||
|
|
||||||
**CRITICAL CONCEPT**: Checklists are **UNIT TESTS FOR REQUIREMENTS WRITING** - they validate the quality, clarity, and completeness of requirements in a given domain.
|
|
||||||
|
|
||||||
**NOT for verification/testing**:
|
|
||||||
|
|
||||||
- ❌ NOT "Verify the button clicks correctly"
|
|
||||||
- ❌ NOT "Test error handling works"
|
|
||||||
- ❌ NOT "Confirm the API returns 200"
|
|
||||||
- ❌ NOT checking if code/implementation matches the spec
|
|
||||||
|
|
||||||
**FOR requirements quality validation**:
|
|
||||||
|
|
||||||
- ✅ "Are visual hierarchy requirements defined for all card types?" (completeness)
|
|
||||||
- ✅ "Is 'prominent display' quantified with specific sizing/positioning?" (clarity)
|
|
||||||
- ✅ "Are hover state requirements consistent across all interactive elements?" (consistency)
|
|
||||||
- ✅ "Are accessibility requirements defined for keyboard navigation?" (coverage)
|
|
||||||
- ✅ "Does the spec define what happens when logo image fails to load?" (edge cases)
|
|
||||||
|
|
||||||
**Metaphor**: If your spec is code written in English, the checklist is its unit test suite. You're testing whether the requirements are well-written, complete, unambiguous, and ready for implementation - NOT whether the implementation works.
|
|
||||||
|
|
||||||
## User Input
|
|
||||||
|
|
||||||
```text
|
|
||||||
$ARGUMENTS
|
|
||||||
```
|
|
||||||
|
|
||||||
You **MUST** consider the user input before proceeding (if not empty).
|
|
||||||
|
|
||||||
## Execution Steps
|
|
||||||
|
|
||||||
1. **Setup**: Run `.specify/scripts/powershell/check-prerequisites.ps1 -Json` from repo root and parse JSON for FEATURE_DIR and AVAILABLE_DOCS list.
|
|
||||||
- All file paths must be absolute.
|
|
||||||
- For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
|
||||||
|
|
||||||
2. **Clarify intent (dynamic)**: Derive up to THREE initial contextual clarifying questions (no pre-baked catalog). They MUST:
|
|
||||||
- Be generated from the user's phrasing + extracted signals from spec/plan/tasks
|
|
||||||
- Only ask about information that materially changes checklist content
|
|
||||||
- Be skipped individually if already unambiguous in `$ARGUMENTS`
|
|
||||||
- Prefer precision over breadth
|
|
||||||
|
|
||||||
Generation algorithm:
|
|
||||||
1. Extract signals: feature domain keywords (e.g., auth, latency, UX, API), risk indicators ("critical", "must", "compliance"), stakeholder hints ("QA", "review", "security team"), and explicit deliverables ("a11y", "rollback", "contracts").
|
|
||||||
2. Cluster signals into candidate focus areas (max 4) ranked by relevance.
|
|
||||||
3. Identify probable audience & timing (author, reviewer, QA, release) if not explicit.
|
|
||||||
4. Detect missing dimensions: scope breadth, depth/rigor, risk emphasis, exclusion boundaries, measurable acceptance criteria.
|
|
||||||
5. Formulate questions chosen from these archetypes:
|
|
||||||
- Scope refinement (e.g., "Should this include integration touchpoints with X and Y or stay limited to local module correctness?")
|
|
||||||
- Risk prioritization (e.g., "Which of these potential risk areas should receive mandatory gating checks?")
|
|
||||||
- Depth calibration (e.g., "Is this a lightweight pre-commit sanity list or a formal release gate?")
|
|
||||||
- Audience framing (e.g., "Will this be used by the author only or peers during PR review?")
|
|
||||||
- Boundary exclusion (e.g., "Should we explicitly exclude performance tuning items this round?")
|
|
||||||
- Scenario class gap (e.g., "No recovery flows detected—are rollback / partial failure paths in scope?")
|
|
||||||
|
|
||||||
Question formatting rules:
|
|
||||||
- If presenting options, generate a compact table with columns: Option | Candidate | Why It Matters
|
|
||||||
- Limit to A–E options maximum; omit table if a free-form answer is clearer
|
|
||||||
- Never ask the user to restate what they already said
|
|
||||||
- Avoid speculative categories (no hallucination). If uncertain, ask explicitly: "Confirm whether X belongs in scope."
|
|
||||||
|
|
||||||
Defaults when interaction impossible:
|
|
||||||
- Depth: Standard
|
|
||||||
- Audience: Reviewer (PR) if code-related; Author otherwise
|
|
||||||
- Focus: Top 2 relevance clusters
|
|
||||||
|
|
||||||
Output the questions (label Q1/Q2/Q3). After answers: if ≥2 scenario classes (Alternate / Exception / Recovery / Non-Functional domain) remain unclear, you MAY ask up to TWO more targeted follow‑ups (Q4/Q5) with a one-line justification each (e.g., "Unresolved recovery path risk"). Do not exceed five total questions. Skip escalation if user explicitly declines more.
|
|
||||||
|
|
||||||
3. **Understand user request**: Combine `$ARGUMENTS` + clarifying answers:
|
|
||||||
- Derive checklist theme (e.g., security, review, deploy, ux)
|
|
||||||
- Consolidate explicit must-have items mentioned by user
|
|
||||||
- Map focus selections to category scaffolding
|
|
||||||
- Infer any missing context from spec/plan/tasks (do NOT hallucinate)
|
|
||||||
|
|
||||||
4. **Load feature context**: Read from FEATURE_DIR:
|
|
||||||
- spec.md: Feature requirements and scope
|
|
||||||
- plan.md (if exists): Technical details, dependencies
|
|
||||||
- tasks.md (if exists): Implementation tasks
|
|
||||||
|
|
||||||
**Context Loading Strategy**:
|
|
||||||
- Load only necessary portions relevant to active focus areas (avoid full-file dumping)
|
|
||||||
- Prefer summarizing long sections into concise scenario/requirement bullets
|
|
||||||
- Use progressive disclosure: add follow-on retrieval only if gaps detected
|
|
||||||
- If source docs are large, generate interim summary items instead of embedding raw text
|
|
||||||
|
|
||||||
5. **Generate checklist** - Create "Unit Tests for Requirements":
|
|
||||||
- Create `FEATURE_DIR/checklists/` directory if it doesn't exist
|
|
||||||
- Generate unique checklist filename:
|
|
||||||
- Use short, descriptive name based on domain (e.g., `ux.md`, `api.md`, `security.md`)
|
|
||||||
- Format: `[domain].md`
|
|
||||||
- If file exists, append to existing file
|
|
||||||
- Number items sequentially starting from CHK001
|
|
||||||
- Each `/speckit.checklist` run creates a NEW file (never overwrites existing checklists)
|
|
||||||
|
|
||||||
**CORE PRINCIPLE - Test the Requirements, Not the Implementation**:
|
|
||||||
Every checklist item MUST evaluate the REQUIREMENTS THEMSELVES for:
|
|
||||||
- **Completeness**: Are all necessary requirements present?
|
|
||||||
- **Clarity**: Are requirements unambiguous and specific?
|
|
||||||
- **Consistency**: Do requirements align with each other?
|
|
||||||
- **Measurability**: Can requirements be objectively verified?
|
|
||||||
- **Coverage**: Are all scenarios/edge cases addressed?
|
|
||||||
|
|
||||||
**Category Structure** - Group items by requirement quality dimensions:
|
|
||||||
- **Requirement Completeness** (Are all necessary requirements documented?)
|
|
||||||
- **Requirement Clarity** (Are requirements specific and unambiguous?)
|
|
||||||
- **Requirement Consistency** (Do requirements align without conflicts?)
|
|
||||||
- **Acceptance Criteria Quality** (Are success criteria measurable?)
|
|
||||||
- **Scenario Coverage** (Are all flows/cases addressed?)
|
|
||||||
- **Edge Case Coverage** (Are boundary conditions defined?)
|
|
||||||
- **Non-Functional Requirements** (Performance, Security, Accessibility, etc. - are they specified?)
|
|
||||||
- **Dependencies & Assumptions** (Are they documented and validated?)
|
|
||||||
- **Ambiguities & Conflicts** (What needs clarification?)
|
|
||||||
|
|
||||||
**HOW TO WRITE CHECKLIST ITEMS - "Unit Tests for English"**:
|
|
||||||
|
|
||||||
❌ **WRONG** (Testing implementation):
|
|
||||||
- "Verify landing page displays 3 episode cards"
|
|
||||||
- "Test hover states work on desktop"
|
|
||||||
- "Confirm logo click navigates home"
|
|
||||||
|
|
||||||
✅ **CORRECT** (Testing requirements quality):
|
|
||||||
- "Are the exact number and layout of featured episodes specified?" [Completeness]
|
|
||||||
- "Is 'prominent display' quantified with specific sizing/positioning?" [Clarity]
|
|
||||||
- "Are hover state requirements consistent across all interactive elements?" [Consistency]
|
|
||||||
- "Are keyboard navigation requirements defined for all interactive UI?" [Coverage]
|
|
||||||
- "Is the fallback behavior specified when logo image fails to load?" [Edge Cases]
|
|
||||||
- "Are loading states defined for asynchronous episode data?" [Completeness]
|
|
||||||
- "Does the spec define visual hierarchy for competing UI elements?" [Clarity]
|
|
||||||
|
|
||||||
**ITEM STRUCTURE**:
|
|
||||||
Each item should follow this pattern:
|
|
||||||
- Question format asking about requirement quality
|
|
||||||
- Focus on what's WRITTEN (or not written) in the spec/plan
|
|
||||||
- Include quality dimension in brackets [Completeness/Clarity/Consistency/etc.]
|
|
||||||
- Reference spec section `[Spec §X.Y]` when checking existing requirements
|
|
||||||
- Use `[Gap]` marker when checking for missing requirements
|
|
||||||
|
|
||||||
**EXAMPLES BY QUALITY DIMENSION**:
|
|
||||||
|
|
||||||
Completeness:
|
|
||||||
- "Are error handling requirements defined for all API failure modes? [Gap]"
|
|
||||||
- "Are accessibility requirements specified for all interactive elements? [Completeness]"
|
|
||||||
- "Are mobile breakpoint requirements defined for responsive layouts? [Gap]"
|
|
||||||
|
|
||||||
Clarity:
|
|
||||||
- "Is 'fast loading' quantified with specific timing thresholds? [Clarity, Spec §NFR-2]"
|
|
||||||
- "Are 'related episodes' selection criteria explicitly defined? [Clarity, Spec §FR-5]"
|
|
||||||
- "Is 'prominent' defined with measurable visual properties? [Ambiguity, Spec §FR-4]"
|
|
||||||
|
|
||||||
Consistency:
|
|
||||||
- "Do navigation requirements align across all pages? [Consistency, Spec §FR-10]"
|
|
||||||
- "Are card component requirements consistent between landing and detail pages? [Consistency]"
|
|
||||||
|
|
||||||
Coverage:
|
|
||||||
- "Are requirements defined for zero-state scenarios (no episodes)? [Coverage, Edge Case]"
|
|
||||||
- "Are concurrent user interaction scenarios addressed? [Coverage, Gap]"
|
|
||||||
- "Are requirements specified for partial data loading failures? [Coverage, Exception Flow]"
|
|
||||||
|
|
||||||
Measurability:
|
|
||||||
- "Are visual hierarchy requirements measurable/testable? [Acceptance Criteria, Spec §FR-1]"
|
|
||||||
- "Can 'balanced visual weight' be objectively verified? [Measurability, Spec §FR-2]"
|
|
||||||
|
|
||||||
**Scenario Classification & Coverage** (Requirements Quality Focus):
|
|
||||||
- Check if requirements exist for: Primary, Alternate, Exception/Error, Recovery, Non-Functional scenarios
|
|
||||||
- For each scenario class, ask: "Are [scenario type] requirements complete, clear, and consistent?"
|
|
||||||
- If scenario class missing: "Are [scenario type] requirements intentionally excluded or missing? [Gap]"
|
|
||||||
- Include resilience/rollback when state mutation occurs: "Are rollback requirements defined for migration failures? [Gap]"
|
|
||||||
|
|
||||||
**Traceability Requirements**:
|
|
||||||
- MINIMUM: ≥80% of items MUST include at least one traceability reference
|
|
||||||
- Each item should reference: spec section `[Spec §X.Y]`, or use markers: `[Gap]`, `[Ambiguity]`, `[Conflict]`, `[Assumption]`
|
|
||||||
- If no ID system exists: "Is a requirement & acceptance criteria ID scheme established? [Traceability]"
|
|
||||||
|
|
||||||
**Surface & Resolve Issues** (Requirements Quality Problems):
|
|
||||||
Ask questions about the requirements themselves:
|
|
||||||
- Ambiguities: "Is the term 'fast' quantified with specific metrics? [Ambiguity, Spec §NFR-1]"
|
|
||||||
- Conflicts: "Do navigation requirements conflict between §FR-10 and §FR-10a? [Conflict]"
|
|
||||||
- Assumptions: "Is the assumption of 'always available podcast API' validated? [Assumption]"
|
|
||||||
- Dependencies: "Are external podcast API requirements documented? [Dependency, Gap]"
|
|
||||||
- Missing definitions: "Is 'visual hierarchy' defined with measurable criteria? [Gap]"
|
|
||||||
|
|
||||||
**Content Consolidation**:
|
|
||||||
- Soft cap: If raw candidate items > 40, prioritize by risk/impact
|
|
||||||
- Merge near-duplicates checking the same requirement aspect
|
|
||||||
- If >5 low-impact edge cases, create one item: "Are edge cases X, Y, Z addressed in requirements? [Coverage]"
|
|
||||||
|
|
||||||
**🚫 ABSOLUTELY PROHIBITED** - These make it an implementation test, not a requirements test:
|
|
||||||
- ❌ Any item starting with "Verify", "Test", "Confirm", "Check" + implementation behavior
|
|
||||||
- ❌ References to code execution, user actions, system behavior
|
|
||||||
- ❌ "Displays correctly", "works properly", "functions as expected"
|
|
||||||
- ❌ "Click", "navigate", "render", "load", "execute"
|
|
||||||
- ❌ Test cases, test plans, QA procedures
|
|
||||||
- ❌ Implementation details (frameworks, APIs, algorithms)
|
|
||||||
|
|
||||||
**✅ REQUIRED PATTERNS** - These test requirements quality:
|
|
||||||
- ✅ "Are [requirement type] defined/specified/documented for [scenario]?"
|
|
||||||
- ✅ "Is [vague term] quantified/clarified with specific criteria?"
|
|
||||||
- ✅ "Are requirements consistent between [section A] and [section B]?"
|
|
||||||
- ✅ "Can [requirement] be objectively measured/verified?"
|
|
||||||
- ✅ "Are [edge cases/scenarios] addressed in requirements?"
|
|
||||||
- ✅ "Does the spec define [missing aspect]?"
|
|
||||||
|
|
||||||
6. **Structure Reference**: Generate the checklist following the canonical template in `.specify/templates/checklist-template.md` for title, meta section, category headings, and ID formatting. If template is unavailable, use: H1 title, purpose/created meta lines, `##` category sections containing `- [ ] CHK### <requirement item>` lines with globally incrementing IDs starting at CHK001.
|
|
||||||
|
|
||||||
7. **Report**: Output full path to created checklist, item count, and remind user that each run creates a new file. Summarize:
|
|
||||||
- Focus areas selected
|
|
||||||
- Depth level
|
|
||||||
- Actor/timing
|
|
||||||
- Any explicit user-specified must-have items incorporated
|
|
||||||
|
|
||||||
**Important**: Each `/speckit.checklist` command invocation creates a checklist file using short, descriptive names unless file already exists. This allows:
|
|
||||||
|
|
||||||
- Multiple checklists of different types (e.g., `ux.md`, `test.md`, `security.md`)
|
|
||||||
- Simple, memorable filenames that indicate checklist purpose
|
|
||||||
- Easy identification and navigation in the `checklists/` folder
|
|
||||||
|
|
||||||
To avoid clutter, use descriptive types and clean up obsolete checklists when done.
|
|
||||||
|
|
||||||
## Example Checklist Types & Sample Items
|
|
||||||
|
|
||||||
**UX Requirements Quality:** `ux.md`
|
|
||||||
|
|
||||||
Sample items (testing the requirements, NOT the implementation):
|
|
||||||
|
|
||||||
- "Are visual hierarchy requirements defined with measurable criteria? [Clarity, Spec §FR-1]"
|
|
||||||
- "Is the number and positioning of UI elements explicitly specified? [Completeness, Spec §FR-1]"
|
|
||||||
- "Are interaction state requirements (hover, focus, active) consistently defined? [Consistency]"
|
|
||||||
- "Are accessibility requirements specified for all interactive elements? [Coverage, Gap]"
|
|
||||||
- "Is fallback behavior defined when images fail to load? [Edge Case, Gap]"
|
|
||||||
- "Can 'prominent display' be objectively measured? [Measurability, Spec §FR-4]"
|
|
||||||
|
|
||||||
**API Requirements Quality:** `api.md`
|
|
||||||
|
|
||||||
Sample items:
|
|
||||||
|
|
||||||
- "Are error response formats specified for all failure scenarios? [Completeness]"
|
|
||||||
- "Are rate limiting requirements quantified with specific thresholds? [Clarity]"
|
|
||||||
- "Are authentication requirements consistent across all endpoints? [Consistency]"
|
|
||||||
- "Are retry/timeout requirements defined for external dependencies? [Coverage, Gap]"
|
|
||||||
- "Is versioning strategy documented in requirements? [Gap]"
|
|
||||||
|
|
||||||
**Performance Requirements Quality:** `performance.md`
|
|
||||||
|
|
||||||
Sample items:
|
|
||||||
|
|
||||||
- "Are performance requirements quantified with specific metrics? [Clarity]"
|
|
||||||
- "Are performance targets defined for all critical user journeys? [Coverage]"
|
|
||||||
- "Are performance requirements under different load conditions specified? [Completeness]"
|
|
||||||
- "Can performance requirements be objectively measured? [Measurability]"
|
|
||||||
- "Are degradation requirements defined for high-load scenarios? [Edge Case, Gap]"
|
|
||||||
|
|
||||||
**Security Requirements Quality:** `security.md`
|
|
||||||
|
|
||||||
Sample items:
|
|
||||||
|
|
||||||
- "Are authentication requirements specified for all protected resources? [Coverage]"
|
|
||||||
- "Are data protection requirements defined for sensitive information? [Completeness]"
|
|
||||||
- "Is the threat model documented and requirements aligned to it? [Traceability]"
|
|
||||||
- "Are security requirements consistent with compliance obligations? [Consistency]"
|
|
||||||
- "Are security failure/breach response requirements defined? [Gap, Exception Flow]"
|
|
||||||
|
|
||||||
## Anti-Examples: What NOT To Do
|
|
||||||
|
|
||||||
**❌ WRONG - These test implementation, not requirements:**
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
- [ ] CHK001 - Verify landing page displays 3 episode cards [Spec §FR-001]
|
|
||||||
- [ ] CHK002 - Test hover states work correctly on desktop [Spec §FR-003]
|
|
||||||
- [ ] CHK003 - Confirm logo click navigates to home page [Spec §FR-010]
|
|
||||||
- [ ] CHK004 - Check that related episodes section shows 3-5 items [Spec §FR-005]
|
|
||||||
```
|
|
||||||
|
|
||||||
**✅ CORRECT - These test requirements quality:**
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
- [ ] CHK001 - Are the number and layout of featured episodes explicitly specified? [Completeness, Spec §FR-001]
|
|
||||||
- [ ] CHK002 - Are hover state requirements consistently defined for all interactive elements? [Consistency, Spec §FR-003]
|
|
||||||
- [ ] CHK003 - Are navigation requirements clear for all clickable brand elements? [Clarity, Spec §FR-010]
|
|
||||||
- [ ] CHK004 - Is the selection criteria for related episodes documented? [Gap, Spec §FR-005]
|
|
||||||
- [ ] CHK005 - Are loading state requirements defined for asynchronous episode data? [Gap]
|
|
||||||
- [ ] CHK006 - Can "visual hierarchy" requirements be objectively measured? [Measurability, Spec §FR-001]
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key Differences:**
|
|
||||||
|
|
||||||
- Wrong: Tests if the system works correctly
|
|
||||||
- Correct: Tests if the requirements are written correctly
|
|
||||||
- Wrong: Verification of behavior
|
|
||||||
- Correct: Validation of requirement quality
|
|
||||||
- Wrong: "Does it do X?"
|
|
||||||
- Correct: "Is X clearly specified?"
|
|
||||||
@@ -1,181 +0,0 @@
|
|||||||
---
|
|
||||||
description: Identify underspecified areas in the current feature spec by asking up to 5 highly targeted clarification questions and encoding answers back into the spec.
|
|
||||||
handoffs:
|
|
||||||
- label: Build Technical Plan
|
|
||||||
agent: speckit.plan
|
|
||||||
prompt: Create a plan for the spec. I am building with...
|
|
||||||
---
|
|
||||||
|
|
||||||
## User Input
|
|
||||||
|
|
||||||
```text
|
|
||||||
$ARGUMENTS
|
|
||||||
```
|
|
||||||
|
|
||||||
You **MUST** consider the user input before proceeding (if not empty).
|
|
||||||
|
|
||||||
## Outline
|
|
||||||
|
|
||||||
Goal: Detect and reduce ambiguity or missing decision points in the active feature specification and record the clarifications directly in the spec file.
|
|
||||||
|
|
||||||
Note: This clarification workflow is expected to run (and be completed) BEFORE invoking `/speckit.plan`. If the user explicitly states they are skipping clarification (e.g., exploratory spike), you may proceed, but must warn that downstream rework risk increases.
|
|
||||||
|
|
||||||
Execution steps:
|
|
||||||
|
|
||||||
1. Run `.specify/scripts/powershell/check-prerequisites.ps1 -Json -PathsOnly` from repo root **once** (combined `--json --paths-only` mode / `-Json -PathsOnly`). Parse minimal JSON payload fields:
|
|
||||||
- `FEATURE_DIR`
|
|
||||||
- `FEATURE_SPEC`
|
|
||||||
- (Optionally capture `IMPL_PLAN`, `TASKS` for future chained flows.)
|
|
||||||
- If JSON parsing fails, abort and instruct user to re-run `/speckit.specify` or verify feature branch environment.
|
|
||||||
- For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
|
||||||
|
|
||||||
2. Load the current spec file. Perform a structured ambiguity & coverage scan using this taxonomy. For each category, mark status: Clear / Partial / Missing. Produce an internal coverage map used for prioritization (do not output raw map unless no questions will be asked).
|
|
||||||
|
|
||||||
Functional Scope & Behavior:
|
|
||||||
- Core user goals & success criteria
|
|
||||||
- Explicit out-of-scope declarations
|
|
||||||
- User roles / personas differentiation
|
|
||||||
|
|
||||||
Domain & Data Model:
|
|
||||||
- Entities, attributes, relationships
|
|
||||||
- Identity & uniqueness rules
|
|
||||||
- Lifecycle/state transitions
|
|
||||||
- Data volume / scale assumptions
|
|
||||||
|
|
||||||
Interaction & UX Flow:
|
|
||||||
- Critical user journeys / sequences
|
|
||||||
- Error/empty/loading states
|
|
||||||
- Accessibility or localization notes
|
|
||||||
|
|
||||||
Non-Functional Quality Attributes:
|
|
||||||
- Performance (latency, throughput targets)
|
|
||||||
- Scalability (horizontal/vertical, limits)
|
|
||||||
- Reliability & availability (uptime, recovery expectations)
|
|
||||||
- Observability (logging, metrics, tracing signals)
|
|
||||||
- Security & privacy (authN/Z, data protection, threat assumptions)
|
|
||||||
- Compliance / regulatory constraints (if any)
|
|
||||||
|
|
||||||
Integration & External Dependencies:
|
|
||||||
- External services/APIs and failure modes
|
|
||||||
- Data import/export formats
|
|
||||||
- Protocol/versioning assumptions
|
|
||||||
|
|
||||||
Edge Cases & Failure Handling:
|
|
||||||
- Negative scenarios
|
|
||||||
- Rate limiting / throttling
|
|
||||||
- Conflict resolution (e.g., concurrent edits)
|
|
||||||
|
|
||||||
Constraints & Tradeoffs:
|
|
||||||
- Technical constraints (language, storage, hosting)
|
|
||||||
- Explicit tradeoffs or rejected alternatives
|
|
||||||
|
|
||||||
Terminology & Consistency:
|
|
||||||
- Canonical glossary terms
|
|
||||||
- Avoided synonyms / deprecated terms
|
|
||||||
|
|
||||||
Completion Signals:
|
|
||||||
- Acceptance criteria testability
|
|
||||||
- Measurable Definition of Done style indicators
|
|
||||||
|
|
||||||
Misc / Placeholders:
|
|
||||||
- TODO markers / unresolved decisions
|
|
||||||
- Ambiguous adjectives ("robust", "intuitive") lacking quantification
|
|
||||||
|
|
||||||
For each category with Partial or Missing status, add a candidate question opportunity unless:
|
|
||||||
- Clarification would not materially change implementation or validation strategy
|
|
||||||
- Information is better deferred to planning phase (note internally)
|
|
||||||
|
|
||||||
3. Generate (internally) a prioritized queue of candidate clarification questions (maximum 5). Do NOT output them all at once. Apply these constraints:
|
|
||||||
- Maximum of 10 total questions across the whole session.
|
|
||||||
- Each question must be answerable with EITHER:
|
|
||||||
- A short multiple‑choice selection (2–5 distinct, mutually exclusive options), OR
|
|
||||||
- A one-word / short‑phrase answer (explicitly constrain: "Answer in <=5 words").
|
|
||||||
- Only include questions whose answers materially impact architecture, data modeling, task decomposition, test design, UX behavior, operational readiness, or compliance validation.
|
|
||||||
- Ensure category coverage balance: attempt to cover the highest impact unresolved categories first; avoid asking two low-impact questions when a single high-impact area (e.g., security posture) is unresolved.
|
|
||||||
- Exclude questions already answered, trivial stylistic preferences, or plan-level execution details (unless blocking correctness).
|
|
||||||
- Favor clarifications that reduce downstream rework risk or prevent misaligned acceptance tests.
|
|
||||||
- If more than 5 categories remain unresolved, select the top 5 by (Impact * Uncertainty) heuristic.
|
|
||||||
|
|
||||||
4. Sequential questioning loop (interactive):
|
|
||||||
- Present EXACTLY ONE question at a time.
|
|
||||||
- For multiple‑choice questions:
|
|
||||||
- **Analyze all options** and determine the **most suitable option** based on:
|
|
||||||
- Best practices for the project type
|
|
||||||
- Common patterns in similar implementations
|
|
||||||
- Risk reduction (security, performance, maintainability)
|
|
||||||
- Alignment with any explicit project goals or constraints visible in the spec
|
|
||||||
- Present your **recommended option prominently** at the top with clear reasoning (1-2 sentences explaining why this is the best choice).
|
|
||||||
- Format as: `**Recommended:** Option [X] - <reasoning>`
|
|
||||||
- Then render all options as a Markdown table:
|
|
||||||
|
|
||||||
| Option | Description |
|
|
||||||
|--------|-------------|
|
|
||||||
| A | <Option A description> |
|
|
||||||
| B | <Option B description> |
|
|
||||||
| C | <Option C description> (add D/E as needed up to 5) |
|
|
||||||
| Short | Provide a different short answer (<=5 words) (Include only if free-form alternative is appropriate) |
|
|
||||||
|
|
||||||
- After the table, add: `You can reply with the option letter (e.g., "A"), accept the recommendation by saying "yes" or "recommended", or provide your own short answer.`
|
|
||||||
- For short‑answer style (no meaningful discrete options):
|
|
||||||
- Provide your **suggested answer** based on best practices and context.
|
|
||||||
- Format as: `**Suggested:** <your proposed answer> - <brief reasoning>`
|
|
||||||
- Then output: `Format: Short answer (<=5 words). You can accept the suggestion by saying "yes" or "suggested", or provide your own answer.`
|
|
||||||
- After the user answers:
|
|
||||||
- If the user replies with "yes", "recommended", or "suggested", use your previously stated recommendation/suggestion as the answer.
|
|
||||||
- Otherwise, validate the answer maps to one option or fits the <=5 word constraint.
|
|
||||||
- If ambiguous, ask for a quick disambiguation (count still belongs to same question; do not advance).
|
|
||||||
- Once satisfactory, record it in working memory (do not yet write to disk) and move to the next queued question.
|
|
||||||
- Stop asking further questions when:
|
|
||||||
- All critical ambiguities resolved early (remaining queued items become unnecessary), OR
|
|
||||||
- User signals completion ("done", "good", "no more"), OR
|
|
||||||
- You reach 5 asked questions.
|
|
||||||
- Never reveal future queued questions in advance.
|
|
||||||
- If no valid questions exist at start, immediately report no critical ambiguities.
|
|
||||||
|
|
||||||
5. Integration after EACH accepted answer (incremental update approach):
|
|
||||||
- Maintain in-memory representation of the spec (loaded once at start) plus the raw file contents.
|
|
||||||
- For the first integrated answer in this session:
|
|
||||||
- Ensure a `## Clarifications` section exists (create it just after the highest-level contextual/overview section per the spec template if missing).
|
|
||||||
- Under it, create (if not present) a `### Session YYYY-MM-DD` subheading for today.
|
|
||||||
- Append a bullet line immediately after acceptance: `- Q: <question> → A: <final answer>`.
|
|
||||||
- Then immediately apply the clarification to the most appropriate section(s):
|
|
||||||
- Functional ambiguity → Update or add a bullet in Functional Requirements.
|
|
||||||
- User interaction / actor distinction → Update User Stories or Actors subsection (if present) with clarified role, constraint, or scenario.
|
|
||||||
- Data shape / entities → Update Data Model (add fields, types, relationships) preserving ordering; note added constraints succinctly.
|
|
||||||
- Non-functional constraint → Add/modify measurable criteria in Non-Functional / Quality Attributes section (convert vague adjective to metric or explicit target).
|
|
||||||
- Edge case / negative flow → Add a new bullet under Edge Cases / Error Handling (or create such subsection if template provides placeholder for it).
|
|
||||||
- Terminology conflict → Normalize term across spec; retain original only if necessary by adding `(formerly referred to as "X")` once.
|
|
||||||
- If the clarification invalidates an earlier ambiguous statement, replace that statement instead of duplicating; leave no obsolete contradictory text.
|
|
||||||
- Save the spec file AFTER each integration to minimize risk of context loss (atomic overwrite).
|
|
||||||
- Preserve formatting: do not reorder unrelated sections; keep heading hierarchy intact.
|
|
||||||
- Keep each inserted clarification minimal and testable (avoid narrative drift).
|
|
||||||
|
|
||||||
6. Validation (performed after EACH write plus final pass):
|
|
||||||
- Clarifications session contains exactly one bullet per accepted answer (no duplicates).
|
|
||||||
- Total asked (accepted) questions ≤ 5.
|
|
||||||
- Updated sections contain no lingering vague placeholders the new answer was meant to resolve.
|
|
||||||
- No contradictory earlier statement remains (scan for now-invalid alternative choices removed).
|
|
||||||
- Markdown structure valid; only allowed new headings: `## Clarifications`, `### Session YYYY-MM-DD`.
|
|
||||||
- Terminology consistency: same canonical term used across all updated sections.
|
|
||||||
|
|
||||||
7. Write the updated spec back to `FEATURE_SPEC`.
|
|
||||||
|
|
||||||
8. Report completion (after questioning loop ends or early termination):
|
|
||||||
- Number of questions asked & answered.
|
|
||||||
- Path to updated spec.
|
|
||||||
- Sections touched (list names).
|
|
||||||
- Coverage summary table listing each taxonomy category with Status: Resolved (was Partial/Missing and addressed), Deferred (exceeds question quota or better suited for planning), Clear (already sufficient), Outstanding (still Partial/Missing but low impact).
|
|
||||||
- If any Outstanding or Deferred remain, recommend whether to proceed to `/speckit.plan` or run `/speckit.clarify` again later post-plan.
|
|
||||||
- Suggested next command.
|
|
||||||
|
|
||||||
Behavior rules:
|
|
||||||
|
|
||||||
- If no meaningful ambiguities found (or all potential questions would be low-impact), respond: "No critical ambiguities detected worth formal clarification." and suggest proceeding.
|
|
||||||
- If spec file missing, instruct user to run `/speckit.specify` first (do not create a new spec here).
|
|
||||||
- Never exceed 5 total asked questions (clarification retries for a single question do not count as new questions).
|
|
||||||
- Avoid speculative tech stack questions unless the absence blocks functional clarity.
|
|
||||||
- Respect user early termination signals ("stop", "done", "proceed").
|
|
||||||
- If no questions asked due to full coverage, output a compact coverage summary (all categories Clear) then suggest advancing.
|
|
||||||
- If quota reached with unresolved high-impact categories remaining, explicitly flag them under Deferred with rationale.
|
|
||||||
|
|
||||||
Context for prioritization: $ARGUMENTS
|
|
||||||
@@ -1,84 +0,0 @@
|
|||||||
---
|
|
||||||
description: Create or update the project constitution from interactive or provided principle inputs, ensuring all dependent templates stay in sync.
|
|
||||||
handoffs:
|
|
||||||
- label: Build Specification
|
|
||||||
agent: speckit.specify
|
|
||||||
prompt: Implement the feature specification based on the updated constitution. I want to build...
|
|
||||||
---
|
|
||||||
|
|
||||||
## User Input
|
|
||||||
|
|
||||||
```text
|
|
||||||
$ARGUMENTS
|
|
||||||
```
|
|
||||||
|
|
||||||
You **MUST** consider the user input before proceeding (if not empty).
|
|
||||||
|
|
||||||
## Outline
|
|
||||||
|
|
||||||
You are updating the project constitution at `.specify/memory/constitution.md`. This file is a TEMPLATE containing placeholder tokens in square brackets (e.g. `[PROJECT_NAME]`, `[PRINCIPLE_1_NAME]`). Your job is to (a) collect/derive concrete values, (b) fill the template precisely, and (c) propagate any amendments across dependent artifacts.
|
|
||||||
|
|
||||||
**Note**: If `.specify/memory/constitution.md` does not exist yet, it should have been initialized from `.specify/templates/constitution-template.md` during project setup. If it's missing, copy the template first.
|
|
||||||
|
|
||||||
Follow this execution flow:
|
|
||||||
|
|
||||||
1. Load the existing constitution at `.specify/memory/constitution.md`.
|
|
||||||
- Identify every placeholder token of the form `[ALL_CAPS_IDENTIFIER]`.
|
|
||||||
**IMPORTANT**: The user might require less or more principles than the ones used in the template. If a number is specified, respect that - follow the general template. You will update the doc accordingly.
|
|
||||||
|
|
||||||
2. Collect/derive values for placeholders:
|
|
||||||
- If user input (conversation) supplies a value, use it.
|
|
||||||
- Otherwise infer from existing repo context (README, docs, prior constitution versions if embedded).
|
|
||||||
- For governance dates: `RATIFICATION_DATE` is the original adoption date (if unknown ask or mark TODO), `LAST_AMENDED_DATE` is today if changes are made, otherwise keep previous.
|
|
||||||
- `CONSTITUTION_VERSION` must increment according to semantic versioning rules:
|
|
||||||
- MAJOR: Backward incompatible governance/principle removals or redefinitions.
|
|
||||||
- MINOR: New principle/section added or materially expanded guidance.
|
|
||||||
- PATCH: Clarifications, wording, typo fixes, non-semantic refinements.
|
|
||||||
- If version bump type ambiguous, propose reasoning before finalizing.
|
|
||||||
|
|
||||||
3. Draft the updated constitution content:
|
|
||||||
- Replace every placeholder with concrete text (no bracketed tokens left except intentionally retained template slots that the project has chosen not to define yet—explicitly justify any left).
|
|
||||||
- Preserve heading hierarchy and comments can be removed once replaced unless they still add clarifying guidance.
|
|
||||||
- Ensure each Principle section: succinct name line, paragraph (or bullet list) capturing non‑negotiable rules, explicit rationale if not obvious.
|
|
||||||
- Ensure Governance section lists amendment procedure, versioning policy, and compliance review expectations.
|
|
||||||
|
|
||||||
4. Consistency propagation checklist (convert prior checklist into active validations):
|
|
||||||
- Read `.specify/templates/plan-template.md` and ensure any "Constitution Check" or rules align with updated principles.
|
|
||||||
- Read `.specify/templates/spec-template.md` for scope/requirements alignment—update if constitution adds/removes mandatory sections or constraints.
|
|
||||||
- Read `.specify/templates/tasks-template.md` and ensure task categorization reflects new or removed principle-driven task types (e.g., observability, versioning, testing discipline).
|
|
||||||
- Read each command file in `.specify/templates/commands/*.md` (including this one) to verify no outdated references (agent-specific names like CLAUDE only) remain when generic guidance is required.
|
|
||||||
- Read any runtime guidance docs (e.g., `README.md`, `docs/quickstart.md`, or agent-specific guidance files if present). Update references to principles changed.
|
|
||||||
|
|
||||||
5. Produce a Sync Impact Report (prepend as an HTML comment at top of the constitution file after update):
|
|
||||||
- Version change: old → new
|
|
||||||
- List of modified principles (old title → new title if renamed)
|
|
||||||
- Added sections
|
|
||||||
- Removed sections
|
|
||||||
- Templates requiring updates (✅ updated / ⚠ pending) with file paths
|
|
||||||
- Follow-up TODOs if any placeholders intentionally deferred.
|
|
||||||
|
|
||||||
6. Validation before final output:
|
|
||||||
- No remaining unexplained bracket tokens.
|
|
||||||
- Version line matches report.
|
|
||||||
- Dates ISO format YYYY-MM-DD.
|
|
||||||
- Principles are declarative, testable, and free of vague language ("should" → replace with MUST/SHOULD rationale where appropriate).
|
|
||||||
|
|
||||||
7. Write the completed constitution back to `.specify/memory/constitution.md` (overwrite).
|
|
||||||
|
|
||||||
8. Output a final summary to the user with:
|
|
||||||
- New version and bump rationale.
|
|
||||||
- Any files flagged for manual follow-up.
|
|
||||||
- Suggested commit message (e.g., `docs: amend constitution to vX.Y.Z (principle additions + governance update)`).
|
|
||||||
|
|
||||||
Formatting & Style Requirements:
|
|
||||||
|
|
||||||
- Use Markdown headings exactly as in the template (do not demote/promote levels).
|
|
||||||
- Wrap long rationale lines to keep readability (<100 chars ideally) but do not hard enforce with awkward breaks.
|
|
||||||
- Keep a single blank line between sections.
|
|
||||||
- Avoid trailing whitespace.
|
|
||||||
|
|
||||||
If the user supplies partial updates (e.g., only one principle revision), still perform validation and version decision steps.
|
|
||||||
|
|
||||||
If critical info missing (e.g., ratification date truly unknown), insert `TODO(<FIELD_NAME>): explanation` and include in the Sync Impact Report under deferred items.
|
|
||||||
|
|
||||||
Do not create a new template; always operate on the existing `.specify/memory/constitution.md` file.
|
|
||||||
@@ -1,135 +0,0 @@
|
|||||||
---
|
|
||||||
description: Execute the implementation plan by processing and executing all tasks defined in tasks.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## User Input
|
|
||||||
|
|
||||||
```text
|
|
||||||
$ARGUMENTS
|
|
||||||
```
|
|
||||||
|
|
||||||
You **MUST** consider the user input before proceeding (if not empty).
|
|
||||||
|
|
||||||
## Outline
|
|
||||||
|
|
||||||
1. Run `.specify/scripts/powershell/check-prerequisites.ps1 -Json -RequireTasks -IncludeTasks` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
|
||||||
|
|
||||||
2. **Check checklists status** (if FEATURE_DIR/checklists/ exists):
|
|
||||||
- Scan all checklist files in the checklists/ directory
|
|
||||||
- For each checklist, count:
|
|
||||||
- Total items: All lines matching `- [ ]` or `- [X]` or `- [x]`
|
|
||||||
- Completed items: Lines matching `- [X]` or `- [x]`
|
|
||||||
- Incomplete items: Lines matching `- [ ]`
|
|
||||||
- Create a status table:
|
|
||||||
|
|
||||||
```text
|
|
||||||
| Checklist | Total | Completed | Incomplete | Status |
|
|
||||||
|-----------|-------|-----------|------------|--------|
|
|
||||||
| ux.md | 12 | 12 | 0 | ✓ PASS |
|
|
||||||
| test.md | 8 | 5 | 3 | ✗ FAIL |
|
|
||||||
| security.md | 6 | 6 | 0 | ✓ PASS |
|
|
||||||
```
|
|
||||||
|
|
||||||
- Calculate overall status:
|
|
||||||
- **PASS**: All checklists have 0 incomplete items
|
|
||||||
- **FAIL**: One or more checklists have incomplete items
|
|
||||||
|
|
||||||
- **If any checklist is incomplete**:
|
|
||||||
- Display the table with incomplete item counts
|
|
||||||
- **STOP** and ask: "Some checklists are incomplete. Do you want to proceed with implementation anyway? (yes/no)"
|
|
||||||
- Wait for user response before continuing
|
|
||||||
- If user says "no" or "wait" or "stop", halt execution
|
|
||||||
- If user says "yes" or "proceed" or "continue", proceed to step 3
|
|
||||||
|
|
||||||
- **If all checklists are complete**:
|
|
||||||
- Display the table showing all checklists passed
|
|
||||||
- Automatically proceed to step 3
|
|
||||||
|
|
||||||
3. Load and analyze the implementation context:
|
|
||||||
- **REQUIRED**: Read tasks.md for the complete task list and execution plan
|
|
||||||
- **REQUIRED**: Read plan.md for tech stack, architecture, and file structure
|
|
||||||
- **IF EXISTS**: Read data-model.md for entities and relationships
|
|
||||||
- **IF EXISTS**: Read contracts/ for API specifications and test requirements
|
|
||||||
- **IF EXISTS**: Read research.md for technical decisions and constraints
|
|
||||||
- **IF EXISTS**: Read quickstart.md for integration scenarios
|
|
||||||
|
|
||||||
4. **Project Setup Verification**:
|
|
||||||
- **REQUIRED**: Create/verify ignore files based on actual project setup:
|
|
||||||
|
|
||||||
**Detection & Creation Logic**:
|
|
||||||
- Check if the following command succeeds to determine if the repository is a git repo (create/verify .gitignore if so):
|
|
||||||
|
|
||||||
```sh
|
|
||||||
git rev-parse --git-dir 2>/dev/null
|
|
||||||
```
|
|
||||||
|
|
||||||
- Check if Dockerfile* exists or Docker in plan.md → create/verify .dockerignore
|
|
||||||
- Check if .eslintrc* exists → create/verify .eslintignore
|
|
||||||
- Check if eslint.config.* exists → ensure the config's `ignores` entries cover required patterns
|
|
||||||
- Check if .prettierrc* exists → create/verify .prettierignore
|
|
||||||
- Check if .npmrc or package.json exists → create/verify .npmignore (if publishing)
|
|
||||||
- Check if terraform files (*.tf) exist → create/verify .terraformignore
|
|
||||||
- Check if .helmignore needed (helm charts present) → create/verify .helmignore
|
|
||||||
|
|
||||||
**If ignore file already exists**: Verify it contains essential patterns, append missing critical patterns only
|
|
||||||
**If ignore file missing**: Create with full pattern set for detected technology
|
|
||||||
|
|
||||||
**Common Patterns by Technology** (from plan.md tech stack):
|
|
||||||
- **Node.js/JavaScript/TypeScript**: `node_modules/`, `dist/`, `build/`, `*.log`, `.env*`
|
|
||||||
- **Python**: `__pycache__/`, `*.pyc`, `.venv/`, `venv/`, `dist/`, `*.egg-info/`
|
|
||||||
- **Java**: `target/`, `*.class`, `*.jar`, `.gradle/`, `build/`
|
|
||||||
- **C#/.NET**: `bin/`, `obj/`, `*.user`, `*.suo`, `packages/`
|
|
||||||
- **Go**: `*.exe`, `*.test`, `vendor/`, `*.out`
|
|
||||||
- **Ruby**: `.bundle/`, `log/`, `tmp/`, `*.gem`, `vendor/bundle/`
|
|
||||||
- **PHP**: `vendor/`, `*.log`, `*.cache`, `*.env`
|
|
||||||
- **Rust**: `target/`, `debug/`, `release/`, `*.rs.bk`, `*.rlib`, `*.prof*`, `.idea/`, `*.log`, `.env*`
|
|
||||||
- **Kotlin**: `build/`, `out/`, `.gradle/`, `.idea/`, `*.class`, `*.jar`, `*.iml`, `*.log`, `.env*`
|
|
||||||
- **C++**: `build/`, `bin/`, `obj/`, `out/`, `*.o`, `*.so`, `*.a`, `*.exe`, `*.dll`, `.idea/`, `*.log`, `.env*`
|
|
||||||
- **C**: `build/`, `bin/`, `obj/`, `out/`, `*.o`, `*.a`, `*.so`, `*.exe`, `Makefile`, `config.log`, `.idea/`, `*.log`, `.env*`
|
|
||||||
- **Swift**: `.build/`, `DerivedData/`, `*.swiftpm/`, `Packages/`
|
|
||||||
- **R**: `.Rproj.user/`, `.Rhistory`, `.RData`, `.Ruserdata`, `*.Rproj`, `packrat/`, `renv/`
|
|
||||||
- **Universal**: `.DS_Store`, `Thumbs.db`, `*.tmp`, `*.swp`, `.vscode/`, `.idea/`
|
|
||||||
|
|
||||||
**Tool-Specific Patterns**:
|
|
||||||
- **Docker**: `node_modules/`, `.git/`, `Dockerfile*`, `.dockerignore`, `*.log*`, `.env*`, `coverage/`
|
|
||||||
- **ESLint**: `node_modules/`, `dist/`, `build/`, `coverage/`, `*.min.js`
|
|
||||||
- **Prettier**: `node_modules/`, `dist/`, `build/`, `coverage/`, `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml`
|
|
||||||
- **Terraform**: `.terraform/`, `*.tfstate*`, `*.tfvars`, `.terraform.lock.hcl`
|
|
||||||
- **Kubernetes/k8s**: `*.secret.yaml`, `secrets/`, `.kube/`, `kubeconfig*`, `*.key`, `*.crt`
|
|
||||||
|
|
||||||
5. Parse tasks.md structure and extract:
|
|
||||||
- **Task phases**: Setup, Tests, Core, Integration, Polish
|
|
||||||
- **Task dependencies**: Sequential vs parallel execution rules
|
|
||||||
- **Task details**: ID, description, file paths, parallel markers [P]
|
|
||||||
- **Execution flow**: Order and dependency requirements
|
|
||||||
|
|
||||||
6. Execute implementation following the task plan:
|
|
||||||
- **Phase-by-phase execution**: Complete each phase before moving to the next
|
|
||||||
- **Respect dependencies**: Run sequential tasks in order, parallel tasks [P] can run together
|
|
||||||
- **Follow TDD approach**: Execute test tasks before their corresponding implementation tasks
|
|
||||||
- **File-based coordination**: Tasks affecting the same files must run sequentially
|
|
||||||
- **Validation checkpoints**: Verify each phase completion before proceeding
|
|
||||||
|
|
||||||
7. Implementation execution rules:
|
|
||||||
- **Setup first**: Initialize project structure, dependencies, configuration
|
|
||||||
- **Tests before code**: If you need to write tests for contracts, entities, and integration scenarios
|
|
||||||
- **Core development**: Implement models, services, CLI commands, endpoints
|
|
||||||
- **Integration work**: Database connections, middleware, logging, external services
|
|
||||||
- **Polish and validation**: Unit tests, performance optimization, documentation
|
|
||||||
|
|
||||||
8. Progress tracking and error handling:
|
|
||||||
- Report progress after each completed task
|
|
||||||
- Halt execution if any non-parallel task fails
|
|
||||||
- For parallel tasks [P], continue with successful tasks, report failed ones
|
|
||||||
- Provide clear error messages with context for debugging
|
|
||||||
- Suggest next steps if implementation cannot proceed
|
|
||||||
- **IMPORTANT** For completed tasks, make sure to mark the task off as [X] in the tasks file.
|
|
||||||
|
|
||||||
9. Completion validation:
|
|
||||||
- Verify all required tasks are completed
|
|
||||||
- Check that implemented features match the original specification
|
|
||||||
- Validate that tests pass and coverage meets requirements
|
|
||||||
- Confirm the implementation follows the technical plan
|
|
||||||
- Report final status with summary of completed work
|
|
||||||
|
|
||||||
Note: This command assumes a complete task breakdown exists in tasks.md. If tasks are incomplete or missing, suggest running `/speckit.tasks` first to regenerate the task list.
|
|
||||||
@@ -1,90 +0,0 @@
|
|||||||
---
|
|
||||||
description: Execute the implementation planning workflow using the plan template to generate design artifacts.
|
|
||||||
handoffs:
|
|
||||||
- label: Create Tasks
|
|
||||||
agent: speckit.tasks
|
|
||||||
prompt: Break the plan into tasks
|
|
||||||
send: true
|
|
||||||
- label: Create Checklist
|
|
||||||
agent: speckit.checklist
|
|
||||||
prompt: Create a checklist for the following domain...
|
|
||||||
---
|
|
||||||
|
|
||||||
## User Input
|
|
||||||
|
|
||||||
```text
|
|
||||||
$ARGUMENTS
|
|
||||||
```
|
|
||||||
|
|
||||||
You **MUST** consider the user input before proceeding (if not empty).
|
|
||||||
|
|
||||||
## Outline
|
|
||||||
|
|
||||||
1. **Setup**: Run `.specify/scripts/powershell/setup-plan.ps1 -Json` from repo root and parse JSON for FEATURE_SPEC, IMPL_PLAN, SPECS_DIR, BRANCH. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
|
||||||
|
|
||||||
2. **Load context**: Read FEATURE_SPEC and `.specify/memory/constitution.md`. Load IMPL_PLAN template (already copied).
|
|
||||||
|
|
||||||
3. **Execute plan workflow**: Follow the structure in IMPL_PLAN template to:
|
|
||||||
- Fill Technical Context (mark unknowns as "NEEDS CLARIFICATION")
|
|
||||||
- Fill Constitution Check section from constitution
|
|
||||||
- Evaluate gates (ERROR if violations unjustified)
|
|
||||||
- Phase 0: Generate research.md (resolve all NEEDS CLARIFICATION)
|
|
||||||
- Phase 1: Generate data-model.md, contracts/, quickstart.md
|
|
||||||
- Phase 1: Update agent context by running the agent script
|
|
||||||
- Re-evaluate Constitution Check post-design
|
|
||||||
|
|
||||||
4. **Stop and report**: Command ends after Phase 2 planning. Report branch, IMPL_PLAN path, and generated artifacts.
|
|
||||||
|
|
||||||
## Phases
|
|
||||||
|
|
||||||
### Phase 0: Outline & Research
|
|
||||||
|
|
||||||
1. **Extract unknowns from Technical Context** above:
|
|
||||||
- For each NEEDS CLARIFICATION → research task
|
|
||||||
- For each dependency → best practices task
|
|
||||||
- For each integration → patterns task
|
|
||||||
|
|
||||||
2. **Generate and dispatch research agents**:
|
|
||||||
|
|
||||||
```text
|
|
||||||
For each unknown in Technical Context:
|
|
||||||
Task: "Research {unknown} for {feature context}"
|
|
||||||
For each technology choice:
|
|
||||||
Task: "Find best practices for {tech} in {domain}"
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Consolidate findings** in `research.md` using format:
|
|
||||||
- Decision: [what was chosen]
|
|
||||||
- Rationale: [why chosen]
|
|
||||||
- Alternatives considered: [what else evaluated]
|
|
||||||
|
|
||||||
**Output**: research.md with all NEEDS CLARIFICATION resolved
|
|
||||||
|
|
||||||
### Phase 1: Design & Contracts
|
|
||||||
|
|
||||||
**Prerequisites:** `research.md` complete
|
|
||||||
|
|
||||||
1. **Extract entities from feature spec** → `data-model.md`:
|
|
||||||
- Entity name, fields, relationships
|
|
||||||
- Validation rules from requirements
|
|
||||||
- State transitions if applicable
|
|
||||||
|
|
||||||
2. **Define interface contracts** (if project has external interfaces) → `/contracts/`:
|
|
||||||
- Identify what interfaces the project exposes to users or other systems
|
|
||||||
- Document the contract format appropriate for the project type
|
|
||||||
- Examples: public APIs for libraries, command schemas for CLI tools, endpoints for web services, grammars for parsers, UI contracts for applications
|
|
||||||
- Skip if project is purely internal (build scripts, one-off tools, etc.)
|
|
||||||
|
|
||||||
3. **Agent context update**:
|
|
||||||
- Run `.specify/scripts/powershell/update-agent-context.ps1 -AgentType claude`
|
|
||||||
- These scripts detect which AI agent is in use
|
|
||||||
- Update the appropriate agent-specific context file
|
|
||||||
- Add only new technology from current plan
|
|
||||||
- Preserve manual additions between markers
|
|
||||||
|
|
||||||
**Output**: data-model.md, /contracts/*, quickstart.md, agent-specific file
|
|
||||||
|
|
||||||
## Key rules
|
|
||||||
|
|
||||||
- Use absolute paths
|
|
||||||
- ERROR on gate failures or unresolved clarifications
|
|
||||||
@@ -1,258 +0,0 @@
|
|||||||
---
|
|
||||||
description: Create or update the feature specification from a natural language feature description.
|
|
||||||
handoffs:
|
|
||||||
- label: Build Technical Plan
|
|
||||||
agent: speckit.plan
|
|
||||||
prompt: Create a plan for the spec. I am building with...
|
|
||||||
- label: Clarify Spec Requirements
|
|
||||||
agent: speckit.clarify
|
|
||||||
prompt: Clarify specification requirements
|
|
||||||
send: true
|
|
||||||
---
|
|
||||||
|
|
||||||
## User Input
|
|
||||||
|
|
||||||
```text
|
|
||||||
$ARGUMENTS
|
|
||||||
```
|
|
||||||
|
|
||||||
You **MUST** consider the user input before proceeding (if not empty).
|
|
||||||
|
|
||||||
## Outline
|
|
||||||
|
|
||||||
The text the user typed after `/speckit.specify` in the triggering message **is** the feature description. Assume you always have it available in this conversation even if `$ARGUMENTS` appears literally below. Do not ask the user to repeat it unless they provided an empty command.
|
|
||||||
|
|
||||||
Given that feature description, do this:
|
|
||||||
|
|
||||||
1. **Generate a concise short name** (2-4 words) for the branch:
|
|
||||||
- Analyze the feature description and extract the most meaningful keywords
|
|
||||||
- Create a 2-4 word short name that captures the essence of the feature
|
|
||||||
- Use action-noun format when possible (e.g., "add-user-auth", "fix-payment-bug")
|
|
||||||
- Preserve technical terms and acronyms (OAuth2, API, JWT, etc.)
|
|
||||||
- Keep it concise but descriptive enough to understand the feature at a glance
|
|
||||||
- Examples:
|
|
||||||
- "I want to add user authentication" → "user-auth"
|
|
||||||
- "Implement OAuth2 integration for the API" → "oauth2-api-integration"
|
|
||||||
- "Create a dashboard for analytics" → "analytics-dashboard"
|
|
||||||
- "Fix payment processing timeout bug" → "fix-payment-timeout"
|
|
||||||
|
|
||||||
2. **Check for existing branches before creating new one**:
|
|
||||||
|
|
||||||
a. First, fetch all remote branches to ensure we have the latest information:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git fetch --all --prune
|
|
||||||
```
|
|
||||||
|
|
||||||
b. Find the highest feature number across all sources for the short-name:
|
|
||||||
- Remote branches: `git ls-remote --heads origin | grep -E 'refs/heads/[0-9]+-<short-name>$'`
|
|
||||||
- Local branches: `git branch | grep -E '^[* ]*[0-9]+-<short-name>$'`
|
|
||||||
- Specs directories: Check for directories matching `specs/[0-9]+-<short-name>`
|
|
||||||
|
|
||||||
c. Determine the next available number:
|
|
||||||
- Extract all numbers from all three sources
|
|
||||||
- Find the highest number N
|
|
||||||
- Use N+1 for the new branch number
|
|
||||||
|
|
||||||
d. Run the script `.specify/scripts/powershell/create-new-feature.ps1 -Json "$ARGUMENTS"` with the calculated number and short-name:
|
|
||||||
- Pass `--number N+1` and `--short-name "your-short-name"` along with the feature description
|
|
||||||
- Bash example: `.specify/scripts/powershell/create-new-feature.ps1 -Json "$ARGUMENTS" --json --number 5 --short-name "user-auth" "Add user authentication"`
|
|
||||||
- PowerShell example: `.specify/scripts/powershell/create-new-feature.ps1 -Json "$ARGUMENTS" -Json -Number 5 -ShortName "user-auth" "Add user authentication"`
|
|
||||||
|
|
||||||
**IMPORTANT**:
|
|
||||||
- Check all three sources (remote branches, local branches, specs directories) to find the highest number
|
|
||||||
- Only match branches/directories with the exact short-name pattern
|
|
||||||
- If no existing branches/directories found with this short-name, start with number 1
|
|
||||||
- You must only ever run this script once per feature
|
|
||||||
- The JSON is provided in the terminal as output - always refer to it to get the actual content you're looking for
|
|
||||||
- The JSON output will contain BRANCH_NAME and SPEC_FILE paths
|
|
||||||
- For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot")
|
|
||||||
|
|
||||||
3. Load `.specify/templates/spec-template.md` to understand required sections.
|
|
||||||
|
|
||||||
4. Follow this execution flow:
|
|
||||||
|
|
||||||
1. Parse user description from Input
|
|
||||||
If empty: ERROR "No feature description provided"
|
|
||||||
2. Extract key concepts from description
|
|
||||||
Identify: actors, actions, data, constraints
|
|
||||||
3. For unclear aspects:
|
|
||||||
- Make informed guesses based on context and industry standards
|
|
||||||
- Only mark with [NEEDS CLARIFICATION: specific question] if:
|
|
||||||
- The choice significantly impacts feature scope or user experience
|
|
||||||
- Multiple reasonable interpretations exist with different implications
|
|
||||||
- No reasonable default exists
|
|
||||||
- **LIMIT: Maximum 3 [NEEDS CLARIFICATION] markers total**
|
|
||||||
- Prioritize clarifications by impact: scope > security/privacy > user experience > technical details
|
|
||||||
4. Fill User Scenarios & Testing section
|
|
||||||
If no clear user flow: ERROR "Cannot determine user scenarios"
|
|
||||||
5. Generate Functional Requirements
|
|
||||||
Each requirement must be testable
|
|
||||||
Use reasonable defaults for unspecified details (document assumptions in Assumptions section)
|
|
||||||
6. Define Success Criteria
|
|
||||||
Create measurable, technology-agnostic outcomes
|
|
||||||
Include both quantitative metrics (time, performance, volume) and qualitative measures (user satisfaction, task completion)
|
|
||||||
Each criterion must be verifiable without implementation details
|
|
||||||
7. Identify Key Entities (if data involved)
|
|
||||||
8. Return: SUCCESS (spec ready for planning)
|
|
||||||
|
|
||||||
5. Write the specification to SPEC_FILE using the template structure, replacing placeholders with concrete details derived from the feature description (arguments) while preserving section order and headings.
|
|
||||||
|
|
||||||
6. **Specification Quality Validation**: After writing the initial spec, validate it against quality criteria:
|
|
||||||
|
|
||||||
a. **Create Spec Quality Checklist**: Generate a checklist file at `FEATURE_DIR/checklists/requirements.md` using the checklist template structure with these validation items:
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
# Specification Quality Checklist: [FEATURE NAME]
|
|
||||||
|
|
||||||
**Purpose**: Validate specification completeness and quality before proceeding to planning
|
|
||||||
**Created**: [DATE]
|
|
||||||
**Feature**: [Link to spec.md]
|
|
||||||
|
|
||||||
## Content Quality
|
|
||||||
|
|
||||||
- [ ] No implementation details (languages, frameworks, APIs)
|
|
||||||
- [ ] Focused on user value and business needs
|
|
||||||
- [ ] Written for non-technical stakeholders
|
|
||||||
- [ ] All mandatory sections completed
|
|
||||||
|
|
||||||
## Requirement Completeness
|
|
||||||
|
|
||||||
- [ ] No [NEEDS CLARIFICATION] markers remain
|
|
||||||
- [ ] Requirements are testable and unambiguous
|
|
||||||
- [ ] Success criteria are measurable
|
|
||||||
- [ ] Success criteria are technology-agnostic (no implementation details)
|
|
||||||
- [ ] All acceptance scenarios are defined
|
|
||||||
- [ ] Edge cases are identified
|
|
||||||
- [ ] Scope is clearly bounded
|
|
||||||
- [ ] Dependencies and assumptions identified
|
|
||||||
|
|
||||||
## Feature Readiness
|
|
||||||
|
|
||||||
- [ ] All functional requirements have clear acceptance criteria
|
|
||||||
- [ ] User scenarios cover primary flows
|
|
||||||
- [ ] Feature meets measurable outcomes defined in Success Criteria
|
|
||||||
- [ ] No implementation details leak into specification
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- Items marked incomplete require spec updates before `/speckit.clarify` or `/speckit.plan`
|
|
||||||
```
|
|
||||||
|
|
||||||
b. **Run Validation Check**: Review the spec against each checklist item:
|
|
||||||
- For each item, determine if it passes or fails
|
|
||||||
- Document specific issues found (quote relevant spec sections)
|
|
||||||
|
|
||||||
c. **Handle Validation Results**:
|
|
||||||
|
|
||||||
- **If all items pass**: Mark checklist complete and proceed to step 6
|
|
||||||
|
|
||||||
- **If items fail (excluding [NEEDS CLARIFICATION])**:
|
|
||||||
1. List the failing items and specific issues
|
|
||||||
2. Update the spec to address each issue
|
|
||||||
3. Re-run validation until all items pass (max 3 iterations)
|
|
||||||
4. If still failing after 3 iterations, document remaining issues in checklist notes and warn user
|
|
||||||
|
|
||||||
- **If [NEEDS CLARIFICATION] markers remain**:
|
|
||||||
1. Extract all [NEEDS CLARIFICATION: ...] markers from the spec
|
|
||||||
2. **LIMIT CHECK**: If more than 3 markers exist, keep only the 3 most critical (by scope/security/UX impact) and make informed guesses for the rest
|
|
||||||
3. For each clarification needed (max 3), present options to user in this format:
|
|
||||||
|
|
||||||
```markdown
|
|
||||||
## Question [N]: [Topic]
|
|
||||||
|
|
||||||
**Context**: [Quote relevant spec section]
|
|
||||||
|
|
||||||
**What we need to know**: [Specific question from NEEDS CLARIFICATION marker]
|
|
||||||
|
|
||||||
**Suggested Answers**:
|
|
||||||
|
|
||||||
| Option | Answer | Implications |
|
|
||||||
|--------|--------|--------------|
|
|
||||||
| A | [First suggested answer] | [What this means for the feature] |
|
|
||||||
| B | [Second suggested answer] | [What this means for the feature] |
|
|
||||||
| C | [Third suggested answer] | [What this means for the feature] |
|
|
||||||
| Custom | Provide your own answer | [Explain how to provide custom input] |
|
|
||||||
|
|
||||||
**Your choice**: _[Wait for user response]_
|
|
||||||
```
|
|
||||||
|
|
||||||
4. **CRITICAL - Table Formatting**: Ensure markdown tables are properly formatted:
|
|
||||||
- Use consistent spacing with pipes aligned
|
|
||||||
- Each cell should have spaces around content: `| Content |` not `|Content|`
|
|
||||||
- Header separator must have at least 3 dashes: `|--------|`
|
|
||||||
- Test that the table renders correctly in markdown preview
|
|
||||||
5. Number questions sequentially (Q1, Q2, Q3 - max 3 total)
|
|
||||||
6. Present all questions together before waiting for responses
|
|
||||||
7. Wait for user to respond with their choices for all questions (e.g., "Q1: A, Q2: Custom - [details], Q3: B")
|
|
||||||
8. Update the spec by replacing each [NEEDS CLARIFICATION] marker with the user's selected or provided answer
|
|
||||||
9. Re-run validation after all clarifications are resolved
|
|
||||||
|
|
||||||
d. **Update Checklist**: After each validation iteration, update the checklist file with current pass/fail status
|
|
||||||
|
|
||||||
7. Report completion with branch name, spec file path, checklist results, and readiness for the next phase (`/speckit.clarify` or `/speckit.plan`).
|
|
||||||
|
|
||||||
**NOTE:** The script creates and checks out the new branch and initializes the spec file before writing.
|
|
||||||
|
|
||||||
## General Guidelines
|
|
||||||
|
|
||||||
## Quick Guidelines
|
|
||||||
|
|
||||||
- Focus on **WHAT** users need and **WHY**.
|
|
||||||
- Avoid HOW to implement (no tech stack, APIs, code structure).
|
|
||||||
- Written for business stakeholders, not developers.
|
|
||||||
- DO NOT create any checklists that are embedded in the spec. That will be a separate command.
|
|
||||||
|
|
||||||
### Section Requirements
|
|
||||||
|
|
||||||
- **Mandatory sections**: Must be completed for every feature
|
|
||||||
- **Optional sections**: Include only when relevant to the feature
|
|
||||||
- When a section doesn't apply, remove it entirely (don't leave as "N/A")
|
|
||||||
|
|
||||||
### For AI Generation
|
|
||||||
|
|
||||||
When creating this spec from a user prompt:
|
|
||||||
|
|
||||||
1. **Make informed guesses**: Use context, industry standards, and common patterns to fill gaps
|
|
||||||
2. **Document assumptions**: Record reasonable defaults in the Assumptions section
|
|
||||||
3. **Limit clarifications**: Maximum 3 [NEEDS CLARIFICATION] markers - use only for critical decisions that:
|
|
||||||
- Significantly impact feature scope or user experience
|
|
||||||
- Have multiple reasonable interpretations with different implications
|
|
||||||
- Lack any reasonable default
|
|
||||||
4. **Prioritize clarifications**: scope > security/privacy > user experience > technical details
|
|
||||||
5. **Think like a tester**: Every vague requirement should fail the "testable and unambiguous" checklist item
|
|
||||||
6. **Common areas needing clarification** (only if no reasonable default exists):
|
|
||||||
- Feature scope and boundaries (include/exclude specific use cases)
|
|
||||||
- User types and permissions (if multiple conflicting interpretations possible)
|
|
||||||
- Security/compliance requirements (when legally/financially significant)
|
|
||||||
|
|
||||||
**Examples of reasonable defaults** (don't ask about these):
|
|
||||||
|
|
||||||
- Data retention: Industry-standard practices for the domain
|
|
||||||
- Performance targets: Standard web/mobile app expectations unless specified
|
|
||||||
- Error handling: User-friendly messages with appropriate fallbacks
|
|
||||||
- Authentication method: Standard session-based or OAuth2 for web apps
|
|
||||||
- Integration patterns: Use project-appropriate patterns (REST/GraphQL for web services, function calls for libraries, CLI args for tools, etc.)
|
|
||||||
|
|
||||||
### Success Criteria Guidelines
|
|
||||||
|
|
||||||
Success criteria must be:
|
|
||||||
|
|
||||||
1. **Measurable**: Include specific metrics (time, percentage, count, rate)
|
|
||||||
2. **Technology-agnostic**: No mention of frameworks, languages, databases, or tools
|
|
||||||
3. **User-focused**: Describe outcomes from user/business perspective, not system internals
|
|
||||||
4. **Verifiable**: Can be tested/validated without knowing implementation details
|
|
||||||
|
|
||||||
**Good examples**:
|
|
||||||
|
|
||||||
- "Users can complete checkout in under 3 minutes"
|
|
||||||
- "System supports 10,000 concurrent users"
|
|
||||||
- "95% of searches return results in under 1 second"
|
|
||||||
- "Task completion rate improves by 40%"
|
|
||||||
|
|
||||||
**Bad examples** (implementation-focused):
|
|
||||||
|
|
||||||
- "API response time is under 200ms" (too technical, use "Users see results instantly")
|
|
||||||
- "Database can handle 1000 TPS" (implementation detail, use user-facing metric)
|
|
||||||
- "React components render efficiently" (framework-specific)
|
|
||||||
- "Redis cache hit rate above 80%" (technology-specific)
|
|
||||||
@@ -1,137 +0,0 @@
|
|||||||
---
|
|
||||||
description: Generate an actionable, dependency-ordered tasks.md for the feature based on available design artifacts.
|
|
||||||
handoffs:
|
|
||||||
- label: Analyze For Consistency
|
|
||||||
agent: speckit.analyze
|
|
||||||
prompt: Run a project analysis for consistency
|
|
||||||
send: true
|
|
||||||
- label: Implement Project
|
|
||||||
agent: speckit.implement
|
|
||||||
prompt: Start the implementation in phases
|
|
||||||
send: true
|
|
||||||
---
|
|
||||||
|
|
||||||
## User Input
|
|
||||||
|
|
||||||
```text
|
|
||||||
$ARGUMENTS
|
|
||||||
```
|
|
||||||
|
|
||||||
You **MUST** consider the user input before proceeding (if not empty).
|
|
||||||
|
|
||||||
## Outline
|
|
||||||
|
|
||||||
1. **Setup**: Run `.specify/scripts/powershell/check-prerequisites.ps1 -Json` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
|
||||||
|
|
||||||
2. **Load design documents**: Read from FEATURE_DIR:
|
|
||||||
- **Required**: plan.md (tech stack, libraries, structure), spec.md (user stories with priorities)
|
|
||||||
- **Optional**: data-model.md (entities), contracts/ (interface contracts), research.md (decisions), quickstart.md (test scenarios)
|
|
||||||
- Note: Not all projects have all documents. Generate tasks based on what's available.
|
|
||||||
|
|
||||||
3. **Execute task generation workflow**:
|
|
||||||
- Load plan.md and extract tech stack, libraries, project structure
|
|
||||||
- Load spec.md and extract user stories with their priorities (P1, P2, P3, etc.)
|
|
||||||
- If data-model.md exists: Extract entities and map to user stories
|
|
||||||
- If contracts/ exists: Map interface contracts to user stories
|
|
||||||
- If research.md exists: Extract decisions for setup tasks
|
|
||||||
- Generate tasks organized by user story (see Task Generation Rules below)
|
|
||||||
- Generate dependency graph showing user story completion order
|
|
||||||
- Create parallel execution examples per user story
|
|
||||||
- Validate task completeness (each user story has all needed tasks, independently testable)
|
|
||||||
|
|
||||||
4. **Generate tasks.md**: Use `.specify/templates/tasks-template.md` as structure, fill with:
|
|
||||||
- Correct feature name from plan.md
|
|
||||||
- Phase 1: Setup tasks (project initialization)
|
|
||||||
- Phase 2: Foundational tasks (blocking prerequisites for all user stories)
|
|
||||||
- Phase 3+: One phase per user story (in priority order from spec.md)
|
|
||||||
- Each phase includes: story goal, independent test criteria, tests (if requested), implementation tasks
|
|
||||||
- Final Phase: Polish & cross-cutting concerns
|
|
||||||
- All tasks must follow the strict checklist format (see Task Generation Rules below)
|
|
||||||
- Clear file paths for each task
|
|
||||||
- Dependencies section showing story completion order
|
|
||||||
- Parallel execution examples per story
|
|
||||||
- Implementation strategy section (MVP first, incremental delivery)
|
|
||||||
|
|
||||||
5. **Report**: Output path to generated tasks.md and summary:
|
|
||||||
- Total task count
|
|
||||||
- Task count per user story
|
|
||||||
- Parallel opportunities identified
|
|
||||||
- Independent test criteria for each story
|
|
||||||
- Suggested MVP scope (typically just User Story 1)
|
|
||||||
- Format validation: Confirm ALL tasks follow the checklist format (checkbox, ID, labels, file paths)
|
|
||||||
|
|
||||||
Context for task generation: $ARGUMENTS
|
|
||||||
|
|
||||||
The tasks.md should be immediately executable - each task must be specific enough that an LLM can complete it without additional context.
|
|
||||||
|
|
||||||
## Task Generation Rules
|
|
||||||
|
|
||||||
**CRITICAL**: Tasks MUST be organized by user story to enable independent implementation and testing.
|
|
||||||
|
|
||||||
**Tests are OPTIONAL**: Only generate test tasks if explicitly requested in the feature specification or if user requests TDD approach.
|
|
||||||
|
|
||||||
### Checklist Format (REQUIRED)
|
|
||||||
|
|
||||||
Every task MUST strictly follow this format:
|
|
||||||
|
|
||||||
```text
|
|
||||||
- [ ] [TaskID] [P?] [Story?] Description with file path
|
|
||||||
```
|
|
||||||
|
|
||||||
**Format Components**:
|
|
||||||
|
|
||||||
1. **Checkbox**: ALWAYS start with `- [ ]` (markdown checkbox)
|
|
||||||
2. **Task ID**: Sequential number (T001, T002, T003...) in execution order
|
|
||||||
3. **[P] marker**: Include ONLY if task is parallelizable (different files, no dependencies on incomplete tasks)
|
|
||||||
4. **[Story] label**: REQUIRED for user story phase tasks only
|
|
||||||
- Format: [US1], [US2], [US3], etc. (maps to user stories from spec.md)
|
|
||||||
- Setup phase: NO story label
|
|
||||||
- Foundational phase: NO story label
|
|
||||||
- User Story phases: MUST have story label
|
|
||||||
- Polish phase: NO story label
|
|
||||||
5. **Description**: Clear action with exact file path
|
|
||||||
|
|
||||||
**Examples**:
|
|
||||||
|
|
||||||
- ✅ CORRECT: `- [ ] T001 Create project structure per implementation plan`
|
|
||||||
- ✅ CORRECT: `- [ ] T005 [P] Implement authentication middleware in src/middleware/auth.py`
|
|
||||||
- ✅ CORRECT: `- [ ] T012 [P] [US1] Create User model in src/models/user.py`
|
|
||||||
- ✅ CORRECT: `- [ ] T014 [US1] Implement UserService in src/services/user_service.py`
|
|
||||||
- ❌ WRONG: `- [ ] Create User model` (missing ID and Story label)
|
|
||||||
- ❌ WRONG: `T001 [US1] Create model` (missing checkbox)
|
|
||||||
- ❌ WRONG: `- [ ] [US1] Create User model` (missing Task ID)
|
|
||||||
- ❌ WRONG: `- [ ] T001 [US1] Create model` (missing file path)
|
|
||||||
|
|
||||||
### Task Organization
|
|
||||||
|
|
||||||
1. **From User Stories (spec.md)** - PRIMARY ORGANIZATION:
|
|
||||||
- Each user story (P1, P2, P3...) gets its own phase
|
|
||||||
- Map all related components to their story:
|
|
||||||
- Models needed for that story
|
|
||||||
- Services needed for that story
|
|
||||||
- Interfaces/UI needed for that story
|
|
||||||
- If tests requested: Tests specific to that story
|
|
||||||
- Mark story dependencies (most stories should be independent)
|
|
||||||
|
|
||||||
2. **From Contracts**:
|
|
||||||
- Map each interface contract → to the user story it serves
|
|
||||||
- If tests requested: Each interface contract → contract test task [P] before implementation in that story's phase
|
|
||||||
|
|
||||||
3. **From Data Model**:
|
|
||||||
- Map each entity to the user story(ies) that need it
|
|
||||||
- If entity serves multiple stories: Put in earliest story or Setup phase
|
|
||||||
- Relationships → service layer tasks in appropriate story phase
|
|
||||||
|
|
||||||
4. **From Setup/Infrastructure**:
|
|
||||||
- Shared infrastructure → Setup phase (Phase 1)
|
|
||||||
- Foundational/blocking tasks → Foundational phase (Phase 2)
|
|
||||||
- Story-specific setup → within that story's phase
|
|
||||||
|
|
||||||
### Phase Structure
|
|
||||||
|
|
||||||
- **Phase 1**: Setup (project initialization)
|
|
||||||
- **Phase 2**: Foundational (blocking prerequisites - MUST complete before user stories)
|
|
||||||
- **Phase 3+**: User Stories in priority order (P1, P2, P3...)
|
|
||||||
- Within each story: Tests (if requested) → Models → Services → Endpoints → Integration
|
|
||||||
- Each phase should be a complete, independently testable increment
|
|
||||||
- **Final Phase**: Polish & Cross-Cutting Concerns
|
|
||||||
@@ -1,30 +0,0 @@
|
|||||||
---
|
|
||||||
description: Convert existing tasks into actionable, dependency-ordered GitHub issues for the feature based on available design artifacts.
|
|
||||||
tools: ['github/github-mcp-server/issue_write']
|
|
||||||
---
|
|
||||||
|
|
||||||
## User Input
|
|
||||||
|
|
||||||
```text
|
|
||||||
$ARGUMENTS
|
|
||||||
```
|
|
||||||
|
|
||||||
You **MUST** consider the user input before proceeding (if not empty).
|
|
||||||
|
|
||||||
## Outline
|
|
||||||
|
|
||||||
1. Run `.specify/scripts/powershell/check-prerequisites.ps1 -Json -RequireTasks -IncludeTasks` from repo root and parse FEATURE_DIR and AVAILABLE_DOCS list. All paths must be absolute. For single quotes in args like "I'm Groot", use escape syntax: e.g 'I'\''m Groot' (or double-quote if possible: "I'm Groot").
|
|
||||||
1. From the executed script, extract the path to **tasks**.
|
|
||||||
1. Get the Git remote by running:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git config --get remote.origin.url
|
|
||||||
```
|
|
||||||
|
|
||||||
> [!CAUTION]
|
|
||||||
> ONLY PROCEED TO NEXT STEPS IF THE REMOTE IS A GITHUB URL
|
|
||||||
|
|
||||||
1. For each task in the list, use the GitHub MCP server to create a new issue in the repository that is representative of the Git remote.
|
|
||||||
|
|
||||||
> [!CAUTION]
|
|
||||||
> UNDER NO CIRCUMSTANCES EVER CREATE ISSUES IN REPOSITORIES THAT DO NOT MATCH THE REMOTE URL
|
|
||||||
85
.env.example
85
.env.example
@@ -1,85 +0,0 @@
|
|||||||
# ImageApi configuration template. Copy to `.env` and fill in for your
|
|
||||||
# deploy. Comments mirror the canonical docs in CLAUDE.md — see there
|
|
||||||
# for the full picture (especially the AI-Insights / Apollo / face
|
|
||||||
# integration sections).
|
|
||||||
|
|
||||||
# ── Required ────────────────────────────────────────────────────────────
|
|
||||||
DATABASE_URL=./database.db
|
|
||||||
BASE_PATH=/path/to/media
|
|
||||||
THUMBNAILS=/path/to/thumbnails
|
|
||||||
VIDEO_PATH=/path/to/video/hls
|
|
||||||
GIFS_DIRECTORY=/path/to/gifs
|
|
||||||
PREVIEW_CLIPS_DIRECTORY=/path/to/preview-clips
|
|
||||||
BIND_URL=0.0.0.0:8080
|
|
||||||
CORS_ALLOWED_ORIGINS=http://localhost:3000
|
|
||||||
SECRET_KEY=replace-me-with-a-long-random-secret
|
|
||||||
RUST_LOG=info
|
|
||||||
|
|
||||||
# ── File watching ───────────────────────────────────────────────────────
|
|
||||||
# Quick scan = recently-modified-files only; full scan = comprehensive walk.
|
|
||||||
WATCH_QUICK_INTERVAL_SECONDS=60
|
|
||||||
WATCH_FULL_INTERVAL_SECONDS=3600
|
|
||||||
# Comma-separated path prefixes / component names to skip in /memories
|
|
||||||
# AND in face detection (e.g. @eaDir, .thumbnails, /private).
|
|
||||||
EXCLUDED_DIRS=
|
|
||||||
|
|
||||||
# ── Video / HLS ─────────────────────────────────────────────────────────
|
|
||||||
HLS_CONCURRENCY=2
|
|
||||||
HLS_TIMEOUT_SECONDS=900
|
|
||||||
PLAYLIST_CLEANUP_INTERVAL_SECONDS=86400
|
|
||||||
|
|
||||||
# ── Telemetry (release builds only) ─────────────────────────────────────
|
|
||||||
# OTLP_OTLS_ENDPOINT=http://localhost:4317
|
|
||||||
|
|
||||||
# ── AI Insights — Ollama (local LLM) ────────────────────────────────────
|
|
||||||
OLLAMA_PRIMARY_URL=http://localhost:11434
|
|
||||||
OLLAMA_PRIMARY_MODEL=nemotron-3-nano:30b
|
|
||||||
# Optional fallback server tried on connection failure.
|
|
||||||
# OLLAMA_FALLBACK_URL=http://server:11434
|
|
||||||
# OLLAMA_FALLBACK_MODEL=llama3.2:3b
|
|
||||||
OLLAMA_REQUEST_TIMEOUT_SECONDS=120
|
|
||||||
# Cap on tool-calling iterations per chat turn / agentic insight.
|
|
||||||
AGENTIC_MAX_ITERATIONS=6
|
|
||||||
AGENTIC_CHAT_MAX_ITERATIONS=6
|
|
||||||
|
|
||||||
# ── AI Insights — OpenRouter (hybrid backend, optional) ─────────────────
|
|
||||||
# Set OPENROUTER_API_KEY to enable the hybrid backend (vision stays
|
|
||||||
# local on Ollama, chat routes to OpenRouter).
|
|
||||||
# OPENROUTER_API_KEY=sk-or-...
|
|
||||||
# OPENROUTER_DEFAULT_MODEL=anthropic/claude-sonnet-4
|
|
||||||
# OPENROUTER_ALLOWED_MODELS=openai/gpt-4o-mini,anthropic/claude-haiku-4-5,google/gemini-2.5-flash
|
|
||||||
# OPENROUTER_BASE_URL=https://openrouter.ai/api/v1
|
|
||||||
# OPENROUTER_EMBEDDING_MODEL=openai/text-embedding-3-small
|
|
||||||
# OPENROUTER_HTTP_REFERER=https://your-site.example
|
|
||||||
# OPENROUTER_APP_TITLE=ImageApi
|
|
||||||
|
|
||||||
# ── AI Insights — sibling services (optional) ───────────────────────────
|
|
||||||
# Apollo (places + face inference). Single Apollo deploys typically set
|
|
||||||
# only APOLLO_API_BASE_URL and let the face client fall back to it.
|
|
||||||
# APOLLO_API_BASE_URL=http://apollo.lan:8000
|
|
||||||
# APOLLO_FACE_API_BASE_URL=http://apollo.lan:8000
|
|
||||||
# SMS_API_URL=http://localhost:8000
|
|
||||||
# SMS_API_TOKEN=
|
|
||||||
|
|
||||||
# Display name used in agentic prompts when the LLM refers to "you".
|
|
||||||
USER_NAME=
|
|
||||||
|
|
||||||
# ── Face detection (Phase 3+) ───────────────────────────────────────────
|
|
||||||
# Cosine-sim floor for auto-binding a detected face to an existing
|
|
||||||
# same-named person on detection. 0.4 ≈ moderate-confidence match.
|
|
||||||
FACE_AUTOBIND_MIN_COS=0.4
|
|
||||||
# Per-scan-tick fan-out into Apollo's detect endpoint. Apollo's GPU
|
|
||||||
# pool serializes server-side; this just overlaps file-IO with
|
|
||||||
# inference RTT.
|
|
||||||
FACE_DETECT_CONCURRENCY=8
|
|
||||||
# Per-detect HTTP timeout. CPU-only Apollo deploys may need higher.
|
|
||||||
FACE_DETECT_TIMEOUT_SEC=60
|
|
||||||
# Per-tick caps on the two backlog drains (independent of WATCH_*
|
|
||||||
# quick / full scans). Tune up if you have a large unscanned backlog
|
|
||||||
# and want it to clear faster; tune down if Apollo is overloaded.
|
|
||||||
FACE_BACKLOG_MAX_PER_TICK=64
|
|
||||||
FACE_HASH_BACKFILL_MAX_PER_TICK=2000
|
|
||||||
|
|
||||||
# ── RAG / search ────────────────────────────────────────────────────────
|
|
||||||
# Set to `1` to enable cross-encoder reranking on /search results.
|
|
||||||
SEARCH_RAG_RERANK=0
|
|
||||||
9
.gitignore
vendored
9
.gitignore
vendored
@@ -1,22 +1,13 @@
|
|||||||
/target
|
/target
|
||||||
database/target
|
database/target
|
||||||
*.db
|
*.db
|
||||||
*.db.bak
|
|
||||||
*.db-shm
|
|
||||||
*.db-wal
|
|
||||||
.env
|
.env
|
||||||
/tmp
|
|
||||||
/docs
|
|
||||||
/specs
|
|
||||||
|
|
||||||
# Default ignored files
|
# Default ignored files
|
||||||
.idea/shelf/
|
.idea/shelf/
|
||||||
.idea/workspace.xml
|
.idea/workspace.xml
|
||||||
.idea/inspectionProfiles/
|
|
||||||
.idea/markdown.xml
|
|
||||||
# Datasource local storage ignored files
|
# Datasource local storage ignored files
|
||||||
.idea/dataSources*
|
.idea/dataSources*
|
||||||
.idea/dataSources.local.xml
|
.idea/dataSources.local.xml
|
||||||
# Editor-based HTTP Client requests
|
# Editor-based HTTP Client requests
|
||||||
.idea/httpRequests/
|
.idea/httpRequests/
|
||||||
/.claude/settings.local.json
|
|
||||||
|
|||||||
1
.idea/image-api.iml
generated
1
.idea/image-api.iml
generated
@@ -3,7 +3,6 @@
|
|||||||
<component name="NewModuleRootManager">
|
<component name="NewModuleRootManager">
|
||||||
<content url="file://$MODULE_DIR$">
|
<content url="file://$MODULE_DIR$">
|
||||||
<sourceFolder url="file://$MODULE_DIR$/src" isTestSource="false" />
|
<sourceFolder url="file://$MODULE_DIR$/src" isTestSource="false" />
|
||||||
<excludeFolder url="file://$MODULE_DIR$/.idea/dataSources" />
|
|
||||||
<excludeFolder url="file://$MODULE_DIR$/target" />
|
<excludeFolder url="file://$MODULE_DIR$/target" />
|
||||||
</content>
|
</content>
|
||||||
<orderEntry type="inheritedJdk" />
|
<orderEntry type="inheritedJdk" />
|
||||||
|
|||||||
@@ -1,149 +0,0 @@
|
|||||||
<!--
|
|
||||||
Sync Impact Report
|
|
||||||
==================
|
|
||||||
Version change: (new) -> 1.0.0
|
|
||||||
Modified principles: N/A (initial ratification)
|
|
||||||
Added sections:
|
|
||||||
- Core Principles (5 principles)
|
|
||||||
- Technology Stack & Constraints
|
|
||||||
- Development Workflow
|
|
||||||
- Governance
|
|
||||||
Removed sections: N/A
|
|
||||||
Templates requiring updates:
|
|
||||||
- .specify/templates/plan-template.md — ✅ no changes needed (Constitution Check section is generic)
|
|
||||||
- .specify/templates/spec-template.md — ✅ no changes needed
|
|
||||||
- .specify/templates/tasks-template.md — ✅ no changes needed
|
|
||||||
- .specify/templates/checklist-template.md — ✅ no changes needed
|
|
||||||
- .specify/templates/agent-file-template.md — ✅ no changes needed
|
|
||||||
Follow-up TODOs: None
|
|
||||||
-->
|
|
||||||
|
|
||||||
# ImageApi Constitution
|
|
||||||
|
|
||||||
## Core Principles
|
|
||||||
|
|
||||||
### I. Layered Architecture
|
|
||||||
|
|
||||||
All features MUST follow the established layered architecture:
|
|
||||||
|
|
||||||
- **HTTP Layer** (`main.rs`, feature modules): Route handlers, request
|
|
||||||
parsing, response formatting. No direct database access.
|
|
||||||
- **Service Layer** (`files.rs`, `exif.rs`, `memories.rs`, etc.): Business
|
|
||||||
logic. No HTTP-specific types.
|
|
||||||
- **DAO Layer** (`database/` trait definitions): Trait-based data access
|
|
||||||
contracts. Every DAO MUST be defined as a trait to enable mock
|
|
||||||
implementations for testing.
|
|
||||||
- **Database Layer** (Diesel ORM, `schema.rs`): Concrete `Sqlite*Dao`
|
|
||||||
implementations. All queries traced with OpenTelemetry.
|
|
||||||
|
|
||||||
New features MUST NOT bypass layers (e.g., HTTP handlers MUST NOT
|
|
||||||
execute raw SQL). Actix actors are permitted for long-running async
|
|
||||||
work (video processing, file watching) but MUST interact with the
|
|
||||||
DAO layer through the established trait interfaces.
|
|
||||||
|
|
||||||
### II. Path Safety (NON-NEGOTIABLE)
|
|
||||||
|
|
||||||
All user-supplied file paths MUST be validated against `BASE_PATH`
|
|
||||||
using `is_valid_full_path()` before any filesystem operation. This
|
|
||||||
prevents directory traversal attacks.
|
|
||||||
|
|
||||||
- Paths stored in the database MUST be relative to `BASE_PATH`.
|
|
||||||
- Paths passed to external tools (ffmpeg, image processing) MUST be
|
|
||||||
fully resolved absolute paths.
|
|
||||||
- Extension detection MUST use the centralized helpers in
|
|
||||||
`file_types.rs` (case-insensitive). Manual string matching on
|
|
||||||
extensions is prohibited.
|
|
||||||
|
|
||||||
### III. Trait-Based Testability
|
|
||||||
|
|
||||||
All data access MUST go through trait-based DAOs so that every
|
|
||||||
handler and service can be tested with mock implementations.
|
|
||||||
|
|
||||||
- Each DAO trait MUST be defined in `src/database/` and require
|
|
||||||
`Sync + Send`.
|
|
||||||
- Mock DAOs for testing MUST live in `src/testhelpers.rs`.
|
|
||||||
- Integration tests against real SQLite MUST use in-memory databases
|
|
||||||
via `in_memory_db_connection()` from `database::test`.
|
|
||||||
- Handler tests MUST use `actix_web::test` utilities with JWT token
|
|
||||||
injection (using `Claims::valid_user()` and the `test_key` secret).
|
|
||||||
- New DAO implementations MUST include a `#[cfg(test)]` constructor
|
|
||||||
(e.g., `from_connection`) accepting an injected connection.
|
|
||||||
|
|
||||||
### IV. Environment-Driven Configuration
|
|
||||||
|
|
||||||
Server behavior MUST be controlled through environment variables
|
|
||||||
loaded from `.env` files. Hard-coded paths, URLs, or secrets are
|
|
||||||
prohibited.
|
|
||||||
|
|
||||||
- Required variables MUST call `.expect()` with a clear message at
|
|
||||||
startup so misconfiguration fails fast.
|
|
||||||
- Optional variables MUST use `.unwrap_or_else()` with sensible
|
|
||||||
defaults and be documented in `README.md`.
|
|
||||||
- Any new environment variable MUST be added to the README
|
|
||||||
environment section before the feature is considered complete.
|
|
||||||
|
|
||||||
### V. Observability
|
|
||||||
|
|
||||||
All database operations and HTTP handlers MUST be instrumented
|
|
||||||
with OpenTelemetry spans via the `trace_db_call` helper or
|
|
||||||
equivalent tracing macros.
|
|
||||||
|
|
||||||
- Release builds export traces to the configured OTLP endpoint.
|
|
||||||
- Debug builds use the basic logger.
|
|
||||||
- Prometheus metrics (`imageserver_image_total`,
|
|
||||||
`imageserver_video_total`) MUST be maintained for key counters.
|
|
||||||
- Errors MUST be logged at `error!` level with sufficient context
|
|
||||||
for debugging without reproducing the issue.
|
|
||||||
|
|
||||||
## Technology Stack & Constraints
|
|
||||||
|
|
||||||
- **Language**: Rust (stable toolchain, Cargo build system)
|
|
||||||
- **HTTP Framework**: Actix-web 4
|
|
||||||
- **ORM**: Diesel 2.2 with SQLite backend
|
|
||||||
- **Auth**: JWT (HS256) via `jsonwebtoken` crate, bcrypt password
|
|
||||||
hashing
|
|
||||||
- **Video Processing**: ffmpeg/ffprobe (CLI, must be on PATH)
|
|
||||||
- **Image Processing**: `image` crate for thumbnails, `kamadak-exif`
|
|
||||||
for EXIF extraction
|
|
||||||
- **Tracing**: OpenTelemetry with OTLP export (release),
|
|
||||||
basic logger (debug)
|
|
||||||
- **Testing**: `cargo test`, `actix_web::test`, in-memory SQLite
|
|
||||||
|
|
||||||
External dependencies (ffmpeg, Ollama) are optional runtime
|
|
||||||
requirements. The server MUST start and serve core functionality
|
|
||||||
(images, thumbnails, tags) without them. Features that depend on
|
|
||||||
optional services MUST degrade gracefully with logged warnings,
|
|
||||||
not panics.
|
|
||||||
|
|
||||||
## Development Workflow
|
|
||||||
|
|
||||||
- `cargo fmt` MUST pass before committing.
|
|
||||||
- `cargo clippy` warnings MUST be resolved or explicitly suppressed
|
|
||||||
with a justification comment.
|
|
||||||
- `cargo test` MUST pass with all tests green before merging to
|
|
||||||
master.
|
|
||||||
- Database schema changes MUST use Diesel migrations
|
|
||||||
(`diesel migration generate`), with hand-written SQL in `up.sql`
|
|
||||||
and `down.sql`, followed by `diesel print-schema` to regenerate
|
|
||||||
`schema.rs`.
|
|
||||||
- Features MUST be developed on named branches
|
|
||||||
(`###-feature-name`) and merged to master via pull request.
|
|
||||||
- File uploads MUST preserve existing files (append timestamp on
|
|
||||||
conflict, never overwrite).
|
|
||||||
|
|
||||||
## Governance
|
|
||||||
|
|
||||||
This constitution defines the non-negotiable architectural and
|
|
||||||
development standards for the ImageApi project. All code changes
|
|
||||||
MUST comply with these principles.
|
|
||||||
|
|
||||||
- **Amendments**: Any change to this constitution MUST be documented
|
|
||||||
with a version bump, rationale, and updated Sync Impact Report.
|
|
||||||
- **Versioning**: MAJOR for principle removals/redefinitions, MINOR
|
|
||||||
for new principles or material expansions, PATCH for wording
|
|
||||||
clarifications.
|
|
||||||
- **Compliance**: Pull request reviews SHOULD verify adherence to
|
|
||||||
these principles. The CLAUDE.md file provides runtime development
|
|
||||||
guidance and MUST remain consistent with this constitution.
|
|
||||||
|
|
||||||
**Version**: 1.0.0 | **Ratified**: 2026-02-26 | **Last Amended**: 2026-02-26
|
|
||||||
@@ -1,148 +0,0 @@
|
|||||||
#!/usr/bin/env pwsh
|
|
||||||
|
|
||||||
# Consolidated prerequisite checking script (PowerShell)
|
|
||||||
#
|
|
||||||
# This script provides unified prerequisite checking for Spec-Driven Development workflow.
|
|
||||||
# It replaces the functionality previously spread across multiple scripts.
|
|
||||||
#
|
|
||||||
# Usage: ./check-prerequisites.ps1 [OPTIONS]
|
|
||||||
#
|
|
||||||
# OPTIONS:
|
|
||||||
# -Json Output in JSON format
|
|
||||||
# -RequireTasks Require tasks.md to exist (for implementation phase)
|
|
||||||
# -IncludeTasks Include tasks.md in AVAILABLE_DOCS list
|
|
||||||
# -PathsOnly Only output path variables (no validation)
|
|
||||||
# -Help, -h Show help message
|
|
||||||
|
|
||||||
[CmdletBinding()]
|
|
||||||
param(
|
|
||||||
[switch]$Json,
|
|
||||||
[switch]$RequireTasks,
|
|
||||||
[switch]$IncludeTasks,
|
|
||||||
[switch]$PathsOnly,
|
|
||||||
[switch]$Help
|
|
||||||
)
|
|
||||||
|
|
||||||
$ErrorActionPreference = 'Stop'
|
|
||||||
|
|
||||||
# Show help if requested
|
|
||||||
if ($Help) {
|
|
||||||
Write-Output @"
|
|
||||||
Usage: check-prerequisites.ps1 [OPTIONS]
|
|
||||||
|
|
||||||
Consolidated prerequisite checking for Spec-Driven Development workflow.
|
|
||||||
|
|
||||||
OPTIONS:
|
|
||||||
-Json Output in JSON format
|
|
||||||
-RequireTasks Require tasks.md to exist (for implementation phase)
|
|
||||||
-IncludeTasks Include tasks.md in AVAILABLE_DOCS list
|
|
||||||
-PathsOnly Only output path variables (no prerequisite validation)
|
|
||||||
-Help, -h Show this help message
|
|
||||||
|
|
||||||
EXAMPLES:
|
|
||||||
# Check task prerequisites (plan.md required)
|
|
||||||
.\check-prerequisites.ps1 -Json
|
|
||||||
|
|
||||||
# Check implementation prerequisites (plan.md + tasks.md required)
|
|
||||||
.\check-prerequisites.ps1 -Json -RequireTasks -IncludeTasks
|
|
||||||
|
|
||||||
# Get feature paths only (no validation)
|
|
||||||
.\check-prerequisites.ps1 -PathsOnly
|
|
||||||
|
|
||||||
"@
|
|
||||||
exit 0
|
|
||||||
}
|
|
||||||
|
|
||||||
# Source common functions
|
|
||||||
. "$PSScriptRoot/common.ps1"
|
|
||||||
|
|
||||||
# Get feature paths and validate branch
|
|
||||||
$paths = Get-FeaturePathsEnv
|
|
||||||
|
|
||||||
if (-not (Test-FeatureBranch -Branch $paths.CURRENT_BRANCH -HasGit:$paths.HAS_GIT)) {
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
# If paths-only mode, output paths and exit (support combined -Json -PathsOnly)
|
|
||||||
if ($PathsOnly) {
|
|
||||||
if ($Json) {
|
|
||||||
[PSCustomObject]@{
|
|
||||||
REPO_ROOT = $paths.REPO_ROOT
|
|
||||||
BRANCH = $paths.CURRENT_BRANCH
|
|
||||||
FEATURE_DIR = $paths.FEATURE_DIR
|
|
||||||
FEATURE_SPEC = $paths.FEATURE_SPEC
|
|
||||||
IMPL_PLAN = $paths.IMPL_PLAN
|
|
||||||
TASKS = $paths.TASKS
|
|
||||||
} | ConvertTo-Json -Compress
|
|
||||||
} else {
|
|
||||||
Write-Output "REPO_ROOT: $($paths.REPO_ROOT)"
|
|
||||||
Write-Output "BRANCH: $($paths.CURRENT_BRANCH)"
|
|
||||||
Write-Output "FEATURE_DIR: $($paths.FEATURE_DIR)"
|
|
||||||
Write-Output "FEATURE_SPEC: $($paths.FEATURE_SPEC)"
|
|
||||||
Write-Output "IMPL_PLAN: $($paths.IMPL_PLAN)"
|
|
||||||
Write-Output "TASKS: $($paths.TASKS)"
|
|
||||||
}
|
|
||||||
exit 0
|
|
||||||
}
|
|
||||||
|
|
||||||
# Validate required directories and files
|
|
||||||
if (-not (Test-Path $paths.FEATURE_DIR -PathType Container)) {
|
|
||||||
Write-Output "ERROR: Feature directory not found: $($paths.FEATURE_DIR)"
|
|
||||||
Write-Output "Run /speckit.specify first to create the feature structure."
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
if (-not (Test-Path $paths.IMPL_PLAN -PathType Leaf)) {
|
|
||||||
Write-Output "ERROR: plan.md not found in $($paths.FEATURE_DIR)"
|
|
||||||
Write-Output "Run /speckit.plan first to create the implementation plan."
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
# Check for tasks.md if required
|
|
||||||
if ($RequireTasks -and -not (Test-Path $paths.TASKS -PathType Leaf)) {
|
|
||||||
Write-Output "ERROR: tasks.md not found in $($paths.FEATURE_DIR)"
|
|
||||||
Write-Output "Run /speckit.tasks first to create the task list."
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
# Build list of available documents
|
|
||||||
$docs = @()
|
|
||||||
|
|
||||||
# Always check these optional docs
|
|
||||||
if (Test-Path $paths.RESEARCH) { $docs += 'research.md' }
|
|
||||||
if (Test-Path $paths.DATA_MODEL) { $docs += 'data-model.md' }
|
|
||||||
|
|
||||||
# Check contracts directory (only if it exists and has files)
|
|
||||||
if ((Test-Path $paths.CONTRACTS_DIR) -and (Get-ChildItem -Path $paths.CONTRACTS_DIR -ErrorAction SilentlyContinue | Select-Object -First 1)) {
|
|
||||||
$docs += 'contracts/'
|
|
||||||
}
|
|
||||||
|
|
||||||
if (Test-Path $paths.QUICKSTART) { $docs += 'quickstart.md' }
|
|
||||||
|
|
||||||
# Include tasks.md if requested and it exists
|
|
||||||
if ($IncludeTasks -and (Test-Path $paths.TASKS)) {
|
|
||||||
$docs += 'tasks.md'
|
|
||||||
}
|
|
||||||
|
|
||||||
# Output results
|
|
||||||
if ($Json) {
|
|
||||||
# JSON output
|
|
||||||
[PSCustomObject]@{
|
|
||||||
FEATURE_DIR = $paths.FEATURE_DIR
|
|
||||||
AVAILABLE_DOCS = $docs
|
|
||||||
} | ConvertTo-Json -Compress
|
|
||||||
} else {
|
|
||||||
# Text output
|
|
||||||
Write-Output "FEATURE_DIR:$($paths.FEATURE_DIR)"
|
|
||||||
Write-Output "AVAILABLE_DOCS:"
|
|
||||||
|
|
||||||
# Show status of each potential document
|
|
||||||
Test-FileExists -Path $paths.RESEARCH -Description 'research.md' | Out-Null
|
|
||||||
Test-FileExists -Path $paths.DATA_MODEL -Description 'data-model.md' | Out-Null
|
|
||||||
Test-DirHasFiles -Path $paths.CONTRACTS_DIR -Description 'contracts/' | Out-Null
|
|
||||||
Test-FileExists -Path $paths.QUICKSTART -Description 'quickstart.md' | Out-Null
|
|
||||||
|
|
||||||
if ($IncludeTasks) {
|
|
||||||
Test-FileExists -Path $paths.TASKS -Description 'tasks.md' | Out-Null
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,137 +0,0 @@
|
|||||||
#!/usr/bin/env pwsh
|
|
||||||
# Common PowerShell functions analogous to common.sh
|
|
||||||
|
|
||||||
function Get-RepoRoot {
|
|
||||||
try {
|
|
||||||
$result = git rev-parse --show-toplevel 2>$null
|
|
||||||
if ($LASTEXITCODE -eq 0) {
|
|
||||||
return $result
|
|
||||||
}
|
|
||||||
} catch {
|
|
||||||
# Git command failed
|
|
||||||
}
|
|
||||||
|
|
||||||
# Fall back to script location for non-git repos
|
|
||||||
return (Resolve-Path (Join-Path $PSScriptRoot "../../..")).Path
|
|
||||||
}
|
|
||||||
|
|
||||||
function Get-CurrentBranch {
|
|
||||||
# First check if SPECIFY_FEATURE environment variable is set
|
|
||||||
if ($env:SPECIFY_FEATURE) {
|
|
||||||
return $env:SPECIFY_FEATURE
|
|
||||||
}
|
|
||||||
|
|
||||||
# Then check git if available
|
|
||||||
try {
|
|
||||||
$result = git rev-parse --abbrev-ref HEAD 2>$null
|
|
||||||
if ($LASTEXITCODE -eq 0) {
|
|
||||||
return $result
|
|
||||||
}
|
|
||||||
} catch {
|
|
||||||
# Git command failed
|
|
||||||
}
|
|
||||||
|
|
||||||
# For non-git repos, try to find the latest feature directory
|
|
||||||
$repoRoot = Get-RepoRoot
|
|
||||||
$specsDir = Join-Path $repoRoot "specs"
|
|
||||||
|
|
||||||
if (Test-Path $specsDir) {
|
|
||||||
$latestFeature = ""
|
|
||||||
$highest = 0
|
|
||||||
|
|
||||||
Get-ChildItem -Path $specsDir -Directory | ForEach-Object {
|
|
||||||
if ($_.Name -match '^(\d{3})-') {
|
|
||||||
$num = [int]$matches[1]
|
|
||||||
if ($num -gt $highest) {
|
|
||||||
$highest = $num
|
|
||||||
$latestFeature = $_.Name
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if ($latestFeature) {
|
|
||||||
return $latestFeature
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
# Final fallback
|
|
||||||
return "main"
|
|
||||||
}
|
|
||||||
|
|
||||||
function Test-HasGit {
|
|
||||||
try {
|
|
||||||
git rev-parse --show-toplevel 2>$null | Out-Null
|
|
||||||
return ($LASTEXITCODE -eq 0)
|
|
||||||
} catch {
|
|
||||||
return $false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function Test-FeatureBranch {
|
|
||||||
param(
|
|
||||||
[string]$Branch,
|
|
||||||
[bool]$HasGit = $true
|
|
||||||
)
|
|
||||||
|
|
||||||
# For non-git repos, we can't enforce branch naming but still provide output
|
|
||||||
if (-not $HasGit) {
|
|
||||||
Write-Warning "[specify] Warning: Git repository not detected; skipped branch validation"
|
|
||||||
return $true
|
|
||||||
}
|
|
||||||
|
|
||||||
if ($Branch -notmatch '^[0-9]{3}-') {
|
|
||||||
Write-Output "ERROR: Not on a feature branch. Current branch: $Branch"
|
|
||||||
Write-Output "Feature branches should be named like: 001-feature-name"
|
|
||||||
return $false
|
|
||||||
}
|
|
||||||
return $true
|
|
||||||
}
|
|
||||||
|
|
||||||
function Get-FeatureDir {
|
|
||||||
param([string]$RepoRoot, [string]$Branch)
|
|
||||||
Join-Path $RepoRoot "specs/$Branch"
|
|
||||||
}
|
|
||||||
|
|
||||||
function Get-FeaturePathsEnv {
|
|
||||||
$repoRoot = Get-RepoRoot
|
|
||||||
$currentBranch = Get-CurrentBranch
|
|
||||||
$hasGit = Test-HasGit
|
|
||||||
$featureDir = Get-FeatureDir -RepoRoot $repoRoot -Branch $currentBranch
|
|
||||||
|
|
||||||
[PSCustomObject]@{
|
|
||||||
REPO_ROOT = $repoRoot
|
|
||||||
CURRENT_BRANCH = $currentBranch
|
|
||||||
HAS_GIT = $hasGit
|
|
||||||
FEATURE_DIR = $featureDir
|
|
||||||
FEATURE_SPEC = Join-Path $featureDir 'spec.md'
|
|
||||||
IMPL_PLAN = Join-Path $featureDir 'plan.md'
|
|
||||||
TASKS = Join-Path $featureDir 'tasks.md'
|
|
||||||
RESEARCH = Join-Path $featureDir 'research.md'
|
|
||||||
DATA_MODEL = Join-Path $featureDir 'data-model.md'
|
|
||||||
QUICKSTART = Join-Path $featureDir 'quickstart.md'
|
|
||||||
CONTRACTS_DIR = Join-Path $featureDir 'contracts'
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function Test-FileExists {
|
|
||||||
param([string]$Path, [string]$Description)
|
|
||||||
if (Test-Path -Path $Path -PathType Leaf) {
|
|
||||||
Write-Output " ✓ $Description"
|
|
||||||
return $true
|
|
||||||
} else {
|
|
||||||
Write-Output " ✗ $Description"
|
|
||||||
return $false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function Test-DirHasFiles {
|
|
||||||
param([string]$Path, [string]$Description)
|
|
||||||
if ((Test-Path -Path $Path -PathType Container) -and (Get-ChildItem -Path $Path -ErrorAction SilentlyContinue | Where-Object { -not $_.PSIsContainer } | Select-Object -First 1)) {
|
|
||||||
Write-Output " ✓ $Description"
|
|
||||||
return $true
|
|
||||||
} else {
|
|
||||||
Write-Output " ✗ $Description"
|
|
||||||
return $false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@@ -1,283 +0,0 @@
|
|||||||
#!/usr/bin/env pwsh
|
|
||||||
# Create a new feature
|
|
||||||
[CmdletBinding()]
|
|
||||||
param(
|
|
||||||
[switch]$Json,
|
|
||||||
[string]$ShortName,
|
|
||||||
[int]$Number = 0,
|
|
||||||
[switch]$Help,
|
|
||||||
[Parameter(ValueFromRemainingArguments = $true)]
|
|
||||||
[string[]]$FeatureDescription
|
|
||||||
)
|
|
||||||
$ErrorActionPreference = 'Stop'
|
|
||||||
|
|
||||||
# Show help if requested
|
|
||||||
if ($Help) {
|
|
||||||
Write-Host "Usage: ./create-new-feature.ps1 [-Json] [-ShortName <name>] [-Number N] <feature description>"
|
|
||||||
Write-Host ""
|
|
||||||
Write-Host "Options:"
|
|
||||||
Write-Host " -Json Output in JSON format"
|
|
||||||
Write-Host " -ShortName <name> Provide a custom short name (2-4 words) for the branch"
|
|
||||||
Write-Host " -Number N Specify branch number manually (overrides auto-detection)"
|
|
||||||
Write-Host " -Help Show this help message"
|
|
||||||
Write-Host ""
|
|
||||||
Write-Host "Examples:"
|
|
||||||
Write-Host " ./create-new-feature.ps1 'Add user authentication system' -ShortName 'user-auth'"
|
|
||||||
Write-Host " ./create-new-feature.ps1 'Implement OAuth2 integration for API'"
|
|
||||||
exit 0
|
|
||||||
}
|
|
||||||
|
|
||||||
# Check if feature description provided
|
|
||||||
if (-not $FeatureDescription -or $FeatureDescription.Count -eq 0) {
|
|
||||||
Write-Error "Usage: ./create-new-feature.ps1 [-Json] [-ShortName <name>] <feature description>"
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
$featureDesc = ($FeatureDescription -join ' ').Trim()
|
|
||||||
|
|
||||||
# Resolve repository root. Prefer git information when available, but fall back
|
|
||||||
# to searching for repository markers so the workflow still functions in repositories that
|
|
||||||
# were initialized with --no-git.
|
|
||||||
function Find-RepositoryRoot {
|
|
||||||
param(
|
|
||||||
[string]$StartDir,
|
|
||||||
[string[]]$Markers = @('.git', '.specify')
|
|
||||||
)
|
|
||||||
$current = Resolve-Path $StartDir
|
|
||||||
while ($true) {
|
|
||||||
foreach ($marker in $Markers) {
|
|
||||||
if (Test-Path (Join-Path $current $marker)) {
|
|
||||||
return $current
|
|
||||||
}
|
|
||||||
}
|
|
||||||
$parent = Split-Path $current -Parent
|
|
||||||
if ($parent -eq $current) {
|
|
||||||
# Reached filesystem root without finding markers
|
|
||||||
return $null
|
|
||||||
}
|
|
||||||
$current = $parent
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function Get-HighestNumberFromSpecs {
|
|
||||||
param([string]$SpecsDir)
|
|
||||||
|
|
||||||
$highest = 0
|
|
||||||
if (Test-Path $SpecsDir) {
|
|
||||||
Get-ChildItem -Path $SpecsDir -Directory | ForEach-Object {
|
|
||||||
if ($_.Name -match '^(\d+)') {
|
|
||||||
$num = [int]$matches[1]
|
|
||||||
if ($num -gt $highest) { $highest = $num }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return $highest
|
|
||||||
}
|
|
||||||
|
|
||||||
function Get-HighestNumberFromBranches {
|
|
||||||
param()
|
|
||||||
|
|
||||||
$highest = 0
|
|
||||||
try {
|
|
||||||
$branches = git branch -a 2>$null
|
|
||||||
if ($LASTEXITCODE -eq 0) {
|
|
||||||
foreach ($branch in $branches) {
|
|
||||||
# Clean branch name: remove leading markers and remote prefixes
|
|
||||||
$cleanBranch = $branch.Trim() -replace '^\*?\s+', '' -replace '^remotes/[^/]+/', ''
|
|
||||||
|
|
||||||
# Extract feature number if branch matches pattern ###-*
|
|
||||||
if ($cleanBranch -match '^(\d+)-') {
|
|
||||||
$num = [int]$matches[1]
|
|
||||||
if ($num -gt $highest) { $highest = $num }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} catch {
|
|
||||||
# If git command fails, return 0
|
|
||||||
Write-Verbose "Could not check Git branches: $_"
|
|
||||||
}
|
|
||||||
return $highest
|
|
||||||
}
|
|
||||||
|
|
||||||
function Get-NextBranchNumber {
|
|
||||||
param(
|
|
||||||
[string]$SpecsDir
|
|
||||||
)
|
|
||||||
|
|
||||||
# Fetch all remotes to get latest branch info (suppress errors if no remotes)
|
|
||||||
try {
|
|
||||||
git fetch --all --prune 2>$null | Out-Null
|
|
||||||
} catch {
|
|
||||||
# Ignore fetch errors
|
|
||||||
}
|
|
||||||
|
|
||||||
# Get highest number from ALL branches (not just matching short name)
|
|
||||||
$highestBranch = Get-HighestNumberFromBranches
|
|
||||||
|
|
||||||
# Get highest number from ALL specs (not just matching short name)
|
|
||||||
$highestSpec = Get-HighestNumberFromSpecs -SpecsDir $SpecsDir
|
|
||||||
|
|
||||||
# Take the maximum of both
|
|
||||||
$maxNum = [Math]::Max($highestBranch, $highestSpec)
|
|
||||||
|
|
||||||
# Return next number
|
|
||||||
return $maxNum + 1
|
|
||||||
}
|
|
||||||
|
|
||||||
function ConvertTo-CleanBranchName {
|
|
||||||
param([string]$Name)
|
|
||||||
|
|
||||||
return $Name.ToLower() -replace '[^a-z0-9]', '-' -replace '-{2,}', '-' -replace '^-', '' -replace '-$', ''
|
|
||||||
}
|
|
||||||
$fallbackRoot = (Find-RepositoryRoot -StartDir $PSScriptRoot)
|
|
||||||
if (-not $fallbackRoot) {
|
|
||||||
Write-Error "Error: Could not determine repository root. Please run this script from within the repository."
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
$repoRoot = git rev-parse --show-toplevel 2>$null
|
|
||||||
if ($LASTEXITCODE -eq 0) {
|
|
||||||
$hasGit = $true
|
|
||||||
} else {
|
|
||||||
throw "Git not available"
|
|
||||||
}
|
|
||||||
} catch {
|
|
||||||
$repoRoot = $fallbackRoot
|
|
||||||
$hasGit = $false
|
|
||||||
}
|
|
||||||
|
|
||||||
Set-Location $repoRoot
|
|
||||||
|
|
||||||
$specsDir = Join-Path $repoRoot 'specs'
|
|
||||||
New-Item -ItemType Directory -Path $specsDir -Force | Out-Null
|
|
||||||
|
|
||||||
# Function to generate branch name with stop word filtering and length filtering
|
|
||||||
function Get-BranchName {
|
|
||||||
param([string]$Description)
|
|
||||||
|
|
||||||
# Common stop words to filter out
|
|
||||||
$stopWords = @(
|
|
||||||
'i', 'a', 'an', 'the', 'to', 'for', 'of', 'in', 'on', 'at', 'by', 'with', 'from',
|
|
||||||
'is', 'are', 'was', 'were', 'be', 'been', 'being', 'have', 'has', 'had',
|
|
||||||
'do', 'does', 'did', 'will', 'would', 'should', 'could', 'can', 'may', 'might', 'must', 'shall',
|
|
||||||
'this', 'that', 'these', 'those', 'my', 'your', 'our', 'their',
|
|
||||||
'want', 'need', 'add', 'get', 'set'
|
|
||||||
)
|
|
||||||
|
|
||||||
# Convert to lowercase and extract words (alphanumeric only)
|
|
||||||
$cleanName = $Description.ToLower() -replace '[^a-z0-9\s]', ' '
|
|
||||||
$words = $cleanName -split '\s+' | Where-Object { $_ }
|
|
||||||
|
|
||||||
# Filter words: remove stop words and words shorter than 3 chars (unless they're uppercase acronyms in original)
|
|
||||||
$meaningfulWords = @()
|
|
||||||
foreach ($word in $words) {
|
|
||||||
# Skip stop words
|
|
||||||
if ($stopWords -contains $word) { continue }
|
|
||||||
|
|
||||||
# Keep words that are length >= 3 OR appear as uppercase in original (likely acronyms)
|
|
||||||
if ($word.Length -ge 3) {
|
|
||||||
$meaningfulWords += $word
|
|
||||||
} elseif ($Description -match "\b$($word.ToUpper())\b") {
|
|
||||||
# Keep short words if they appear as uppercase in original (likely acronyms)
|
|
||||||
$meaningfulWords += $word
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
# If we have meaningful words, use first 3-4 of them
|
|
||||||
if ($meaningfulWords.Count -gt 0) {
|
|
||||||
$maxWords = if ($meaningfulWords.Count -eq 4) { 4 } else { 3 }
|
|
||||||
$result = ($meaningfulWords | Select-Object -First $maxWords) -join '-'
|
|
||||||
return $result
|
|
||||||
} else {
|
|
||||||
# Fallback to original logic if no meaningful words found
|
|
||||||
$result = ConvertTo-CleanBranchName -Name $Description
|
|
||||||
$fallbackWords = ($result -split '-') | Where-Object { $_ } | Select-Object -First 3
|
|
||||||
return [string]::Join('-', $fallbackWords)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
# Generate branch name
|
|
||||||
if ($ShortName) {
|
|
||||||
# Use provided short name, just clean it up
|
|
||||||
$branchSuffix = ConvertTo-CleanBranchName -Name $ShortName
|
|
||||||
} else {
|
|
||||||
# Generate from description with smart filtering
|
|
||||||
$branchSuffix = Get-BranchName -Description $featureDesc
|
|
||||||
}
|
|
||||||
|
|
||||||
# Determine branch number
|
|
||||||
if ($Number -eq 0) {
|
|
||||||
if ($hasGit) {
|
|
||||||
# Check existing branches on remotes
|
|
||||||
$Number = Get-NextBranchNumber -SpecsDir $specsDir
|
|
||||||
} else {
|
|
||||||
# Fall back to local directory check
|
|
||||||
$Number = (Get-HighestNumberFromSpecs -SpecsDir $specsDir) + 1
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
$featureNum = ('{0:000}' -f $Number)
|
|
||||||
$branchName = "$featureNum-$branchSuffix"
|
|
||||||
|
|
||||||
# GitHub enforces a 244-byte limit on branch names
|
|
||||||
# Validate and truncate if necessary
|
|
||||||
$maxBranchLength = 244
|
|
||||||
if ($branchName.Length -gt $maxBranchLength) {
|
|
||||||
# Calculate how much we need to trim from suffix
|
|
||||||
# Account for: feature number (3) + hyphen (1) = 4 chars
|
|
||||||
$maxSuffixLength = $maxBranchLength - 4
|
|
||||||
|
|
||||||
# Truncate suffix
|
|
||||||
$truncatedSuffix = $branchSuffix.Substring(0, [Math]::Min($branchSuffix.Length, $maxSuffixLength))
|
|
||||||
# Remove trailing hyphen if truncation created one
|
|
||||||
$truncatedSuffix = $truncatedSuffix -replace '-$', ''
|
|
||||||
|
|
||||||
$originalBranchName = $branchName
|
|
||||||
$branchName = "$featureNum-$truncatedSuffix"
|
|
||||||
|
|
||||||
Write-Warning "[specify] Branch name exceeded GitHub's 244-byte limit"
|
|
||||||
Write-Warning "[specify] Original: $originalBranchName ($($originalBranchName.Length) bytes)"
|
|
||||||
Write-Warning "[specify] Truncated to: $branchName ($($branchName.Length) bytes)"
|
|
||||||
}
|
|
||||||
|
|
||||||
if ($hasGit) {
|
|
||||||
try {
|
|
||||||
git checkout -b $branchName | Out-Null
|
|
||||||
} catch {
|
|
||||||
Write-Warning "Failed to create git branch: $branchName"
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
Write-Warning "[specify] Warning: Git repository not detected; skipped branch creation for $branchName"
|
|
||||||
}
|
|
||||||
|
|
||||||
$featureDir = Join-Path $specsDir $branchName
|
|
||||||
New-Item -ItemType Directory -Path $featureDir -Force | Out-Null
|
|
||||||
|
|
||||||
$template = Join-Path $repoRoot '.specify/templates/spec-template.md'
|
|
||||||
$specFile = Join-Path $featureDir 'spec.md'
|
|
||||||
if (Test-Path $template) {
|
|
||||||
Copy-Item $template $specFile -Force
|
|
||||||
} else {
|
|
||||||
New-Item -ItemType File -Path $specFile | Out-Null
|
|
||||||
}
|
|
||||||
|
|
||||||
# Set the SPECIFY_FEATURE environment variable for the current session
|
|
||||||
$env:SPECIFY_FEATURE = $branchName
|
|
||||||
|
|
||||||
if ($Json) {
|
|
||||||
$obj = [PSCustomObject]@{
|
|
||||||
BRANCH_NAME = $branchName
|
|
||||||
SPEC_FILE = $specFile
|
|
||||||
FEATURE_NUM = $featureNum
|
|
||||||
HAS_GIT = $hasGit
|
|
||||||
}
|
|
||||||
$obj | ConvertTo-Json -Compress
|
|
||||||
} else {
|
|
||||||
Write-Output "BRANCH_NAME: $branchName"
|
|
||||||
Write-Output "SPEC_FILE: $specFile"
|
|
||||||
Write-Output "FEATURE_NUM: $featureNum"
|
|
||||||
Write-Output "HAS_GIT: $hasGit"
|
|
||||||
Write-Output "SPECIFY_FEATURE environment variable set to: $branchName"
|
|
||||||
}
|
|
||||||
|
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
#!/usr/bin/env pwsh
|
|
||||||
# Setup implementation plan for a feature
|
|
||||||
|
|
||||||
[CmdletBinding()]
|
|
||||||
param(
|
|
||||||
[switch]$Json,
|
|
||||||
[switch]$Help
|
|
||||||
)
|
|
||||||
|
|
||||||
$ErrorActionPreference = 'Stop'
|
|
||||||
|
|
||||||
# Show help if requested
|
|
||||||
if ($Help) {
|
|
||||||
Write-Output "Usage: ./setup-plan.ps1 [-Json] [-Help]"
|
|
||||||
Write-Output " -Json Output results in JSON format"
|
|
||||||
Write-Output " -Help Show this help message"
|
|
||||||
exit 0
|
|
||||||
}
|
|
||||||
|
|
||||||
# Load common functions
|
|
||||||
. "$PSScriptRoot/common.ps1"
|
|
||||||
|
|
||||||
# Get all paths and variables from common functions
|
|
||||||
$paths = Get-FeaturePathsEnv
|
|
||||||
|
|
||||||
# Check if we're on a proper feature branch (only for git repos)
|
|
||||||
if (-not (Test-FeatureBranch -Branch $paths.CURRENT_BRANCH -HasGit $paths.HAS_GIT)) {
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
# Ensure the feature directory exists
|
|
||||||
New-Item -ItemType Directory -Path $paths.FEATURE_DIR -Force | Out-Null
|
|
||||||
|
|
||||||
# Copy plan template if it exists, otherwise note it or create empty file
|
|
||||||
$template = Join-Path $paths.REPO_ROOT '.specify/templates/plan-template.md'
|
|
||||||
if (Test-Path $template) {
|
|
||||||
Copy-Item $template $paths.IMPL_PLAN -Force
|
|
||||||
Write-Output "Copied plan template to $($paths.IMPL_PLAN)"
|
|
||||||
} else {
|
|
||||||
Write-Warning "Plan template not found at $template"
|
|
||||||
# Create a basic plan file if template doesn't exist
|
|
||||||
New-Item -ItemType File -Path $paths.IMPL_PLAN -Force | Out-Null
|
|
||||||
}
|
|
||||||
|
|
||||||
# Output results
|
|
||||||
if ($Json) {
|
|
||||||
$result = [PSCustomObject]@{
|
|
||||||
FEATURE_SPEC = $paths.FEATURE_SPEC
|
|
||||||
IMPL_PLAN = $paths.IMPL_PLAN
|
|
||||||
SPECS_DIR = $paths.FEATURE_DIR
|
|
||||||
BRANCH = $paths.CURRENT_BRANCH
|
|
||||||
HAS_GIT = $paths.HAS_GIT
|
|
||||||
}
|
|
||||||
$result | ConvertTo-Json -Compress
|
|
||||||
} else {
|
|
||||||
Write-Output "FEATURE_SPEC: $($paths.FEATURE_SPEC)"
|
|
||||||
Write-Output "IMPL_PLAN: $($paths.IMPL_PLAN)"
|
|
||||||
Write-Output "SPECS_DIR: $($paths.FEATURE_DIR)"
|
|
||||||
Write-Output "BRANCH: $($paths.CURRENT_BRANCH)"
|
|
||||||
Write-Output "HAS_GIT: $($paths.HAS_GIT)"
|
|
||||||
}
|
|
||||||
@@ -1,452 +0,0 @@
|
|||||||
#!/usr/bin/env pwsh
|
|
||||||
<#!
|
|
||||||
.SYNOPSIS
|
|
||||||
Update agent context files with information from plan.md (PowerShell version)
|
|
||||||
|
|
||||||
.DESCRIPTION
|
|
||||||
Mirrors the behavior of scripts/bash/update-agent-context.sh:
|
|
||||||
1. Environment Validation
|
|
||||||
2. Plan Data Extraction
|
|
||||||
3. Agent File Management (create from template or update existing)
|
|
||||||
4. Content Generation (technology stack, recent changes, timestamp)
|
|
||||||
5. Multi-Agent Support (claude, gemini, copilot, cursor-agent, qwen, opencode, codex, windsurf, kilocode, auggie, roo, codebuddy, amp, shai, q, agy, bob, qodercli)
|
|
||||||
|
|
||||||
.PARAMETER AgentType
|
|
||||||
Optional agent key to update a single agent. If omitted, updates all existing agent files (creating a default Claude file if none exist).
|
|
||||||
|
|
||||||
.EXAMPLE
|
|
||||||
./update-agent-context.ps1 -AgentType claude
|
|
||||||
|
|
||||||
.EXAMPLE
|
|
||||||
./update-agent-context.ps1 # Updates all existing agent files
|
|
||||||
|
|
||||||
.NOTES
|
|
||||||
Relies on common helper functions in common.ps1
|
|
||||||
#>
|
|
||||||
param(
|
|
||||||
[Parameter(Position=0)]
|
|
||||||
[ValidateSet('claude','gemini','copilot','cursor-agent','qwen','opencode','codex','windsurf','kilocode','auggie','roo','codebuddy','amp','shai','q','agy','bob','qodercli','generic')]
|
|
||||||
[string]$AgentType
|
|
||||||
)
|
|
||||||
|
|
||||||
$ErrorActionPreference = 'Stop'
|
|
||||||
|
|
||||||
# Import common helpers
|
|
||||||
$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
|
|
||||||
. (Join-Path $ScriptDir 'common.ps1')
|
|
||||||
|
|
||||||
# Acquire environment paths
|
|
||||||
$envData = Get-FeaturePathsEnv
|
|
||||||
$REPO_ROOT = $envData.REPO_ROOT
|
|
||||||
$CURRENT_BRANCH = $envData.CURRENT_BRANCH
|
|
||||||
$HAS_GIT = $envData.HAS_GIT
|
|
||||||
$IMPL_PLAN = $envData.IMPL_PLAN
|
|
||||||
$NEW_PLAN = $IMPL_PLAN
|
|
||||||
|
|
||||||
# Agent file paths
|
|
||||||
$CLAUDE_FILE = Join-Path $REPO_ROOT 'CLAUDE.md'
|
|
||||||
$GEMINI_FILE = Join-Path $REPO_ROOT 'GEMINI.md'
|
|
||||||
$COPILOT_FILE = Join-Path $REPO_ROOT '.github/agents/copilot-instructions.md'
|
|
||||||
$CURSOR_FILE = Join-Path $REPO_ROOT '.cursor/rules/specify-rules.mdc'
|
|
||||||
$QWEN_FILE = Join-Path $REPO_ROOT 'QWEN.md'
|
|
||||||
$AGENTS_FILE = Join-Path $REPO_ROOT 'AGENTS.md'
|
|
||||||
$WINDSURF_FILE = Join-Path $REPO_ROOT '.windsurf/rules/specify-rules.md'
|
|
||||||
$KILOCODE_FILE = Join-Path $REPO_ROOT '.kilocode/rules/specify-rules.md'
|
|
||||||
$AUGGIE_FILE = Join-Path $REPO_ROOT '.augment/rules/specify-rules.md'
|
|
||||||
$ROO_FILE = Join-Path $REPO_ROOT '.roo/rules/specify-rules.md'
|
|
||||||
$CODEBUDDY_FILE = Join-Path $REPO_ROOT 'CODEBUDDY.md'
|
|
||||||
$QODER_FILE = Join-Path $REPO_ROOT 'QODER.md'
|
|
||||||
$AMP_FILE = Join-Path $REPO_ROOT 'AGENTS.md'
|
|
||||||
$SHAI_FILE = Join-Path $REPO_ROOT 'SHAI.md'
|
|
||||||
$Q_FILE = Join-Path $REPO_ROOT 'AGENTS.md'
|
|
||||||
$AGY_FILE = Join-Path $REPO_ROOT '.agent/rules/specify-rules.md'
|
|
||||||
$BOB_FILE = Join-Path $REPO_ROOT 'AGENTS.md'
|
|
||||||
|
|
||||||
$TEMPLATE_FILE = Join-Path $REPO_ROOT '.specify/templates/agent-file-template.md'
|
|
||||||
|
|
||||||
# Parsed plan data placeholders
|
|
||||||
$script:NEW_LANG = ''
|
|
||||||
$script:NEW_FRAMEWORK = ''
|
|
||||||
$script:NEW_DB = ''
|
|
||||||
$script:NEW_PROJECT_TYPE = ''
|
|
||||||
|
|
||||||
function Write-Info {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$Message
|
|
||||||
)
|
|
||||||
Write-Host "INFO: $Message"
|
|
||||||
}
|
|
||||||
|
|
||||||
function Write-Success {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$Message
|
|
||||||
)
|
|
||||||
Write-Host "$([char]0x2713) $Message"
|
|
||||||
}
|
|
||||||
|
|
||||||
function Write-WarningMsg {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$Message
|
|
||||||
)
|
|
||||||
Write-Warning $Message
|
|
||||||
}
|
|
||||||
|
|
||||||
function Write-Err {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$Message
|
|
||||||
)
|
|
||||||
Write-Host "ERROR: $Message" -ForegroundColor Red
|
|
||||||
}
|
|
||||||
|
|
||||||
function Validate-Environment {
|
|
||||||
if (-not $CURRENT_BRANCH) {
|
|
||||||
Write-Err 'Unable to determine current feature'
|
|
||||||
if ($HAS_GIT) { Write-Info "Make sure you're on a feature branch" } else { Write-Info 'Set SPECIFY_FEATURE environment variable or create a feature first' }
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
if (-not (Test-Path $NEW_PLAN)) {
|
|
||||||
Write-Err "No plan.md found at $NEW_PLAN"
|
|
||||||
Write-Info 'Ensure you are working on a feature with a corresponding spec directory'
|
|
||||||
if (-not $HAS_GIT) { Write-Info 'Use: $env:SPECIFY_FEATURE=your-feature-name or create a new feature first' }
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
if (-not (Test-Path $TEMPLATE_FILE)) {
|
|
||||||
Write-Err "Template file not found at $TEMPLATE_FILE"
|
|
||||||
Write-Info 'Run specify init to scaffold .specify/templates, or add agent-file-template.md there.'
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function Extract-PlanField {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$FieldPattern,
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$PlanFile
|
|
||||||
)
|
|
||||||
if (-not (Test-Path $PlanFile)) { return '' }
|
|
||||||
# Lines like **Language/Version**: Python 3.12
|
|
||||||
$regex = "^\*\*$([Regex]::Escape($FieldPattern))\*\*: (.+)$"
|
|
||||||
Get-Content -LiteralPath $PlanFile -Encoding utf8 | ForEach-Object {
|
|
||||||
if ($_ -match $regex) {
|
|
||||||
$val = $Matches[1].Trim()
|
|
||||||
if ($val -notin @('NEEDS CLARIFICATION','N/A')) { return $val }
|
|
||||||
}
|
|
||||||
} | Select-Object -First 1
|
|
||||||
}
|
|
||||||
|
|
||||||
function Parse-PlanData {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$PlanFile
|
|
||||||
)
|
|
||||||
if (-not (Test-Path $PlanFile)) { Write-Err "Plan file not found: $PlanFile"; return $false }
|
|
||||||
Write-Info "Parsing plan data from $PlanFile"
|
|
||||||
$script:NEW_LANG = Extract-PlanField -FieldPattern 'Language/Version' -PlanFile $PlanFile
|
|
||||||
$script:NEW_FRAMEWORK = Extract-PlanField -FieldPattern 'Primary Dependencies' -PlanFile $PlanFile
|
|
||||||
$script:NEW_DB = Extract-PlanField -FieldPattern 'Storage' -PlanFile $PlanFile
|
|
||||||
$script:NEW_PROJECT_TYPE = Extract-PlanField -FieldPattern 'Project Type' -PlanFile $PlanFile
|
|
||||||
|
|
||||||
if ($NEW_LANG) { Write-Info "Found language: $NEW_LANG" } else { Write-WarningMsg 'No language information found in plan' }
|
|
||||||
if ($NEW_FRAMEWORK) { Write-Info "Found framework: $NEW_FRAMEWORK" }
|
|
||||||
if ($NEW_DB -and $NEW_DB -ne 'N/A') { Write-Info "Found database: $NEW_DB" }
|
|
||||||
if ($NEW_PROJECT_TYPE) { Write-Info "Found project type: $NEW_PROJECT_TYPE" }
|
|
||||||
return $true
|
|
||||||
}
|
|
||||||
|
|
||||||
function Format-TechnologyStack {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$false)]
|
|
||||||
[string]$Lang,
|
|
||||||
[Parameter(Mandatory=$false)]
|
|
||||||
[string]$Framework
|
|
||||||
)
|
|
||||||
$parts = @()
|
|
||||||
if ($Lang -and $Lang -ne 'NEEDS CLARIFICATION') { $parts += $Lang }
|
|
||||||
if ($Framework -and $Framework -notin @('NEEDS CLARIFICATION','N/A')) { $parts += $Framework }
|
|
||||||
if (-not $parts) { return '' }
|
|
||||||
return ($parts -join ' + ')
|
|
||||||
}
|
|
||||||
|
|
||||||
function Get-ProjectStructure {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$false)]
|
|
||||||
[string]$ProjectType
|
|
||||||
)
|
|
||||||
if ($ProjectType -match 'web') { return "backend/`nfrontend/`ntests/" } else { return "src/`ntests/" }
|
|
||||||
}
|
|
||||||
|
|
||||||
function Get-CommandsForLanguage {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$false)]
|
|
||||||
[string]$Lang
|
|
||||||
)
|
|
||||||
switch -Regex ($Lang) {
|
|
||||||
'Python' { return "cd src; pytest; ruff check ." }
|
|
||||||
'Rust' { return "cargo test; cargo clippy" }
|
|
||||||
'JavaScript|TypeScript' { return "npm test; npm run lint" }
|
|
||||||
default { return "# Add commands for $Lang" }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function Get-LanguageConventions {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$false)]
|
|
||||||
[string]$Lang
|
|
||||||
)
|
|
||||||
if ($Lang) { "${Lang}: Follow standard conventions" } else { 'General: Follow standard conventions' }
|
|
||||||
}
|
|
||||||
|
|
||||||
function New-AgentFile {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$TargetFile,
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$ProjectName,
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[datetime]$Date
|
|
||||||
)
|
|
||||||
if (-not (Test-Path $TEMPLATE_FILE)) { Write-Err "Template not found at $TEMPLATE_FILE"; return $false }
|
|
||||||
$temp = New-TemporaryFile
|
|
||||||
Copy-Item -LiteralPath $TEMPLATE_FILE -Destination $temp -Force
|
|
||||||
|
|
||||||
$projectStructure = Get-ProjectStructure -ProjectType $NEW_PROJECT_TYPE
|
|
||||||
$commands = Get-CommandsForLanguage -Lang $NEW_LANG
|
|
||||||
$languageConventions = Get-LanguageConventions -Lang $NEW_LANG
|
|
||||||
|
|
||||||
$escaped_lang = $NEW_LANG
|
|
||||||
$escaped_framework = $NEW_FRAMEWORK
|
|
||||||
$escaped_branch = $CURRENT_BRANCH
|
|
||||||
|
|
||||||
$content = Get-Content -LiteralPath $temp -Raw -Encoding utf8
|
|
||||||
$content = $content -replace '\[PROJECT NAME\]',$ProjectName
|
|
||||||
$content = $content -replace '\[DATE\]',$Date.ToString('yyyy-MM-dd')
|
|
||||||
|
|
||||||
# Build the technology stack string safely
|
|
||||||
$techStackForTemplate = ""
|
|
||||||
if ($escaped_lang -and $escaped_framework) {
|
|
||||||
$techStackForTemplate = "- $escaped_lang + $escaped_framework ($escaped_branch)"
|
|
||||||
} elseif ($escaped_lang) {
|
|
||||||
$techStackForTemplate = "- $escaped_lang ($escaped_branch)"
|
|
||||||
} elseif ($escaped_framework) {
|
|
||||||
$techStackForTemplate = "- $escaped_framework ($escaped_branch)"
|
|
||||||
}
|
|
||||||
|
|
||||||
$content = $content -replace '\[EXTRACTED FROM ALL PLAN.MD FILES\]',$techStackForTemplate
|
|
||||||
# For project structure we manually embed (keep newlines)
|
|
||||||
$escapedStructure = [Regex]::Escape($projectStructure)
|
|
||||||
$content = $content -replace '\[ACTUAL STRUCTURE FROM PLANS\]',$escapedStructure
|
|
||||||
# Replace escaped newlines placeholder after all replacements
|
|
||||||
$content = $content -replace '\[ONLY COMMANDS FOR ACTIVE TECHNOLOGIES\]',$commands
|
|
||||||
$content = $content -replace '\[LANGUAGE-SPECIFIC, ONLY FOR LANGUAGES IN USE\]',$languageConventions
|
|
||||||
|
|
||||||
# Build the recent changes string safely
|
|
||||||
$recentChangesForTemplate = ""
|
|
||||||
if ($escaped_lang -and $escaped_framework) {
|
|
||||||
$recentChangesForTemplate = "- ${escaped_branch}: Added ${escaped_lang} + ${escaped_framework}"
|
|
||||||
} elseif ($escaped_lang) {
|
|
||||||
$recentChangesForTemplate = "- ${escaped_branch}: Added ${escaped_lang}"
|
|
||||||
} elseif ($escaped_framework) {
|
|
||||||
$recentChangesForTemplate = "- ${escaped_branch}: Added ${escaped_framework}"
|
|
||||||
}
|
|
||||||
|
|
||||||
$content = $content -replace '\[LAST 3 FEATURES AND WHAT THEY ADDED\]',$recentChangesForTemplate
|
|
||||||
# Convert literal \n sequences introduced by Escape to real newlines
|
|
||||||
$content = $content -replace '\\n',[Environment]::NewLine
|
|
||||||
|
|
||||||
$parent = Split-Path -Parent $TargetFile
|
|
||||||
if (-not (Test-Path $parent)) { New-Item -ItemType Directory -Path $parent | Out-Null }
|
|
||||||
Set-Content -LiteralPath $TargetFile -Value $content -NoNewline -Encoding utf8
|
|
||||||
Remove-Item $temp -Force
|
|
||||||
return $true
|
|
||||||
}
|
|
||||||
|
|
||||||
function Update-ExistingAgentFile {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$TargetFile,
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[datetime]$Date
|
|
||||||
)
|
|
||||||
if (-not (Test-Path $TargetFile)) { return (New-AgentFile -TargetFile $TargetFile -ProjectName (Split-Path $REPO_ROOT -Leaf) -Date $Date) }
|
|
||||||
|
|
||||||
$techStack = Format-TechnologyStack -Lang $NEW_LANG -Framework $NEW_FRAMEWORK
|
|
||||||
$newTechEntries = @()
|
|
||||||
if ($techStack) {
|
|
||||||
$escapedTechStack = [Regex]::Escape($techStack)
|
|
||||||
if (-not (Select-String -Pattern $escapedTechStack -Path $TargetFile -Quiet)) {
|
|
||||||
$newTechEntries += "- $techStack ($CURRENT_BRANCH)"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if ($NEW_DB -and $NEW_DB -notin @('N/A','NEEDS CLARIFICATION')) {
|
|
||||||
$escapedDB = [Regex]::Escape($NEW_DB)
|
|
||||||
if (-not (Select-String -Pattern $escapedDB -Path $TargetFile -Quiet)) {
|
|
||||||
$newTechEntries += "- $NEW_DB ($CURRENT_BRANCH)"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
$newChangeEntry = ''
|
|
||||||
if ($techStack) { $newChangeEntry = "- ${CURRENT_BRANCH}: Added ${techStack}" }
|
|
||||||
elseif ($NEW_DB -and $NEW_DB -notin @('N/A','NEEDS CLARIFICATION')) { $newChangeEntry = "- ${CURRENT_BRANCH}: Added ${NEW_DB}" }
|
|
||||||
|
|
||||||
$lines = Get-Content -LiteralPath $TargetFile -Encoding utf8
|
|
||||||
$output = New-Object System.Collections.Generic.List[string]
|
|
||||||
$inTech = $false; $inChanges = $false; $techAdded = $false; $changeAdded = $false; $existingChanges = 0
|
|
||||||
|
|
||||||
for ($i=0; $i -lt $lines.Count; $i++) {
|
|
||||||
$line = $lines[$i]
|
|
||||||
if ($line -eq '## Active Technologies') {
|
|
||||||
$output.Add($line)
|
|
||||||
$inTech = $true
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if ($inTech -and $line -match '^##\s') {
|
|
||||||
if (-not $techAdded -and $newTechEntries.Count -gt 0) { $newTechEntries | ForEach-Object { $output.Add($_) }; $techAdded = $true }
|
|
||||||
$output.Add($line); $inTech = $false; continue
|
|
||||||
}
|
|
||||||
if ($inTech -and [string]::IsNullOrWhiteSpace($line)) {
|
|
||||||
if (-not $techAdded -and $newTechEntries.Count -gt 0) { $newTechEntries | ForEach-Object { $output.Add($_) }; $techAdded = $true }
|
|
||||||
$output.Add($line); continue
|
|
||||||
}
|
|
||||||
if ($line -eq '## Recent Changes') {
|
|
||||||
$output.Add($line)
|
|
||||||
if ($newChangeEntry) { $output.Add($newChangeEntry); $changeAdded = $true }
|
|
||||||
$inChanges = $true
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if ($inChanges -and $line -match '^##\s') { $output.Add($line); $inChanges = $false; continue }
|
|
||||||
if ($inChanges -and $line -match '^- ') {
|
|
||||||
if ($existingChanges -lt 2) { $output.Add($line); $existingChanges++ }
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if ($line -match '\*\*Last updated\*\*: .*\d{4}-\d{2}-\d{2}') {
|
|
||||||
$output.Add(($line -replace '\d{4}-\d{2}-\d{2}',$Date.ToString('yyyy-MM-dd')))
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
$output.Add($line)
|
|
||||||
}
|
|
||||||
|
|
||||||
# Post-loop check: if we're still in the Active Technologies section and haven't added new entries
|
|
||||||
if ($inTech -and -not $techAdded -and $newTechEntries.Count -gt 0) {
|
|
||||||
$newTechEntries | ForEach-Object { $output.Add($_) }
|
|
||||||
}
|
|
||||||
|
|
||||||
Set-Content -LiteralPath $TargetFile -Value ($output -join [Environment]::NewLine) -Encoding utf8
|
|
||||||
return $true
|
|
||||||
}
|
|
||||||
|
|
||||||
function Update-AgentFile {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$TargetFile,
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$AgentName
|
|
||||||
)
|
|
||||||
if (-not $TargetFile -or -not $AgentName) { Write-Err 'Update-AgentFile requires TargetFile and AgentName'; return $false }
|
|
||||||
Write-Info "Updating $AgentName context file: $TargetFile"
|
|
||||||
$projectName = Split-Path $REPO_ROOT -Leaf
|
|
||||||
$date = Get-Date
|
|
||||||
|
|
||||||
$dir = Split-Path -Parent $TargetFile
|
|
||||||
if (-not (Test-Path $dir)) { New-Item -ItemType Directory -Path $dir | Out-Null }
|
|
||||||
|
|
||||||
if (-not (Test-Path $TargetFile)) {
|
|
||||||
if (New-AgentFile -TargetFile $TargetFile -ProjectName $projectName -Date $date) { Write-Success "Created new $AgentName context file" } else { Write-Err 'Failed to create new agent file'; return $false }
|
|
||||||
} else {
|
|
||||||
try {
|
|
||||||
if (Update-ExistingAgentFile -TargetFile $TargetFile -Date $date) { Write-Success "Updated existing $AgentName context file" } else { Write-Err 'Failed to update agent file'; return $false }
|
|
||||||
} catch {
|
|
||||||
Write-Err "Cannot access or update existing file: $TargetFile. $_"
|
|
||||||
return $false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return $true
|
|
||||||
}
|
|
||||||
|
|
||||||
function Update-SpecificAgent {
|
|
||||||
param(
|
|
||||||
[Parameter(Mandatory=$true)]
|
|
||||||
[string]$Type
|
|
||||||
)
|
|
||||||
switch ($Type) {
|
|
||||||
'claude' { Update-AgentFile -TargetFile $CLAUDE_FILE -AgentName 'Claude Code' }
|
|
||||||
'gemini' { Update-AgentFile -TargetFile $GEMINI_FILE -AgentName 'Gemini CLI' }
|
|
||||||
'copilot' { Update-AgentFile -TargetFile $COPILOT_FILE -AgentName 'GitHub Copilot' }
|
|
||||||
'cursor-agent' { Update-AgentFile -TargetFile $CURSOR_FILE -AgentName 'Cursor IDE' }
|
|
||||||
'qwen' { Update-AgentFile -TargetFile $QWEN_FILE -AgentName 'Qwen Code' }
|
|
||||||
'opencode' { Update-AgentFile -TargetFile $AGENTS_FILE -AgentName 'opencode' }
|
|
||||||
'codex' { Update-AgentFile -TargetFile $AGENTS_FILE -AgentName 'Codex CLI' }
|
|
||||||
'windsurf' { Update-AgentFile -TargetFile $WINDSURF_FILE -AgentName 'Windsurf' }
|
|
||||||
'kilocode' { Update-AgentFile -TargetFile $KILOCODE_FILE -AgentName 'Kilo Code' }
|
|
||||||
'auggie' { Update-AgentFile -TargetFile $AUGGIE_FILE -AgentName 'Auggie CLI' }
|
|
||||||
'roo' { Update-AgentFile -TargetFile $ROO_FILE -AgentName 'Roo Code' }
|
|
||||||
'codebuddy' { Update-AgentFile -TargetFile $CODEBUDDY_FILE -AgentName 'CodeBuddy CLI' }
|
|
||||||
'qodercli' { Update-AgentFile -TargetFile $QODER_FILE -AgentName 'Qoder CLI' }
|
|
||||||
'amp' { Update-AgentFile -TargetFile $AMP_FILE -AgentName 'Amp' }
|
|
||||||
'shai' { Update-AgentFile -TargetFile $SHAI_FILE -AgentName 'SHAI' }
|
|
||||||
'q' { Update-AgentFile -TargetFile $Q_FILE -AgentName 'Amazon Q Developer CLI' }
|
|
||||||
'agy' { Update-AgentFile -TargetFile $AGY_FILE -AgentName 'Antigravity' }
|
|
||||||
'bob' { Update-AgentFile -TargetFile $BOB_FILE -AgentName 'IBM Bob' }
|
|
||||||
'generic' { Write-Info 'Generic agent: no predefined context file. Use the agent-specific update script for your agent.' }
|
|
||||||
default { Write-Err "Unknown agent type '$Type'"; Write-Err 'Expected: claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|kilocode|auggie|roo|codebuddy|amp|shai|q|agy|bob|qodercli|generic'; return $false }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function Update-AllExistingAgents {
|
|
||||||
$found = $false
|
|
||||||
$ok = $true
|
|
||||||
if (Test-Path $CLAUDE_FILE) { if (-not (Update-AgentFile -TargetFile $CLAUDE_FILE -AgentName 'Claude Code')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $GEMINI_FILE) { if (-not (Update-AgentFile -TargetFile $GEMINI_FILE -AgentName 'Gemini CLI')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $COPILOT_FILE) { if (-not (Update-AgentFile -TargetFile $COPILOT_FILE -AgentName 'GitHub Copilot')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $CURSOR_FILE) { if (-not (Update-AgentFile -TargetFile $CURSOR_FILE -AgentName 'Cursor IDE')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $QWEN_FILE) { if (-not (Update-AgentFile -TargetFile $QWEN_FILE -AgentName 'Qwen Code')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $AGENTS_FILE) { if (-not (Update-AgentFile -TargetFile $AGENTS_FILE -AgentName 'Codex/opencode')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $WINDSURF_FILE) { if (-not (Update-AgentFile -TargetFile $WINDSURF_FILE -AgentName 'Windsurf')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $KILOCODE_FILE) { if (-not (Update-AgentFile -TargetFile $KILOCODE_FILE -AgentName 'Kilo Code')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $AUGGIE_FILE) { if (-not (Update-AgentFile -TargetFile $AUGGIE_FILE -AgentName 'Auggie CLI')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $ROO_FILE) { if (-not (Update-AgentFile -TargetFile $ROO_FILE -AgentName 'Roo Code')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $CODEBUDDY_FILE) { if (-not (Update-AgentFile -TargetFile $CODEBUDDY_FILE -AgentName 'CodeBuddy CLI')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $QODER_FILE) { if (-not (Update-AgentFile -TargetFile $QODER_FILE -AgentName 'Qoder CLI')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $SHAI_FILE) { if (-not (Update-AgentFile -TargetFile $SHAI_FILE -AgentName 'SHAI')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $Q_FILE) { if (-not (Update-AgentFile -TargetFile $Q_FILE -AgentName 'Amazon Q Developer CLI')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $AGY_FILE) { if (-not (Update-AgentFile -TargetFile $AGY_FILE -AgentName 'Antigravity')) { $ok = $false }; $found = $true }
|
|
||||||
if (Test-Path $BOB_FILE) { if (-not (Update-AgentFile -TargetFile $BOB_FILE -AgentName 'IBM Bob')) { $ok = $false }; $found = $true }
|
|
||||||
if (-not $found) {
|
|
||||||
Write-Info 'No existing agent files found, creating default Claude file...'
|
|
||||||
if (-not (Update-AgentFile -TargetFile $CLAUDE_FILE -AgentName 'Claude Code')) { $ok = $false }
|
|
||||||
}
|
|
||||||
return $ok
|
|
||||||
}
|
|
||||||
|
|
||||||
function Print-Summary {
|
|
||||||
Write-Host ''
|
|
||||||
Write-Info 'Summary of changes:'
|
|
||||||
if ($NEW_LANG) { Write-Host " - Added language: $NEW_LANG" }
|
|
||||||
if ($NEW_FRAMEWORK) { Write-Host " - Added framework: $NEW_FRAMEWORK" }
|
|
||||||
if ($NEW_DB -and $NEW_DB -ne 'N/A') { Write-Host " - Added database: $NEW_DB" }
|
|
||||||
Write-Host ''
|
|
||||||
Write-Info 'Usage: ./update-agent-context.ps1 [-AgentType claude|gemini|copilot|cursor-agent|qwen|opencode|codex|windsurf|kilocode|auggie|roo|codebuddy|amp|shai|q|agy|bob|qodercli|generic]'
|
|
||||||
}
|
|
||||||
|
|
||||||
function Main {
|
|
||||||
Validate-Environment
|
|
||||||
Write-Info "=== Updating agent context files for feature $CURRENT_BRANCH ==="
|
|
||||||
if (-not (Parse-PlanData -PlanFile $NEW_PLAN)) { Write-Err 'Failed to parse plan data'; exit 1 }
|
|
||||||
$success = $true
|
|
||||||
if ($AgentType) {
|
|
||||||
Write-Info "Updating specific agent: $AgentType"
|
|
||||||
if (-not (Update-SpecificAgent -Type $AgentType)) { $success = $false }
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
Write-Info 'No agent specified, updating all existing agent files...'
|
|
||||||
if (-not (Update-AllExistingAgents)) { $success = $false }
|
|
||||||
}
|
|
||||||
Print-Summary
|
|
||||||
if ($success) { Write-Success 'Agent context update completed successfully'; exit 0 } else { Write-Err 'Agent context update completed with errors'; exit 1 }
|
|
||||||
}
|
|
||||||
|
|
||||||
Main
|
|
||||||
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
# [PROJECT NAME] Development Guidelines
|
|
||||||
|
|
||||||
Auto-generated from all feature plans. Last updated: [DATE]
|
|
||||||
|
|
||||||
## Active Technologies
|
|
||||||
|
|
||||||
[EXTRACTED FROM ALL PLAN.MD FILES]
|
|
||||||
|
|
||||||
## Project Structure
|
|
||||||
|
|
||||||
```text
|
|
||||||
[ACTUAL STRUCTURE FROM PLANS]
|
|
||||||
```
|
|
||||||
|
|
||||||
## Commands
|
|
||||||
|
|
||||||
[ONLY COMMANDS FOR ACTIVE TECHNOLOGIES]
|
|
||||||
|
|
||||||
## Code Style
|
|
||||||
|
|
||||||
[LANGUAGE-SPECIFIC, ONLY FOR LANGUAGES IN USE]
|
|
||||||
|
|
||||||
## Recent Changes
|
|
||||||
|
|
||||||
[LAST 3 FEATURES AND WHAT THEY ADDED]
|
|
||||||
|
|
||||||
<!-- MANUAL ADDITIONS START -->
|
|
||||||
<!-- MANUAL ADDITIONS END -->
|
|
||||||
@@ -1,40 +0,0 @@
|
|||||||
# [CHECKLIST TYPE] Checklist: [FEATURE NAME]
|
|
||||||
|
|
||||||
**Purpose**: [Brief description of what this checklist covers]
|
|
||||||
**Created**: [DATE]
|
|
||||||
**Feature**: [Link to spec.md or relevant documentation]
|
|
||||||
|
|
||||||
**Note**: This checklist is generated by the `/speckit.checklist` command based on feature context and requirements.
|
|
||||||
|
|
||||||
<!--
|
|
||||||
============================================================================
|
|
||||||
IMPORTANT: The checklist items below are SAMPLE ITEMS for illustration only.
|
|
||||||
|
|
||||||
The /speckit.checklist command MUST replace these with actual items based on:
|
|
||||||
- User's specific checklist request
|
|
||||||
- Feature requirements from spec.md
|
|
||||||
- Technical context from plan.md
|
|
||||||
- Implementation details from tasks.md
|
|
||||||
|
|
||||||
DO NOT keep these sample items in the generated checklist file.
|
|
||||||
============================================================================
|
|
||||||
-->
|
|
||||||
|
|
||||||
## [Category 1]
|
|
||||||
|
|
||||||
- [ ] CHK001 First checklist item with clear action
|
|
||||||
- [ ] CHK002 Second checklist item
|
|
||||||
- [ ] CHK003 Third checklist item
|
|
||||||
|
|
||||||
## [Category 2]
|
|
||||||
|
|
||||||
- [ ] CHK004 Another category item
|
|
||||||
- [ ] CHK005 Item with specific criteria
|
|
||||||
- [ ] CHK006 Final item in this category
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- Check items off as completed: `[x]`
|
|
||||||
- Add comments or findings inline
|
|
||||||
- Link to relevant resources or documentation
|
|
||||||
- Items are numbered sequentially for easy reference
|
|
||||||
@@ -1,50 +0,0 @@
|
|||||||
# [PROJECT_NAME] Constitution
|
|
||||||
<!-- Example: Spec Constitution, TaskFlow Constitution, etc. -->
|
|
||||||
|
|
||||||
## Core Principles
|
|
||||||
|
|
||||||
### [PRINCIPLE_1_NAME]
|
|
||||||
<!-- Example: I. Library-First -->
|
|
||||||
[PRINCIPLE_1_DESCRIPTION]
|
|
||||||
<!-- Example: Every feature starts as a standalone library; Libraries must be self-contained, independently testable, documented; Clear purpose required - no organizational-only libraries -->
|
|
||||||
|
|
||||||
### [PRINCIPLE_2_NAME]
|
|
||||||
<!-- Example: II. CLI Interface -->
|
|
||||||
[PRINCIPLE_2_DESCRIPTION]
|
|
||||||
<!-- Example: Every library exposes functionality via CLI; Text in/out protocol: stdin/args → stdout, errors → stderr; Support JSON + human-readable formats -->
|
|
||||||
|
|
||||||
### [PRINCIPLE_3_NAME]
|
|
||||||
<!-- Example: III. Test-First (NON-NEGOTIABLE) -->
|
|
||||||
[PRINCIPLE_3_DESCRIPTION]
|
|
||||||
<!-- Example: TDD mandatory: Tests written → User approved → Tests fail → Then implement; Red-Green-Refactor cycle strictly enforced -->
|
|
||||||
|
|
||||||
### [PRINCIPLE_4_NAME]
|
|
||||||
<!-- Example: IV. Integration Testing -->
|
|
||||||
[PRINCIPLE_4_DESCRIPTION]
|
|
||||||
<!-- Example: Focus areas requiring integration tests: New library contract tests, Contract changes, Inter-service communication, Shared schemas -->
|
|
||||||
|
|
||||||
### [PRINCIPLE_5_NAME]
|
|
||||||
<!-- Example: V. Observability, VI. Versioning & Breaking Changes, VII. Simplicity -->
|
|
||||||
[PRINCIPLE_5_DESCRIPTION]
|
|
||||||
<!-- Example: Text I/O ensures debuggability; Structured logging required; Or: MAJOR.MINOR.BUILD format; Or: Start simple, YAGNI principles -->
|
|
||||||
|
|
||||||
## [SECTION_2_NAME]
|
|
||||||
<!-- Example: Additional Constraints, Security Requirements, Performance Standards, etc. -->
|
|
||||||
|
|
||||||
[SECTION_2_CONTENT]
|
|
||||||
<!-- Example: Technology stack requirements, compliance standards, deployment policies, etc. -->
|
|
||||||
|
|
||||||
## [SECTION_3_NAME]
|
|
||||||
<!-- Example: Development Workflow, Review Process, Quality Gates, etc. -->
|
|
||||||
|
|
||||||
[SECTION_3_CONTENT]
|
|
||||||
<!-- Example: Code review requirements, testing gates, deployment approval process, etc. -->
|
|
||||||
|
|
||||||
## Governance
|
|
||||||
<!-- Example: Constitution supersedes all other practices; Amendments require documentation, approval, migration plan -->
|
|
||||||
|
|
||||||
[GOVERNANCE_RULES]
|
|
||||||
<!-- Example: All PRs/reviews must verify compliance; Complexity must be justified; Use [GUIDANCE_FILE] for runtime development guidance -->
|
|
||||||
|
|
||||||
**Version**: [CONSTITUTION_VERSION] | **Ratified**: [RATIFICATION_DATE] | **Last Amended**: [LAST_AMENDED_DATE]
|
|
||||||
<!-- Example: Version: 2.1.1 | Ratified: 2025-06-13 | Last Amended: 2025-07-16 -->
|
|
||||||
@@ -1,104 +0,0 @@
|
|||||||
# Implementation Plan: [FEATURE]
|
|
||||||
|
|
||||||
**Branch**: `[###-feature-name]` | **Date**: [DATE] | **Spec**: [link]
|
|
||||||
**Input**: Feature specification from `/specs/[###-feature-name]/spec.md`
|
|
||||||
|
|
||||||
**Note**: This template is filled in by the `/speckit.plan` command. See `.specify/templates/plan-template.md` for the execution workflow.
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
[Extract from feature spec: primary requirement + technical approach from research]
|
|
||||||
|
|
||||||
## Technical Context
|
|
||||||
|
|
||||||
<!--
|
|
||||||
ACTION REQUIRED: Replace the content in this section with the technical details
|
|
||||||
for the project. The structure here is presented in advisory capacity to guide
|
|
||||||
the iteration process.
|
|
||||||
-->
|
|
||||||
|
|
||||||
**Language/Version**: [e.g., Python 3.11, Swift 5.9, Rust 1.75 or NEEDS CLARIFICATION]
|
|
||||||
**Primary Dependencies**: [e.g., FastAPI, UIKit, LLVM or NEEDS CLARIFICATION]
|
|
||||||
**Storage**: [if applicable, e.g., PostgreSQL, CoreData, files or N/A]
|
|
||||||
**Testing**: [e.g., pytest, XCTest, cargo test or NEEDS CLARIFICATION]
|
|
||||||
**Target Platform**: [e.g., Linux server, iOS 15+, WASM or NEEDS CLARIFICATION]
|
|
||||||
**Project Type**: [e.g., library/cli/web-service/mobile-app/compiler/desktop-app or NEEDS CLARIFICATION]
|
|
||||||
**Performance Goals**: [domain-specific, e.g., 1000 req/s, 10k lines/sec, 60 fps or NEEDS CLARIFICATION]
|
|
||||||
**Constraints**: [domain-specific, e.g., <200ms p95, <100MB memory, offline-capable or NEEDS CLARIFICATION]
|
|
||||||
**Scale/Scope**: [domain-specific, e.g., 10k users, 1M LOC, 50 screens or NEEDS CLARIFICATION]
|
|
||||||
|
|
||||||
## Constitution Check
|
|
||||||
|
|
||||||
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
|
|
||||||
|
|
||||||
[Gates determined based on constitution file]
|
|
||||||
|
|
||||||
## Project Structure
|
|
||||||
|
|
||||||
### Documentation (this feature)
|
|
||||||
|
|
||||||
```text
|
|
||||||
specs/[###-feature]/
|
|
||||||
├── plan.md # This file (/speckit.plan command output)
|
|
||||||
├── research.md # Phase 0 output (/speckit.plan command)
|
|
||||||
├── data-model.md # Phase 1 output (/speckit.plan command)
|
|
||||||
├── quickstart.md # Phase 1 output (/speckit.plan command)
|
|
||||||
├── contracts/ # Phase 1 output (/speckit.plan command)
|
|
||||||
└── tasks.md # Phase 2 output (/speckit.tasks command - NOT created by /speckit.plan)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Source Code (repository root)
|
|
||||||
<!--
|
|
||||||
ACTION REQUIRED: Replace the placeholder tree below with the concrete layout
|
|
||||||
for this feature. Delete unused options and expand the chosen structure with
|
|
||||||
real paths (e.g., apps/admin, packages/something). The delivered plan must
|
|
||||||
not include Option labels.
|
|
||||||
-->
|
|
||||||
|
|
||||||
```text
|
|
||||||
# [REMOVE IF UNUSED] Option 1: Single project (DEFAULT)
|
|
||||||
src/
|
|
||||||
├── models/
|
|
||||||
├── services/
|
|
||||||
├── cli/
|
|
||||||
└── lib/
|
|
||||||
|
|
||||||
tests/
|
|
||||||
├── contract/
|
|
||||||
├── integration/
|
|
||||||
└── unit/
|
|
||||||
|
|
||||||
# [REMOVE IF UNUSED] Option 2: Web application (when "frontend" + "backend" detected)
|
|
||||||
backend/
|
|
||||||
├── src/
|
|
||||||
│ ├── models/
|
|
||||||
│ ├── services/
|
|
||||||
│ └── api/
|
|
||||||
└── tests/
|
|
||||||
|
|
||||||
frontend/
|
|
||||||
├── src/
|
|
||||||
│ ├── components/
|
|
||||||
│ ├── pages/
|
|
||||||
│ └── services/
|
|
||||||
└── tests/
|
|
||||||
|
|
||||||
# [REMOVE IF UNUSED] Option 3: Mobile + API (when "iOS/Android" detected)
|
|
||||||
api/
|
|
||||||
└── [same as backend above]
|
|
||||||
|
|
||||||
ios/ or android/
|
|
||||||
└── [platform-specific structure: feature modules, UI flows, platform tests]
|
|
||||||
```
|
|
||||||
|
|
||||||
**Structure Decision**: [Document the selected structure and reference the real
|
|
||||||
directories captured above]
|
|
||||||
|
|
||||||
## Complexity Tracking
|
|
||||||
|
|
||||||
> **Fill ONLY if Constitution Check has violations that must be justified**
|
|
||||||
|
|
||||||
| Violation | Why Needed | Simpler Alternative Rejected Because |
|
|
||||||
|-----------|------------|-------------------------------------|
|
|
||||||
| [e.g., 4th project] | [current need] | [why 3 projects insufficient] |
|
|
||||||
| [e.g., Repository pattern] | [specific problem] | [why direct DB access insufficient] |
|
|
||||||
@@ -1,115 +0,0 @@
|
|||||||
# Feature Specification: [FEATURE NAME]
|
|
||||||
|
|
||||||
**Feature Branch**: `[###-feature-name]`
|
|
||||||
**Created**: [DATE]
|
|
||||||
**Status**: Draft
|
|
||||||
**Input**: User description: "$ARGUMENTS"
|
|
||||||
|
|
||||||
## User Scenarios & Testing *(mandatory)*
|
|
||||||
|
|
||||||
<!--
|
|
||||||
IMPORTANT: User stories should be PRIORITIZED as user journeys ordered by importance.
|
|
||||||
Each user story/journey must be INDEPENDENTLY TESTABLE - meaning if you implement just ONE of them,
|
|
||||||
you should still have a viable MVP (Minimum Viable Product) that delivers value.
|
|
||||||
|
|
||||||
Assign priorities (P1, P2, P3, etc.) to each story, where P1 is the most critical.
|
|
||||||
Think of each story as a standalone slice of functionality that can be:
|
|
||||||
- Developed independently
|
|
||||||
- Tested independently
|
|
||||||
- Deployed independently
|
|
||||||
- Demonstrated to users independently
|
|
||||||
-->
|
|
||||||
|
|
||||||
### User Story 1 - [Brief Title] (Priority: P1)
|
|
||||||
|
|
||||||
[Describe this user journey in plain language]
|
|
||||||
|
|
||||||
**Why this priority**: [Explain the value and why it has this priority level]
|
|
||||||
|
|
||||||
**Independent Test**: [Describe how this can be tested independently - e.g., "Can be fully tested by [specific action] and delivers [specific value]"]
|
|
||||||
|
|
||||||
**Acceptance Scenarios**:
|
|
||||||
|
|
||||||
1. **Given** [initial state], **When** [action], **Then** [expected outcome]
|
|
||||||
2. **Given** [initial state], **When** [action], **Then** [expected outcome]
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### User Story 2 - [Brief Title] (Priority: P2)
|
|
||||||
|
|
||||||
[Describe this user journey in plain language]
|
|
||||||
|
|
||||||
**Why this priority**: [Explain the value and why it has this priority level]
|
|
||||||
|
|
||||||
**Independent Test**: [Describe how this can be tested independently]
|
|
||||||
|
|
||||||
**Acceptance Scenarios**:
|
|
||||||
|
|
||||||
1. **Given** [initial state], **When** [action], **Then** [expected outcome]
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### User Story 3 - [Brief Title] (Priority: P3)
|
|
||||||
|
|
||||||
[Describe this user journey in plain language]
|
|
||||||
|
|
||||||
**Why this priority**: [Explain the value and why it has this priority level]
|
|
||||||
|
|
||||||
**Independent Test**: [Describe how this can be tested independently]
|
|
||||||
|
|
||||||
**Acceptance Scenarios**:
|
|
||||||
|
|
||||||
1. **Given** [initial state], **When** [action], **Then** [expected outcome]
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
[Add more user stories as needed, each with an assigned priority]
|
|
||||||
|
|
||||||
### Edge Cases
|
|
||||||
|
|
||||||
<!--
|
|
||||||
ACTION REQUIRED: The content in this section represents placeholders.
|
|
||||||
Fill them out with the right edge cases.
|
|
||||||
-->
|
|
||||||
|
|
||||||
- What happens when [boundary condition]?
|
|
||||||
- How does system handle [error scenario]?
|
|
||||||
|
|
||||||
## Requirements *(mandatory)*
|
|
||||||
|
|
||||||
<!--
|
|
||||||
ACTION REQUIRED: The content in this section represents placeholders.
|
|
||||||
Fill them out with the right functional requirements.
|
|
||||||
-->
|
|
||||||
|
|
||||||
### Functional Requirements
|
|
||||||
|
|
||||||
- **FR-001**: System MUST [specific capability, e.g., "allow users to create accounts"]
|
|
||||||
- **FR-002**: System MUST [specific capability, e.g., "validate email addresses"]
|
|
||||||
- **FR-003**: Users MUST be able to [key interaction, e.g., "reset their password"]
|
|
||||||
- **FR-004**: System MUST [data requirement, e.g., "persist user preferences"]
|
|
||||||
- **FR-005**: System MUST [behavior, e.g., "log all security events"]
|
|
||||||
|
|
||||||
*Example of marking unclear requirements:*
|
|
||||||
|
|
||||||
- **FR-006**: System MUST authenticate users via [NEEDS CLARIFICATION: auth method not specified - email/password, SSO, OAuth?]
|
|
||||||
- **FR-007**: System MUST retain user data for [NEEDS CLARIFICATION: retention period not specified]
|
|
||||||
|
|
||||||
### Key Entities *(include if feature involves data)*
|
|
||||||
|
|
||||||
- **[Entity 1]**: [What it represents, key attributes without implementation]
|
|
||||||
- **[Entity 2]**: [What it represents, relationships to other entities]
|
|
||||||
|
|
||||||
## Success Criteria *(mandatory)*
|
|
||||||
|
|
||||||
<!--
|
|
||||||
ACTION REQUIRED: Define measurable success criteria.
|
|
||||||
These must be technology-agnostic and measurable.
|
|
||||||
-->
|
|
||||||
|
|
||||||
### Measurable Outcomes
|
|
||||||
|
|
||||||
- **SC-001**: [Measurable metric, e.g., "Users can complete account creation in under 2 minutes"]
|
|
||||||
- **SC-002**: [Measurable metric, e.g., "System handles 1000 concurrent users without degradation"]
|
|
||||||
- **SC-003**: [User satisfaction metric, e.g., "90% of users successfully complete primary task on first attempt"]
|
|
||||||
- **SC-004**: [Business metric, e.g., "Reduce support tickets related to [X] by 50%"]
|
|
||||||
@@ -1,251 +0,0 @@
|
|||||||
---
|
|
||||||
|
|
||||||
description: "Task list template for feature implementation"
|
|
||||||
---
|
|
||||||
|
|
||||||
# Tasks: [FEATURE NAME]
|
|
||||||
|
|
||||||
**Input**: Design documents from `/specs/[###-feature-name]/`
|
|
||||||
**Prerequisites**: plan.md (required), spec.md (required for user stories), research.md, data-model.md, contracts/
|
|
||||||
|
|
||||||
**Tests**: The examples below include test tasks. Tests are OPTIONAL - only include them if explicitly requested in the feature specification.
|
|
||||||
|
|
||||||
**Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story.
|
|
||||||
|
|
||||||
## Format: `[ID] [P?] [Story] Description`
|
|
||||||
|
|
||||||
- **[P]**: Can run in parallel (different files, no dependencies)
|
|
||||||
- **[Story]**: Which user story this task belongs to (e.g., US1, US2, US3)
|
|
||||||
- Include exact file paths in descriptions
|
|
||||||
|
|
||||||
## Path Conventions
|
|
||||||
|
|
||||||
- **Single project**: `src/`, `tests/` at repository root
|
|
||||||
- **Web app**: `backend/src/`, `frontend/src/`
|
|
||||||
- **Mobile**: `api/src/`, `ios/src/` or `android/src/`
|
|
||||||
- Paths shown below assume single project - adjust based on plan.md structure
|
|
||||||
|
|
||||||
<!--
|
|
||||||
============================================================================
|
|
||||||
IMPORTANT: The tasks below are SAMPLE TASKS for illustration purposes only.
|
|
||||||
|
|
||||||
The /speckit.tasks command MUST replace these with actual tasks based on:
|
|
||||||
- User stories from spec.md (with their priorities P1, P2, P3...)
|
|
||||||
- Feature requirements from plan.md
|
|
||||||
- Entities from data-model.md
|
|
||||||
- Endpoints from contracts/
|
|
||||||
|
|
||||||
Tasks MUST be organized by user story so each story can be:
|
|
||||||
- Implemented independently
|
|
||||||
- Tested independently
|
|
||||||
- Delivered as an MVP increment
|
|
||||||
|
|
||||||
DO NOT keep these sample tasks in the generated tasks.md file.
|
|
||||||
============================================================================
|
|
||||||
-->
|
|
||||||
|
|
||||||
## Phase 1: Setup (Shared Infrastructure)
|
|
||||||
|
|
||||||
**Purpose**: Project initialization and basic structure
|
|
||||||
|
|
||||||
- [ ] T001 Create project structure per implementation plan
|
|
||||||
- [ ] T002 Initialize [language] project with [framework] dependencies
|
|
||||||
- [ ] T003 [P] Configure linting and formatting tools
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 2: Foundational (Blocking Prerequisites)
|
|
||||||
|
|
||||||
**Purpose**: Core infrastructure that MUST be complete before ANY user story can be implemented
|
|
||||||
|
|
||||||
**⚠️ CRITICAL**: No user story work can begin until this phase is complete
|
|
||||||
|
|
||||||
Examples of foundational tasks (adjust based on your project):
|
|
||||||
|
|
||||||
- [ ] T004 Setup database schema and migrations framework
|
|
||||||
- [ ] T005 [P] Implement authentication/authorization framework
|
|
||||||
- [ ] T006 [P] Setup API routing and middleware structure
|
|
||||||
- [ ] T007 Create base models/entities that all stories depend on
|
|
||||||
- [ ] T008 Configure error handling and logging infrastructure
|
|
||||||
- [ ] T009 Setup environment configuration management
|
|
||||||
|
|
||||||
**Checkpoint**: Foundation ready - user story implementation can now begin in parallel
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 3: User Story 1 - [Title] (Priority: P1) 🎯 MVP
|
|
||||||
|
|
||||||
**Goal**: [Brief description of what this story delivers]
|
|
||||||
|
|
||||||
**Independent Test**: [How to verify this story works on its own]
|
|
||||||
|
|
||||||
### Tests for User Story 1 (OPTIONAL - only if tests requested) ⚠️
|
|
||||||
|
|
||||||
> **NOTE: Write these tests FIRST, ensure they FAIL before implementation**
|
|
||||||
|
|
||||||
- [ ] T010 [P] [US1] Contract test for [endpoint] in tests/contract/test_[name].py
|
|
||||||
- [ ] T011 [P] [US1] Integration test for [user journey] in tests/integration/test_[name].py
|
|
||||||
|
|
||||||
### Implementation for User Story 1
|
|
||||||
|
|
||||||
- [ ] T012 [P] [US1] Create [Entity1] model in src/models/[entity1].py
|
|
||||||
- [ ] T013 [P] [US1] Create [Entity2] model in src/models/[entity2].py
|
|
||||||
- [ ] T014 [US1] Implement [Service] in src/services/[service].py (depends on T012, T013)
|
|
||||||
- [ ] T015 [US1] Implement [endpoint/feature] in src/[location]/[file].py
|
|
||||||
- [ ] T016 [US1] Add validation and error handling
|
|
||||||
- [ ] T017 [US1] Add logging for user story 1 operations
|
|
||||||
|
|
||||||
**Checkpoint**: At this point, User Story 1 should be fully functional and testable independently
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 4: User Story 2 - [Title] (Priority: P2)
|
|
||||||
|
|
||||||
**Goal**: [Brief description of what this story delivers]
|
|
||||||
|
|
||||||
**Independent Test**: [How to verify this story works on its own]
|
|
||||||
|
|
||||||
### Tests for User Story 2 (OPTIONAL - only if tests requested) ⚠️
|
|
||||||
|
|
||||||
- [ ] T018 [P] [US2] Contract test for [endpoint] in tests/contract/test_[name].py
|
|
||||||
- [ ] T019 [P] [US2] Integration test for [user journey] in tests/integration/test_[name].py
|
|
||||||
|
|
||||||
### Implementation for User Story 2
|
|
||||||
|
|
||||||
- [ ] T020 [P] [US2] Create [Entity] model in src/models/[entity].py
|
|
||||||
- [ ] T021 [US2] Implement [Service] in src/services/[service].py
|
|
||||||
- [ ] T022 [US2] Implement [endpoint/feature] in src/[location]/[file].py
|
|
||||||
- [ ] T023 [US2] Integrate with User Story 1 components (if needed)
|
|
||||||
|
|
||||||
**Checkpoint**: At this point, User Stories 1 AND 2 should both work independently
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase 5: User Story 3 - [Title] (Priority: P3)
|
|
||||||
|
|
||||||
**Goal**: [Brief description of what this story delivers]
|
|
||||||
|
|
||||||
**Independent Test**: [How to verify this story works on its own]
|
|
||||||
|
|
||||||
### Tests for User Story 3 (OPTIONAL - only if tests requested) ⚠️
|
|
||||||
|
|
||||||
- [ ] T024 [P] [US3] Contract test for [endpoint] in tests/contract/test_[name].py
|
|
||||||
- [ ] T025 [P] [US3] Integration test for [user journey] in tests/integration/test_[name].py
|
|
||||||
|
|
||||||
### Implementation for User Story 3
|
|
||||||
|
|
||||||
- [ ] T026 [P] [US3] Create [Entity] model in src/models/[entity].py
|
|
||||||
- [ ] T027 [US3] Implement [Service] in src/services/[service].py
|
|
||||||
- [ ] T028 [US3] Implement [endpoint/feature] in src/[location]/[file].py
|
|
||||||
|
|
||||||
**Checkpoint**: All user stories should now be independently functional
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
[Add more user story phases as needed, following the same pattern]
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Phase N: Polish & Cross-Cutting Concerns
|
|
||||||
|
|
||||||
**Purpose**: Improvements that affect multiple user stories
|
|
||||||
|
|
||||||
- [ ] TXXX [P] Documentation updates in docs/
|
|
||||||
- [ ] TXXX Code cleanup and refactoring
|
|
||||||
- [ ] TXXX Performance optimization across all stories
|
|
||||||
- [ ] TXXX [P] Additional unit tests (if requested) in tests/unit/
|
|
||||||
- [ ] TXXX Security hardening
|
|
||||||
- [ ] TXXX Run quickstart.md validation
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Dependencies & Execution Order
|
|
||||||
|
|
||||||
### Phase Dependencies
|
|
||||||
|
|
||||||
- **Setup (Phase 1)**: No dependencies - can start immediately
|
|
||||||
- **Foundational (Phase 2)**: Depends on Setup completion - BLOCKS all user stories
|
|
||||||
- **User Stories (Phase 3+)**: All depend on Foundational phase completion
|
|
||||||
- User stories can then proceed in parallel (if staffed)
|
|
||||||
- Or sequentially in priority order (P1 → P2 → P3)
|
|
||||||
- **Polish (Final Phase)**: Depends on all desired user stories being complete
|
|
||||||
|
|
||||||
### User Story Dependencies
|
|
||||||
|
|
||||||
- **User Story 1 (P1)**: Can start after Foundational (Phase 2) - No dependencies on other stories
|
|
||||||
- **User Story 2 (P2)**: Can start after Foundational (Phase 2) - May integrate with US1 but should be independently testable
|
|
||||||
- **User Story 3 (P3)**: Can start after Foundational (Phase 2) - May integrate with US1/US2 but should be independently testable
|
|
||||||
|
|
||||||
### Within Each User Story
|
|
||||||
|
|
||||||
- Tests (if included) MUST be written and FAIL before implementation
|
|
||||||
- Models before services
|
|
||||||
- Services before endpoints
|
|
||||||
- Core implementation before integration
|
|
||||||
- Story complete before moving to next priority
|
|
||||||
|
|
||||||
### Parallel Opportunities
|
|
||||||
|
|
||||||
- All Setup tasks marked [P] can run in parallel
|
|
||||||
- All Foundational tasks marked [P] can run in parallel (within Phase 2)
|
|
||||||
- Once Foundational phase completes, all user stories can start in parallel (if team capacity allows)
|
|
||||||
- All tests for a user story marked [P] can run in parallel
|
|
||||||
- Models within a story marked [P] can run in parallel
|
|
||||||
- Different user stories can be worked on in parallel by different team members
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Parallel Example: User Story 1
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Launch all tests for User Story 1 together (if tests requested):
|
|
||||||
Task: "Contract test for [endpoint] in tests/contract/test_[name].py"
|
|
||||||
Task: "Integration test for [user journey] in tests/integration/test_[name].py"
|
|
||||||
|
|
||||||
# Launch all models for User Story 1 together:
|
|
||||||
Task: "Create [Entity1] model in src/models/[entity1].py"
|
|
||||||
Task: "Create [Entity2] model in src/models/[entity2].py"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Implementation Strategy
|
|
||||||
|
|
||||||
### MVP First (User Story 1 Only)
|
|
||||||
|
|
||||||
1. Complete Phase 1: Setup
|
|
||||||
2. Complete Phase 2: Foundational (CRITICAL - blocks all stories)
|
|
||||||
3. Complete Phase 3: User Story 1
|
|
||||||
4. **STOP and VALIDATE**: Test User Story 1 independently
|
|
||||||
5. Deploy/demo if ready
|
|
||||||
|
|
||||||
### Incremental Delivery
|
|
||||||
|
|
||||||
1. Complete Setup + Foundational → Foundation ready
|
|
||||||
2. Add User Story 1 → Test independently → Deploy/Demo (MVP!)
|
|
||||||
3. Add User Story 2 → Test independently → Deploy/Demo
|
|
||||||
4. Add User Story 3 → Test independently → Deploy/Demo
|
|
||||||
5. Each story adds value without breaking previous stories
|
|
||||||
|
|
||||||
### Parallel Team Strategy
|
|
||||||
|
|
||||||
With multiple developers:
|
|
||||||
|
|
||||||
1. Team completes Setup + Foundational together
|
|
||||||
2. Once Foundational is done:
|
|
||||||
- Developer A: User Story 1
|
|
||||||
- Developer B: User Story 2
|
|
||||||
- Developer C: User Story 3
|
|
||||||
3. Stories complete and integrate independently
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- [P] tasks = different files, no dependencies
|
|
||||||
- [Story] label maps task to specific user story for traceability
|
|
||||||
- Each user story should be independently completable and testable
|
|
||||||
- Verify tests fail before implementing
|
|
||||||
- Commit after each task or logical group
|
|
||||||
- Stop at any checkpoint to validate story independently
|
|
||||||
- Avoid: vague tasks, same file conflicts, cross-story dependencies that break independence
|
|
||||||
772
CLAUDE.md
772
CLAUDE.md
@@ -1,772 +0,0 @@
|
|||||||
# CLAUDE.md
|
|
||||||
|
|
||||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
||||||
|
|
||||||
## Project Overview
|
|
||||||
|
|
||||||
An Actix-web REST API for serving images and videos from a filesystem with automatic thumbnail generation, EXIF extraction, tag organization, and a memories feature for browsing photos by date. Uses SQLite/Diesel ORM for data persistence and ffmpeg for video processing.
|
|
||||||
|
|
||||||
## Development Commands
|
|
||||||
|
|
||||||
### Building & Running
|
|
||||||
```bash
|
|
||||||
# Build for development
|
|
||||||
cargo build
|
|
||||||
|
|
||||||
# Build for release (uses thin LTO optimization)
|
|
||||||
cargo build --release
|
|
||||||
|
|
||||||
# Run the server (requires .env file with DATABASE_URL, BASE_PATH, THUMBNAILS, VIDEO_PATH, BIND_URL, SECRET_KEY)
|
|
||||||
cargo run
|
|
||||||
|
|
||||||
# Run with specific log level
|
|
||||||
RUST_LOG=debug cargo run
|
|
||||||
```
|
|
||||||
|
|
||||||
### Testing
|
|
||||||
```bash
|
|
||||||
# Run all tests (requires BASE_PATH in .env)
|
|
||||||
cargo test
|
|
||||||
|
|
||||||
# Run specific test
|
|
||||||
cargo test test_name
|
|
||||||
|
|
||||||
# Run tests with output
|
|
||||||
cargo test -- --nocapture
|
|
||||||
```
|
|
||||||
|
|
||||||
### Database Migrations
|
|
||||||
```bash
|
|
||||||
# Install diesel CLI (one-time setup)
|
|
||||||
cargo install diesel_cli --no-default-features --features sqlite
|
|
||||||
|
|
||||||
# Create new migration
|
|
||||||
diesel migration generate migration_name
|
|
||||||
|
|
||||||
# Run migrations (also runs automatically on app startup)
|
|
||||||
diesel migration run
|
|
||||||
|
|
||||||
# Revert last migration
|
|
||||||
diesel migration revert
|
|
||||||
|
|
||||||
# Regenerate schema.rs after manual migration changes
|
|
||||||
diesel print-schema > src/database/schema.rs
|
|
||||||
```
|
|
||||||
|
|
||||||
### Code Quality
|
|
||||||
```bash
|
|
||||||
# Format code
|
|
||||||
cargo fmt
|
|
||||||
|
|
||||||
# Run clippy linter
|
|
||||||
cargo clippy
|
|
||||||
|
|
||||||
# Fix automatically fixable issues
|
|
||||||
cargo fix
|
|
||||||
```
|
|
||||||
|
|
||||||
### Utility Binaries
|
|
||||||
```bash
|
|
||||||
# Two-phase cleanup: resolve missing files and validate file types
|
|
||||||
cargo run --bin cleanup_files -- --base-path /path/to/media --database-url ./database.db
|
|
||||||
```
|
|
||||||
|
|
||||||
## Architecture Overview
|
|
||||||
|
|
||||||
### Core Components
|
|
||||||
|
|
||||||
**Layered Architecture:**
|
|
||||||
- **Startup wiring** (`main.rs`): only ~350 lines — env load, migrations, AppState, route registration, server bind. Background jobs are kicked off here but defined elsewhere.
|
|
||||||
- **HTTP Layer** (`handlers/{image,video,favorites}.rs`, `files.rs`, `tags.rs`, `faces.rs`, `memories.rs`, `ai/handlers.rs`): the route handlers, grouped by domain.
|
|
||||||
- **Background loops** (`watcher.rs`): the file-watcher tick (`watch_files`, `process_new_files`) and the orphaned-playlist cleanup (`cleanup_orphaned_playlists`). Per-tick drains are factored into `backfill.rs` (`backfill_unhashed_backlog`, `backfill_missing_date_taken`, `backfill_missing_content_hashes`, `process_face_backlog`, `build_face_candidates`).
|
|
||||||
- **Thumbnails** (`thumbnails.rs`): generation pipeline + the `IMAGE_GAUGE` / `VIDEO_GAUGE` Prometheus metrics.
|
|
||||||
- **Auth Layer** (`auth.rs`): JWT token validation, Claims extraction via FromRequest trait
|
|
||||||
- **Service Layer** (`files.rs`, `exif.rs`, `memories.rs`): Business logic for file operations and EXIF extraction
|
|
||||||
- **DAO Layer** (`database/mod.rs`): Trait-based data access (ExifDao, UserDao, FavoriteDao, TagDao)
|
|
||||||
- **Database Layer**: Diesel ORM with SQLite, schema in `database/schema.rs`
|
|
||||||
|
|
||||||
**Async Actor System (Actix):**
|
|
||||||
- `StreamActor`: Manages ffmpeg video processing lifecycle
|
|
||||||
- `VideoPlaylistManager`: Scans directories and queues videos
|
|
||||||
- `PlaylistGenerator`: Creates HLS playlists for video streaming
|
|
||||||
|
|
||||||
### Database Schema & Patterns
|
|
||||||
|
|
||||||
**Tables:**
|
|
||||||
- `users`: Authentication (id, username, password_hash)
|
|
||||||
- `favorites`: User-specific favorites (userid, path)
|
|
||||||
- `tags`: Custom labels with timestamps
|
|
||||||
- `tagged_photo`: Many-to-many photo-tag relationships
|
|
||||||
- `image_exif`: Rich metadata (file_path + 16 EXIF fields: camera, GPS, dates, exposure settings)
|
|
||||||
|
|
||||||
**DAO Pattern:**
|
|
||||||
All database access goes through trait-based DAOs (e.g., `ExifDao`, `SqliteExifDao`). Connection pooling uses `Arc<Mutex<SqliteConnection>>`. All DB operations are traced with OpenTelemetry in release builds.
|
|
||||||
|
|
||||||
**Key DAO Methods:**
|
|
||||||
- `store_exif()`, `get_exif()`, `get_exif_batch()`: EXIF CRUD operations
|
|
||||||
- `query_by_exif()`: Complex filtering by camera, GPS bounds, date ranges
|
|
||||||
- Batch operations minimize DB hits during file watching
|
|
||||||
|
|
||||||
### Multi-library data model
|
|
||||||
|
|
||||||
ImageApi supports more than one library (a library = a `(name, root_path)`
|
|
||||||
row in the `libraries` table that maps to a mounted directory tree). The
|
|
||||||
same bytes may exist under more than one library — typical case is an
|
|
||||||
"active" library plus an "archive" library that ingests files as they age
|
|
||||||
out — and the data model is designed so that derived data follows the
|
|
||||||
**bytes**, not the path, while user-managed data does the same.
|
|
||||||
|
|
||||||
**The principle.** A photo's identity is its `content_hash` (blake3, see
|
|
||||||
`src/content_hash.rs`). Anything we compute from or attach to a photo is
|
|
||||||
keyed on that hash so it survives:
|
|
||||||
- the same file appearing in a second library (backup / archive / mirror),
|
|
||||||
- the file moving between libraries (recent → archive handoff),
|
|
||||||
- the file moving within a library (re-organized rel_path),
|
|
||||||
- intra-library duplicates (same bytes at two paths).
|
|
||||||
|
|
||||||
**Table classification.** Three categories drive the keying decision:
|
|
||||||
|
|
||||||
| Category | Key | Rationale | Tables |
|
|
||||||
|---|---|---|---|
|
|
||||||
| Intrinsic to bytes | `content_hash` | Rerunning is wasted work (or LLM cost) | `face_detections` ✓, `image_exif` (target), `photo_insights` (target), `video_preview_clips` (target) |
|
|
||||||
| User intent about a photo | `content_hash` | "Tag this photo" means the bytes, not a path | `tagged_photo` (target), `favorites` (target) |
|
|
||||||
| Library administrative | `(library_id, rel_path)` | Tied to a specific filesystem location | `libraries`, `entity_photo_links`, the `rel_path` back-ref columns on hash-keyed tables |
|
|
||||||
|
|
||||||
✓ = already implemented this way. *(target)* = today still keyed on
|
|
||||||
`(library_id, rel_path)` and slated for migration. The migration adds a
|
|
||||||
nullable `content_hash` column, populates it from `image_exif` where
|
|
||||||
known, and read paths fall back to rel_path while the hash is null.
|
|
||||||
|
|
||||||
**Carrying a `rel_path` even when hash-keyed.** Hash-keyed tables retain
|
|
||||||
`(library_id, rel_path)` columns as a denormalized **back-reference**, not
|
|
||||||
as the key. This lets a single query answer "what is at this path right
|
|
||||||
now" without joining through `image_exif`, and supports the path-only
|
|
||||||
endpoints that predate the hash. `face_detections` is the reference
|
|
||||||
implementation: hash is the truth, path is a hint.
|
|
||||||
|
|
||||||
**Merge semantics on read.** When the same hash has rows under more than
|
|
||||||
one library:
|
|
||||||
- Set-valued data (tags, favorites, faces, entity links) → **union**.
|
|
||||||
- Scalar data (current insight, EXIF row, video preview clip) → earliest
|
|
||||||
`generated_at` / `created_time` wins. The historical lib1 row beats a
|
|
||||||
re-generated lib2 row, so the user's curated insight isn't shadowed by
|
|
||||||
a re-run on archive ingest.
|
|
||||||
|
|
||||||
**Write attribution.** A new tag/favorite/insight created while viewing
|
|
||||||
under lib2 binds to the bytes, not to lib2 — so it shows up under lib1
|
|
||||||
too. This is by design, but it's the most surprising rule on first
|
|
||||||
encounter; clients should not assume tags are library-scoped.
|
|
||||||
|
|
||||||
**Hash-less rows (transitional state).** During and immediately after a
|
|
||||||
new mount, `image_exif.content_hash` is being populated by
|
|
||||||
`backfill_unhashed_backlog` (capped per tick). Rules during this window:
|
|
||||||
- Writes: if the hash is known, write hash-keyed. If not, write
|
|
||||||
`(library_id, rel_path)`-keyed and let the reconciliation job collapse
|
|
||||||
duplicates once the hash lands.
|
|
||||||
- Reads: prefer hash key, fall back to `(library_id, rel_path)`.
|
|
||||||
- Reconciliation: a one-shot pass after every backfill tick collapses
|
|
||||||
rows that now share a hash, applying the merge semantics above.
|
|
||||||
Idempotent — safe to re-run.
|
|
||||||
|
|
||||||
**Library handoff (recent → archive).** When a file moves between
|
|
||||||
libraries (e.g. operator moves `~/photos/2024/IMG.nef` to the archive
|
|
||||||
mount), the file watcher sees the disappearance under lib1 and the
|
|
||||||
appearance under lib2. Hash-keyed rows don't need migration; the
|
|
||||||
`(library_id, rel_path)` back-ref columns are updated to point to the new
|
|
||||||
location. Library administrative rows (`entity_photo_links`,
|
|
||||||
`(library_id, rel_path)` rows in `image_exif` for hash-less items) are
|
|
||||||
re-keyed by the move detector, which matches a disappearance to an
|
|
||||||
appearance by `content_hash` within a configurable window.
|
|
||||||
|
|
||||||
**Orphans (source deleted while a copy survives).** When the only
|
|
||||||
`image_exif` row for a hash is deleted (file removed from disk), the
|
|
||||||
hash-keyed derived rows survive **as long as another `image_exif` row
|
|
||||||
references the same hash**. If the last reference is gone, derived rows
|
|
||||||
are eligible for GC (deferred — the GC job runs on a slow schedule so
|
|
||||||
that a brief unmount or rename doesn't wipe history).
|
|
||||||
|
|
||||||
**Stats and counts.** When reporting "how many photos do you have," count
|
|
||||||
`DISTINCT content_hash` over `image_exif`, not row count. Faces stats
|
|
||||||
already does this (`FaceDao::stats` in `src/faces.rs`); other counters
|
|
||||||
should follow suit. Numerator and denominator must live in the same
|
|
||||||
domain — see the face-stats commentary below for the cautionary tale.
|
|
||||||
|
|
||||||
**Per-library scoping when the user asks for it.** A request scoped to
|
|
||||||
`?library=N` filters the `image_exif` view to that library, and the
|
|
||||||
hash-keyed derived data is joined through that view. The user sees only
|
|
||||||
photos that have a copy under lib N, but the derived data attached to
|
|
||||||
those photos is the merged hash-keyed view. This is the answer to "show
|
|
||||||
me archive photos with their original tags."
|
|
||||||
|
|
||||||
**Operator kill switch (`libraries.enabled`).** Setting `enabled=0` on a
|
|
||||||
library is a hard pause: the watcher skips it entirely — before the
|
|
||||||
probe, before ingest, before any maintenance pass — and the orphan-GC
|
|
||||||
all-online consensus check filters disabled libraries out (they don't
|
|
||||||
keep the GC window closed). Reads / serving are unaffected; nothing
|
|
||||||
prevents `/image?path=...` from resolving against a disabled library's
|
|
||||||
root if the file is on disk. The existing `image_exif` rows for a
|
|
||||||
disabled library are **not deleted** — they continue to anchor
|
|
||||||
hash-keyed derived data, so cross-library duplicates survive the
|
|
||||||
disable. Toggle via SQL; there is intentionally no HTTP endpoint for
|
|
||||||
library mutation (single-user tool, no role / permission story).
|
|
||||||
Typical workflows: stage a new mount with `enabled=0` then flip to `1`;
|
|
||||||
quiet a flaky NAS during maintenance without disturbing the rest of
|
|
||||||
the system.
|
|
||||||
|
|
||||||
**Per-library excludes (`libraries.excluded_dirs`).** A
|
|
||||||
comma-separated column, same shape as the global `EXCLUDED_DIRS` env
|
|
||||||
var, that's applied **in union** with the env-var globals when a
|
|
||||||
walker scans this library. Use case: mount a parent directory as a
|
|
||||||
new library while a sibling library covers a child subtree, and
|
|
||||||
exclude that child subtree from the parent so the two libraries
|
|
||||||
don't double-walk and double-write `image_exif`. Two entry forms
|
|
||||||
(parsed by `memories::PathExcluder`):
|
|
||||||
- `/sub/path` — leading slash flags it as a path under the library
|
|
||||||
root. Joins to root + matches by `path.starts_with(...)`. Works
|
|
||||||
at any depth (`/photos`, `/media/2024/raw`).
|
|
||||||
- `name` — no leading slash flags it as a component name to skip
|
|
||||||
anywhere in the tree (`@eaDir`, `.thumbnails`). Single segment
|
|
||||||
only — `media/photos/a` without a leading slash never matches
|
|
||||||
anything. Hash-keyed derived
|
|
||||||
data (faces, tags, insights) is unaffected either way — those
|
|
||||||
follow the bytes — but `image_exif` row count, walker CPU, and
|
|
||||||
thumbnail disk usage all drop to 1× instead of 2× for the overlap.
|
|
||||||
Affects: file-watch ingest (`process_new_files`), thumbnail
|
|
||||||
generation, media-count gauges, the orphaned-playlist cleanup walk,
|
|
||||||
and the `/memories` endpoint. The face-detection backlog drain
|
|
||||||
inherits via `face_watch::filter_excluded`. NULL = no extras (only
|
|
||||||
the global env var applies).
|
|
||||||
|
|
||||||
**Library availability and safety.** Libraries can be on network shares
|
|
||||||
or removable media; the file watcher must not interpret a temporary
|
|
||||||
unavailability as a mass-deletion event. Every tick begins with a
|
|
||||||
**presence probe** per library: the library is considered online iff
|
|
||||||
its `root_path` exists, is readable, and a top-level scan returns at
|
|
||||||
least one expected entry (or matches a recent file-count high-water
|
|
||||||
mark within a tolerance). The probe result gates which actions are safe
|
|
||||||
to run on that library this tick:
|
|
||||||
|
|
||||||
| Action | Requires online? |
|
|
||||||
|---|---|
|
|
||||||
| Quick / full scan ingest of new files | yes |
|
|
||||||
| EXIF / face / insight backlog drains | yes — but the work runs against any online library |
|
|
||||||
| Move-handoff detection (lib1 disappearance ↔ lib2 appearance match) | **both** libraries online |
|
|
||||||
| `(library_id, rel_path)` re-keying on detected move | **both** libraries online |
|
|
||||||
| Orphan GC of hash-keyed derived data | all libraries that have *ever* held the hash must be online and confirmed-clean for two consecutive ticks |
|
|
||||||
| Reads / serving | always allowed; falls back to whichever library is online |
|
|
||||||
|
|
||||||
A library that fails the probe enters a "stale" state: writes scoped to
|
|
||||||
it are paused, its rows are flagged stale (not deleted) in
|
|
||||||
`/libraries` status, and the watcher logs at `warn` once per
|
|
||||||
state-transition (not per tick). A library that recovers re-enters the
|
|
||||||
online set automatically; no operator action required for transient
|
|
||||||
outages. The intent is that pulling a USB drive, rebooting a NAS, or
|
|
||||||
losing a VPN never triggers a destructive code path — the worst case is
|
|
||||||
that derived-data work pauses until the share returns.
|
|
||||||
|
|
||||||
The same rule constrains the move-handoff matcher: a disappearance
|
|
||||||
under lib1 only counts as a "move" if there is a matching appearance
|
|
||||||
under another **online** library within the window. A bare
|
|
||||||
disappearance with no matching appearance is treated as
|
|
||||||
"unavailable-or-deleted, defer judgment" — it does not re-key any rows
|
|
||||||
and does not enqueue GC.
|
|
||||||
|
|
||||||
**Maintenance pipeline (`src/library_maintenance.rs`).** The watcher
|
|
||||||
runs three maintenance passes per tick that together implement the
|
|
||||||
move/handoff and orphan rules:
|
|
||||||
|
|
||||||
1. **Missing-file scan** — per online library, paginated. A page of
|
|
||||||
`image_exif` rows is loaded (`IMAGE_EXIF_MISSING_SCAN_PAGE_SIZE`,
|
|
||||||
default 500), each row's `(root_path/rel_path)` is `stat()`-ed,
|
|
||||||
and confirmed-not-found rows are deleted from `image_exif`
|
|
||||||
(capped at `IMAGE_EXIF_MISSING_DELETE_CAP_PER_TICK`, default 200).
|
|
||||||
Permission/IO errors are skipped, never deleted — only `NotFound`
|
|
||||||
triggers a deletion. The cursor wraps every time a partial page
|
|
||||||
comes back, so the whole library is swept across consecutive ticks.
|
|
||||||
Skipped wholesale for Stale libraries via the per-library probe
|
|
||||||
gate at the top of the loop iteration.
|
|
||||||
|
|
||||||
2. **Back-ref refresh** — DB-only. For `face_detections`,
|
|
||||||
`tagged_photo`, and `photo_insights`: any hash-keyed row whose
|
|
||||||
`(library_id, rel_path)` no longer matches an `image_exif` row
|
|
||||||
*but whose `content_hash` does* is repointed at the surviving
|
|
||||||
`image_exif` location. Idempotent SQL; no health gate needed.
|
|
||||||
This is what makes the recent → archive handoff invisible to
|
|
||||||
read paths: when the missing-file scan retires the lib-A row,
|
|
||||||
tags/faces/insights pivot to lib-B's path before any user
|
|
||||||
notices.
|
|
||||||
|
|
||||||
3. **Orphan GC** — destructive. Hash-keyed derived rows whose
|
|
||||||
`content_hash` no longer has any `image_exif` row are eligible.
|
|
||||||
Two-tick consensus: a hash must be observed orphaned on two
|
|
||||||
consecutive ticks AND every library must be online for both. A
|
|
||||||
single Stale tick within the window cancels all pending deletes.
|
|
||||||
The pending set is held in memory (`OrphanGcState`) — restart
|
|
||||||
resets it, which only delays a delete, never causes one. Tags,
|
|
||||||
faces, and insights for orphaned hashes are deleted in one batch
|
|
||||||
per tick.
|
|
||||||
|
|
||||||
A backup library that briefly disappears, then returns within two
|
|
||||||
ticks, never loses any derived data. A move from lib-A to lib-B
|
|
||||||
without disappearance flips through pass 1 (lib-A row retired) and
|
|
||||||
pass 2 (back-refs follow), with pass 3 noting nothing because the
|
|
||||||
hash is still present in `image_exif` (lib-B's row).
|
|
||||||
|
|
||||||
**Known gap: in-place content changes (future Branch D).** The
|
|
||||||
maintenance pipeline assumes a `(library_id, rel_path)`'s bytes are
|
|
||||||
stable for as long as the file exists at that path. If a user edits
|
|
||||||
a file in place (crop, re-export) without renaming, the watcher's
|
|
||||||
quick scan walks the file (mtime is recent) but `process_new_files`
|
|
||||||
short-circuits because `(library_id, rel_path)` already has an
|
|
||||||
`image_exif` row — no re-hash, no re-EXIF, no face redetection. The
|
|
||||||
row's `content_hash` keeps pointing at the original bytes. Tags /
|
|
||||||
faces / insights stay attached to the original hash and continue to
|
|
||||||
display because the rel_path back-ref still resolves; new faces
|
|
||||||
introduced by the edit are never detected.
|
|
||||||
|
|
||||||
The right place to fix this is a **stale-content detection pass**
|
|
||||||
that compares `image_exif.last_modified` / `size_bytes` to
|
|
||||||
`fs::metadata` for rows the quick scan would otherwise skip. On
|
|
||||||
mismatch, recompute the hash, update `image_exif`, and apply the
|
|
||||||
"content branched" semantics:
|
|
||||||
- **Faces** re-run (faces are fully derived from bytes).
|
|
||||||
- **Tags** migrate to the new hash (user intent — "this photo is
|
|
||||||
vacation" survives a crop). Insights migrate forward as a
|
|
||||||
starting point and are flagged for re-generation.
|
|
||||||
- **Favorites** (when migrated to hash-keyed) follow the path /
|
|
||||||
user intent.
|
|
||||||
|
|
||||||
The interesting case is the operator who keeps an unedited copy in
|
|
||||||
the archive library and edits the local copy: post-detection, the
|
|
||||||
archive copy stays on the original hash, the local copy branches to
|
|
||||||
the new hash, and the two histories cleanly split. Apollo's
|
|
||||||
`derived.db` cache will need an invalidation hook for the changed
|
|
||||||
hash — design it alongside Branch D.
|
|
||||||
|
|
||||||
### File Processing Pipeline
|
|
||||||
|
|
||||||
**Thumbnail Generation:**
|
|
||||||
1. Startup scan: Rayon parallel walk of BASE_PATH
|
|
||||||
2. Creates 200x200 thumbnails in THUMBNAILS directory (mirrors source structure)
|
|
||||||
3. Videos: extracts frame at 3-second mark via ffmpeg
|
|
||||||
4. Images: uses `image` crate for JPEG/PNG processing
|
|
||||||
5. RAW formats (NEF/CR2/ARW/DNG/etc.): the `image` crate can't decode RAW
|
|
||||||
pixel data, so the pipeline pulls an embedded JPEG preview instead. Fast
|
|
||||||
path is `exif::read_jpeg_at_ifd` against IFD0 (PRIMARY) and IFD1
|
|
||||||
(THUMBNAIL) — covers most older bodies and DNGs. Slow-path fallback shells
|
|
||||||
out to **`exiftool`** for `PreviewImage` / `JpgFromRaw` / `OtherImage`,
|
|
||||||
which reaches MakerNote / SubIFD-hosted previews kamadak-exif can't see
|
|
||||||
(e.g. Nikon's `PreviewIFD`, where modern Nikon bodies store the full-res
|
|
||||||
review JPEG). All candidates are pooled and the largest valid JPEG wins.
|
|
||||||
See `src/exif.rs::extract_embedded_jpeg_preview`.
|
|
||||||
|
|
||||||
**File Watching:**
|
|
||||||
Runs in background thread with two-tier strategy:
|
|
||||||
- **Quick scan** (default 60s): Recently modified files only
|
|
||||||
- **Full scan** (default 3600s): Comprehensive directory check
|
|
||||||
- Batch queries EXIF DB to detect new files
|
|
||||||
- Configurable via `WATCH_QUICK_INTERVAL_SECONDS` and `WATCH_FULL_INTERVAL_SECONDS`
|
|
||||||
|
|
||||||
**Canonical date_taken pipeline (`src/date_resolver.rs`).** Every row's
|
|
||||||
`image_exif.date_taken` is populated at ingest by a four-step waterfall;
|
|
||||||
which step won is recorded in `image_exif.date_taken_source` so the
|
|
||||||
per-tick drain can re-resolve weak entries when better tools become
|
|
||||||
available, and so the UI/debug surface can answer "why did this photo
|
|
||||||
land on this date?". Order:
|
|
||||||
|
|
||||||
1. **`exif`** — kamadak-exif `DateTime` / `DateTimeOriginal`. Fast,
|
|
||||||
in-process, image-only.
|
|
||||||
2. **`exiftool`** — shell-out fallback for tags kamadak can't reach:
|
|
||||||
QuickTime/MP4 (`MediaCreateDate`, `TrackCreateDate`, `CreateDate`),
|
|
||||||
Apple's `ContentCreateDate`, MakerNote sub-IFDs. Required for
|
|
||||||
videos to land a real date. Single-file at ingest; the per-tick
|
|
||||||
drain feeds the whole batch through one `exiftool -@ -` subprocess.
|
|
||||||
Degrades silently when `exiftool` isn't on PATH (resolver caches the
|
|
||||||
"available" check via `OnceLock`).
|
|
||||||
3. **`filename`** — `extract_date_from_filename` in `memories.rs`
|
|
||||||
matches screenshot, chat-export, and timestamp-named patterns.
|
|
||||||
4. **`fs_time`** — `earliest_fs_time(metadata)` (earlier of created /
|
|
||||||
modified). Last resort.
|
|
||||||
|
|
||||||
Notable behavior change vs. the pre-2026-05 request-time logic:
|
|
||||||
**EXIF beats filename when both are present.** A photo named
|
|
||||||
`Screenshot_2014-06-01.png` whose EXIF `DateTime` is 2021 now appears
|
|
||||||
under 2021, not 2014 — on the theory that EXIF is more reliable than
|
|
||||||
import-named filenames. The reverse case (no EXIF, filename has a
|
|
||||||
date) is unchanged.
|
|
||||||
|
|
||||||
The `backfill_missing_date_taken` drain (`src/backfill.rs`) runs every
|
|
||||||
watcher tick alongside `backfill_unhashed_backlog` (also `src/backfill.rs`). It loads up to
|
|
||||||
`DATE_BACKFILL_MAX_PER_TICK` rows (default 500) where
|
|
||||||
`date_taken IS NULL` (backed by the `idx_image_exif_date_backfill`
|
|
||||||
partial index), runs the waterfall batch via `resolve_dates_batch`,
|
|
||||||
and writes results via the `backfill_date_taken` DAO method (touches
|
|
||||||
only `date_taken` + `date_taken_source` so EXIF / hash / perceptual
|
|
||||||
columns are preserved). Resolved rows — including the ones the
|
|
||||||
waterfall could only resolve via `fs_time` — are not re-eligible:
|
|
||||||
the resolver is deterministic on file bytes + filename + fs metadata,
|
|
||||||
so re-running on the same inputs lands on the same source every time.
|
|
||||||
An earlier version included `date_taken_source = 'fs_time'` in the
|
|
||||||
eligibility predicate, but with `ORDER BY id ASC LIMIT 500` it spun on
|
|
||||||
the same lowest-id rows in perpetuity and held the SQLite write lock
|
|
||||||
long enough to starve face-PATCH writers (5s busy_timeout → 500). If
|
|
||||||
a stronger tool comes online (exiftool install, new filename regex),
|
|
||||||
re-resolve out-of-band rather than re-introducing the steady-state
|
|
||||||
eligibility.
|
|
||||||
|
|
||||||
`/memories` is a single SQL query against this column
|
|
||||||
(`get_memories_in_window` in `src/database/mod.rs`), using
|
|
||||||
`strftime('%m-%d' | '%W' | '%m', date_taken, 'unixepoch', tz)` for
|
|
||||||
calendar matching with the client's timezone offset. The pre-rewrite
|
|
||||||
version stat'd every row and walked the entire library tree — at
|
|
||||||
~14k photos this took 10–15 s; the rewrite is single-digit ms.
|
|
||||||
|
|
||||||
**EXIF Extraction:**
|
|
||||||
- Uses `kamadak-exif` crate
|
|
||||||
- Supports: JPEG, TIFF, RAW (NEF, CR2, CR3), HEIF/HEIC, PNG, WebP
|
|
||||||
- Extracts: camera make/model, lens, dimensions, GPS coordinates, focal length, aperture, shutter speed, ISO, date taken
|
|
||||||
- Triggered on upload and during file watching
|
|
||||||
|
|
||||||
**File Upload Behavior:**
|
|
||||||
If file exists, appends timestamp to filename (`photo_1735124234.jpg`) to preserve history without overwrites.
|
|
||||||
|
|
||||||
### Authentication Flow
|
|
||||||
|
|
||||||
**Login:**
|
|
||||||
1. POST `/login` with username/password
|
|
||||||
2. Verify with `bcrypt::verify()` against password_hash
|
|
||||||
3. Generate JWT with claims: `{ sub: user_id, exp: 5_days_from_now }`
|
|
||||||
4. Sign with HS256 using `SECRET_KEY` environment variable
|
|
||||||
|
|
||||||
**Authorization:**
|
|
||||||
All protected endpoints extract `Claims` via `FromRequest` trait implementation. Token passed as `Authorization: Bearer <token>` header.
|
|
||||||
|
|
||||||
### API Structure
|
|
||||||
|
|
||||||
**Key Endpoint Patterns:**
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// Image serving & upload
|
|
||||||
GET /image?path=...&size=...&format=...
|
|
||||||
POST /image (multipart file upload)
|
|
||||||
|
|
||||||
// Metadata & EXIF
|
|
||||||
GET /image/metadata?path=...
|
|
||||||
|
|
||||||
// Advanced search with filters
|
|
||||||
GET /photos?path=...&recursive=true&sort=DateTakenDesc&camera_make=Canon&gps_lat=...&gps_lon=...&gps_radius_km=10&date_from=...&date_to=...&tag_ids=1,2,3&media_type=Photo
|
|
||||||
|
|
||||||
// Video streaming (HLS)
|
|
||||||
POST /video/generate (creates .m3u8 playlist + .ts segments)
|
|
||||||
GET /video/stream?path=... (serves playlist)
|
|
||||||
|
|
||||||
// Tags
|
|
||||||
GET /image/tags/all
|
|
||||||
POST /image/tags (add tag to file)
|
|
||||||
DELETE /image/tags (remove tag from file)
|
|
||||||
POST /image/tags/batch (bulk tag updates)
|
|
||||||
|
|
||||||
// Memories (week-based grouping)
|
|
||||||
GET /memories?path=...&recursive=true
|
|
||||||
|
|
||||||
// AI Insights
|
|
||||||
POST /insights/generate (non-agentic single-shot)
|
|
||||||
POST /insights/generate/agentic (tool-calling loop; body: { file_path, backend?, model?, ... })
|
|
||||||
GET /insights?path=...&library=...
|
|
||||||
GET /insights/models (local Ollama models + capabilities)
|
|
||||||
GET /insights/openrouter/models (curated OpenRouter allowlist)
|
|
||||||
POST /insights/rate (thumbs up/down for training data)
|
|
||||||
|
|
||||||
// Insight Chat Continuation
|
|
||||||
POST /insights/chat (single-turn reply, non-streaming)
|
|
||||||
POST /insights/chat/stream (SSE: text / tool_call / tool_result / truncated / done)
|
|
||||||
GET /insights/chat/history?path=... (rendered transcript with tool invocations)
|
|
||||||
POST /insights/chat/rewind (truncate transcript at a rendered index)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Request Types:**
|
|
||||||
- `FilesRequest`: Supports complex filtering (tags, EXIF fields, GPS radius, date ranges)
|
|
||||||
- `SortType`: Shuffle, NameAsc/Desc, TagCountAsc/Desc, DateTakenAsc/Desc
|
|
||||||
|
|
||||||
### Important Patterns
|
|
||||||
|
|
||||||
**Service Builder Pattern:**
|
|
||||||
Routes are registered via composable `ServiceBuilder` trait in `service.rs`. Allows modular feature addition.
|
|
||||||
|
|
||||||
**Path Validation:**
|
|
||||||
Always use `is_valid_full_path(&base_path, &requested_path, check_exists)` to prevent directory traversal attacks.
|
|
||||||
|
|
||||||
**File Type Detection:**
|
|
||||||
Centralized in `file_types.rs` with constants `IMAGE_EXTENSIONS` and `VIDEO_EXTENSIONS`. Provides both `Path` and `DirEntry` variants for performance.
|
|
||||||
|
|
||||||
**OpenTelemetry Tracing:**
|
|
||||||
All database operations and HTTP handlers wrapped in spans. In release builds, exports to OTLP endpoint via `OTLP_OTLS_ENDPOINT`. Debug builds use basic logger.
|
|
||||||
|
|
||||||
**Memory Exclusion:**
|
|
||||||
`PathExcluder` in `memories.rs` filters out directories from memories API via `EXCLUDED_DIRS` environment variable (comma-separated paths or substring patterns). The same excluder is applied to face-detection candidates (`face_watch::filter_excluded`) so junk directories like `@eaDir` / `.thumbnails` don't burn detect calls on Apollo.
|
|
||||||
|
|
||||||
### Face detection system
|
|
||||||
|
|
||||||
ImageApi owns the face data; Apollo (sibling repo) hosts the insightface inference service. Inference is triggered automatically by the file watcher and persisted into two tables:
|
|
||||||
|
|
||||||
- `persons(id, name UNIQUE COLLATE NOCASE, cover_face_id, entity_id, created_from_tag, notes, ...)` — operator-managed, name is the user-visible identity.
|
|
||||||
- `face_detections(id, library_id, content_hash, rel_path, bbox_*, embedding BLOB, confidence, source, person_id, status, model_version, ...)` — keyed on `content_hash` so a photo duplicated across libraries is detected once. Marker rows for `status IN ('no_faces','failed')` carry NULL bbox/embedding (CHECK constraint enforces this).
|
|
||||||
|
|
||||||
**Why content_hash and not (library_id, rel_path):** ties face data to the bytes, not the path. A backup mount that copies files from the primary library naturally inherits the existing detections without re-running inference. This is the reference implementation of the multi-library data model — see "Multi-library data model" above.
|
|
||||||
|
|
||||||
**File-watch hook** (`src/watcher.rs::process_new_files`): for each photo with a populated `content_hash`, check `FaceDao::already_scanned(hash)`; if not, send bytes (or embedded JPEG preview for RAW via `exif::extract_embedded_jpeg_preview`) to Apollo's `/api/internal/faces/detect`. K=`FACE_DETECT_CONCURRENCY` (default 8) parallel calls per scan tick; Apollo serializes them via its single-worker GPU pool. `face_watch.rs` is the Tokio orchestration layer.
|
|
||||||
|
|
||||||
**Per-tick backlog drain** (`src/backfill.rs`): two passes that run on every watcher tick regardless of quick-vs-full scan:
|
|
||||||
- `backfill_unhashed_backlog` — populates `image_exif.content_hash` for photos that arrived before the hash field was retroactive. Capped by `FACE_HASH_BACKFILL_MAX_PER_TICK` (default 2000); errors don't burn the cap.
|
|
||||||
- `process_face_backlog` — runs detection on photos that have a hash but no `face_detections` row. Capped by `FACE_BACKLOG_MAX_PER_TICK` (default 64). Selected via a SQL anti-join (`FaceDao::list_unscanned_candidates`); videos and EXCLUDED_DIRS paths filtered out client-side via `face_watch::filter_excluded` so they never reach Apollo.
|
|
||||||
|
|
||||||
**Auto-bind on detection:** when a photo carries a tag whose name matches a `persons.name` (case-insensitive), the new face binds automatically iff cosine similarity to the person's existing-face mean is ≥ `FACE_AUTOBIND_MIN_COS` (default 0.4). Persons with no existing faces bind unconditionally and the new face becomes the cover.
|
|
||||||
|
|
||||||
**Manual face create** (`POST /image/faces`): crops the image to the user-supplied bbox, applies EXIF orientation via `exif::apply_orientation` (the `image` crate hands raw pre-rotation pixels — without this, manually-drawn bboxes never resolved a face on re-detection), pads to ~50% of bbox dims (RetinaFace anchor scales need ~50% face-fill at det_size=640), then calls Apollo's embed endpoint. A `force` flag lets the operator save a face the detector couldn't see (e.g. profile shots, occluded faces) — the row gets a zero-vector embedding so it's manually-bound only and won't participate in clustering.
|
|
||||||
|
|
||||||
**Rerun preserves manual rows** (`POST /image/faces/{id}/rerun`): only `source='auto'` rows are deleted before re-running detection. `already_scanned` returns true on ANY row, so a photo whose only faces are manually drawn never auto-redetects.
|
|
||||||
|
|
||||||
**Stats domain — content_hash, not file rows** (`FaceDao::stats` in `src/faces.rs`): `total_photos` counts `DISTINCT content_hash` over `image_exif` (filtered to image extensions, `content_hash IS NOT NULL`), and so do `scanned` / `with_faces` / `no_faces` / `failed` over `face_detections`. Numerator and denominator must live in the same domain — `face_detections` is keyed on content_hash, so the same JPEG present at two rel_paths or in two libraries scans once. Counting `image_exif` rows in the denominator inflated total by one per duplicate file and produced a permanent gap (e.g. 1101/1103 with nothing actually pending). Hash-less rows are excluded from total_photos while they sit in the `backfill_unhashed_backlog` queue; otherwise the bar pins below 100% for the duration of that backfill even though those rows aren't pending detection yet — they're pending hashing.
|
|
||||||
|
|
||||||
Module map:
|
|
||||||
- `src/faces.rs` — `FaceDao` trait + `SqliteFaceDao` impl, route handlers for `/faces/*`, `/image/faces/*`, `/persons/*`. Mirror of `tags.rs` layout.
|
|
||||||
- `src/face_watch.rs` — Tokio orchestration for the file-watch detect pass; `filter_excluded` (PathExcluder + image-extension filter), `read_image_bytes_for_detect` (RAW preview fallback).
|
|
||||||
- `src/backfill.rs` — per-tick drains (unhashed-hash, date_taken, face-backlog, etc.) called from `watcher::watch_files` and `watcher::process_new_files`.
|
|
||||||
- `src/watcher.rs` — the watcher loop itself and `process_new_files` (file walk → EXIF write → face-candidate build).
|
|
||||||
- `src/ai/face_client.rs` — HTTP client for Apollo's inference. Configured by `APOLLO_FACE_API_BASE_URL`, falls back to `APOLLO_API_BASE_URL`. Both unset → feature disabled, file-watch hook is a no-op.
|
|
||||||
- `migrations/2026-04-29-000000_add_faces/` — schema.
|
|
||||||
|
|
||||||
### Startup Sequence
|
|
||||||
|
|
||||||
1. Load `.env` file
|
|
||||||
2. Run embedded Diesel migrations
|
|
||||||
3. Spawn file watcher thread
|
|
||||||
4. Create initial thumbnails (parallel scan)
|
|
||||||
5. Generate video GIF thumbnails
|
|
||||||
6. Initialize AppState with Actix actors
|
|
||||||
7. Set up Prometheus metrics (`imageserver_image_total`, `imageserver_video_total`)
|
|
||||||
8. Scan directory for videos and queue HLS processing
|
|
||||||
9. Start HTTP server on `BIND_URL` + localhost:8088
|
|
||||||
|
|
||||||
## Testing Patterns
|
|
||||||
|
|
||||||
Tests require `BASE_PATH` environment variable. Many integration tests create temporary directories and files.
|
|
||||||
|
|
||||||
When testing database code:
|
|
||||||
- Use in-memory SQLite: `DATABASE_URL=":memory:"`
|
|
||||||
- Run migrations in test setup
|
|
||||||
- Clean up with `DROP TABLE` or use `#[serial]` from `serial_test` crate if parallel tests conflict
|
|
||||||
|
|
||||||
## Common Gotchas
|
|
||||||
|
|
||||||
**EXIF Date Parsing:**
|
|
||||||
Multiple formats supported (EXIF DateTime, ISO8601, Unix timestamp). Fallback chain attempts multiple parsers.
|
|
||||||
|
|
||||||
**Video Processing:**
|
|
||||||
ffmpeg processes run asynchronously via actors. Use `StreamActor` to track completion. HLS segments written to `VIDEO_PATH`.
|
|
||||||
|
|
||||||
**File Extensions:**
|
|
||||||
Extension detection is case-insensitive. Use `file_types.rs` helpers rather than manual string matching.
|
|
||||||
|
|
||||||
**Migration Workflow:**
|
|
||||||
After creating a migration, manually edit the SQL, then regenerate `schema.rs` with `diesel print-schema`. Migrations auto-run on startup via `embedded_migrations!()` macro.
|
|
||||||
|
|
||||||
**Path Absolutization:**
|
|
||||||
Use `path-absolutize` crate's `.absolutize()` method when converting user-provided paths to ensure they're within `BASE_PATH`.
|
|
||||||
|
|
||||||
## Required Environment Variables
|
|
||||||
|
|
||||||
```bash
|
|
||||||
DATABASE_URL=./database.db # SQLite database path
|
|
||||||
BASE_PATH=/path/to/media # Root media directory
|
|
||||||
THUMBNAILS=/path/to/thumbnails # Thumbnail storage
|
|
||||||
VIDEO_PATH=/path/to/video/hls # HLS playlist output
|
|
||||||
GIFS_DIRECTORY=/path/to/gifs # Video GIF thumbnails
|
|
||||||
BIND_URL=0.0.0.0:8080 # Server binding
|
|
||||||
CORS_ALLOWED_ORIGINS=http://localhost:3000
|
|
||||||
SECRET_KEY=your-secret-key-here # JWT signing secret
|
|
||||||
RUST_LOG=info # Log level
|
|
||||||
EXCLUDED_DIRS=/private,/archive # Comma-separated paths to exclude from memories
|
|
||||||
```
|
|
||||||
|
|
||||||
Optional:
|
|
||||||
```bash
|
|
||||||
WATCH_QUICK_INTERVAL_SECONDS=60 # Quick scan interval
|
|
||||||
WATCH_FULL_INTERVAL_SECONDS=3600 # Full scan interval
|
|
||||||
DATE_BACKFILL_MAX_PER_TICK=500 # Cap on canonical-date drain per watcher tick
|
|
||||||
OTLP_OTLS_ENDPOINT=http://... # OpenTelemetry collector (release builds)
|
|
||||||
|
|
||||||
# AI Insights Configuration
|
|
||||||
OLLAMA_PRIMARY_URL=http://desktop:11434 # Primary Ollama server (e.g., desktop)
|
|
||||||
OLLAMA_FALLBACK_URL=http://server:11434 # Fallback Ollama server (optional, always-on)
|
|
||||||
OLLAMA_PRIMARY_MODEL=nemotron-3-nano:30b # Model for primary server (default: nemotron-3-nano:30b)
|
|
||||||
OLLAMA_FALLBACK_MODEL=llama3.2:3b # Model for fallback server (optional, uses primary if not set)
|
|
||||||
OLLAMA_REQUEST_TIMEOUT_SECONDS=120 # Per-request generation timeout (default 120). Increase for slow CPU-offloaded models.
|
|
||||||
SMS_API_URL=http://localhost:8000 # SMS message API endpoint (default: localhost:8000)
|
|
||||||
SMS_API_TOKEN=your-api-token # SMS API authentication token (optional)
|
|
||||||
|
|
||||||
# Apollo Places integration (optional). When set, photo-insight enrichment
|
|
||||||
# folds the user's personal place name (Home, Work, Cabin, ...) into the
|
|
||||||
# location string fed to the LLM, and the agentic loop gains a
|
|
||||||
# `get_personal_place_at` tool. Unset = legacy Nominatim-only path.
|
|
||||||
APOLLO_API_BASE_URL=http://apollo.lan:8000 # Base URL of the sibling Apollo backend
|
|
||||||
|
|
||||||
# Face inference (optional). Apollo also hosts the insightface inference
|
|
||||||
# service; ImageApi calls it from the file-watch hook (Phase 3) and from
|
|
||||||
# the manual face-create endpoint. Falls back to APOLLO_API_BASE_URL when
|
|
||||||
# unset (typical single-Apollo deploy). Both unset = feature disabled.
|
|
||||||
APOLLO_FACE_API_BASE_URL=http://apollo.lan:8000 # Override if face service runs separately
|
|
||||||
FACE_AUTOBIND_MIN_COS=0.4 # Phase 3: cosine-sim floor for tag-name auto-bind
|
|
||||||
FACE_DETECT_CONCURRENCY=8 # Phase 3: per-scan-tick parallel detect calls
|
|
||||||
FACE_DETECT_TIMEOUT_SEC=60 # reqwest client timeout (CPU inference can be slow)
|
|
||||||
|
|
||||||
# OpenRouter (Hybrid Backend) - keeps embeddings + vision local, routes chat to OpenRouter
|
|
||||||
OPENROUTER_API_KEY=sk-or-... # Required to enable hybrid backend
|
|
||||||
OPENROUTER_DEFAULT_MODEL=anthropic/claude-sonnet-4 # Used when client doesn't pick a model
|
|
||||||
OPENROUTER_ALLOWED_MODELS=openai/gpt-4o-mini,anthropic/claude-haiku-4-5,google/gemini-2.5-flash
|
|
||||||
# Curated allowlist exposed to clients via
|
|
||||||
# GET /insights/openrouter/models. Empty = no picker.
|
|
||||||
OPENROUTER_BASE_URL=https://openrouter.ai/api/v1 # Override base URL (optional)
|
|
||||||
OPENROUTER_EMBEDDING_MODEL=openai/text-embedding-3-small # Optional, embeddings stay local today
|
|
||||||
OPENROUTER_HTTP_REFERER=https://your-site.example # Optional attribution header
|
|
||||||
OPENROUTER_APP_TITLE=ImageApi # Optional attribution header
|
|
||||||
|
|
||||||
# Insight Chat Continuation
|
|
||||||
AGENTIC_CHAT_MAX_ITERATIONS=6 # Cap on tool-calling iterations per chat turn (default 6)
|
|
||||||
```
|
|
||||||
|
|
||||||
**AI Insights Fallback Behavior:**
|
|
||||||
- Primary server is tried first with its configured model (5-second connection timeout)
|
|
||||||
- On connection failure, automatically falls back to secondary server with its model (if configured)
|
|
||||||
- If `OLLAMA_FALLBACK_MODEL` not set, uses same model as primary server on fallback
|
|
||||||
- Total request timeout is 120 seconds to accommodate slow LLM inference
|
|
||||||
- Logs indicate which server and model was used (info level) and failover attempts (warn level)
|
|
||||||
- Backwards compatible: `OLLAMA_URL` and `OLLAMA_MODEL` still supported as fallbacks
|
|
||||||
|
|
||||||
**Model Discovery:**
|
|
||||||
The `OllamaClient` provides methods to query available models:
|
|
||||||
- `OllamaClient::list_models(url)` - Returns list of all models on a server
|
|
||||||
- `OllamaClient::is_model_available(url, model_name)` - Checks if a specific model exists
|
|
||||||
|
|
||||||
This allows runtime verification of model availability before generating insights.
|
|
||||||
|
|
||||||
**Hybrid Backend (OpenRouter):**
|
|
||||||
- Per-request opt-in via `backend=hybrid` on `POST /insights/generate/agentic`.
|
|
||||||
- Local Ollama still describes the image (vision); the description is inlined
|
|
||||||
into the chat prompt and the agentic loop runs on OpenRouter.
|
|
||||||
- `request.model` (if provided) overrides `OPENROUTER_DEFAULT_MODEL` for that
|
|
||||||
call. The mobile picker reads from `OPENROUTER_ALLOWED_MODELS`.
|
|
||||||
- No live capability precheck — the operator-curated allowlist is trusted.
|
|
||||||
A bad model id surfaces as a chat-call error.
|
|
||||||
- `GET /insights/openrouter/models` returns `{ models, default_model, configured }`
|
|
||||||
for client picker UIs.
|
|
||||||
|
|
||||||
**Insight Chat Continuation:**
|
|
||||||
|
|
||||||
After an agentic insight is generated, the full `Vec<ChatMessage>` transcript is
|
|
||||||
stored in `photo_insights.training_messages` and can be continued via the
|
|
||||||
chat endpoints. The `PhotoInsightResponse.has_training_messages` flag tells
|
|
||||||
clients whether chat is available for a given insight.
|
|
||||||
|
|
||||||
- `POST /insights/chat` runs one turn of the agentic loop against the replayed
|
|
||||||
history. Body: `{ file_path, library?, user_message, model?, backend?, num_ctx?,
|
|
||||||
temperature?, top_p?, top_k?, min_p?, max_iterations?, system_prompt?, amend? }`.
|
|
||||||
`system_prompt` is a per-turn override: in append mode (default) it's applied
|
|
||||||
ephemerally — the original system message is restored before persistence so
|
|
||||||
the stored transcript keeps its baked persona. In amend mode the override
|
|
||||||
stays in place and becomes the new insight row's system message. Mirrors the
|
|
||||||
internal `annotate_system_with_budget` swap-and-restore pattern.
|
|
||||||
- `POST /insights/chat/stream` is the SSE variant — same request body, response
|
|
||||||
is `text/event-stream` with events: `iteration_start`, `text` (delta), `tool_call`,
|
|
||||||
`tool_result`, `truncated`, `done`, plus a server-emitted `error_message` on
|
|
||||||
failure. Preferred by the mobile client for live tool-chip updates.
|
|
||||||
- `GET /insights/chat/history?path=...&library=...` returns the rendered
|
|
||||||
transcript. Each assistant message carries a `tools: [{name, arguments, result,
|
|
||||||
result_truncated?}]` array with the tool invocations that led up to it. Tool
|
|
||||||
results over 2000 chars are truncated with `result_truncated: true`.
|
|
||||||
- `POST /insights/chat/rewind` truncates the transcript at a given rendered
|
|
||||||
index (drops that message + any tool-call scaffolding that preceded it + all
|
|
||||||
later turns). Index 0 is protected. Used for "try again from here" flows.
|
|
||||||
|
|
||||||
Backend routing rules (matches agentic-insight generation):
|
|
||||||
- Stored `backend` on the insight row is authoritative by default.
|
|
||||||
- `request.backend` may override per-turn. `local -> hybrid` is rejected in
|
|
||||||
v1 (would require on-the-fly visual-description rewrite); `hybrid -> local`
|
|
||||||
replays verbatim since the description is already inlined as text.
|
|
||||||
- `request.model` overrides the chat model (an Ollama id in local mode, an
|
|
||||||
OpenRouter id in hybrid mode).
|
|
||||||
|
|
||||||
Persistence:
|
|
||||||
- Append mode (default): re-serialize the full history and `UPDATE` the same
|
|
||||||
row's `training_messages`.
|
|
||||||
- Amend mode (`amend: true`): regenerate the title, insert a new insight row
|
|
||||||
via `store_insight` (auto-flips prior rows' `is_current=false`). Response
|
|
||||||
surfaces the new row's id as `amended_insight_id`.
|
|
||||||
|
|
||||||
Per-`(library_id, file_path)` async mutex (`AppState.insight_chat.chat_locks`)
|
|
||||||
serialises concurrent turns on the same insight so the JSON blob doesn't race.
|
|
||||||
|
|
||||||
Context management is a soft bound: if the serialized history exceeds
|
|
||||||
`num_ctx - 2048` tokens (cheap 4-byte/token heuristic), the oldest
|
|
||||||
assistant-tool_call + tool_result pairs are dropped until under budget. The
|
|
||||||
initial user message (with any images) and system prompt are always preserved.
|
|
||||||
The `truncated` event / flag is surfaced to the client when a drop occurred.
|
|
||||||
|
|
||||||
Configurable env:
|
|
||||||
- `AGENTIC_CHAT_MAX_ITERATIONS` — cap on tool-calling iterations per turn
|
|
||||||
(default 6). Per-request `max_iterations` is clamped to this cap.
|
|
||||||
|
|
||||||
**Apollo Places integration (optional):**
|
|
||||||
|
|
||||||
The sibling Apollo project (personal location-history viewer) owns
|
|
||||||
user-defined Places: `name + lat/lon + radius_m + description (+ optional
|
|
||||||
category)`. When `APOLLO_API_BASE_URL` is set, ImageApi queries
|
|
||||||
`/api/places/contains?lat=&lon=` to enrich the LLM prompt's location
|
|
||||||
string. See `src/ai/apollo_client.rs` and `src/ai/insight_generator.rs`:
|
|
||||||
|
|
||||||
- **Auto-enrichment** (always on when configured): the per-photo location
|
|
||||||
resolver folds the most-specific containing Place ("Home — near
|
|
||||||
Cambridge, MA" or "Home (My house in Cambridge) — near Cambridge, MA"
|
|
||||||
when a description is set) into the location field of `combine_contexts`.
|
|
||||||
Smallest-radius wins — Apollo sorts server-side, this code takes `[0]`.
|
|
||||||
- **Agentic tool** `get_personal_place_at(latitude, longitude)`: registered
|
|
||||||
alongside `reverse_geocode` only when `apollo_enabled()` returns true.
|
|
||||||
Returns "- Name [category]: description (radius N m)" lines, smallest
|
|
||||||
radius first. The tool is **deliberately narrow** — no enumerate-all
|
|
||||||
variant; auto-enrichment covers the photo-context path and the agentic
|
|
||||||
tool covers ad-hoc lat/lon questions in chat continuation.
|
|
||||||
|
|
||||||
Failure modes degrade silently to the legacy Nominatim path: 5 s timeout,
|
|
||||||
errors logged at `warn`, empty results returned. Apollo's routes are
|
|
||||||
unauthenticated (single-user, LAN-trust); add JWT auth here + on Apollo's
|
|
||||||
side if exposing beyond a trusted network.
|
|
||||||
|
|
||||||
## Dependencies of Note
|
|
||||||
|
|
||||||
### Rust crates
|
|
||||||
|
|
||||||
- **actix-web**: HTTP framework
|
|
||||||
- **diesel**: ORM for SQLite
|
|
||||||
- **jsonwebtoken**: JWT implementation
|
|
||||||
- **kamadak-exif**: EXIF parsing
|
|
||||||
- **image**: Thumbnail generation
|
|
||||||
- **walkdir**: Directory traversal
|
|
||||||
- **rayon**: Parallel processing
|
|
||||||
- **opentelemetry**: Distributed tracing
|
|
||||||
- **bcrypt**: Password hashing
|
|
||||||
- **infer**: Magic number file type detection
|
|
||||||
|
|
||||||
### External binaries (must be on `PATH`)
|
|
||||||
|
|
||||||
- **`ffmpeg`** — video thumbnail extraction (`StreamActor`, HLS pipeline) and
|
|
||||||
the HEIF/HEIC/NEF/ARW thumbnail fallback in `generate_image_thumbnail_ffmpeg`.
|
|
||||||
Required for any deploy that holds video or HEIF files.
|
|
||||||
- **`exiftool`** — optional but strongly recommended for RAW-heavy libraries.
|
|
||||||
The thumbnail pipeline shells out to it as the slow-path fallback for
|
|
||||||
embedded preview extraction (Nikon MakerNote `PreviewIFD`, Canon SubIFDs,
|
|
||||||
etc. — anything kamadak-exif's IFD0/IFD1 readers can't reach). Without
|
|
||||||
exiftool installed, RAWs whose preview lives outside IFD0/IFD1 will fall
|
|
||||||
through to ffmpeg, which often produces black thumbnails. Install via
|
|
||||||
package manager: `apt install libimage-exiftool-perl`,
|
|
||||||
`brew install exiftool`, `winget install OliverBetz.ExifTool`, or
|
|
||||||
`choco install exiftool`.
|
|
||||||
5034
Cargo.lock
generated
5034
Cargo.lock
generated
File diff suppressed because it is too large
Load Diff
79
Cargo.toml
79
Cargo.toml
@@ -1,73 +1,38 @@
|
|||||||
[package]
|
[package]
|
||||||
name = "image-api"
|
name = "image-api"
|
||||||
version = "1.1.0"
|
version = "0.1.0"
|
||||||
authors = ["Cameron Cordes <cameronc.dev@gmail.com>"]
|
authors = ["Cameron Cordes <cameronc.dev@gmail.com>"]
|
||||||
edition = "2024"
|
edition = "2018"
|
||||||
|
|
||||||
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
|
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
|
||||||
|
|
||||||
[profile.release]
|
[profile.release]
|
||||||
lto = "thin"
|
lto = true
|
||||||
|
|
||||||
[profile.dev]
|
|
||||||
debug = "line-tables-only"
|
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
actix = "0.13.1"
|
actix = "0.10"
|
||||||
actix-web = "4"
|
actix-web = "3"
|
||||||
actix-rt = "2.6"
|
actix-rt = "1"
|
||||||
tokio = { version = "1.42.0", features = ["default", "process", "sync", "macros", "rt-multi-thread"] }
|
actix-files = "0.5"
|
||||||
actix-files = "0.6"
|
actix-multipart = "0.3.0"
|
||||||
actix-cors = "0.7"
|
|
||||||
actix-multipart = "0.7.2"
|
|
||||||
actix-governor = "0.5"
|
|
||||||
futures = "0.3.5"
|
futures = "0.3.5"
|
||||||
jsonwebtoken = "9.3.0"
|
jsonwebtoken = "7.2.0"
|
||||||
serde = "1"
|
serde = "1"
|
||||||
serde_json = "1"
|
serde_json = "1"
|
||||||
diesel = { version = "2.2.10", features = ["sqlite"] }
|
diesel = { version = "1.4.8", features = ["sqlite"] }
|
||||||
libsqlite3-sys = "0.35"
|
hmac = "0.11"
|
||||||
diesel_migrations = "2.2.0"
|
sha2 = "0.9"
|
||||||
chrono = "0.4"
|
chrono = "0.4"
|
||||||
clap = { version = "4.5", features = ["derive"] }
|
|
||||||
dotenv = "0.15"
|
dotenv = "0.15"
|
||||||
bcrypt = "0.17.1"
|
bcrypt = "0.9"
|
||||||
image = { version = "0.25.5", default-features = false, features = ["jpeg", "png", "rayon", "webp", "tiff", "avif"] }
|
image = { version = "0.23", default-features = false, features = ["jpeg", "png", "jpeg_rayon"] }
|
||||||
infer = "0.16"
|
walkdir = "2"
|
||||||
walkdir = "2.4.0"
|
|
||||||
rayon = "1.5"
|
rayon = "1.5"
|
||||||
path-absolutize = "3.1"
|
notify = "4.0"
|
||||||
log = "0.4"
|
path-absolutize = "3.0"
|
||||||
env_logger = "0.11.5"
|
log="0.4"
|
||||||
actix-web-prom = "0.9.0"
|
env_logger="0.8"
|
||||||
prometheus = "0.13"
|
actix-web-prom = "0.5.1"
|
||||||
lazy_static = "1.5"
|
prometheus = "0.11"
|
||||||
|
lazy_static = "1.1"
|
||||||
anyhow = "1.0"
|
anyhow = "1.0"
|
||||||
rand = "0.8.5"
|
|
||||||
opentelemetry = { version = "0.31.0", features = ["default", "metrics", "tracing"] }
|
|
||||||
opentelemetry_sdk = { version = "0.31.0", features = ["default", "rt-tokio-current-thread", "metrics"] }
|
|
||||||
opentelemetry-otlp = { version = "0.31.0", features = ["default", "metrics", "tracing", "grpc-tonic"] }
|
|
||||||
opentelemetry-stdout = "0.31.0"
|
|
||||||
opentelemetry-appender-log = "0.31.0"
|
|
||||||
tempfile = "3.20.0"
|
|
||||||
regex = "1.11.1"
|
|
||||||
exif = { package = "kamadak-exif", version = "0.6.1" }
|
|
||||||
reqwest = { version = "0.12", features = ["json", "stream", "multipart"] }
|
|
||||||
async-stream = "0.3"
|
|
||||||
tokio-util = { version = "0.7", features = ["io"] }
|
|
||||||
bytes = "1"
|
|
||||||
urlencoding = "2.1"
|
|
||||||
zerocopy = "0.8"
|
|
||||||
ical = "0.11"
|
|
||||||
scraper = "0.20"
|
|
||||||
base64 = "0.22"
|
|
||||||
blake3 = "1.5"
|
|
||||||
image_hasher = "3.0"
|
|
||||||
bk-tree = "0.5"
|
|
||||||
async-trait = "0.1"
|
|
||||||
indicatif = "0.17"
|
|
||||||
|
|
||||||
# Windows lacks system sqlite3, so re-enable the bundled C build there.
|
|
||||||
# Linux/macOS use the system library (faster builds, smaller binary).
|
|
||||||
[target.'cfg(windows)'.dependencies]
|
|
||||||
libsqlite3-sys = { version = "0.35", features = ["bundled"] }
|
|
||||||
|
|||||||
2
Jenkinsfile
vendored
2
Jenkinsfile
vendored
@@ -1,7 +1,7 @@
|
|||||||
pipeline {
|
pipeline {
|
||||||
agent {
|
agent {
|
||||||
docker {
|
docker {
|
||||||
image 'rust:1.59'
|
image 'rust:1.51'
|
||||||
args '-v "$PWD":/usr/src/image-api'
|
args '-v "$PWD":/usr/src/image-api'
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
179
README.md
179
README.md
@@ -2,191 +2,14 @@
|
|||||||
This is an Actix-web server for serving images and videos from a filesystem.
|
This is an Actix-web server for serving images and videos from a filesystem.
|
||||||
Upon first run it will generate thumbnails for all images and videos at `BASE_PATH`.
|
Upon first run it will generate thumbnails for all images and videos at `BASE_PATH`.
|
||||||
|
|
||||||
## Features
|
|
||||||
- Automatic thumbnail generation for images and videos
|
|
||||||
- EXIF data extraction and storage for photos
|
|
||||||
- File watching with NFS support (polling-based)
|
|
||||||
- Video streaming with HLS
|
|
||||||
- Tag-based organization
|
|
||||||
- Memories API for browsing photos by date
|
|
||||||
- **Video Wall** - Auto-generated short preview clips for videos, served via a grid view
|
|
||||||
- **AI-Powered Photo Insights** - Generate contextual insights from photos using LLMs
|
|
||||||
- **RAG-based Context Retrieval** - Semantic search over daily conversation summaries
|
|
||||||
- **Automatic Daily Summaries** - LLM-generated summaries of daily conversations with embeddings
|
|
||||||
|
|
||||||
## External Dependencies
|
|
||||||
|
|
||||||
### ffmpeg (required)
|
|
||||||
`ffmpeg` must be on `PATH`. It is used for:
|
|
||||||
- **HLS video streaming** — transcoding/segmenting source videos into `.m3u8` + `.ts` playlists
|
|
||||||
- **Video thumbnails** — extracting a frame at the 3-second mark
|
|
||||||
- **Video preview clips** — short looping previews for the Video Wall
|
|
||||||
- **HEIC / HEIF thumbnails** — decoding Apple's HEIC format (your ffmpeg build must include
|
|
||||||
`libheif`; most modern builds do)
|
|
||||||
|
|
||||||
Builds used in development: the `gyan.dev` full build on Windows, and distro `ffmpeg`
|
|
||||||
packages on Linux work fine. If HEIC thumbnails silently fail, check
|
|
||||||
`ffmpeg -formats | grep heif` to confirm HEIF support.
|
|
||||||
|
|
||||||
### RAW photo thumbnails
|
|
||||||
RAW formats (ARW, NEF, CR2, CR3, DNG, RAF, ORF, RW2, PEF, SRW, TIFF) are thumbnailed
|
|
||||||
by reading an embedded JPEG preview out of the TIFF container — no external RAW
|
|
||||||
decoder (libraw / dcraw) is involved. The pipeline tries two layers in order and
|
|
||||||
keeps the largest valid JPEG:
|
|
||||||
|
|
||||||
1. **Fast path (no extra dependency)** — `kamadak-exif` reads
|
|
||||||
`JPEGInterchangeFormat` from IFD0 / IFD1 directly. Covers older bodies and
|
|
||||||
most DNGs.
|
|
||||||
2. **`exiftool` fallback (recommended for RAW-heavy libraries)** — shells out
|
|
||||||
to extract `PreviewImage` / `JpgFromRaw` / `OtherImage`, which reaches
|
|
||||||
MakerNote and SubIFD-hosted previews kamadak-exif can't see (e.g. Nikon's
|
|
||||||
`PreviewIFD`, where modern Nikon bodies stash the full-res review JPEG).
|
|
||||||
If `exiftool` isn't on `PATH` this layer is skipped silently and only the
|
|
||||||
fast-path result is used.
|
|
||||||
|
|
||||||
Install `exiftool` via your package manager:
|
|
||||||
- macOS: `brew install exiftool`
|
|
||||||
- Linux (Debian/Ubuntu): `apt install libimage-exiftool-perl`
|
|
||||||
- Windows: `winget install OliverBetz.ExifTool` or `choco install exiftool`
|
|
||||||
|
|
||||||
Files where neither layer produces a valid preview fall back to ffmpeg. Anything
|
|
||||||
that still can't be decoded is marked with a `<thumb>.unsupported` sentinel in
|
|
||||||
the thumbnail directory so we don't retry it every scan. Delete those sentinels
|
|
||||||
(and any cached black thumbnails) to force retries after a tooling upgrade.
|
|
||||||
|
|
||||||
## Environment
|
## Environment
|
||||||
There are a handful of required environment variables to have the API run.
|
There are a handful of required environment variables to have the API run.
|
||||||
They should be defined where the binary is located or above it in an `.env` file.
|
They should be defined where the binary is located or above it in an `.env` file.
|
||||||
|
|
||||||
- `DATABASE_URL` is a path or url to a database (currently only SQLite is tested)
|
- `DATABASE_URL` is a path or url to a database (currently only SQLite is tested)
|
||||||
- `BASE_PATH` is the root from which you want to serve images and videos
|
- `BASE_PATH` is the root from which you want to serve images and videos
|
||||||
- `THUMBNAILS` is a path where generated thumbnails should be stored. Thumbnails
|
- `THUMBNAILS` is a path where generated thumbnails should be stored
|
||||||
mirror the source tree under `BASE_PATH` and keep the source's original
|
|
||||||
extension (e.g. `foo.arw` or `bar.mp4`), though the file contents are always
|
|
||||||
JPEG bytes — browsers content-sniff. Files that can't be thumbnailed by the
|
|
||||||
`image` crate, ffmpeg, or an embedded RAW preview get a zero-byte
|
|
||||||
`<thumb_path>.unsupported` sentinel in this directory so subsequent scans
|
|
||||||
skip them. Delete the `*.unsupported` files to force retries (for example
|
|
||||||
after upgrading ffmpeg or adding libheif)
|
|
||||||
- `VIDEO_PATH` is a path where HLS playlists and video parts should be stored
|
|
||||||
- `GIFS_DIRECTORY` is a path where generated video GIF thumbnails should be stored
|
|
||||||
- `BIND_URL` is the url and port to bind to (typically your own IP address)
|
- `BIND_URL` is the url and port to bind to (typically your own IP address)
|
||||||
- `SECRET_KEY` is the *hopefully* random string to sign Tokens with
|
- `SECRET_KEY` is the *hopefully* random string to sign Tokens with
|
||||||
- `RUST_LOG` is one of `off, error, warn, info, debug, trace`, from least to most noisy [error is default]
|
- `RUST_LOG` is one of `off, error, warn, info, debug, trace`, from least to most noisy [error is default]
|
||||||
- `EXCLUDED_DIRS` is a comma separated list of directories to exclude from the Memories API
|
|
||||||
- `PREVIEW_CLIPS_DIRECTORY` (optional) is a path where generated video preview clips should be stored [default: `preview_clips`]
|
|
||||||
- `WATCH_QUICK_INTERVAL_SECONDS` (optional) is the interval in seconds for quick file scans [default: 60]
|
|
||||||
- `WATCH_FULL_INTERVAL_SECONDS` (optional) is the interval in seconds for full file scans [default: 3600]
|
|
||||||
|
|
||||||
### AI Insights Configuration (Optional)
|
|
||||||
|
|
||||||
The following environment variables configure AI-powered photo insights and daily conversation summaries:
|
|
||||||
|
|
||||||
#### Ollama Configuration
|
|
||||||
- `OLLAMA_PRIMARY_URL` - Primary Ollama server URL [default: `http://localhost:11434`]
|
|
||||||
- Example: `http://desktop:11434` (your main/powerful server)
|
|
||||||
- `OLLAMA_FALLBACK_URL` - Fallback Ollama server URL (optional)
|
|
||||||
- Example: `http://server:11434` (always-on backup server)
|
|
||||||
- `OLLAMA_PRIMARY_MODEL` - Model to use on primary server [default: `nemotron-3-nano:30b`]
|
|
||||||
- Example: `nemotron-3-nano:30b`, `llama3.2:3b`, etc.
|
|
||||||
- `OLLAMA_FALLBACK_MODEL` - Model to use on fallback server (optional)
|
|
||||||
- If not set, uses `OLLAMA_PRIMARY_MODEL` on fallback server
|
|
||||||
|
|
||||||
**Legacy Variables** (still supported):
|
|
||||||
- `OLLAMA_URL` - Used if `OLLAMA_PRIMARY_URL` not set
|
|
||||||
- `OLLAMA_MODEL` - Used if `OLLAMA_PRIMARY_MODEL` not set
|
|
||||||
|
|
||||||
#### OpenRouter Configuration (Hybrid Backend)
|
|
||||||
The hybrid agentic backend keeps embeddings + vision local (Ollama) while routing
|
|
||||||
chat + tool-calling to OpenRouter. Enabled per-request when the client sends
|
|
||||||
`backend=hybrid`.
|
|
||||||
|
|
||||||
- `OPENROUTER_API_KEY` - OpenRouter API key. Required to enable the hybrid backend.
|
|
||||||
- `OPENROUTER_DEFAULT_MODEL` - Model id used when the client doesn't specify one
|
|
||||||
[default: `anthropic/claude-sonnet-4`]
|
|
||||||
- Example: `openai/gpt-4o-mini`, `google/gemini-2.5-flash`
|
|
||||||
- `OPENROUTER_ALLOWED_MODELS` - Comma-separated curated allowlist exposed to
|
|
||||||
clients via `GET /insights/openrouter/models`. The mobile picker shows only
|
|
||||||
these. Empty/unset = no picker, server default is used.
|
|
||||||
- Example: `openai/gpt-4o-mini,anthropic/claude-haiku-4-5,google/gemini-2.5-flash`
|
|
||||||
- `OPENROUTER_BASE_URL` - Override base URL [default: `https://openrouter.ai/api/v1`]
|
|
||||||
- `OPENROUTER_EMBEDDING_MODEL` - Embedding model for OpenRouter
|
|
||||||
[default: `openai/text-embedding-3-small`]. Only used if/when embeddings are
|
|
||||||
routed through OpenRouter (currently embeddings stay local).
|
|
||||||
- `OPENROUTER_HTTP_REFERER` - Optional `HTTP-Referer` for OpenRouter attribution
|
|
||||||
- `OPENROUTER_APP_TITLE` - Optional `X-Title` for OpenRouter attribution
|
|
||||||
|
|
||||||
Capability checks are skipped for the curated allowlist — bad model ids surface
|
|
||||||
as a 4xx from the chat call. Pick tool-capable models.
|
|
||||||
|
|
||||||
#### SMS API Configuration
|
|
||||||
- `SMS_API_URL` - URL to SMS message API [default: `http://localhost:8000`]
|
|
||||||
- Used to fetch conversation data for context in insights
|
|
||||||
- `SMS_API_TOKEN` - Authentication token for SMS API (optional)
|
|
||||||
|
|
||||||
#### Agentic Insight Generation
|
|
||||||
- `AGENTIC_MAX_ITERATIONS` - Maximum tool-call iterations per agentic insight request [default: `10`]
|
|
||||||
- Controls how many times the model can invoke tools before being forced to produce a final answer
|
|
||||||
- Increase for more thorough context gathering; decrease to limit response time
|
|
||||||
|
|
||||||
#### Insight Chat Continuation
|
|
||||||
After an agentic insight is generated, the conversation can be continued. Endpoints:
|
|
||||||
- `POST /insights/chat` — single-turn reply (non-streaming)
|
|
||||||
- `POST /insights/chat/stream` — SSE variant with live `text` deltas and
|
|
||||||
`tool_call` / `tool_result` events. Mobile client uses this.
|
|
||||||
- `GET /insights/chat/history?path=...&library=...` — rendered transcript;
|
|
||||||
each assistant message carries a `tools: [{name, arguments, result}]` array
|
|
||||||
- `POST /insights/chat/rewind` — truncate transcript at a rendered index
|
|
||||||
(drops that message + any preceding tool scaffolding + later turns). Used
|
|
||||||
for "try again from here" flows. The initial user message is protected.
|
|
||||||
|
|
||||||
Amend mode (`amend: true` in the chat request body) regenerates the insight's
|
|
||||||
title and inserts a new row instead of appending to the existing transcript,
|
|
||||||
so you can rewrite the saved summary from within chat.
|
|
||||||
|
|
||||||
- `AGENTIC_CHAT_MAX_ITERATIONS` - Cap on tool-calling iterations per chat turn [default: `6`]
|
|
||||||
- Per-request `max_iterations` (when sent by the client) is clamped to this cap
|
|
||||||
|
|
||||||
#### Fallback Behavior
|
|
||||||
- Primary server is tried first with 5-second connection timeout
|
|
||||||
- On failure, automatically falls back to secondary server (if configured)
|
|
||||||
- Total request timeout is 120 seconds to accommodate LLM inference
|
|
||||||
- Logs indicate which server/model was used and any failover attempts
|
|
||||||
|
|
||||||
#### Daily Summary Generation
|
|
||||||
Daily conversation summaries are generated automatically on server startup. Configure in `src/main.rs`:
|
|
||||||
- Date range for summary generation
|
|
||||||
- Contacts to process
|
|
||||||
- Model version used for embeddings: `nomic-embed-text:v1.5`
|
|
||||||
|
|
||||||
### Apollo + Face Recognition (Optional)
|
|
||||||
|
|
||||||
Apollo (sibling project) hosts both the Places API and the local insightface
|
|
||||||
inference service. Both integrations are optional and degrade gracefully when
|
|
||||||
unset.
|
|
||||||
|
|
||||||
- `APOLLO_API_BASE_URL` - Base URL of the sibling Apollo backend.
|
|
||||||
- When set, photo-insight enrichment folds the user's personal place name
|
|
||||||
(Home, Work, Cabin, ...) into the location string, and the agentic loop
|
|
||||||
gains a `get_personal_place_at` tool. Unset = legacy Nominatim-only path.
|
|
||||||
- `APOLLO_FACE_API_BASE_URL` - Base URL for the face-detection service.
|
|
||||||
- Falls back to `APOLLO_API_BASE_URL` when unset (typical single-Apollo
|
|
||||||
deploy). Both unset = face feature disabled (file-watch hook and
|
|
||||||
manual-face endpoints short-circuit silently).
|
|
||||||
- `FACE_AUTOBIND_MIN_COS` (Phase 3) - Cosine-sim floor for auto-binding a
|
|
||||||
detected face to an existing same-named person via people-tag bootstrap
|
|
||||||
[default: `0.4`].
|
|
||||||
- `FACE_DETECT_CONCURRENCY` (Phase 3) - Per-scan-tick concurrent detect
|
|
||||||
calls fired by the file watcher [default: `8`]. Apollo serializes them
|
|
||||||
via its single-worker GPU pool.
|
|
||||||
- `FACE_DETECT_TIMEOUT_SEC` - reqwest client timeout per detect call
|
|
||||||
[default: `60`]. CPU inference on a backlog can take many seconds.
|
|
||||||
- `FACE_BACKLOG_MAX_PER_TICK` - Cap on the per-tick backlog drain (photos
|
|
||||||
with a content_hash but no face_detections row) [default: `64`]. Runs
|
|
||||||
every watcher tick regardless of quick-vs-full scan, so the unscanned
|
|
||||||
set drains independently of the file walk.
|
|
||||||
- `FACE_HASH_BACKFILL_MAX_PER_TICK` - Cap on the per-tick content_hash
|
|
||||||
backfill (photos that were registered before the hash field was
|
|
||||||
populated retroactively) [default: `2000`]. Errors don't burn the cap;
|
|
||||||
only successful hashes count.
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,2 +0,0 @@
|
|||||||
DROP INDEX IF EXISTS idx_image_exif_file_path;
|
|
||||||
DROP TABLE IF EXISTS image_exif;
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
CREATE TABLE image_exif (
|
|
||||||
id INTEGER PRIMARY KEY NOT NULL,
|
|
||||||
file_path TEXT NOT NULL UNIQUE,
|
|
||||||
|
|
||||||
-- Camera Information
|
|
||||||
camera_make TEXT,
|
|
||||||
camera_model TEXT,
|
|
||||||
lens_model TEXT,
|
|
||||||
|
|
||||||
-- Image Properties
|
|
||||||
width INTEGER,
|
|
||||||
height INTEGER,
|
|
||||||
orientation INTEGER,
|
|
||||||
|
|
||||||
-- GPS Coordinates
|
|
||||||
gps_latitude REAL,
|
|
||||||
gps_longitude REAL,
|
|
||||||
gps_altitude REAL,
|
|
||||||
|
|
||||||
-- Capture Settings
|
|
||||||
focal_length REAL,
|
|
||||||
aperture REAL,
|
|
||||||
shutter_speed TEXT,
|
|
||||||
iso INTEGER,
|
|
||||||
date_taken BIGINT,
|
|
||||||
|
|
||||||
-- Housekeeping
|
|
||||||
created_time BIGINT NOT NULL,
|
|
||||||
last_modified BIGINT NOT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX idx_image_exif_file_path ON image_exif(file_path);
|
|
||||||
@@ -1,9 +0,0 @@
|
|||||||
-- Rollback indexes
|
|
||||||
|
|
||||||
DROP INDEX IF EXISTS idx_favorites_userid;
|
|
||||||
DROP INDEX IF EXISTS idx_favorites_path;
|
|
||||||
DROP INDEX IF EXISTS idx_tags_name;
|
|
||||||
DROP INDEX IF EXISTS idx_tagged_photo_photo_name;
|
|
||||||
DROP INDEX IF EXISTS idx_tagged_photo_tag_id;
|
|
||||||
DROP INDEX IF EXISTS idx_image_exif_camera;
|
|
||||||
DROP INDEX IF EXISTS idx_image_exif_gps;
|
|
||||||
@@ -1,17 +0,0 @@
|
|||||||
-- Add indexes for improved query performance
|
|
||||||
|
|
||||||
-- Favorites table indexes
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_favorites_userid ON favorites(userid);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_favorites_path ON favorites(path);
|
|
||||||
|
|
||||||
-- Tags table indexes
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_tags_name ON tags(name);
|
|
||||||
|
|
||||||
-- Tagged photos indexes
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_tagged_photo_photo_name ON tagged_photo(photo_name);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_tagged_photo_tag_id ON tagged_photo(tag_id);
|
|
||||||
|
|
||||||
-- EXIF table indexes (date_taken already has index from previous migration)
|
|
||||||
-- Adding composite index for common EXIF queries
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_image_exif_camera ON image_exif(camera_make, camera_model);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_image_exif_gps ON image_exif(gps_latitude, gps_longitude);
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
-- Rollback unique constraint on favorites
|
|
||||||
|
|
||||||
DROP INDEX IF EXISTS idx_favorites_unique;
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
-- Add unique constraint to prevent duplicate favorites per user
|
|
||||||
|
|
||||||
-- First, remove any existing duplicates (keep the oldest one)
|
|
||||||
DELETE FROM favorites
|
|
||||||
WHERE rowid NOT IN (
|
|
||||||
SELECT MIN(rowid)
|
|
||||||
FROM favorites
|
|
||||||
GROUP BY userid, path
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Add unique index to enforce constraint
|
|
||||||
CREATE UNIQUE INDEX idx_favorites_unique ON favorites(userid, path);
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
-- Remove date_taken index
|
|
||||||
DROP INDEX IF EXISTS idx_image_exif_date_taken;
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
-- Add index on date_taken for efficient date range queries
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_image_exif_date_taken ON image_exif(date_taken);
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
-- Rollback AI insights table
|
|
||||||
DROP INDEX IF EXISTS idx_photo_insights_path;
|
|
||||||
DROP TABLE IF EXISTS photo_insights;
|
|
||||||
@@ -1,11 +0,0 @@
|
|||||||
-- AI-generated insights for individual photos
|
|
||||||
CREATE TABLE IF NOT EXISTS photo_insights (
|
|
||||||
id INTEGER PRIMARY KEY NOT NULL,
|
|
||||||
file_path TEXT NOT NULL UNIQUE, -- Full path to the photo
|
|
||||||
title TEXT NOT NULL, -- "At the beach with Sarah"
|
|
||||||
summary TEXT NOT NULL, -- 2-3 sentence description
|
|
||||||
generated_at BIGINT NOT NULL,
|
|
||||||
model_version TEXT NOT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_photo_insights_path ON photo_insights(file_path);
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
DROP TABLE daily_conversation_summaries;
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
-- Daily conversation summaries for improved RAG quality
|
|
||||||
-- Each row = one day's conversation with a contact, summarized by LLM and embedded
|
|
||||||
|
|
||||||
CREATE TABLE daily_conversation_summaries (
|
|
||||||
id INTEGER PRIMARY KEY NOT NULL,
|
|
||||||
date TEXT NOT NULL, -- ISO date "2024-08-15"
|
|
||||||
contact TEXT NOT NULL, -- Contact name
|
|
||||||
summary TEXT NOT NULL, -- LLM-generated 3-5 sentence summary
|
|
||||||
message_count INTEGER NOT NULL, -- Number of messages in this day
|
|
||||||
embedding BLOB NOT NULL, -- 768-dim vector of the summary
|
|
||||||
created_at BIGINT NOT NULL, -- When this summary was generated
|
|
||||||
model_version TEXT NOT NULL, -- "nomic-embed-text:v1.5"
|
|
||||||
UNIQUE(date, contact)
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes for efficient querying
|
|
||||||
CREATE INDEX idx_daily_summaries_date ON daily_conversation_summaries(date);
|
|
||||||
CREATE INDEX idx_daily_summaries_contact ON daily_conversation_summaries(contact);
|
|
||||||
CREATE INDEX idx_daily_summaries_date_contact ON daily_conversation_summaries(date, contact);
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
DROP TABLE IF EXISTS calendar_events;
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
CREATE TABLE calendar_events (
|
|
||||||
id INTEGER PRIMARY KEY NOT NULL,
|
|
||||||
event_uid TEXT,
|
|
||||||
summary TEXT NOT NULL,
|
|
||||||
description TEXT,
|
|
||||||
location TEXT,
|
|
||||||
start_time BIGINT NOT NULL,
|
|
||||||
end_time BIGINT NOT NULL,
|
|
||||||
all_day BOOLEAN NOT NULL DEFAULT 0,
|
|
||||||
organizer TEXT,
|
|
||||||
attendees TEXT,
|
|
||||||
embedding BLOB,
|
|
||||||
created_at BIGINT NOT NULL,
|
|
||||||
source_file TEXT,
|
|
||||||
UNIQUE(event_uid, start_time)
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX idx_calendar_start_time ON calendar_events(start_time);
|
|
||||||
CREATE INDEX idx_calendar_end_time ON calendar_events(end_time);
|
|
||||||
CREATE INDEX idx_calendar_time_range ON calendar_events(start_time, end_time);
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
DROP TABLE IF EXISTS location_history;
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
CREATE TABLE location_history (
|
|
||||||
id INTEGER PRIMARY KEY NOT NULL,
|
|
||||||
timestamp BIGINT NOT NULL,
|
|
||||||
latitude REAL NOT NULL,
|
|
||||||
longitude REAL NOT NULL,
|
|
||||||
accuracy INTEGER,
|
|
||||||
activity TEXT,
|
|
||||||
activity_confidence INTEGER,
|
|
||||||
place_name TEXT,
|
|
||||||
place_category TEXT,
|
|
||||||
embedding BLOB,
|
|
||||||
created_at BIGINT NOT NULL,
|
|
||||||
source_file TEXT,
|
|
||||||
UNIQUE(timestamp, latitude, longitude)
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX idx_location_timestamp ON location_history(timestamp);
|
|
||||||
CREATE INDEX idx_location_coords ON location_history(latitude, longitude);
|
|
||||||
CREATE INDEX idx_location_activity ON location_history(activity);
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
DROP TABLE IF EXISTS search_history;
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
CREATE TABLE search_history (
|
|
||||||
id INTEGER PRIMARY KEY NOT NULL,
|
|
||||||
timestamp BIGINT NOT NULL,
|
|
||||||
query TEXT NOT NULL,
|
|
||||||
search_engine TEXT,
|
|
||||||
embedding BLOB NOT NULL,
|
|
||||||
created_at BIGINT NOT NULL,
|
|
||||||
source_file TEXT,
|
|
||||||
UNIQUE(timestamp, query)
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX idx_search_timestamp ON search_history(timestamp);
|
|
||||||
CREATE INDEX idx_search_query ON search_history(query);
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
-- Revert search performance optimization indexes
|
|
||||||
|
|
||||||
DROP INDEX IF EXISTS idx_image_exif_date_path;
|
|
||||||
DROP INDEX IF EXISTS idx_tagged_photo_count;
|
|
||||||
@@ -1,15 +0,0 @@
|
|||||||
-- Add composite indexes for search performance optimization
|
|
||||||
-- This migration addresses N+1 query issues and enables database-level sorting
|
|
||||||
|
|
||||||
-- Covering index for date-sorted queries (supports ORDER BY + pagination)
|
|
||||||
-- Enables efficient date-based sorting without loading all files into memory
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_image_exif_date_path
|
|
||||||
ON image_exif(date_taken DESC, file_path);
|
|
||||||
|
|
||||||
-- Optimize batch tag count queries with GROUP BY
|
|
||||||
-- Reduces N individual queries to a single batch query
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_tagged_photo_count
|
|
||||||
ON tagged_photo(photo_name, tag_id);
|
|
||||||
|
|
||||||
-- Update query planner statistics to optimize query execution
|
|
||||||
ANALYZE;
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
DROP TABLE IF EXISTS video_preview_clips;
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
CREATE TABLE video_preview_clips (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
file_path TEXT NOT NULL UNIQUE,
|
|
||||||
status TEXT NOT NULL DEFAULT 'pending',
|
|
||||||
duration_seconds REAL,
|
|
||||||
file_size_bytes INTEGER,
|
|
||||||
error_message TEXT,
|
|
||||||
created_at TEXT NOT NULL,
|
|
||||||
updated_at TEXT NOT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX idx_preview_clips_file_path ON video_preview_clips(file_path);
|
|
||||||
CREATE INDEX idx_preview_clips_status ON video_preview_clips(status);
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
-- Restore original schema, retaining only the current insight per file.
|
|
||||||
CREATE TABLE photo_insights_old (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
file_path TEXT NOT NULL UNIQUE,
|
|
||||||
title TEXT NOT NULL,
|
|
||||||
summary TEXT NOT NULL,
|
|
||||||
generated_at BIGINT NOT NULL,
|
|
||||||
model_version TEXT NOT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO photo_insights_old (id, file_path, title, summary, generated_at, model_version)
|
|
||||||
SELECT id, file_path, title, summary, generated_at, model_version
|
|
||||||
FROM photo_insights
|
|
||||||
WHERE is_current = 1;
|
|
||||||
|
|
||||||
DROP TABLE photo_insights;
|
|
||||||
ALTER TABLE photo_insights_old RENAME TO photo_insights;
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_photo_insights_path ON photo_insights(file_path);
|
|
||||||
@@ -1,25 +0,0 @@
|
|||||||
-- Convert photo_insights to an append-only history table.
|
|
||||||
-- SQLite cannot drop a UNIQUE constraint via ALTER TABLE, so we recreate the table.
|
|
||||||
-- This preserves existing insight IDs so that future entity_facts.source_insight_id
|
|
||||||
-- FK references remain valid.
|
|
||||||
|
|
||||||
CREATE TABLE photo_insights_new (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
file_path TEXT NOT NULL,
|
|
||||||
title TEXT NOT NULL,
|
|
||||||
summary TEXT NOT NULL,
|
|
||||||
generated_at BIGINT NOT NULL,
|
|
||||||
model_version TEXT NOT NULL,
|
|
||||||
is_current BOOLEAN NOT NULL DEFAULT 0
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Migrate existing rows; mark them all as current (one row per path currently).
|
|
||||||
INSERT INTO photo_insights_new (id, file_path, title, summary, generated_at, model_version, is_current)
|
|
||||||
SELECT id, file_path, title, summary, generated_at, model_version, 1
|
|
||||||
FROM photo_insights;
|
|
||||||
|
|
||||||
DROP TABLE photo_insights;
|
|
||||||
ALTER TABLE photo_insights_new RENAME TO photo_insights;
|
|
||||||
|
|
||||||
CREATE INDEX idx_photo_insights_file_path ON photo_insights(file_path);
|
|
||||||
CREATE INDEX idx_photo_insights_current ON photo_insights(file_path, is_current);
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
DROP TABLE IF EXISTS entity_photo_links;
|
|
||||||
DROP TABLE IF EXISTS entity_facts;
|
|
||||||
DROP TABLE IF EXISTS entities;
|
|
||||||
@@ -1,55 +0,0 @@
|
|||||||
-- Entity-relationship knowledge memory tables.
|
|
||||||
-- Entities are the nodes (people, places, events, things).
|
|
||||||
-- entity_facts are typed claims about or between entities.
|
|
||||||
-- entity_photo_links connect entities to specific photos.
|
|
||||||
|
|
||||||
CREATE TABLE entities (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
name TEXT NOT NULL,
|
|
||||||
entity_type TEXT NOT NULL, -- 'person' | 'place' | 'event' | 'thing'
|
|
||||||
description TEXT NOT NULL DEFAULT '',
|
|
||||||
embedding BLOB, -- 768-dim f32 vector; nullable if embedding service was unavailable
|
|
||||||
confidence REAL NOT NULL DEFAULT 0.5,
|
|
||||||
status TEXT NOT NULL DEFAULT 'active', -- 'active' | 'reviewed' | 'rejected'
|
|
||||||
created_at BIGINT NOT NULL,
|
|
||||||
updated_at BIGINT NOT NULL,
|
|
||||||
UNIQUE(name, entity_type)
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX idx_entities_type ON entities(entity_type);
|
|
||||||
CREATE INDEX idx_entities_status ON entities(status);
|
|
||||||
CREATE INDEX idx_entities_name ON entities(name);
|
|
||||||
|
|
||||||
CREATE TABLE entity_facts (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
subject_entity_id INTEGER NOT NULL,
|
|
||||||
predicate TEXT NOT NULL,
|
|
||||||
object_entity_id INTEGER, -- nullable: entity-to-entity relationship target
|
|
||||||
object_value TEXT, -- nullable: free-text attribute value
|
|
||||||
source_photo TEXT, -- photo path that prompted extraction (injected server-side)
|
|
||||||
source_insight_id INTEGER, -- backfilled after insight is stored
|
|
||||||
confidence REAL NOT NULL DEFAULT 0.6,
|
|
||||||
status TEXT NOT NULL DEFAULT 'active', -- 'active' | 'reviewed' | 'rejected'
|
|
||||||
created_at BIGINT NOT NULL,
|
|
||||||
CONSTRAINT fk_ef_subject FOREIGN KEY (subject_entity_id) REFERENCES entities(id) ON DELETE CASCADE,
|
|
||||||
CONSTRAINT fk_ef_object FOREIGN KEY (object_entity_id) REFERENCES entities(id) ON DELETE SET NULL,
|
|
||||||
CONSTRAINT fk_ef_insight FOREIGN KEY (source_insight_id) REFERENCES photo_insights(id) ON DELETE SET NULL,
|
|
||||||
CHECK (object_entity_id IS NOT NULL OR object_value IS NOT NULL)
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX idx_entity_facts_subject ON entity_facts(subject_entity_id);
|
|
||||||
CREATE INDEX idx_entity_facts_predicate ON entity_facts(predicate);
|
|
||||||
CREATE INDEX idx_entity_facts_status ON entity_facts(status);
|
|
||||||
CREATE INDEX idx_entity_facts_source_photo ON entity_facts(source_photo);
|
|
||||||
|
|
||||||
CREATE TABLE entity_photo_links (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
entity_id INTEGER NOT NULL,
|
|
||||||
file_path TEXT NOT NULL,
|
|
||||||
role TEXT NOT NULL, -- 'subject' | 'location' | 'event' | 'thing'
|
|
||||||
CONSTRAINT fk_epl_entity FOREIGN KEY (entity_id) REFERENCES entities(id) ON DELETE CASCADE,
|
|
||||||
UNIQUE(entity_id, file_path, role)
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX idx_entity_photo_links_entity ON entity_photo_links(entity_id);
|
|
||||||
CREATE INDEX idx_entity_photo_links_photo ON entity_photo_links(file_path);
|
|
||||||
@@ -1,14 +0,0 @@
|
|||||||
-- SQLite doesn't support DROP COLUMN directly, so we recreate the table
|
|
||||||
CREATE TABLE photo_insights_backup AS SELECT id, file_path, title, summary, generated_at, model_version, is_current FROM photo_insights;
|
|
||||||
DROP TABLE photo_insights;
|
|
||||||
CREATE TABLE photo_insights (
|
|
||||||
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
|
|
||||||
file_path TEXT NOT NULL,
|
|
||||||
title TEXT NOT NULL,
|
|
||||||
summary TEXT NOT NULL,
|
|
||||||
generated_at BIGINT NOT NULL,
|
|
||||||
model_version TEXT NOT NULL,
|
|
||||||
is_current BOOLEAN NOT NULL DEFAULT TRUE
|
|
||||||
);
|
|
||||||
INSERT INTO photo_insights SELECT * FROM photo_insights_backup;
|
|
||||||
DROP TABLE photo_insights_backup;
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
ALTER TABLE photo_insights ADD COLUMN training_messages TEXT;
|
|
||||||
ALTER TABLE photo_insights ADD COLUMN approved BOOLEAN;
|
|
||||||
@@ -1,155 +0,0 @@
|
|||||||
-- Revert multi-library support.
|
|
||||||
-- Drops library_id/content_hash/size_bytes, renames rel_path back to the
|
|
||||||
-- original column names, and drops the libraries table. Rows originally
|
|
||||||
-- from non-primary libraries (id > 1) would be orphaned, so the rollback
|
|
||||||
-- keeps only rows from library_id=1.
|
|
||||||
|
|
||||||
PRAGMA foreign_keys=OFF;
|
|
||||||
|
|
||||||
-- tagged_photo: rel_path → photo_name.
|
|
||||||
DROP INDEX IF EXISTS idx_tagged_photo_relpath_tag;
|
|
||||||
DROP INDEX IF EXISTS idx_tagged_photo_rel_path;
|
|
||||||
ALTER TABLE tagged_photo RENAME COLUMN rel_path TO photo_name;
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_tagged_photo_photo_name ON tagged_photo(photo_name);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_tagged_photo_count ON tagged_photo(photo_name, tag_id);
|
|
||||||
|
|
||||||
-- favorites: rel_path → path.
|
|
||||||
DROP INDEX IF EXISTS idx_favorites_unique;
|
|
||||||
DROP INDEX IF EXISTS idx_favorites_rel_path;
|
|
||||||
ALTER TABLE favorites RENAME COLUMN rel_path TO path;
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_favorites_path ON favorites(path);
|
|
||||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_favorites_unique ON favorites(userid, path);
|
|
||||||
|
|
||||||
-- video_preview_clips: drop library_id, rel_path → file_path.
|
|
||||||
CREATE TABLE video_preview_clips_old (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
file_path TEXT NOT NULL UNIQUE,
|
|
||||||
status TEXT NOT NULL DEFAULT 'pending',
|
|
||||||
duration_seconds REAL,
|
|
||||||
file_size_bytes INTEGER,
|
|
||||||
error_message TEXT,
|
|
||||||
created_at TEXT NOT NULL,
|
|
||||||
updated_at TEXT NOT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO video_preview_clips_old (
|
|
||||||
id, file_path, status, duration_seconds, file_size_bytes,
|
|
||||||
error_message, created_at, updated_at
|
|
||||||
)
|
|
||||||
SELECT
|
|
||||||
id, rel_path, status, duration_seconds, file_size_bytes,
|
|
||||||
error_message, created_at, updated_at
|
|
||||||
FROM video_preview_clips
|
|
||||||
WHERE library_id = 1;
|
|
||||||
|
|
||||||
DROP TABLE video_preview_clips;
|
|
||||||
ALTER TABLE video_preview_clips_old RENAME TO video_preview_clips;
|
|
||||||
|
|
||||||
CREATE INDEX idx_preview_clips_file_path ON video_preview_clips(file_path);
|
|
||||||
CREATE INDEX idx_preview_clips_status ON video_preview_clips(status);
|
|
||||||
|
|
||||||
-- entity_photo_links: drop library_id, rel_path → file_path.
|
|
||||||
CREATE TABLE entity_photo_links_old (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
entity_id INTEGER NOT NULL,
|
|
||||||
file_path TEXT NOT NULL,
|
|
||||||
role TEXT NOT NULL,
|
|
||||||
CONSTRAINT fk_epl_entity FOREIGN KEY (entity_id) REFERENCES entities(id) ON DELETE CASCADE,
|
|
||||||
UNIQUE(entity_id, file_path, role)
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO entity_photo_links_old (id, entity_id, file_path, role)
|
|
||||||
SELECT id, entity_id, rel_path, role
|
|
||||||
FROM entity_photo_links
|
|
||||||
WHERE library_id = 1;
|
|
||||||
|
|
||||||
DROP TABLE entity_photo_links;
|
|
||||||
ALTER TABLE entity_photo_links_old RENAME TO entity_photo_links;
|
|
||||||
|
|
||||||
CREATE INDEX idx_entity_photo_links_entity ON entity_photo_links(entity_id);
|
|
||||||
CREATE INDEX idx_entity_photo_links_photo ON entity_photo_links(file_path);
|
|
||||||
|
|
||||||
-- photo_insights: drop library_id, rel_path → file_path.
|
|
||||||
CREATE TABLE photo_insights_old (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
file_path TEXT NOT NULL,
|
|
||||||
title TEXT NOT NULL,
|
|
||||||
summary TEXT NOT NULL,
|
|
||||||
generated_at BIGINT NOT NULL,
|
|
||||||
model_version TEXT NOT NULL,
|
|
||||||
is_current BOOLEAN NOT NULL DEFAULT 0,
|
|
||||||
training_messages TEXT,
|
|
||||||
approved BOOLEAN
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO photo_insights_old (
|
|
||||||
id, file_path, title, summary, generated_at, model_version, is_current,
|
|
||||||
training_messages, approved
|
|
||||||
)
|
|
||||||
SELECT
|
|
||||||
id, rel_path, title, summary, generated_at, model_version, is_current,
|
|
||||||
training_messages, approved
|
|
||||||
FROM photo_insights
|
|
||||||
WHERE library_id = 1;
|
|
||||||
|
|
||||||
DROP TABLE photo_insights;
|
|
||||||
ALTER TABLE photo_insights_old RENAME TO photo_insights;
|
|
||||||
|
|
||||||
CREATE INDEX idx_photo_insights_file_path ON photo_insights(file_path);
|
|
||||||
CREATE INDEX idx_photo_insights_current ON photo_insights(file_path, is_current);
|
|
||||||
|
|
||||||
-- image_exif: drop library_id/content_hash/size_bytes, rel_path → file_path.
|
|
||||||
CREATE TABLE image_exif_old (
|
|
||||||
id INTEGER PRIMARY KEY NOT NULL,
|
|
||||||
file_path TEXT NOT NULL UNIQUE,
|
|
||||||
camera_make TEXT,
|
|
||||||
camera_model TEXT,
|
|
||||||
lens_model TEXT,
|
|
||||||
width INTEGER,
|
|
||||||
height INTEGER,
|
|
||||||
orientation INTEGER,
|
|
||||||
gps_latitude REAL,
|
|
||||||
gps_longitude REAL,
|
|
||||||
gps_altitude REAL,
|
|
||||||
focal_length REAL,
|
|
||||||
aperture REAL,
|
|
||||||
shutter_speed TEXT,
|
|
||||||
iso INTEGER,
|
|
||||||
date_taken BIGINT,
|
|
||||||
created_time BIGINT NOT NULL,
|
|
||||||
last_modified BIGINT NOT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO image_exif_old (
|
|
||||||
id, file_path,
|
|
||||||
camera_make, camera_model, lens_model,
|
|
||||||
width, height, orientation,
|
|
||||||
gps_latitude, gps_longitude, gps_altitude,
|
|
||||||
focal_length, aperture, shutter_speed, iso, date_taken,
|
|
||||||
created_time, last_modified
|
|
||||||
)
|
|
||||||
SELECT
|
|
||||||
id, rel_path,
|
|
||||||
camera_make, camera_model, lens_model,
|
|
||||||
width, height, orientation,
|
|
||||||
gps_latitude, gps_longitude, gps_altitude,
|
|
||||||
focal_length, aperture, shutter_speed, iso, date_taken,
|
|
||||||
created_time, last_modified
|
|
||||||
FROM image_exif
|
|
||||||
WHERE library_id = 1;
|
|
||||||
|
|
||||||
DROP TABLE image_exif;
|
|
||||||
ALTER TABLE image_exif_old RENAME TO image_exif;
|
|
||||||
|
|
||||||
CREATE INDEX idx_image_exif_file_path ON image_exif(file_path);
|
|
||||||
CREATE INDEX idx_image_exif_camera ON image_exif(camera_make, camera_model);
|
|
||||||
CREATE INDEX idx_image_exif_gps ON image_exif(gps_latitude, gps_longitude);
|
|
||||||
CREATE INDEX idx_image_exif_date_taken ON image_exif(date_taken);
|
|
||||||
CREATE INDEX idx_image_exif_date_path ON image_exif(date_taken DESC, file_path);
|
|
||||||
|
|
||||||
-- Finally, drop the libraries registry.
|
|
||||||
DROP TABLE libraries;
|
|
||||||
|
|
||||||
PRAGMA foreign_keys=ON;
|
|
||||||
|
|
||||||
ANALYZE;
|
|
||||||
@@ -1,216 +0,0 @@
|
|||||||
-- Multi-library support.
|
|
||||||
-- Adds `libraries` registry table and a `library_id` column on per-instance
|
|
||||||
-- metadata tables. Renames `file_path` / `photo_name` to `rel_path` for
|
|
||||||
-- semantic clarity (values already stored relative to BASE_PATH).
|
|
||||||
-- Adds `content_hash` + `size_bytes` to `image_exif` to support
|
|
||||||
-- content-based dedup of thumbnails and HLS output across libraries.
|
|
||||||
--
|
|
||||||
-- SQLite cannot alter column constraints in place, so per-instance tables
|
|
||||||
-- are recreated following the idiom established in
|
|
||||||
-- 2026-04-02-000000_photo_insights_history/up.sql. Existing row `id`s are
|
|
||||||
-- preserved so foreign keys (entity_facts.source_insight_id, etc.) remain
|
|
||||||
-- valid after migration.
|
|
||||||
|
|
||||||
PRAGMA foreign_keys=OFF;
|
|
||||||
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
-- 1. Libraries registry.
|
|
||||||
-- Seeded with a placeholder for the primary library; AppState patches
|
|
||||||
-- `root_path` from the BASE_PATH env var on first boot. Subsequent
|
|
||||||
-- prod-to-dev DB syncs update this row via a single SQL UPDATE.
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
CREATE TABLE libraries (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
name TEXT NOT NULL UNIQUE,
|
|
||||||
root_path TEXT NOT NULL,
|
|
||||||
created_at BIGINT NOT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO libraries (id, name, root_path, created_at)
|
|
||||||
VALUES (1, 'main', 'BASE_PATH_PLACEHOLDER', strftime('%s','now'));
|
|
||||||
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
-- 2. image_exif: + library_id, file_path → rel_path, + content_hash/size_bytes.
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
CREATE TABLE image_exif_new (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
library_id INTEGER NOT NULL REFERENCES libraries(id),
|
|
||||||
rel_path TEXT NOT NULL,
|
|
||||||
|
|
||||||
-- Camera information
|
|
||||||
camera_make TEXT,
|
|
||||||
camera_model TEXT,
|
|
||||||
lens_model TEXT,
|
|
||||||
|
|
||||||
-- Image properties
|
|
||||||
width INTEGER,
|
|
||||||
height INTEGER,
|
|
||||||
orientation INTEGER,
|
|
||||||
|
|
||||||
-- GPS
|
|
||||||
gps_latitude REAL,
|
|
||||||
gps_longitude REAL,
|
|
||||||
gps_altitude REAL,
|
|
||||||
|
|
||||||
-- Capture settings
|
|
||||||
focal_length REAL,
|
|
||||||
aperture REAL,
|
|
||||||
shutter_speed TEXT,
|
|
||||||
iso INTEGER,
|
|
||||||
date_taken BIGINT,
|
|
||||||
|
|
||||||
-- Housekeeping
|
|
||||||
created_time BIGINT NOT NULL,
|
|
||||||
last_modified BIGINT NOT NULL,
|
|
||||||
|
|
||||||
-- Content identity (backfilled by the `backfill_hashes` binary and by the watcher for new files)
|
|
||||||
content_hash TEXT,
|
|
||||||
size_bytes BIGINT,
|
|
||||||
|
|
||||||
UNIQUE(library_id, rel_path)
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO image_exif_new (
|
|
||||||
id, library_id, rel_path,
|
|
||||||
camera_make, camera_model, lens_model,
|
|
||||||
width, height, orientation,
|
|
||||||
gps_latitude, gps_longitude, gps_altitude,
|
|
||||||
focal_length, aperture, shutter_speed, iso, date_taken,
|
|
||||||
created_time, last_modified
|
|
||||||
)
|
|
||||||
SELECT
|
|
||||||
id, 1, file_path,
|
|
||||||
camera_make, camera_model, lens_model,
|
|
||||||
width, height, orientation,
|
|
||||||
gps_latitude, gps_longitude, gps_altitude,
|
|
||||||
focal_length, aperture, shutter_speed, iso, date_taken,
|
|
||||||
created_time, last_modified
|
|
||||||
FROM image_exif;
|
|
||||||
|
|
||||||
DROP TABLE image_exif;
|
|
||||||
ALTER TABLE image_exif_new RENAME TO image_exif;
|
|
||||||
|
|
||||||
CREATE INDEX idx_image_exif_rel_path ON image_exif(rel_path);
|
|
||||||
CREATE INDEX idx_image_exif_camera ON image_exif(camera_make, camera_model);
|
|
||||||
CREATE INDEX idx_image_exif_gps ON image_exif(gps_latitude, gps_longitude);
|
|
||||||
CREATE INDEX idx_image_exif_date_taken ON image_exif(date_taken);
|
|
||||||
CREATE INDEX idx_image_exif_date_path ON image_exif(date_taken DESC, rel_path);
|
|
||||||
CREATE INDEX idx_image_exif_lib_date ON image_exif(library_id, date_taken);
|
|
||||||
CREATE INDEX idx_image_exif_content_hash ON image_exif(content_hash);
|
|
||||||
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
-- 3. photo_insights: + library_id, file_path → rel_path.
|
|
||||||
-- Preserve `id` so entity_facts.source_insight_id FKs remain valid.
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
CREATE TABLE photo_insights_new (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
library_id INTEGER NOT NULL REFERENCES libraries(id),
|
|
||||||
rel_path TEXT NOT NULL,
|
|
||||||
title TEXT NOT NULL,
|
|
||||||
summary TEXT NOT NULL,
|
|
||||||
generated_at BIGINT NOT NULL,
|
|
||||||
model_version TEXT NOT NULL,
|
|
||||||
is_current BOOLEAN NOT NULL DEFAULT 0,
|
|
||||||
training_messages TEXT,
|
|
||||||
approved BOOLEAN
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO photo_insights_new (
|
|
||||||
id, library_id, rel_path, title, summary, generated_at, model_version,
|
|
||||||
is_current, training_messages, approved
|
|
||||||
)
|
|
||||||
SELECT
|
|
||||||
id, 1, file_path, title, summary, generated_at, model_version,
|
|
||||||
is_current, training_messages, approved
|
|
||||||
FROM photo_insights;
|
|
||||||
|
|
||||||
DROP TABLE photo_insights;
|
|
||||||
ALTER TABLE photo_insights_new RENAME TO photo_insights;
|
|
||||||
|
|
||||||
CREATE INDEX idx_photo_insights_rel_path ON photo_insights(rel_path);
|
|
||||||
CREATE INDEX idx_photo_insights_current ON photo_insights(library_id, rel_path, is_current);
|
|
||||||
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
-- 4. entity_photo_links: + library_id, file_path → rel_path.
|
|
||||||
-- Preserves entity FK; UNIQUE now includes library_id to allow the same
|
|
||||||
-- rel_path to link entities in multiple libraries independently.
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
CREATE TABLE entity_photo_links_new (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
entity_id INTEGER NOT NULL,
|
|
||||||
library_id INTEGER NOT NULL REFERENCES libraries(id),
|
|
||||||
rel_path TEXT NOT NULL,
|
|
||||||
role TEXT NOT NULL,
|
|
||||||
CONSTRAINT fk_epl_entity FOREIGN KEY (entity_id) REFERENCES entities(id) ON DELETE CASCADE,
|
|
||||||
UNIQUE(entity_id, library_id, rel_path, role)
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO entity_photo_links_new (id, entity_id, library_id, rel_path, role)
|
|
||||||
SELECT id, entity_id, 1, file_path, role FROM entity_photo_links;
|
|
||||||
|
|
||||||
DROP TABLE entity_photo_links;
|
|
||||||
ALTER TABLE entity_photo_links_new RENAME TO entity_photo_links;
|
|
||||||
|
|
||||||
CREATE INDEX idx_entity_photo_links_entity ON entity_photo_links(entity_id);
|
|
||||||
CREATE INDEX idx_entity_photo_links_photo ON entity_photo_links(library_id, rel_path);
|
|
||||||
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
-- 5. video_preview_clips: + library_id, file_path → rel_path.
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
CREATE TABLE video_preview_clips_new (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
library_id INTEGER NOT NULL REFERENCES libraries(id),
|
|
||||||
rel_path TEXT NOT NULL,
|
|
||||||
status TEXT NOT NULL DEFAULT 'pending',
|
|
||||||
duration_seconds REAL,
|
|
||||||
file_size_bytes INTEGER,
|
|
||||||
error_message TEXT,
|
|
||||||
created_at TEXT NOT NULL,
|
|
||||||
updated_at TEXT NOT NULL,
|
|
||||||
UNIQUE(library_id, rel_path)
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO video_preview_clips_new (
|
|
||||||
id, library_id, rel_path, status, duration_seconds, file_size_bytes,
|
|
||||||
error_message, created_at, updated_at
|
|
||||||
)
|
|
||||||
SELECT
|
|
||||||
id, 1, file_path, status, duration_seconds, file_size_bytes,
|
|
||||||
error_message, created_at, updated_at
|
|
||||||
FROM video_preview_clips;
|
|
||||||
|
|
||||||
DROP TABLE video_preview_clips;
|
|
||||||
ALTER TABLE video_preview_clips_new RENAME TO video_preview_clips;
|
|
||||||
|
|
||||||
CREATE INDEX idx_preview_clips_rel_path ON video_preview_clips(rel_path);
|
|
||||||
CREATE INDEX idx_preview_clips_status ON video_preview_clips(status);
|
|
||||||
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
-- 6. favorites: path → rel_path. Library-agnostic (cross-library sharing).
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
ALTER TABLE favorites RENAME COLUMN path TO rel_path;
|
|
||||||
|
|
||||||
DROP INDEX IF EXISTS idx_favorites_path;
|
|
||||||
DROP INDEX IF EXISTS idx_favorites_unique;
|
|
||||||
CREATE INDEX idx_favorites_rel_path ON favorites(rel_path);
|
|
||||||
CREATE UNIQUE INDEX idx_favorites_unique ON favorites(userid, rel_path);
|
|
||||||
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
-- 7. tagged_photo: photo_name → rel_path. Library-agnostic.
|
|
||||||
-- Dedup first so the (rel_path, tag_id) unique index can be created safely.
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
ALTER TABLE tagged_photo RENAME COLUMN photo_name TO rel_path;
|
|
||||||
|
|
||||||
DELETE FROM tagged_photo
|
|
||||||
WHERE id NOT IN (
|
|
||||||
SELECT MIN(id) FROM tagged_photo GROUP BY rel_path, tag_id
|
|
||||||
);
|
|
||||||
|
|
||||||
DROP INDEX IF EXISTS idx_tagged_photo_photo_name;
|
|
||||||
DROP INDEX IF EXISTS idx_tagged_photo_count;
|
|
||||||
CREATE INDEX idx_tagged_photo_rel_path ON tagged_photo(rel_path);
|
|
||||||
CREATE UNIQUE INDEX idx_tagged_photo_relpath_tag ON tagged_photo(rel_path, tag_id);
|
|
||||||
|
|
||||||
PRAGMA foreign_keys=ON;
|
|
||||||
|
|
||||||
ANALYZE;
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
-- No-op: there's no sensible way to recover which rows originally used
|
|
||||||
-- backslashes, and there's no reason to want backslashes back. The
|
|
||||||
-- deleted duplicates are also gone.
|
|
||||||
SELECT 1;
|
|
||||||
@@ -1,85 +0,0 @@
|
|||||||
-- Normalize `rel_path` columns to forward slashes. Windows ingest
|
|
||||||
-- historically produced a mix of `\` and `/`, which broke lookups and
|
|
||||||
-- caused spurious UNIQUE-constraint violations on re-registration.
|
|
||||||
--
|
|
||||||
-- SQLite enforces UNIQUE per-row during UPDATE, so we have to drop
|
|
||||||
-- losing duplicates BEFORE normalizing. For each table that has a
|
|
||||||
-- UNIQUE on rel_path, we delete rows whose normalized form already
|
|
||||||
-- exists in canonical (forward-slash) form — keeping the existing
|
|
||||||
-- forward-slash row as the survivor. Then a flat UPDATE finishes the
|
|
||||||
-- job for remaining backslash rows.
|
|
||||||
|
|
||||||
-- image_exif: UNIQUE(library_id, rel_path)
|
|
||||||
DELETE FROM image_exif
|
|
||||||
WHERE rel_path LIKE '%\%'
|
|
||||||
AND EXISTS (
|
|
||||||
SELECT 1 FROM image_exif AS other
|
|
||||||
WHERE other.library_id = image_exif.library_id
|
|
||||||
AND other.rel_path = REPLACE(image_exif.rel_path, '\', '/')
|
|
||||||
AND other.id != image_exif.id
|
|
||||||
);
|
|
||||||
UPDATE image_exif
|
|
||||||
SET rel_path = REPLACE(rel_path, '\', '/')
|
|
||||||
WHERE rel_path LIKE '%\%';
|
|
||||||
|
|
||||||
-- favorites: UNIQUE(userid, rel_path)
|
|
||||||
DELETE FROM favorites
|
|
||||||
WHERE rel_path LIKE '%\%'
|
|
||||||
AND EXISTS (
|
|
||||||
SELECT 1 FROM favorites AS other
|
|
||||||
WHERE other.userid = favorites.userid
|
|
||||||
AND other.rel_path = REPLACE(favorites.rel_path, '\', '/')
|
|
||||||
AND other.id != favorites.id
|
|
||||||
);
|
|
||||||
UPDATE favorites
|
|
||||||
SET rel_path = REPLACE(rel_path, '\', '/')
|
|
||||||
WHERE rel_path LIKE '%\%';
|
|
||||||
|
|
||||||
-- tagged_photo: UNIQUE(rel_path, tag_id)
|
|
||||||
DELETE FROM tagged_photo
|
|
||||||
WHERE rel_path LIKE '%\%'
|
|
||||||
AND EXISTS (
|
|
||||||
SELECT 1 FROM tagged_photo AS other
|
|
||||||
WHERE other.tag_id = tagged_photo.tag_id
|
|
||||||
AND other.rel_path = REPLACE(tagged_photo.rel_path, '\', '/')
|
|
||||||
AND other.id != tagged_photo.id
|
|
||||||
);
|
|
||||||
UPDATE tagged_photo
|
|
||||||
SET rel_path = REPLACE(rel_path, '\', '/')
|
|
||||||
WHERE rel_path LIKE '%\%';
|
|
||||||
|
|
||||||
-- entity_photo_links: UNIQUE(entity_id, library_id, rel_path, role)
|
|
||||||
DELETE FROM entity_photo_links
|
|
||||||
WHERE rel_path LIKE '%\%'
|
|
||||||
AND EXISTS (
|
|
||||||
SELECT 1 FROM entity_photo_links AS other
|
|
||||||
WHERE other.entity_id = entity_photo_links.entity_id
|
|
||||||
AND other.library_id = entity_photo_links.library_id
|
|
||||||
AND other.role = entity_photo_links.role
|
|
||||||
AND other.rel_path = REPLACE(entity_photo_links.rel_path, '\', '/')
|
|
||||||
AND other.id != entity_photo_links.id
|
|
||||||
);
|
|
||||||
UPDATE entity_photo_links
|
|
||||||
SET rel_path = REPLACE(rel_path, '\', '/')
|
|
||||||
WHERE rel_path LIKE '%\%';
|
|
||||||
|
|
||||||
-- video_preview_clips: UNIQUE(library_id, rel_path)
|
|
||||||
DELETE FROM video_preview_clips
|
|
||||||
WHERE rel_path LIKE '%\%'
|
|
||||||
AND EXISTS (
|
|
||||||
SELECT 1 FROM video_preview_clips AS other
|
|
||||||
WHERE other.library_id = video_preview_clips.library_id
|
|
||||||
AND other.rel_path = REPLACE(video_preview_clips.rel_path, '\', '/')
|
|
||||||
AND other.id != video_preview_clips.id
|
|
||||||
);
|
|
||||||
UPDATE video_preview_clips
|
|
||||||
SET rel_path = REPLACE(rel_path, '\', '/')
|
|
||||||
WHERE rel_path LIKE '%\%';
|
|
||||||
|
|
||||||
-- photo_insights has no UNIQUE on rel_path (history table), so a plain
|
|
||||||
-- normalize is safe.
|
|
||||||
UPDATE photo_insights
|
|
||||||
SET rel_path = REPLACE(rel_path, '\', '/')
|
|
||||||
WHERE rel_path LIKE '%\%';
|
|
||||||
|
|
||||||
ANALYZE;
|
|
||||||
@@ -1,23 +0,0 @@
|
|||||||
-- SQLite can't DROP COLUMN cleanly on older versions; rebuild the table.
|
|
||||||
CREATE TABLE photo_insights_backup AS
|
|
||||||
SELECT id, library_id, rel_path, title, summary, generated_at, model_version,
|
|
||||||
is_current, training_messages, approved
|
|
||||||
FROM photo_insights;
|
|
||||||
DROP TABLE photo_insights;
|
|
||||||
CREATE TABLE photo_insights (
|
|
||||||
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
|
|
||||||
library_id INTEGER NOT NULL REFERENCES libraries(id),
|
|
||||||
rel_path TEXT NOT NULL,
|
|
||||||
title TEXT NOT NULL,
|
|
||||||
summary TEXT NOT NULL,
|
|
||||||
generated_at BIGINT NOT NULL,
|
|
||||||
model_version TEXT NOT NULL,
|
|
||||||
is_current BOOLEAN NOT NULL DEFAULT TRUE,
|
|
||||||
training_messages TEXT,
|
|
||||||
approved BOOLEAN
|
|
||||||
);
|
|
||||||
INSERT INTO photo_insights
|
|
||||||
SELECT id, library_id, rel_path, title, summary, generated_at, model_version,
|
|
||||||
is_current, training_messages, approved
|
|
||||||
FROM photo_insights_backup;
|
|
||||||
DROP TABLE photo_insights_backup;
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
ALTER TABLE photo_insights ADD COLUMN backend TEXT NOT NULL DEFAULT 'local';
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
-- SQLite can't DROP COLUMN cleanly on older versions; rebuild the table.
|
|
||||||
CREATE TABLE photo_insights_backup AS
|
|
||||||
SELECT id, library_id, rel_path, title, summary, generated_at, model_version,
|
|
||||||
is_current, training_messages, approved, backend
|
|
||||||
FROM photo_insights;
|
|
||||||
DROP TABLE photo_insights;
|
|
||||||
CREATE TABLE photo_insights (
|
|
||||||
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
|
|
||||||
library_id INTEGER NOT NULL REFERENCES libraries(id),
|
|
||||||
rel_path TEXT NOT NULL,
|
|
||||||
title TEXT NOT NULL,
|
|
||||||
summary TEXT NOT NULL,
|
|
||||||
generated_at BIGINT NOT NULL,
|
|
||||||
model_version TEXT NOT NULL,
|
|
||||||
is_current BOOLEAN NOT NULL DEFAULT TRUE,
|
|
||||||
training_messages TEXT,
|
|
||||||
approved BOOLEAN,
|
|
||||||
backend TEXT NOT NULL DEFAULT 'local'
|
|
||||||
);
|
|
||||||
INSERT INTO photo_insights
|
|
||||||
SELECT id, library_id, rel_path, title, summary, generated_at, model_version,
|
|
||||||
is_current, training_messages, approved, backend
|
|
||||||
FROM photo_insights_backup;
|
|
||||||
DROP TABLE photo_insights_backup;
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
ALTER TABLE photo_insights ADD COLUMN fewshot_source_ids TEXT;
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
DROP TABLE IF EXISTS face_detections;
|
|
||||||
DROP TABLE IF EXISTS persons;
|
|
||||||
@@ -1,67 +0,0 @@
|
|||||||
-- Local face recognition tables.
|
|
||||||
--
|
|
||||||
-- `persons` are visual identities (the "who" of a face). The optional
|
|
||||||
-- `entity_id` bridges to the existing knowledge graph `entities` table —
|
|
||||||
-- when set, this person is the visual side of an LLM-extracted entity.
|
|
||||||
-- Don't auto-create entities from persons; the entity table represents
|
|
||||||
-- LLM-extracted knowledge with its own confidence semantics, and silently
|
|
||||||
-- filling it from face detections muddies the provenance.
|
|
||||||
--
|
|
||||||
-- `face_detections` carries one row per detected face on a content_hash,
|
|
||||||
-- plus marker rows with `status='no_faces'` or `status='failed'` so the
|
|
||||||
-- file watcher knows not to re-scan a hash. Keying on `content_hash`
|
|
||||||
-- (cross-library dedup) rather than `(library_id, rel_path)` means the
|
|
||||||
-- same JPEG in two libraries is scanned once. The denormalized `rel_path`
|
|
||||||
-- carries the most-recently-seen path — useful for cluster-thumb URL
|
|
||||||
-- generation; canonical path lookup goes through image_exif.
|
|
||||||
|
|
||||||
CREATE TABLE persons (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
name TEXT NOT NULL,
|
|
||||||
cover_face_id INTEGER, -- backfilled when the first face binds
|
|
||||||
entity_id INTEGER, -- optional bridge to entities(id)
|
|
||||||
created_from_tag BOOLEAN NOT NULL DEFAULT 0,
|
|
||||||
notes TEXT,
|
|
||||||
created_at BIGINT NOT NULL,
|
|
||||||
updated_at BIGINT NOT NULL,
|
|
||||||
CONSTRAINT fk_persons_entity FOREIGN KEY (entity_id) REFERENCES entities(id) ON DELETE SET NULL,
|
|
||||||
UNIQUE(name COLLATE NOCASE)
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX idx_persons_entity ON persons(entity_id);
|
|
||||||
|
|
||||||
CREATE TABLE face_detections (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
library_id INTEGER NOT NULL,
|
|
||||||
content_hash TEXT NOT NULL, -- canonical key (cross-library dedup)
|
|
||||||
rel_path TEXT NOT NULL, -- denormalized; most recently seen
|
|
||||||
bbox_x REAL, -- normalized 0..1; NULL on marker rows
|
|
||||||
bbox_y REAL,
|
|
||||||
bbox_w REAL,
|
|
||||||
bbox_h REAL,
|
|
||||||
embedding BLOB, -- 512×f32 = 2048 bytes; NULL on marker rows
|
|
||||||
confidence REAL, -- detector score
|
|
||||||
source TEXT NOT NULL, -- 'auto' | 'manual'
|
|
||||||
person_id INTEGER,
|
|
||||||
status TEXT NOT NULL DEFAULT 'detected', -- 'detected' | 'no_faces' | 'failed'
|
|
||||||
model_version TEXT NOT NULL, -- e.g. 'buffalo_l'; embedding lineage
|
|
||||||
created_at BIGINT NOT NULL,
|
|
||||||
CONSTRAINT fk_fd_library FOREIGN KEY (library_id) REFERENCES libraries(id),
|
|
||||||
CONSTRAINT fk_fd_person FOREIGN KEY (person_id) REFERENCES persons(id) ON DELETE SET NULL,
|
|
||||||
-- Detected rows carry geometry + embedding; marker rows ('no_faces',
|
|
||||||
-- 'failed') carry neither. CHECK enforces the invariant so manual
|
|
||||||
-- inserts can't slip through with half a row.
|
|
||||||
CONSTRAINT chk_marker CHECK (
|
|
||||||
(status = 'detected' AND bbox_x IS NOT NULL AND embedding IS NOT NULL)
|
|
||||||
OR (status IN ('no_faces','failed') AND bbox_x IS NULL AND embedding IS NULL)
|
|
||||||
)
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX idx_face_detections_hash ON face_detections(content_hash);
|
|
||||||
CREATE INDEX idx_face_detections_lib_path ON face_detections(library_id, rel_path);
|
|
||||||
CREATE INDEX idx_face_detections_person ON face_detections(person_id);
|
|
||||||
CREATE INDEX idx_face_detections_status ON face_detections(status);
|
|
||||||
-- One marker row per (content_hash, status='no_faces') so the file watcher
|
|
||||||
-- doesn't double-mark when a hash is seen on multiple full-scan passes.
|
|
||||||
CREATE UNIQUE INDEX idx_face_detections_no_faces_unique
|
|
||||||
ON face_detections(content_hash) WHERE status = 'no_faces';
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
DROP INDEX IF EXISTS idx_persons_is_ignored;
|
|
||||||
ALTER TABLE persons DROP COLUMN is_ignored;
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
-- IGNORE / junk bucket for the face recognition feature.
|
|
||||||
--
|
|
||||||
-- An "Ignored" person is the destination for strangers, faces the user
|
|
||||||
-- doesn't want tagged, and false detections. It looks like any other
|
|
||||||
-- person row (so face_detections.person_id stays a clean foreign key)
|
|
||||||
-- but `is_ignored=1` flags it for special UI treatment:
|
|
||||||
-- - hidden from the persons list by default
|
|
||||||
-- - excluded from `find_persons_by_names_ci` so a tag-name match
|
|
||||||
-- can never auto-bind a real face to the ignore bucket
|
|
||||||
-- - cluster-suggest already filters by `person_id IS NULL`, so faces
|
|
||||||
-- bound to an ignored person are naturally excluded from future
|
|
||||||
-- re-clustering
|
|
||||||
--
|
|
||||||
-- Partial index because the WHERE-clause is small (typically 1 row),
|
|
||||||
-- and we only ever query for `is_ignored = 1` to find the bucket.
|
|
||||||
|
|
||||||
ALTER TABLE persons ADD COLUMN is_ignored BOOLEAN NOT NULL DEFAULT 0;
|
|
||||||
|
|
||||||
CREATE INDEX idx_persons_is_ignored
|
|
||||||
ON persons(is_ignored) WHERE is_ignored = 1;
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
DROP INDEX IF EXISTS idx_tags_name_nocase;
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
-- Tags only enforced uniqueness in application code (the add_tag handler
|
|
||||||
-- looks up by name before inserting). The schema itself accepted dupes,
|
|
||||||
-- so a divergent code path could land two tags with the same name. Now
|
|
||||||
-- that we expose a rename endpoint we want a hard guarantee: case-
|
|
||||||
-- insensitive UNIQUE on tags.name.
|
|
||||||
|
|
||||||
-- Pre-flight: collapse exact-name duplicates (case-insensitive) onto the
|
|
||||||
-- lowest-id row before adding the constraint, otherwise the index
|
|
||||||
-- creation fails on any DB that ever produced dupes. On a clean DB this
|
|
||||||
-- is a no-op.
|
|
||||||
UPDATE tagged_photo
|
|
||||||
SET tag_id = (
|
|
||||||
SELECT MIN(t2.id) FROM tags t2
|
|
||||||
WHERE LOWER(t2.name) = LOWER((SELECT name FROM tags WHERE id = tagged_photo.tag_id))
|
|
||||||
)
|
|
||||||
WHERE tag_id IN (
|
|
||||||
SELECT t.id FROM tags t
|
|
||||||
WHERE t.id <> (
|
|
||||||
SELECT MIN(t2.id) FROM tags t2 WHERE LOWER(t2.name) = LOWER(t.name)
|
|
||||||
)
|
|
||||||
);
|
|
||||||
|
|
||||||
DELETE FROM tags
|
|
||||||
WHERE id <> (
|
|
||||||
SELECT MIN(t2.id) FROM tags t2 WHERE LOWER(t2.name) = LOWER(tags.name)
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE UNIQUE INDEX idx_tags_name_nocase ON tags (name COLLATE NOCASE);
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
DROP INDEX IF EXISTS idx_photo_insights_content_hash;
|
|
||||||
ALTER TABLE photo_insights DROP COLUMN content_hash;
|
|
||||||
|
|
||||||
DROP INDEX IF EXISTS idx_tagged_photo_content_hash;
|
|
||||||
ALTER TABLE tagged_photo DROP COLUMN content_hash;
|
|
||||||
@@ -1,64 +0,0 @@
|
|||||||
-- Phase B of the multi-library data-model rollout: add a nullable
|
|
||||||
-- `content_hash` column to derived/user-intent tables that should follow
|
|
||||||
-- the bytes rather than the path. Reads will prefer hash-key joins and
|
|
||||||
-- fall back to rel_path while the column is null. A separate
|
|
||||||
-- reconciliation pass collapses duplicates as the column populates.
|
|
||||||
--
|
|
||||||
-- See CLAUDE.md → "Multi-library data model" for the policy. The
|
|
||||||
-- reference implementation is `face_detections`, which has been
|
|
||||||
-- hash-keyed since it was introduced.
|
|
||||||
--
|
|
||||||
-- Tables in this migration:
|
|
||||||
-- * tagged_photo — user-intent (tags follow the bytes)
|
|
||||||
-- * photo_insights — intrinsic to bytes (LLM-generated description)
|
|
||||||
--
|
|
||||||
-- favorites is the natural third candidate but its DAO is barely used in
|
|
||||||
-- v1 and the row count is tiny; deferring lets this migration stay
|
|
||||||
-- focused on the high-volume tables that drive cross-library overhead.
|
|
||||||
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
-- tagged_photo
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
ALTER TABLE tagged_photo ADD COLUMN content_hash TEXT;
|
|
||||||
|
|
||||||
-- Backfill: for each tagged_photo row, find the content_hash for its
|
|
||||||
-- rel_path. tagged_photo doesn't carry a library_id, so a rel_path that
|
|
||||||
-- exists under multiple libraries with different content is genuinely
|
|
||||||
-- ambiguous — we take the first matching image_exif row. The
|
|
||||||
-- reconciliation pass at runtime cleans up any rows that resolve
|
|
||||||
-- differently once a hash is known per library.
|
|
||||||
UPDATE tagged_photo
|
|
||||||
SET content_hash = (
|
|
||||||
SELECT content_hash FROM image_exif
|
|
||||||
WHERE image_exif.rel_path = tagged_photo.rel_path
|
|
||||||
AND image_exif.content_hash IS NOT NULL
|
|
||||||
LIMIT 1
|
|
||||||
)
|
|
||||||
WHERE content_hash IS NULL;
|
|
||||||
|
|
||||||
-- Hash-key index. Partial (only non-null rows) to keep the index small
|
|
||||||
-- during the transitional window where most rows are still null.
|
|
||||||
CREATE INDEX idx_tagged_photo_content_hash
|
|
||||||
ON tagged_photo (content_hash)
|
|
||||||
WHERE content_hash IS NOT NULL;
|
|
||||||
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
-- photo_insights
|
|
||||||
-- ---------------------------------------------------------------------------
|
|
||||||
ALTER TABLE photo_insights ADD COLUMN content_hash TEXT;
|
|
||||||
|
|
||||||
-- Backfill keyed on (library_id, rel_path) — photo_insights already
|
|
||||||
-- carries library_id, so the resolution is unambiguous.
|
|
||||||
UPDATE photo_insights
|
|
||||||
SET content_hash = (
|
|
||||||
SELECT content_hash FROM image_exif
|
|
||||||
WHERE image_exif.library_id = photo_insights.library_id
|
|
||||||
AND image_exif.rel_path = photo_insights.rel_path
|
|
||||||
AND image_exif.content_hash IS NOT NULL
|
|
||||||
LIMIT 1
|
|
||||||
)
|
|
||||||
WHERE content_hash IS NULL;
|
|
||||||
|
|
||||||
CREATE INDEX idx_photo_insights_content_hash
|
|
||||||
ON photo_insights (content_hash)
|
|
||||||
WHERE content_hash IS NOT NULL;
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
-- Requires SQLite 3.35+ for ALTER TABLE DROP COLUMN.
|
|
||||||
ALTER TABLE libraries DROP COLUMN enabled;
|
|
||||||
@@ -1,14 +0,0 @@
|
|||||||
-- Operator-controlled kill switch for a library. When `enabled = 0` the
|
|
||||||
-- watcher tick skips that library entirely — before the availability
|
|
||||||
-- probe, before ingest, before any maintenance pass — and the orphan-GC
|
|
||||||
-- all-online check treats it as out-of-scope rather than as a blocker.
|
|
||||||
--
|
|
||||||
-- The intended workflow is staging a new mount: insert with enabled=0,
|
|
||||||
-- verify the row appears in /libraries with enabled=false, then UPDATE
|
|
||||||
-- to 1 to start ingest. Same toggle works as a maintenance kill switch
|
|
||||||
-- after the fact ("don't keep probing this NAS while I'm rebooting it").
|
|
||||||
--
|
|
||||||
-- Default 1 so every existing library stays running on upgrade — no
|
|
||||||
-- behavior change without an explicit flip.
|
|
||||||
|
|
||||||
ALTER TABLE libraries ADD COLUMN enabled BOOLEAN NOT NULL DEFAULT 1;
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
-- Requires SQLite 3.35+ for ALTER TABLE DROP COLUMN.
|
|
||||||
ALTER TABLE libraries DROP COLUMN excluded_dirs;
|
|
||||||
@@ -1,14 +0,0 @@
|
|||||||
-- Per-library excluded directories.
|
|
||||||
--
|
|
||||||
-- The global EXCLUDED_DIRS env var is the right knob for excludes that
|
|
||||||
-- every library shares (Synology @eaDir, .thumbnails, etc.). It's a
|
|
||||||
-- poor fit for "exclude this subtree from THIS library only", which
|
|
||||||
-- the natural use case for is mounting a parent directory while
|
|
||||||
-- another library already covers a child subtree underneath.
|
|
||||||
--
|
|
||||||
-- This column is parsed comma-separated, same shape as the env var,
|
|
||||||
-- and the watcher / memories / thumbnail walks each apply
|
|
||||||
-- (env_globals ∪ library.excluded_dirs) when scanning the library.
|
|
||||||
-- NULL = no extra excludes; the global env var still applies.
|
|
||||||
|
|
||||||
ALTER TABLE libraries ADD COLUMN excluded_dirs TEXT;
|
|
||||||
@@ -1,8 +0,0 @@
|
|||||||
DROP INDEX IF EXISTS idx_image_exif_duplicate_of_hash;
|
|
||||||
DROP INDEX IF EXISTS idx_image_exif_dhash;
|
|
||||||
DROP INDEX IF EXISTS idx_image_exif_phash;
|
|
||||||
|
|
||||||
ALTER TABLE image_exif DROP COLUMN duplicate_decided_at;
|
|
||||||
ALTER TABLE image_exif DROP COLUMN duplicate_of_hash;
|
|
||||||
ALTER TABLE image_exif DROP COLUMN dhash_64;
|
|
||||||
ALTER TABLE image_exif DROP COLUMN phash_64;
|
|
||||||
@@ -1,41 +0,0 @@
|
|||||||
-- Adds perceptual-hash signals + soft-mark resolution state to image_exif so
|
|
||||||
-- the duplicates surface in Apollo can group near-duplicates (re-encoded,
|
|
||||||
-- resized, format-converted copies) and let the user demote losers without
|
|
||||||
-- touching the file on disk. Image-only for v1: phash_64/dhash_64 are NULL
|
|
||||||
-- on videos and on images that fail to decode. See Apollo CLAUDE.md →
|
|
||||||
-- Duplicate detection / Caching layer for the policy.
|
|
||||||
--
|
|
||||||
-- Soft-mark columns are media-type-agnostic — when video perceptual hashing
|
|
||||||
-- arrives, it lives in a separate hash-keyed companion table and reuses the
|
|
||||||
-- same duplicate_of_hash / duplicate_decided_at machinery.
|
|
||||||
|
|
||||||
-- pHash (DCT, 64-bit) packed as i64 for fast XOR + popcount Hamming.
|
|
||||||
ALTER TABLE image_exif ADD COLUMN phash_64 BIGINT;
|
|
||||||
|
|
||||||
-- dHash (gradient, 64-bit). Cheap, robust to compression/resize. Stored
|
|
||||||
-- alongside pHash so the query layer can fall back if either is null.
|
|
||||||
ALTER TABLE image_exif ADD COLUMN dhash_64 BIGINT;
|
|
||||||
|
|
||||||
-- When non-null, this row is a soft-marked duplicate of the row whose
|
|
||||||
-- content_hash matches. The duplicate file stays on disk; the default
|
|
||||||
-- /photos listing filters it out. /photos?include_duplicates=true opts
|
|
||||||
-- back in (the Apollo duplicates modal uses this).
|
|
||||||
ALTER TABLE image_exif ADD COLUMN duplicate_of_hash TEXT;
|
|
||||||
|
|
||||||
-- Unix seconds of the resolve. Distinguishes "never reviewed" from
|
|
||||||
-- "reviewed and resolved" for the Apollo include_resolved toggle.
|
|
||||||
ALTER TABLE image_exif ADD COLUMN duplicate_decided_at BIGINT;
|
|
||||||
|
|
||||||
-- Partial indexes — the columns are NULL for the vast majority of rows
|
|
||||||
-- during the transitional window and forever for videos / decode failures.
|
|
||||||
CREATE INDEX idx_image_exif_phash
|
|
||||||
ON image_exif (phash_64)
|
|
||||||
WHERE phash_64 IS NOT NULL;
|
|
||||||
|
|
||||||
CREATE INDEX idx_image_exif_dhash
|
|
||||||
ON image_exif (dhash_64)
|
|
||||||
WHERE dhash_64 IS NOT NULL;
|
|
||||||
|
|
||||||
CREATE INDEX idx_image_exif_duplicate_of_hash
|
|
||||||
ON image_exif (duplicate_of_hash)
|
|
||||||
WHERE duplicate_of_hash IS NOT NULL;
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
DROP INDEX IF EXISTS idx_image_exif_date_backfill;
|
|
||||||
ALTER TABLE image_exif DROP COLUMN date_taken_source;
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
-- Tracks where a row's `date_taken` was sourced so the canonical-date
|
|
||||||
-- waterfall (kamadak-exif → exiftool → filename → earliest_fs_time) is
|
|
||||||
-- visible to debugging and to the per-tick backfill drain that re-runs
|
|
||||||
-- weak sources once stronger ones become available (e.g. exiftool gets
|
|
||||||
-- installed on a deploy that didn't have it). See CLAUDE.md → Memories
|
|
||||||
-- canonical-date pipeline.
|
|
||||||
--
|
|
||||||
-- Values:
|
|
||||||
-- 'exif' — kamadak-exif read DateTime/DateTimeOriginal directly
|
|
||||||
-- 'exiftool' — exiftool fallback caught a video / MakerNote / QuickTime tag
|
|
||||||
-- 'filename' — extract_date_from_filename matched a known pattern
|
|
||||||
-- 'fs_time' — fell through to earliest_fs_time(metadata)
|
|
||||||
--
|
|
||||||
-- NULL when `date_taken` itself is NULL (no source resolved the date).
|
|
||||||
ALTER TABLE image_exif ADD COLUMN date_taken_source TEXT;
|
|
||||||
|
|
||||||
-- Partial index for the per-tick backfill drain: targets rows that need
|
|
||||||
-- re-resolution (no date yet, or only the weakest source resolved it).
|
|
||||||
-- Filename-sourced rows are intentionally excluded — the regex is
|
|
||||||
-- authoritative when it matches and re-running exiftool wouldn't change
|
|
||||||
-- the answer.
|
|
||||||
CREATE INDEX idx_image_exif_date_backfill
|
|
||||||
ON image_exif (library_id, id)
|
|
||||||
WHERE date_taken IS NULL OR date_taken_source = 'fs_time';
|
|
||||||
@@ -1,9 +0,0 @@
|
|||||||
-- Reverting this migration is a no-op: the labels we wrote in `up.sql`
|
|
||||||
-- are correct under any state of the schema (every dated row was indeed
|
|
||||||
-- exif-sourced before the resolver landed), and there's no signal that
|
|
||||||
-- distinguishes "labelled by this migration" from "labelled by the
|
|
||||||
-- ingest path post-resolver". Clearing them would break the drain's
|
|
||||||
-- eligibility filter again.
|
|
||||||
--
|
|
||||||
-- The companion migration `2026-05-06-000000_add_date_taken_source` is
|
|
||||||
-- the one to revert if you need to remove the column entirely.
|
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
-- Backfill `date_taken_source` for rows that pre-date the canonical-date
|
|
||||||
-- pipeline. Before the resolver landed, `image_exif.date_taken` could
|
|
||||||
-- only be populated via `exif::extract_exif_from_path` (kamadak-exif)
|
|
||||||
-- on the file-watcher, upload, or GPS-write paths. The resolver column
|
|
||||||
-- migration added `date_taken_source` defaulting to NULL, so every
|
|
||||||
-- historical row with a date is currently unlabelled — and the
|
|
||||||
-- per-tick drain skips them because its eligibility predicate is
|
|
||||||
-- `date_taken IS NULL OR date_taken_source = 'fs_time'`.
|
|
||||||
--
|
|
||||||
-- Label them `'exif'` once and let the drain take over from here. Safe
|
|
||||||
-- because every code path that wrote `date_taken` prior to the
|
|
||||||
-- resolver was a kamadak-exif read — there was no other source.
|
|
||||||
--
|
|
||||||
-- Idempotent: re-running this migration on a DB that has already been
|
|
||||||
-- backfilled is a no-op (the WHERE clause matches nothing the second
|
|
||||||
-- time around).
|
|
||||||
UPDATE image_exif
|
|
||||||
SET date_taken_source = 'exif'
|
|
||||||
WHERE date_taken IS NOT NULL
|
|
||||||
AND date_taken_source IS NULL;
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
ALTER TABLE image_exif DROP COLUMN original_date_taken_source;
|
|
||||||
ALTER TABLE image_exif DROP COLUMN original_date_taken;
|
|
||||||
@@ -1,15 +0,0 @@
|
|||||||
-- Manual date_taken override: when an operator overrides a row's date via
|
|
||||||
-- POST /image/exif/date, the prior `(date_taken, date_taken_source)` is
|
|
||||||
-- snapshotted into these columns and the live columns hold the new value
|
|
||||||
-- with `date_taken_source = 'manual'`. POST /image/exif/date/clear restores
|
|
||||||
-- the pair and nulls the originals.
|
|
||||||
--
|
|
||||||
-- The waterfall source-name set is now:
|
|
||||||
-- 'exif' | 'exiftool' | 'filename' | 'fs_time' | 'manual'
|
|
||||||
--
|
|
||||||
-- The `idx_image_exif_date_backfill` partial index already filters to
|
|
||||||
-- `date_taken IS NULL OR date_taken_source = 'fs_time'`, so 'manual' rows
|
|
||||||
-- are naturally excluded from the per-tick backfill drain — no index
|
|
||||||
-- change needed.
|
|
||||||
ALTER TABLE image_exif ADD COLUMN original_date_taken BIGINT;
|
|
||||||
ALTER TABLE image_exif ADD COLUMN original_date_taken_source TEXT;
|
|
||||||
@@ -1,43 +0,0 @@
|
|||||||
-- Drop the persona-scoping column on entity_facts via the table-rebuild
|
|
||||||
-- dance for SQLite-version portability (matches the pattern in
|
|
||||||
-- 2026-04-20-000000_add_backend_to_insights/down.sql).
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_persona;
|
|
||||||
|
|
||||||
CREATE TABLE entity_facts_backup AS
|
|
||||||
SELECT id, subject_entity_id, predicate, object_entity_id, object_value,
|
|
||||||
source_photo, source_insight_id, confidence, status, created_at
|
|
||||||
FROM entity_facts;
|
|
||||||
|
|
||||||
DROP TABLE entity_facts;
|
|
||||||
|
|
||||||
CREATE TABLE entity_facts (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
subject_entity_id INTEGER NOT NULL,
|
|
||||||
predicate TEXT NOT NULL,
|
|
||||||
object_entity_id INTEGER,
|
|
||||||
object_value TEXT,
|
|
||||||
source_photo TEXT,
|
|
||||||
source_insight_id INTEGER,
|
|
||||||
confidence REAL NOT NULL DEFAULT 0.6,
|
|
||||||
status TEXT NOT NULL DEFAULT 'active',
|
|
||||||
created_at BIGINT NOT NULL,
|
|
||||||
CONSTRAINT fk_ef_subject FOREIGN KEY (subject_entity_id) REFERENCES entities(id) ON DELETE CASCADE,
|
|
||||||
CONSTRAINT fk_ef_object FOREIGN KEY (object_entity_id) REFERENCES entities(id) ON DELETE SET NULL,
|
|
||||||
CONSTRAINT fk_ef_insight FOREIGN KEY (source_insight_id) REFERENCES photo_insights(id) ON DELETE SET NULL,
|
|
||||||
CHECK (object_entity_id IS NOT NULL OR object_value IS NOT NULL)
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO entity_facts
|
|
||||||
SELECT id, subject_entity_id, predicate, object_entity_id, object_value,
|
|
||||||
source_photo, source_insight_id, confidence, status, created_at
|
|
||||||
FROM entity_facts_backup;
|
|
||||||
|
|
||||||
DROP TABLE entity_facts_backup;
|
|
||||||
|
|
||||||
CREATE INDEX idx_entity_facts_subject ON entity_facts(subject_entity_id);
|
|
||||||
CREATE INDEX idx_entity_facts_predicate ON entity_facts(predicate);
|
|
||||||
CREATE INDEX idx_entity_facts_status ON entity_facts(status);
|
|
||||||
CREATE INDEX idx_entity_facts_source_photo ON entity_facts(source_photo);
|
|
||||||
|
|
||||||
DROP INDEX IF EXISTS idx_personas_user;
|
|
||||||
DROP TABLE IF EXISTS personas;
|
|
||||||
@@ -1,64 +0,0 @@
|
|||||||
-- Personas live server-side now (mobile previously stored them in
|
|
||||||
-- AsyncStorage only). Each user gets the three built-ins seeded; custom
|
|
||||||
-- personas land here too via POST /personas or POST /personas/migrate.
|
|
||||||
--
|
|
||||||
-- `entity_facts` gains a persona_id so each persona accumulates its own
|
|
||||||
-- voice over a shared entity graph (entities themselves stay unscoped).
|
|
||||||
-- Existing rows backfill to 'default' via the column DEFAULT — that
|
|
||||||
-- becomes the historical baseline. The `include_all_memories` flag on
|
|
||||||
-- personas lets any persona opt back into reading the full pool.
|
|
||||||
|
|
||||||
CREATE TABLE personas (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
user_id INTEGER NOT NULL,
|
|
||||||
persona_id TEXT NOT NULL,
|
|
||||||
name TEXT NOT NULL,
|
|
||||||
system_prompt TEXT NOT NULL,
|
|
||||||
is_built_in BOOLEAN NOT NULL DEFAULT FALSE,
|
|
||||||
include_all_memories BOOLEAN NOT NULL DEFAULT FALSE,
|
|
||||||
created_at BIGINT NOT NULL,
|
|
||||||
updated_at BIGINT NOT NULL,
|
|
||||||
UNIQUE(user_id, persona_id),
|
|
||||||
CONSTRAINT fk_personas_user FOREIGN KEY (user_id) REFERENCES users(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX idx_personas_user ON personas(user_id);
|
|
||||||
|
|
||||||
-- Seed built-ins for every existing user. System prompts copied verbatim
|
|
||||||
-- from FileViewer-React/hooks/usePersonas.tsx so server and client agree
|
|
||||||
-- on the canonical voice for each built-in.
|
|
||||||
INSERT INTO personas (user_id, persona_id, name, system_prompt, is_built_in, created_at, updated_at)
|
|
||||||
SELECT
|
|
||||||
u.id,
|
|
||||||
'default',
|
|
||||||
'Default Assistant',
|
|
||||||
'You are my long-term memory assistant. Use only the information provided. Do not invent details. Respond in 3–6 sentences in third person, leading with the most concrete moment from the photo and the surrounding context. Plain prose, no headings.',
|
|
||||||
TRUE,
|
|
||||||
strftime('%s', 'now') * 1000,
|
|
||||||
strftime('%s', 'now') * 1000
|
|
||||||
FROM users u
|
|
||||||
UNION ALL
|
|
||||||
SELECT
|
|
||||||
u.id,
|
|
||||||
'journal',
|
|
||||||
'Personal Journal',
|
|
||||||
'You are a personal journal writer. Write in first person, present tense, with warmth and reflection — focusing on emotions and meaningful moments. Use only the information provided; do not invent details. Aim for 4–8 sentences in a single flowing paragraph, no headings.',
|
|
||||||
TRUE,
|
|
||||||
strftime('%s', 'now') * 1000,
|
|
||||||
strftime('%s', 'now') * 1000
|
|
||||||
FROM users u
|
|
||||||
UNION ALL
|
|
||||||
SELECT
|
|
||||||
u.id,
|
|
||||||
'factual',
|
|
||||||
'Factual Reporter',
|
|
||||||
'You are a factual memory recorder. Be precise, objective, and concise. Lead with the date and place, then list what / when / who in 2–4 short sentences. Use only the information provided; if a detail is unknown, say so rather than guessing.',
|
|
||||||
TRUE,
|
|
||||||
strftime('%s', 'now') * 1000,
|
|
||||||
strftime('%s', 'now') * 1000
|
|
||||||
FROM users u;
|
|
||||||
|
|
||||||
-- Persona scoping on facts only. Entities and entity_photo_links stay
|
|
||||||
-- shared (real-world referents and shared photo ↔ entity associations).
|
|
||||||
ALTER TABLE entity_facts ADD COLUMN persona_id TEXT NOT NULL DEFAULT 'default';
|
|
||||||
CREATE INDEX idx_entity_facts_persona ON entity_facts(persona_id);
|
|
||||||
@@ -1,47 +0,0 @@
|
|||||||
-- Reverse 2026-05-10-000000_entity_facts_persona_fk: drop the
|
|
||||||
-- composite FK and the user_id column via the same rebuild pattern.
|
|
||||||
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_user_persona;
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_persona;
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_source_photo;
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_status;
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_predicate;
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_subject;
|
|
||||||
|
|
||||||
ALTER TABLE entity_facts RENAME TO entity_facts_old;
|
|
||||||
|
|
||||||
CREATE TABLE entity_facts (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
subject_entity_id INTEGER NOT NULL,
|
|
||||||
predicate TEXT NOT NULL,
|
|
||||||
object_entity_id INTEGER,
|
|
||||||
object_value TEXT,
|
|
||||||
source_photo TEXT,
|
|
||||||
source_insight_id INTEGER,
|
|
||||||
confidence REAL NOT NULL DEFAULT 0.6,
|
|
||||||
status TEXT NOT NULL DEFAULT 'active',
|
|
||||||
created_at BIGINT NOT NULL,
|
|
||||||
persona_id TEXT NOT NULL DEFAULT 'default',
|
|
||||||
CONSTRAINT fk_ef_subject FOREIGN KEY (subject_entity_id) REFERENCES entities(id) ON DELETE CASCADE,
|
|
||||||
CONSTRAINT fk_ef_object FOREIGN KEY (object_entity_id) REFERENCES entities(id) ON DELETE SET NULL,
|
|
||||||
CONSTRAINT fk_ef_insight FOREIGN KEY (source_insight_id) REFERENCES photo_insights(id) ON DELETE SET NULL,
|
|
||||||
CHECK (object_entity_id IS NOT NULL OR object_value IS NOT NULL)
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT INTO entity_facts
|
|
||||||
(id, subject_entity_id, predicate, object_entity_id, object_value,
|
|
||||||
source_photo, source_insight_id, confidence, status, created_at,
|
|
||||||
persona_id)
|
|
||||||
SELECT
|
|
||||||
id, subject_entity_id, predicate, object_entity_id, object_value,
|
|
||||||
source_photo, source_insight_id, confidence, status, created_at,
|
|
||||||
persona_id
|
|
||||||
FROM entity_facts_old;
|
|
||||||
|
|
||||||
DROP TABLE entity_facts_old;
|
|
||||||
|
|
||||||
CREATE INDEX idx_entity_facts_subject ON entity_facts(subject_entity_id);
|
|
||||||
CREATE INDEX idx_entity_facts_predicate ON entity_facts(predicate);
|
|
||||||
CREATE INDEX idx_entity_facts_status ON entity_facts(status);
|
|
||||||
CREATE INDEX idx_entity_facts_source_photo ON entity_facts(source_photo);
|
|
||||||
CREATE INDEX idx_entity_facts_persona ON entity_facts(persona_id);
|
|
||||||
@@ -1,82 +0,0 @@
|
|||||||
-- Add a real foreign key from entity_facts to personas. Until now,
|
|
||||||
-- entity_facts.persona_id was a free-form string with no integrity
|
|
||||||
-- guarantee — deleting a persona orphaned its facts, which then sat
|
|
||||||
-- forever in the readable-only-via-PersonaFilter::All hive-mind view.
|
|
||||||
--
|
|
||||||
-- personas is keyed (user_id, persona_id) so the FK has to be
|
|
||||||
-- composite. That requires entity_facts to carry user_id too, which
|
|
||||||
-- has the side benefit of fixing multi-user fact leakage on the read
|
|
||||||
-- path (without it, two users with the same 'default' persona would
|
|
||||||
-- see each other's default-scoped facts).
|
|
||||||
--
|
|
||||||
-- SQLite can't ALTER TABLE to add an FK; the table-rebuild dance is
|
|
||||||
-- the only way. Pattern matches 2026-05-09's down.sql and the older
|
|
||||||
-- 2026-04-20-000000 migration.
|
|
||||||
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_subject;
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_predicate;
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_status;
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_source_photo;
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_persona;
|
|
||||||
|
|
||||||
ALTER TABLE entity_facts RENAME TO entity_facts_old;
|
|
||||||
|
|
||||||
CREATE TABLE entity_facts (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
|
|
||||||
subject_entity_id INTEGER NOT NULL,
|
|
||||||
predicate TEXT NOT NULL,
|
|
||||||
object_entity_id INTEGER,
|
|
||||||
object_value TEXT,
|
|
||||||
source_photo TEXT,
|
|
||||||
source_insight_id INTEGER,
|
|
||||||
confidence REAL NOT NULL DEFAULT 0.6,
|
|
||||||
status TEXT NOT NULL DEFAULT 'active',
|
|
||||||
created_at BIGINT NOT NULL,
|
|
||||||
persona_id TEXT NOT NULL DEFAULT 'default',
|
|
||||||
user_id INTEGER NOT NULL DEFAULT 1,
|
|
||||||
CONSTRAINT fk_ef_subject FOREIGN KEY (subject_entity_id) REFERENCES entities(id) ON DELETE CASCADE,
|
|
||||||
CONSTRAINT fk_ef_object FOREIGN KEY (object_entity_id) REFERENCES entities(id) ON DELETE SET NULL,
|
|
||||||
CONSTRAINT fk_ef_insight FOREIGN KEY (source_insight_id) REFERENCES photo_insights(id) ON DELETE SET NULL,
|
|
||||||
CONSTRAINT fk_ef_persona FOREIGN KEY (user_id, persona_id) REFERENCES personas(user_id, persona_id) ON DELETE CASCADE,
|
|
||||||
CHECK (object_entity_id IS NOT NULL OR object_value IS NOT NULL)
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Backfill: assign each legacy fact to the user that owns the matching
|
|
||||||
-- persona. Built-ins are seeded per-user with the same persona_id
|
|
||||||
-- string for everyone, so MIN(user_id) deterministically picks the
|
|
||||||
-- earliest registered user (typically user 1, the operator). Custom
|
|
||||||
-- persona_ids exist for at most one user, so MIN is also unique.
|
|
||||||
-- Falls back to user_id=1 when no matching persona row exists; in that
|
|
||||||
-- case the FK below would still fail, but legacy rows shouldn't be in
|
|
||||||
-- that state because 2026-05-09 ADD COLUMN defaulted persona_id to
|
|
||||||
-- 'default', which is seeded for every user.
|
|
||||||
INSERT INTO entity_facts
|
|
||||||
(id, subject_entity_id, predicate, object_entity_id, object_value,
|
|
||||||
source_photo, source_insight_id, confidence, status, created_at,
|
|
||||||
persona_id, user_id)
|
|
||||||
SELECT
|
|
||||||
old.id,
|
|
||||||
old.subject_entity_id,
|
|
||||||
old.predicate,
|
|
||||||
old.object_entity_id,
|
|
||||||
old.object_value,
|
|
||||||
old.source_photo,
|
|
||||||
old.source_insight_id,
|
|
||||||
old.confidence,
|
|
||||||
old.status,
|
|
||||||
old.created_at,
|
|
||||||
old.persona_id,
|
|
||||||
COALESCE(
|
|
||||||
(SELECT MIN(p.user_id) FROM personas p WHERE p.persona_id = old.persona_id),
|
|
||||||
1
|
|
||||||
)
|
|
||||||
FROM entity_facts_old old;
|
|
||||||
|
|
||||||
DROP TABLE entity_facts_old;
|
|
||||||
|
|
||||||
CREATE INDEX idx_entity_facts_subject ON entity_facts(subject_entity_id);
|
|
||||||
CREATE INDEX idx_entity_facts_predicate ON entity_facts(predicate);
|
|
||||||
CREATE INDEX idx_entity_facts_status ON entity_facts(status);
|
|
||||||
CREATE INDEX idx_entity_facts_source_photo ON entity_facts(source_photo);
|
|
||||||
CREATE INDEX idx_entity_facts_persona ON entity_facts(persona_id);
|
|
||||||
CREATE INDEX idx_entity_facts_user_persona ON entity_facts(user_id, persona_id);
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
-- SQLite can drop columns since 3.35 (March 2021); embedded
|
|
||||||
-- libsqlite3-sys is well past that. Drop in reverse insert order so
|
|
||||||
-- a partial down still leaves the schema valid.
|
|
||||||
ALTER TABLE entity_facts DROP COLUMN valid_until;
|
|
||||||
ALTER TABLE entity_facts DROP COLUMN valid_from;
|
|
||||||
@@ -1,25 +0,0 @@
|
|||||||
-- Add valid-time columns to entity_facts.
|
|
||||||
--
|
|
||||||
-- entity_facts already has created_at — *transaction time*, the
|
|
||||||
-- moment WE recorded the fact. That's not the same as the real-world
|
|
||||||
-- period the fact was true. "Cameron is_in_relationship_with X" was
|
|
||||||
-- only true during a window; recording it in 2026 doesn't make it
|
|
||||||
-- true today. Without the distinction, every former relationship,
|
|
||||||
-- former job, former address reads as currently-true.
|
|
||||||
--
|
|
||||||
-- Adding two BIGINT NULL columns: valid_from / valid_until (unix
|
|
||||||
-- seconds). NULL means "unbounded on that side" — `valid_from IS
|
|
||||||
-- NULL` reads as "always-true-back-to-the-beginning",
|
|
||||||
-- `valid_until IS NULL` as "still-true-now-or-unknown". Both NULL =
|
|
||||||
-- temporal validity unknown (current state of all legacy rows).
|
|
||||||
--
|
|
||||||
-- Conflict detection refines accordingly: same-predicate facts with
|
|
||||||
-- different objects stop flagging when their intervals are disjoint
|
|
||||||
-- ("lives_in NYC 2018-2020" and "lives_in SF 2020-present" are both
|
|
||||||
-- valid, just at different times).
|
|
||||||
|
|
||||||
ALTER TABLE entity_facts ADD COLUMN valid_from BIGINT;
|
|
||||||
ALTER TABLE entity_facts ADD COLUMN valid_until BIGINT;
|
|
||||||
|
|
||||||
-- Optional partial index for time-bounded scans. Skipped for now —
|
|
||||||
-- conflict detection runs per-entity (small N) and doesn't need it.
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
DROP INDEX IF EXISTS idx_entity_facts_superseded_by;
|
|
||||||
ALTER TABLE entity_facts DROP COLUMN superseded_by;
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
-- Add a supersession pointer to entity_facts.
|
|
||||||
--
|
|
||||||
-- Status alone is a one-way trapdoor: 'rejected' loses the link
|
|
||||||
-- between the rejected fact and the one that replaced it. For
|
|
||||||
-- evolving facts (Cameron's relationship, employer, address) the
|
|
||||||
-- curator wants to *replace* a stale fact with a new one and keep
|
|
||||||
-- the history readable: "from 2018 until 2022 this was true, then
|
|
||||||
-- it became this other thing".
|
|
||||||
--
|
|
||||||
-- A nullable INTEGER column pointing at another entity_facts.id —
|
|
||||||
-- no FK constraint because SQLite can't ALTER ADD COLUMN with REFs;
|
|
||||||
-- the DAO's delete_fact clears dangling pointers in the same
|
|
||||||
-- transaction as the parent delete to keep the column honest.
|
|
||||||
--
|
|
||||||
-- A status of 'superseded' on the old fact (alongside the existing
|
|
||||||
-- active / reviewed / rejected) signals "replaced by a newer
|
|
||||||
-- claim". Read paths already filter 'rejected' out of the active
|
|
||||||
-- view; the curation UI will treat 'superseded' the same way for
|
|
||||||
-- conflict detection so they don't keep flagging.
|
|
||||||
--
|
|
||||||
-- Pairs with the valid-time columns from 2026-05-10-000100: the
|
|
||||||
-- supersede action auto-stamps the old fact's `valid_until` from
|
|
||||||
-- the new fact's `valid_from`, closing the interval cleanly.
|
|
||||||
|
|
||||||
ALTER TABLE entity_facts ADD COLUMN superseded_by INTEGER;
|
|
||||||
|
|
||||||
-- Helpful index for "show me what superseded this fact" walks
|
|
||||||
-- (rare today; cheap to add now while the table is small).
|
|
||||||
CREATE INDEX idx_entity_facts_superseded_by
|
|
||||||
ON entity_facts(superseded_by)
|
|
||||||
WHERE superseded_by IS NOT NULL;
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
DROP INDEX IF EXISTS idx_entity_facts_created_by_backend;
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_created_by_model;
|
|
||||||
ALTER TABLE entity_facts DROP COLUMN created_by_backend;
|
|
||||||
ALTER TABLE entity_facts DROP COLUMN created_by_model;
|
|
||||||
@@ -1,30 +0,0 @@
|
|||||||
-- Track which model + backend generated each fact so the curator
|
|
||||||
-- can audit which configurations produce trustworthy knowledge.
|
|
||||||
--
|
|
||||||
-- photo_insights already carries `model_version` + `backend`, and
|
|
||||||
-- entity_facts.source_insight_id links to it — but:
|
|
||||||
-- 1. source_insight_id is only set after an insight is stored
|
|
||||||
-- (post-loop), so chat-continuation facts and facts whose insight
|
|
||||||
-- was regenerated lose the link.
|
|
||||||
-- 2. JOINing for every read is more friction than just embedding the
|
|
||||||
-- provenance on the fact row itself.
|
|
||||||
-- 3. Manual facts (POST /knowledge/facts) have no insight at all and
|
|
||||||
-- need to record "manual" as their provenance.
|
|
||||||
--
|
|
||||||
-- Two nullable TEXT columns are enough for the audit use case: model
|
|
||||||
-- (e.g. "qwen2.5:7b", "anthropic/claude-sonnet-4") and backend
|
|
||||||
-- ("local", "hybrid", "manual"). Pre-existing rows leave both NULL —
|
|
||||||
-- legacy facts predate this tracking and can't be back-filled
|
|
||||||
-- reliably from training_messages without burning compute.
|
|
||||||
|
|
||||||
ALTER TABLE entity_facts ADD COLUMN created_by_model TEXT;
|
|
||||||
ALTER TABLE entity_facts ADD COLUMN created_by_backend TEXT;
|
|
||||||
|
|
||||||
-- Indexes are cheap and useful for "show me all facts from model X"
|
|
||||||
-- audit queries — partial so the legacy NULL rows don't bloat them.
|
|
||||||
CREATE INDEX idx_entity_facts_created_by_model
|
|
||||||
ON entity_facts(created_by_model)
|
|
||||||
WHERE created_by_model IS NOT NULL;
|
|
||||||
CREATE INDEX idx_entity_facts_created_by_backend
|
|
||||||
ON entity_facts(created_by_backend)
|
|
||||||
WHERE created_by_backend IS NOT NULL;
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
ALTER TABLE personas DROP COLUMN reviewed_only_facts;
|
|
||||||
@@ -1,16 +0,0 @@
|
|||||||
-- Per-persona toggle: when true, agent reads only see facts whose
|
|
||||||
-- status is exactly 'reviewed' (human-verified). When false (the
|
|
||||||
-- default), agent reads see 'active' OR 'reviewed' — everything not
|
|
||||||
-- rejected or superseded.
|
|
||||||
--
|
|
||||||
-- The mobile app surfaces this as "Strict mode" on the persona
|
|
||||||
-- editor: useful when you want a persona's chat to be grounded
|
|
||||||
-- exclusively on the curated subset, e.g. for tasks where
|
|
||||||
-- hallucinated agent claims are particularly costly.
|
|
||||||
--
|
|
||||||
-- Note: this is separate from `include_all_memories` (which unions
|
|
||||||
-- across personas for hive-mind reads). Reviewed-only operates on
|
|
||||||
-- the status axis; include_all_memories operates on the persona-
|
|
||||||
-- scope axis. They compose freely.
|
|
||||||
|
|
||||||
ALTER TABLE personas ADD COLUMN reviewed_only_facts BOOLEAN NOT NULL DEFAULT 0;
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
ALTER TABLE personas DROP COLUMN allow_agent_corrections;
|
|
||||||
DROP INDEX IF EXISTS idx_entity_facts_last_modified_at;
|
|
||||||
ALTER TABLE entity_facts DROP COLUMN last_modified_at;
|
|
||||||
ALTER TABLE entity_facts DROP COLUMN last_modified_by_backend;
|
|
||||||
ALTER TABLE entity_facts DROP COLUMN last_modified_by_model;
|
|
||||||
@@ -1,30 +0,0 @@
|
|||||||
-- Three coupled changes for agent self-correction safety:
|
|
||||||
--
|
|
||||||
-- 1. `entity_facts.last_modified_by_*` + `last_modified_at` track who
|
|
||||||
-- most recently mutated each fact. `created_by_*` from migration
|
|
||||||
-- 2026-05-10-000300 records who first wrote the row; this records
|
|
||||||
-- who last *changed* it. Separate columns so the create vs update
|
|
||||||
-- audit is independently grep-able ("show me every fact gpt-5
|
|
||||||
-- altered last week" stays a single index scan).
|
|
||||||
--
|
|
||||||
-- 2. `personas.allow_agent_corrections` is the gate for the new
|
|
||||||
-- agent-side `update_fact` / `supersede_fact` tools. Default OFF —
|
|
||||||
-- a fresh persona's agent can create but can't alter or replace.
|
|
||||||
-- Operator opts in per-persona after the model has earned trust,
|
|
||||||
-- typically via the strict-mode flow (curate, then ratchet up
|
|
||||||
-- agent autonomy as confidence rises). Parallel in shape to
|
|
||||||
-- `reviewed_only_facts` from 2026-05-10-000400; they compose.
|
|
||||||
--
|
|
||||||
-- 3. Index on `last_modified_at` (partial, NOT NULL) for the
|
|
||||||
-- audit-feed reads in the curation UI ("show recent agent edits
|
|
||||||
-- sorted newest first").
|
|
||||||
|
|
||||||
ALTER TABLE entity_facts ADD COLUMN last_modified_by_model TEXT;
|
|
||||||
ALTER TABLE entity_facts ADD COLUMN last_modified_by_backend TEXT;
|
|
||||||
ALTER TABLE entity_facts ADD COLUMN last_modified_at BIGINT;
|
|
||||||
|
|
||||||
CREATE INDEX idx_entity_facts_last_modified_at
|
|
||||||
ON entity_facts(last_modified_at)
|
|
||||||
WHERE last_modified_at IS NOT NULL;
|
|
||||||
|
|
||||||
ALTER TABLE personas ADD COLUMN allow_agent_corrections BOOLEAN NOT NULL DEFAULT 0;
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user