# Private Cloud NAS --- Master Project Plan & Engineering Specification **Project:** Synology-like Private Cloud NAS for Raspberry Pi 4\ **Target users:** \~50 registered users\ **Primary clients:** Responsive Web App + Android/iOS Mobile App\ **Primary goal:** Provide a simple, secure, cloud-drive-style experience for files, users, sharing and remote access. ------------------------------------------------------------------------ ## 0. Non-Negotiable Engineering Rules These rules apply to every implementation decision. 1. **Do not expose SMB, SSH, MySQL, Redis, Docker, or internal services to the public Internet.** 2. Only the reverse proxy is publicly reachable, normally on TCP 443. 3. Every API endpoint must enforce authentication and authorization server-side. Never trust frontend permissions. 4. Users must never be able to escape their authorized storage root through path traversal, symlinks, crafted filenames, or manipulated IDs. 5. File names are untrusted input. 6. Uploaded files are untrusted input. 7. Never load an entire large file into application memory. 8. Large uploads/downloads must be streamed and/or chunked. 9. The database stores file metadata; the filesystem stores file contents. 10. Passwords must use a modern password hashing algorithm through the framework's password hasher. Never store plaintext passwords. 11. Sensitive tokens must be hashed at rest when practical and must never be logged. 12. All state-changing operations must have server-side validation. 13. Destructive operations must be recoverable through Trash where applicable. 14. Every important feature must have automated tests before it is considered complete. 15. Do not mark a feature complete merely because the UI works. API, authorization, failure handling, logging and tests are required. 16. Use migrations for database changes. Never manually edit production database structure. 17. Use environment variables/secrets for credentials and deployment-specific configuration. 18. Production must run with debug/development features disabled. 19. The application must work on ARM64 because the target is Raspberry Pi 4. 20. Keep the architecture modular so the storage layer can later move from Raspberry Pi to a more powerful server without rewriting the product. ------------------------------------------------------------------------ # 1. Product Definition The product is a private cloud storage platform similar in user experience to a simplified Synology Drive/Google Drive/Dropbox. A user should be able to: - Create an account or receive an invitation. - Log in securely. - Browse authorized files and folders. - Upload files. - Download files. - Create folders. - Rename, move, copy and delete files/folders. - Restore deleted items. - Search files. - Preview common file types. - Generate share links. - Revoke share links. - Set share-link expiration. - Optionally password-protect a share link. - Access the system remotely through HTTPS. - Use the same account through the web and mobile app. - Receive notifications. - View active devices/sessions. - Change password. - Enable two-factor authentication. - Use biometric app unlock on supported mobile devices. - Upload/download files from the mobile app. - Automatically back up photos in a later milestone. Administrators should be able to: - Manage users. - Invite users. - Disable/enable accounts. - Reset accounts. - Manage roles. - Manage shared folders. - Set permissions. - View storage usage. - View system health. - View security/audit logs. - Configure retention policies. - Manage share links. - Review active sessions/devices. - Monitor failed logins. - Manage application settings. ------------------------------------------------------------------------ # 2. Recommended Technology Stack ## Infrastructure / Server Roles See §50 for the full deployment architecture. Summary: ### Public Application VPS - Ubuntu 24.04 LTS (or equivalent supported Linux) - Docker + Docker Compose - Caddy as the public reverse proxy, terminating TLS - Public DNS (e.g. `cloud.example.com`) - Laravel + PHP 8.3+, React production build, MySQL, Redis, queue workers - WireGuard client endpoint for the private link to the storage node - Firewall: only required public ports exposed (normally 80/443, plus SSH as administratively required) ### Raspberry Pi Storage Node - Raspberry Pi 4 Model B, 64-bit Raspberry Pi OS - USB 3 HDD/SSD for user data, Ethernet strongly recommended - WireGuard peer to the VPS - Runs only the Storage Agent (§50) — no Laravel, MySQL, Redis, or React - No public Internet-facing ports at all - UPS strongly recommended for production ## Backend - Laravel - PHP 8.3+ - REST JSON API - Laravel Sanctum or an equivalent secure token/session mechanism - Laravel Queue - Redis ## Database - MySQL ## Frontend - React - TypeScript - Vite - Tailwind CSS - A consistent component library/design system ## Mobile - React Native - TypeScript - Android - iOS ## Reverse proxy / Public edge - Caddy, running on the VPS - Automatic HTTPS through Let's Encrypt - The VPS is the only public entry point; it reaches the storage node only over the private WireGuard link (§50) ## Monitoring Initial: - Application health endpoints - Structured application logs - Docker health checks Later: - Prometheus - Grafana - Node/system metrics ## Storage - MySQL stores file/folder metadata only - Actual file contents live on the Raspberry Pi storage node's HDD/SSD, reached from the VPS only through the private WireGuard link and a dedicated Storage Agent (§50) — never a publicly exposed SMB/NFS mount - Laravel accesses storage exclusively through a `StorageService` abstraction, so the physical backend (Pi + Storage Agent today) can later be replaced — by a bigger server, or S3-compatible object storage — without changing any user-facing API - Storage paths are never user-controlled; the application resolves opaque storage keys server-side ------------------------------------------------------------------------ # 3. High-Level Architecture See §50 for the full two-node (VPS + Raspberry Pi) deployment architecture, which supersedes the single-node picture this section originally described. Summary: ``` text INTERNET | HTTPS :443 | v +---------------+ | VPS | | | | [ Caddy ] | | | | | +----+----+ | | | | | | v v | | [Web] [Laravel API] | | | | +-------+ | | | | | | v v | | [MySQL] [Redis] | +--------|--------+ | WireGuard (private) | v +-----------------+ | Raspberry Pi | | | | Storage Agent | | | | | v | | HDD/SSD | +-----------------+ ``` Mobile: ``` text React Native Mobile App | HTTPS | v Caddy / API | Laravel ``` The frontend and mobile app must never directly access MySQL or the filesystem. ------------------------------------------------------------------------ # 4. Storage Architecture Recommended logical structure: ``` text /mnt/nas-data/ users/ {user-uuid}/ files/ trash/ shared/ {shared-folder-uuid}/ uploads/ temporary/ system/ thumbnails/ previews/ application/ ``` The exact physical layout can change, but the storage service must abstract it from the rest of the application. As of §50, this layout lives on the Raspberry Pi storage node and is reached from the Laravel application only through the Storage Agent over the private WireGuard link — never as a local or network filesystem mount on the VPS. ## File metadata Minimum fields: - id - owner_id - parent_id - storage_key - original_name - mime_type - extension - size - checksum - type (file/folder) - status - created_at - updated_at - deleted_at - deleted_by - version information Never use the original filename as the sole physical storage identifier. Prefer a generated internal storage key/UUID. ------------------------------------------------------------------------ # 5. Core User Roles ## Super Admin Can: - Manage all users - Manage all storage - Manage shared folders - View audit logs - Configure system settings - Disable users - Revoke sessions - Manage security settings ## User Can: - Access own files - Access explicitly shared folders - Create share links according to policy - Manage own profile/security - Manage own sessions/devices ## Optional Read-only role Can: - View/download permitted files - Cannot modify/delete Permissions must be calculated server-side. ------------------------------------------------------------------------ # 6. Authentication & Identity ## Required - Login - Logout - Password hashing - Password reset - Email verification - Session/token expiration - Revoke individual session - Revoke all sessions - Login rate limiting - Brute-force protection - Account disable/enable - Device/session list ## Security - Passwords never logged. - Authentication tokens never logged. - Password reset tokens expire. - Email verification tokens expire. - Rate-limit login attempts. - Rate-limit password reset requests. - Return safe error messages that do not leak account existence where appropriate. ## 2FA Phase 2: - TOTP - Recovery codes - Enforce 2FA for administrators Later: - Passkeys/WebAuthn ------------------------------------------------------------------------ # 7. User Management Admin UI: ``` text Users ------------------------------------------------ Name Email Role Status John ... User Active Mike ... User Disabled Admin ... Admin Active ``` Actions: - Invite user - Create user - Edit user - Disable user - Enable user - Delete user - Force password reset - Revoke all sessions - Change role - View storage usage - View login history User invitation: 1. Admin creates invitation. 2. System sends email. 3. User opens invitation. 4. User creates password. 5. Email is verified. 6. Account becomes active. ------------------------------------------------------------------------ # 8. File Manager Requirements ## Navigation - Home - My Files - Shared with me - Recent - Starred - Trash ## File operations Required: - Create folder - Upload one file - Upload multiple files - Folder upload - Download one file - Download multiple files as ZIP - Rename - Move - Copy - Delete - Restore - Empty Trash - Star/unstar - Share ## UI - Breadcrumbs - Grid view - List view - Sort - Search - File type icons - Size - Modified date - Owner - Selection state - Context menu - Drag/drop upload - Upload progress - Download progress - Error state - Empty state - Loading state ## File names Support Unicode filenames. Normalize filenames safely. Prevent: - `../` - absolute paths - null bytes - control characters that cause filesystem issues - reserved/system path collisions ------------------------------------------------------------------------ # 9. Upload System This is a critical subsystem. ## Small uploads Can use multipart HTTP uploads. ## Large uploads Use resumable/chunked uploads. Flow: ``` text 1. Create upload session 2. Server returns upload ID 3. Client sends chunks 4. Server records received chunks 5. Client retries failed chunks 6. Server verifies completion 7. Server calculates/validates checksum 8. Temporary upload becomes final file 9. Metadata transaction is committed 10. Client receives final file ID ``` Required: - Pause/resume - Retry failed chunks - Progress percentage - Cancel upload - Expired upload cleanup - Duplicate/chunk protection - Maximum file size configuration - Per-user storage quota - Server-side validation Never trust client-provided file size or MIME type without verification. ------------------------------------------------------------------------ # 10. Download System Required: - Stream files - Support HTTP range requests where appropriate - Resume interrupted downloads - Correct Content-Type - Correct Content-Disposition - Authorization before download - Prevent downloading deleted/unowned files - Generate ZIP archives asynchronously for large multi-file downloads Never read a huge file entirely into PHP memory. ------------------------------------------------------------------------ # 11. File Preview V1: - Images - PDF - Plain text - Common video/audio metadata where practical Later: - Office documents - Spreadsheet preview - Presentation preview Previews must run through a controlled service/process. Never execute uploaded files. ------------------------------------------------------------------------ # 12. Trash / Recycle Bin Delete behavior: ``` text Active | Delete v Trash | +--> Restore | +--> Permanent delete ``` Admin-configurable retention: - Default 30 days - Configurable later Background worker permanently removes expired trash. ------------------------------------------------------------------------ # 13. File Versioning Phase 2. For important file changes: ``` text report.docx | +-- v1 +-- v2 +-- v3 +-- current ``` Required: - Version history - Restore version - Delete old version according to retention policy Do not implement versioning by blindly duplicating massive files forever. Design storage/version retention carefully. ------------------------------------------------------------------------ # 14. Search V1: - Filename - Folder - MIME type - Owner - Date - Size Use MySQL indexes/FULLTEXT features initially. Later: - Full text inside documents - OCR - Advanced search engine Search must only return files the authenticated user is authorized to see. ------------------------------------------------------------------------ # 15. Sharing System ## Share links Each share link should have: - Random unguessable token - File/folder reference - Created by - Created date - Expiry - Optional password - Download permission - Optional upload permission - Maximum downloads - Revoked state - Last accessed - Access count Example: ``` text https://cloud.example.com/s/Ab8Kp92xQ ``` Never use sequential IDs as public share tokens. ## Share management Users can: - Create share - Copy share URL - Set expiry - Set password - Revoke share - View active shares Admins can revoke any share. ------------------------------------------------------------------------ # 16. Shared Folders Admin can create: ``` text Company Documents Project Alpha Public Resources ``` Permissions can be: - Read - Write - Delete - Share Membership can be assigned to users/groups. ------------------------------------------------------------------------ # 17. Remote Access Production URL: ``` text https://cloud.example.com ``` Recommended path: ``` text Domain | DNS | Public IP | Router port 443 | Caddy | Application ``` Only: ``` text TCP 443 ``` should normally be publicly exposed. ## Important network limitation If the ISP uses CGNAT, normal port forwarding may not work. The deployment plan must therefore support two modes: ### Mode A --- Direct HTTPS Best performance. ``` text Internet → Router → Caddy → NAS ``` ### Mode B --- Outbound tunnel For networks where inbound connectivity is impossible. The tunnel provider should be selected and configured separately from the application so the product does not become tightly coupled to one provider. The application must still behave exactly the same to users. ------------------------------------------------------------------------ # 18. DNS / TLS Required: - Custom domain - DNS record - Automatic TLS - Automatic certificate renewal - HTTP → HTTPS redirect - HSTS after deployment is verified - Secure cookies - TLS-only production API Never build your own certificate management. ------------------------------------------------------------------------ # 19. API Design Use versioned API: ``` text /api/v1/ ``` Examples: ### Authentication ``` text POST /api/v1/auth/login POST /api/v1/auth/logout POST /api/v1/auth/refresh POST /api/v1/auth/forgot-password POST /api/v1/auth/reset-password POST /api/v1/auth/verify-email ``` ### User ``` text GET /api/v1/me PATCH /api/v1/me GET /api/v1/me/sessions DELETE /api/v1/me/sessions/{id} ``` ### Files ``` text GET /api/v1/files POST /api/v1/files/folders GET /api/v1/files/{id} PATCH /api/v1/files/{id} DELETE /api/v1/files/{id} POST /api/v1/files/{id}/move POST /api/v1/files/{id}/copy POST /api/v1/files/{id}/star ``` ### Upload ``` text POST /api/v1/uploads POST /api/v1/uploads/{id}/chunks POST /api/v1/uploads/{id}/complete DELETE /api/v1/uploads/{id} ``` ### Download ``` text GET /api/v1/files/{id}/download ``` ### Shares ``` text POST /api/v1/shares GET /api/v1/shares DELETE /api/v1/shares/{id} GET /api/v1/public/shares/{token} ``` ### Admin ``` text GET /api/v1/admin/users POST /api/v1/admin/users PATCH /api/v1/admin/users/{id} DELETE /api/v1/admin/users/{id} GET /api/v1/admin/storage GET /api/v1/admin/audit-logs GET /api/v1/admin/system-health ``` API naming can evolve, but it must be consistent and documented with OpenAPI. ------------------------------------------------------------------------ # 20. Database Design Minimum entities: ``` text users roles permissions role_permissions user_roles sessions devices folders files file_versions shared_folders shared_folder_members share_links uploads upload_chunks favorites notifications audit_logs password_reset_tokens email_verifications system_settings ``` Every table must have proper: - Primary keys - Foreign keys - Unique constraints - Indexes - Timestamps - Soft deletion where appropriate Use UUID/ULID identifiers for externally exposed resources. ------------------------------------------------------------------------ # 21. Audit Logging Record security-sensitive events: - Login success - Login failure - Logout - Password change - Password reset - 2FA changes - User created - User disabled - Permission changed - File uploaded - File downloaded - File deleted - File restored - Share created - Share revoked - Session revoked Do not log: - Passwords - Authentication tokens - Private file contents - Share passwords Audit logs should contain: - actor - event - target - timestamp - IP where appropriate - user agent/device where appropriate - success/failure - metadata ------------------------------------------------------------------------ # 22. Notifications V1: - In-app notifications Later: - Push notifications - Email notifications Examples: - File shared with you - Upload completed - New login - Password changed - Share link revoked - Storage quota warning ------------------------------------------------------------------------ # 23. Mobile Application Use React Native (TypeScript) with a single codebase targeting Android and iOS. ## V1 screens ``` text Splash Login Forgot Password Home My Files Folder File Details Upload Download Shared With Me Share Notifications Profile Security Devices Settings ``` ## Mobile features Required: - Login - Logout - Browse files - Upload - Download - Create folders - Rename - Delete - Restore - Share - Search - Preview images/PDF - Upload progress - Download progress - Retry failed transfer - Secure local session storage - Biometric unlock where supported ## Mobile security Tokens must be stored in platform-secure storage (e.g. `react-native-keychain` or equivalent), never in AsyncStorage or plain local storage: - Android Keystore-backed secure storage - iOS Keychain Do not store authentication secrets in plain local storage. ------------------------------------------------------------------------ # 24. Automatic Photo Backup Phase 3. User can enable: ``` text [✓] Automatically back up photos ``` Options: - Wi-Fi only - Charging only - Selected albums - Background upload - Duplicate detection - Retry failed uploads Architecture: ``` text Phone Camera | Photo Backup Worker | HTTPS chunked upload | NAS | Photos library ``` Do not promise unlimited background execution on mobile OSes. Respect Android/iOS background execution rules. ------------------------------------------------------------------------ # 25. Web UX Requirements The web application must feel like a polished commercial product. Required: - Responsive layout - Desktop sidebar - Mobile navigation - Dark/light theme if practical - Keyboard navigation - Accessible buttons/forms - Loading skeletons - Empty states - Error states - Toast notifications - Confirmation dialogs for destructive actions - Upload progress - Network failure recovery - Offline-friendly UI state Never expose raw backend errors to normal users. ------------------------------------------------------------------------ # 26. Security Requirements Mandatory: - HTTPS - Secure cookies/tokens - CSRF protection where cookie authentication is used - CORS restricted to known origins - Rate limiting - Login throttling - Strong password policy - Password hashing - 2FA for admin - Authorization middleware - Resource ownership checks - Path traversal protection - File upload validation - Filename sanitization - MIME validation - No arbitrary command execution from uploads - SQL parameterization/ORM - XSS protection - Content Security Policy where practical - Security headers - Secrets outside source control - Dependency updates - Audit logging - Session revocation - Account disable enforcement ------------------------------------------------------------------------ # 27. Threat Model The system must explicitly defend against: ### Unauthorized file access Attacker changes: ``` text /files/123 ``` to: ``` text /files/124 ``` Server must verify authorization. ### Path traversal Attacker uploads or requests: ``` text ../../etc/passwd ``` Must be rejected. ### Malicious filenames Treat filenames as untrusted strings. ### Broken share tokens Tokens must be cryptographically random and long enough to prevent guessing. ### Brute force Rate limit authentication. ### Session theft Use secure session handling, expiration and revocation. ### Malicious uploads Never execute uploaded files. ### ZIP extraction attacks If ZIP extraction is ever added, protect against: - Zip Slip - decompression bombs - huge file counts - excessive extraction size ------------------------------------------------------------------------ # 28. Backup Strategy RAID is not backup. Minimum production strategy: ``` text NAS primary storage | +---- Local backup | +---- Offsite backup ``` At minimum: - Database backups - Application configuration backup - Storage backup - Encryption keys/secrets recovery plan Test restoration. A backup that has never been restored is not considered verified. ------------------------------------------------------------------------ # 29. Database Backup Automate: - Daily full MySQL backup - Retention - Compression - Off-device copy Example policy: ``` text Daily: 7 Weekly: 4 Monthly: 6 ``` Exact retention can be configurable. ------------------------------------------------------------------------ # 30. Disaster Recovery Document how to recover from: - SD card failure - HDD failure - Raspberry Pi failure - Database corruption - Accidental deletion - Ransomware - Lost credentials - Power outage The product is not production-ready until the recovery procedure has been tested. ------------------------------------------------------------------------ # 31. Raspberry Pi Deployment As of §50, the Pi runs only the Storage Agent and its data volume — not the rest of the stack. Recommended: - 64-bit OS - Ethernet, not Wi-Fi - Quality power supply - UPS - USB 3 storage - Good cooling - Disable unnecessary services - Automatic security updates where appropriate - Docker restart policies - Health checks - WireGuard configured and healthy before the Storage Agent is considered deployed Do not put the storage volume on a cheap microSD card — use SSD/HDD for persistent data. ------------------------------------------------------------------------ # 32. Docker Services Two separate Compose stacks now (§50) — they are never deployed together on the same host. VPS stack: ``` text caddy frontend backend worker scheduler mysql redis ``` Raspberry Pi stack: ``` text storage-agent ``` Optional later (VPS): ``` text thumbnail-worker monitoring prometheus grafana ``` Persistent volumes must be explicit on both hosts. Never store critical production data only inside ephemeral containers. ------------------------------------------------------------------------ # 33. Background Jobs Use queue workers for: - Email - ZIP generation - Thumbnail generation - Trash cleanup - Expired upload cleanup - Expired share cleanup - Notifications - Storage calculations - File checksum processing - Photo indexing Never make a normal HTTP request wait several minutes for a heavy job. ------------------------------------------------------------------------ # 34. File Integrity For important file operations: - Calculate checksum - Verify upload completion - Detect incomplete uploads - Detect orphan metadata - Detect orphan storage objects Build an admin integrity-check command later: ``` text Check database records Check filesystem objects Report missing objects Report orphan objects ``` Do not automatically delete anything during the first integrity check. ------------------------------------------------------------------------ # 35. Storage Quotas Per-user quota: ``` text User: John Quota: 100 GB Used: 63 GB Available: 37 GB ``` Enforce quota server-side. A user must not be able to bypass quota by: - Concurrent uploads - Multiple upload sessions - Folder copies - Duplicate requests Quota calculations must be concurrency-safe. ------------------------------------------------------------------------ # 36. Performance Targets Target, not a guarantee: - Web UI initial usable state: fast on LAN and reasonable over Internet - Small API requests: normally sub-second on LAN - File transfer speed: limited primarily by disk, network, Raspberry Pi, ISP and remote network - Support at least 50 registered users - Design for a smaller number of simultaneous heavy transfers on Raspberry Pi 4 Do not benchmark only on localhost. Test: - LAN - Mobile network - High-latency connection - Multiple concurrent uploads - Multiple concurrent downloads ------------------------------------------------------------------------ # 37. Testing Strategy ## Backend unit tests Test: - Authentication - Permissions - Quotas - Folder hierarchy - File ownership - Share expiry - Share password - Trash - Versioning - Upload sessions ## API integration tests Test every endpoint. ## Security tests Attempt: - IDOR - Path traversal - Unauthorized download - Unauthorized share access - Brute force - Expired tokens - Disabled user access - Malicious filenames - Oversized uploads ## Frontend tests Test: - Login - Navigation - Upload - File actions - Sharing - Error handling ## Mobile tests Test: - Login - Session restore - Upload - Download - Retry - Share - Logout ## End-to-end At minimum: ``` text Create user → Login → Create folder → Upload file → Download file → Share file → Open share from another browser → Revoke share → Delete file → Restore file → Logout ``` ------------------------------------------------------------------------ # 38. CI/CD Use Git. Every pull request should run: ``` text Lint Unit tests Integration tests Build frontend Build backend Build React Native (Android + iOS) Security/dependency checks ``` Deployment should use versioned releases. Never deploy untested code directly to production. ------------------------------------------------------------------------ # 39. Environment Separation At minimum: ``` text development staging production ``` Do not use production data during development. Example: ``` text .env .env.example ``` Never commit: - passwords - API keys - database credentials - production tokens - private certificates ------------------------------------------------------------------------ # 40. Project Phases ## Phase 0 --- Requirements & Architecture Deliverables: - Final architecture - Database ERD - API specification - UI wireframes - Security model - Storage model - Deployment design Exit criteria: - Architecture reviewed - No major unresolved design decisions ------------------------------------------------------------------------ ## Phase 1 --- Infrastructure Build: - Raspberry Pi OS - Docker - Storage mount - MySQL - Redis - Caddy - Basic CI/CD Exit criteria: - All services start reliably - Data survives container recreation - Health checks work - HTTPS works in staging ------------------------------------------------------------------------ ## Phase 2 --- Authentication Build: - Registration/invitation - Login - Logout - Password reset - Email verification - Sessions - User roles - Admin user management Exit criteria: - Authentication tests pass - Disabled users cannot authenticate - Sessions can be revoked ------------------------------------------------------------------------ ## Phase 3 --- Storage Engine Build: - Folder model - File model - Storage abstraction - Upload - Download - Rename - Move - Copy - Delete - Trash - Quotas Exit criteria: - All file operations tested - No path traversal - Large-file upload works - Restart does not corrupt metadata ------------------------------------------------------------------------ ## Phase 4 --- Web File Manager Build: - Dashboard - File browser - Grid/list - Drag/drop upload - Progress - Search - Preview - Context menus - Trash - Sharing UI Exit criteria: - Complete core workflow works in browser ------------------------------------------------------------------------ ## Phase 5 --- Sharing & Remote Access Build: - Share links - Expiration - Password - Revoke - HTTPS - Domain - Reverse proxy - Remote access - CGNAT-compatible deployment option Exit criteria: - User can access files securely from another network - Public share works without account when configured - Private files remain private ------------------------------------------------------------------------ ## Phase 6 --- Mobile App Build React Native app: - Authentication - File browser - Upload - Download - Folder creation - Rename - Delete - Share - Search - Preview - Secure session storage Exit criteria: - Android and iOS builds connect to production API - Mobile upload/download works over cellular network ------------------------------------------------------------------------ ## Phase 7 --- Security Hardening Build: - 2FA - Rate limiting - Audit logs - Device management - Security headers - CORS policy - CSP - Dependency scanning - Backup verification - Recovery documentation Exit criteria: - Security test suite passes - Admin account protected by 2FA - Recovery process tested ------------------------------------------------------------------------ ## Phase 8 --- Production Readiness Test: - 50 users - Concurrent uploads - Concurrent downloads - Large files - Storage near capacity - Database backup - Restore - Power failure - Network interruption - Internet outage - Raspberry Pi reboot - HDD disconnection/replacement procedure Exit criteria: - No critical security bugs - No critical data-loss bugs - Recovery procedure succeeds - Monitoring/alerts work ------------------------------------------------------------------------ # 41. Feature Priority ## Must Have --- V1 - Authentication - Users - Roles - File manager - Upload - Download - Folder management - Delete - Trash - Search - Sharing - HTTPS - Remote access - Responsive web UI - React Native mobile app - Audit logs - Basic admin dashboard - Backup - Security hardening ## Should Have --- V1.5 - 2FA - File versioning - Favorites - Recent files - Notifications - Device/session management - Storage quotas - PDF/image preview - Resumable uploads - Better admin analytics ## Later - Automatic photo backup - Desktop synchronization client - Passkeys - Advanced search - Office previews - AI photo indexing - Multi-NAS replication - Plugin/app system ------------------------------------------------------------------------ # 42. Definition of Done A feature is NOT complete when: - the screen exists - the button works - the API returns 200 A feature is complete only when: 1. Backend logic exists. 2. Authorization exists. 3. Validation exists. 4. Error handling exists. 5. Logging exists where appropriate. 6. Database migration exists if needed. 7. API documentation exists. 8. Automated tests exist. 9. Frontend implementation exists. 10. Mobile implementation exists if applicable. 11. Security behavior has been tested. 12. Failure/retry behavior has been tested. 13. Documentation exists. 14. It works on ARM64/Raspberry Pi deployment. ------------------------------------------------------------------------ # 43. Claude Code Development Rules Claude Code must follow these rules. ## Rule 1 --- Inspect before modifying Before changing existing files: - Inspect repository structure. - Identify framework versions. - Identify existing environment configuration. - Identify existing database structure. - Identify current deployment setup. - Do not overwrite unrelated work. ## Rule 2 --- Small incremental changes Do not attempt to build the entire system in one giant change. Implement one phase/subsystem at a time. ## Rule 3 --- Test after every major change After implementation: - Run tests. - Run lint. - Run build. - Check migrations. - Check API behavior. If something fails, fix it before continuing. ## Rule 4 --- Never silently weaken security Do not: - disable authentication to make something work - disable authorization - expose database ports - expose storage directories - hardcode secrets - disable TLS - bypass validation ## Rule 5 --- Preserve existing functionality Do not modify unrelated modules merely to make a new feature easier. ## Rule 6 --- Explain assumptions When requirements are ambiguous: - choose the safest reasonable behavior - document the assumption - do not invent dangerous defaults ## Rule 7 --- No fake functionality Do not create UI buttons that pretend a feature works when the backend does not implement it. Do not return fake/mock production data. ## Rule 8 --- No TODO masquerading as completion A placeholder must be clearly marked as incomplete. ------------------------------------------------------------------------ # 44. Recommended Repository Structure ``` text private-cloud-nas/ │ ├── backend/ │ ├── app/ │ ├── config/ │ ├── database/ │ ├── routes/ │ ├── tests/ │ └── ... │ ├── web/ │ ├── src/ │ ├── public/ │ └── ... │ ├── mobile/ │ ├── src/ │ ├── ios/ │ ├── android/ │ ├── __tests__/ │ └── ... │ ├── storage-agent/ # runs on the Raspberry Pi only — see §50 │ ├── src/ │ ├── test/ │ └── ... │ ├── infrastructure/ │ ├── docker/ │ ├── caddy/ │ ├── wireguard/ # example configs, key-generation scripts — see §50 │ ├── scripts/ │ └── monitoring/ │ ├── docs/ │ ├── architecture/ │ ├── api/ │ ├── deployment/ │ ├── security/ │ └── disaster-recovery/ │ ├── docker-compose.yml # VPS stack ├── docker-compose.pi.yml # Raspberry Pi stack (storage-agent only) ├── .env.example ├── README.md └── LICENSE ``` ------------------------------------------------------------------------ # 45. Documentation Required Claude Code must maintain: - README - Architecture document - API documentation - Database ERD - Deployment guide - Raspberry Pi setup guide - Backup guide - Restore guide - Security guide - Troubleshooting guide - Mobile build guide - Production checklist - Disaster recovery guide - Changelog ------------------------------------------------------------------------ # 46. Final Acceptance Test The project is considered successful only if this complete scenario works. ### Administrator 1. Deploys system on Raspberry Pi. 2. Opens HTTPS domain. 3. Logs into admin dashboard. 4. Creates/invites 50 users. 5. Creates shared folders. 6. Assigns permissions. 7. Views storage usage. ### User 1. Receives invitation. 2. Creates account. 3. Logs in. 4. Opens My Files. 5. Creates a folder. 6. Uploads a large file. 7. Upload resumes after network interruption. 8. Downloads file. 9. Renames file. 10. Moves file. 11. Deletes file. 12. Restores file. 13. Searches for file. 14. Creates share link. 15. Sets expiration. 16. Sends link to another person. ### External user 1. Opens share link from another network. 2. Downloads permitted file. 3. Cannot access unrelated private files. ### Mobile user 1. Installs app. 2. Logs in. 3. Browses files. 4. Uploads a file over cellular. 5. Downloads a file. 6. Creates folder. 7. Shares a file. 8. Logs out. 9. Session is correctly invalidated. ### Security 1. Unauthorized API requests fail. 2. User A cannot access User B's files. 3. Disabled user cannot log in. 4. Expired share links fail. 5. Revoked share links fail. 6. Path traversal attempts fail. 7. Brute-force attempts are rate limited. 8. Admin 2FA works. 9. HTTPS is enforced. 10. No internal service is publicly exposed. ### Recovery 1. Database backup is created. 2. Storage backup is created. 3. Raspberry Pi is restarted. 4. Services recover automatically. 5. Data remains intact. 6. A test restore succeeds. ------------------------------------------------------------------------ # 47. First Implementation Order Claude Code should implement in exactly this broad order: ``` text 01. Repository + architecture 02. Docker infrastructure 03. MySQL + Redis 04. Laravel backend skeleton 05. React web skeleton 06. Authentication 07. User/role system 08. StorageService abstraction (initial local-disk implementation) 09. Folder/file database models 10. Upload engine 11. Download engine 12. File manager API 13. File manager web UI 14. Trash 15. Search 16. Share links 17. Admin dashboard 18. Audit logs 19. Caddy + HTTPS (VPS) 20. WireGuard: VPS <-> Raspberry Pi (see §50) 21. Raspberry Pi Storage Agent (see §50) 22. Rewire StorageService onto the Storage Agent, retire the local-disk implementation from step 08 23. Remote-access deployment 24. React Native project 25. React Native authentication 26. React Native file manager 27. Mobile upload/download 28. Mobile sharing 29. 2FA 30. Notifications 31. Versioning 32. Backup automation 33. Monitoring 34. Security testing 35. Load/concurrency testing 36. Disaster recovery testing 37. Production release ``` Do not skip directly to the mobile app before the API and web workflows are stable. ------------------------------------------------------------------------ # 48. Product Philosophy The user should never need to know that the backend is a Raspberry Pi. The experience should be: ``` text Open app ↓ Login ↓ My Files ↓ Upload / Download / Share ``` It should NOT feel like: ``` text Linux server SSH SMB IP addresses Port numbers VPN configuration manual file permissions ``` The complexity belongs behind the product. The user should see a simple private cloud. ------------------------------------------------------------------------ # 49. Final Product Goal The finished system should provide: ``` text PRIVATE CLOUD NAS ┌───────────────────────────────────┐ │ │ │ WEB APP │ │ │ └────────────────┬──────────────────┘ │ │ ┌────────────────▼──────────────────┐ │ BACKEND API │ │ │ │ Auth │ Users │ Files │ Sharing │ │ Quota│ Search│ Audit │ Jobs │ └────────────────┬──────────────────┘ │ ┌─────────┴─────────┐ │ │ Database Redis } same VPS as the app (§50) │ │ StorageService, over WireGuard (§50) ▼ ┌─────────────────┐ │ Raspberry Pi 4 │ │ Storage Agent + │ │ HDD/SSD │ └─────────────────┘ ▲ │ HTTPS / Internet │ ┌────────┴────────┐ │ │ Browser Mobile React Native ``` The project should be treated as a **real software product with security, testing, backups and recovery**, not simply a collection of scripts running on a Raspberry Pi. ------------------------------------------------------------------------ # 50. Deployment Architecture: VPS + Raspberry Pi Storage **This supersedes every earlier section that described a single-node deployment with Laravel, MySQL, and Redis running directly on the Raspberry Pi** (in particular the server/storage/reverse-proxy subsections of §2, the diagrams in §3 and §49, §4's physical-layout note, and §31–32). Sections §0, §1, §5–§29, §33–§48 are unaffected — the product definition, roles, auth, file manager, sharing, security, testing, and CI requirements are all identical; only *where the bytes physically live and how the app reaches them* has changed. The production architecture is split into two roles: ```text Internet | v cloud.example.com | v +-----------------------------+ | VPS | | | | Reverse Proxy / TLS (Caddy) | | Laravel API | | React Web | | MySQL | | Redis | | Queue Workers + Scheduler | +--------------+--------------+ | WireGuard private link | v +-----------------------------+ | Raspberry Pi 4 | | | | Storage Agent | | HDD / SSD | +-----------------------------+ ``` ## Mandatory networking rules 1. The VPS is the only public application entry point. 2. The production domain resolves to the VPS public IP. 3. The Raspberry Pi must not require inbound Internet access at all — WireGuard is the *only* connection it needs, and it can be the one initiating that tunnel outbound if the Pi is behind CGNAT or a restrictive home router (consistent with §17's Mode B reasoning, now applied to the storage link instead of the whole app). 4. The VPS and Pi communicate only over WireGuard/private networking. 5. The Storage Agent binds only to the private WireGuard interface — never `0.0.0.0`, never a public port. 6. MySQL and Redis are never exposed to the public Internet (unchanged from §0 rule 1 — they just now live on the VPS instead of the Pi). 7. SMB/NFS, if ever used for administration or migration, must remain private and must never be the public or VPS↔Pi application protocol — that protocol is the Storage Agent's own API. 8. Users never receive the Pi's IP address, storage paths, or WireGuard details. Per §48, they should not even know a Raspberry Pi is involved. ## Storage Agent contract The Storage Agent is a small, dedicated service running only on the Raspberry Pi, reachable only over the WireGuard interface, that provides authenticated operations for: - `PUT` — write an object by storage key - `GET` — read an object by storage key (must support HTTP Range, since Laravel's own download path proxies Range requests through to here — see §10) - `HEAD` — object metadata (existence, size) without transferring the body - `DELETE` — remove an object - `MOVE`/rename — server-side rename where the caller doesn't need to move actual bytes over the link - checksum verification on write - a health/status endpoint (used by the VPS to detect the Pi being offline — see Failure behavior below) Authentication is a shared secret (bearer token) even though the link is already private — defense in depth, matching §0's general posture of never trusting network position alone. The Laravel application remains the sole authority for authentication, authorization, ownership, sharing, and quotas — none of that logic exists on the Pi. The Storage Agent is intentionally "dumb": it has no concept of users, folders, permissions, or shares. It stores bytes under opaque keys and answers PUT/GET/HEAD/DELETE/MOVE/health — nothing more. This keeps the Storage Agent small enough to audit completely and means all product logic changes happen in one place (Laravel), not two. ## StorageService integration `StorageService` (§2, §4) is the only class in the application that talks to the Storage Agent. It is responsible for: - Resolving a file's storage key to a Storage Agent request - Streaming uploads/downloads through to the Agent without buffering a whole file in PHP memory (§0 rule 7) — for downloads specifically, this means proxying the Storage Agent's response (including `Content-Range`) back to the client rather than downloading the whole file to the VPS first - Translating Storage Agent errors/timeouts into the failure behavior described below - Everything above `StorageService` (upload sessions, file/folder metadata, sharing, quotas) must not need to change if the Storage Agent is ever replaced by something else (a bigger server, S3- compatible object storage) — this was already a requirement before the VPS split and remains one. Chunked-upload staging (§9) happens on the VPS's own local disk; only the final, checksummed, assembled file is sent to the Storage Agent in one write. This avoids a network round trip to the Pi per chunk and keeps upload retry/resume logic fast and entirely local to the VPS. ## Failure behavior If the Pi/Storage Agent is unreachable or unhealthy: - The VPS application must remain healthy and responsive for everything that doesn't need file bytes (auth, listing metadata, admin, sharing management). - Requests that do need file bytes (upload completion, download, copy) must fail fast and safely with a clear, retryable error — never hang waiting on a dead connection. - Queued/background operations that touch storage must be safe to retry (idempotent), since a queue worker may retry a job after a transient Pi outage. - The admin "system health" view (§1, §19) must surface Storage Agent reachability as its own signal, distinct from "the app is down." ------------------------------------------------------------------------ # 51. Deployment Decision Record For the initial production deployment, the canonical architecture is: - **VPS** = public gateway + Laravel application + React + MySQL + Redis + workers - **Raspberry Pi** = private storage appliance + Storage Agent + HDD/SSD - **WireGuard** = private transport between VPS and Raspberry Pi This decision supersedes any earlier architecture in this document that placed Laravel directly on the Raspberry Pi (see §50). The application must remain written so the physical storage backend is replaceable without changing user-facing APIs — that constraint predates this decision and isn't new because of it. ------------------------------------------------------------------------ # 52. Comprehensive File Lifecycle, Deletion, Restoration & Recovery Specification (v2 §0B) This section is mandatory. "Delete", "restore", and "backup" are separate concepts and must never be implemented as a single boolean flag. Internal subsection labels below (0B.1, 0B.2, ...) are kept exactly as in the source document so cross-references in code comments (e.g. `App\Console\Commands\ReconcileStorage`) and this document stay in sync. See §55 for what's actually implemented versus deliberately simplified or deferred against every subsection here. ## 0B.1 File lifecycle states A file or folder can move through these logical states: ``` text CREATING | v ACTIVE <---------------------------+ | | | delete | restore v | TRASHED ---------------------------+ | | permanent delete v PURGED ACTIVE | | version replaced/changed v VERSIONED (if versioning enabled) Any persisted state | | backup snapshot v BACKED-UP (backup is an independent recovery source) ``` The database must record enough information to distinguish: - currently active object - trashed object - permanently purged object - storage operation in progress - backup availability - restoration source - restoration result Do not use "deleted = true" as the entire lifecycle model. ## 0B.2 Metadata required for recoverable deletion For every file/folder object, maintain or derive: - immutable object ID - owner/user ID - current parent folder ID - original parent folder ID when trashed - current name - original name when trashed - storage key - object type: file/folder - size - MIME type - checksum - created timestamp - updated timestamp - deleted timestamp - deleted-by user/admin ID - purge timestamp when permanently removed - retention deadline - version ID where applicable - storage operation status - last verified storage timestamp - restore count / relevant audit references Do not trust a path stored from a browser. Store logical identifiers and generate paths on the server. ## 0B.3 Normal single-file deletion flow ``` text User clicks Delete | v Frontend asks API for deletion | v Laravel authenticates user | v Laravel checks ownership/permission | v Check object exists and is ACTIVE | +---- no ---> return current state / controlled error | v Create transactional deletion intent | v Record original parent + name + storage key | v Move object into TRASHED logical state | v Storage operation moves/renames object to trash namespace | +---- success ---> mark storage operation complete | +---- failure ---> rollback/reconcile state; do not pretend deleted | v Write audit event | v Return new state to client ``` For a user-visible Trash system, the preferred behavior is a logical move to Trash rather than immediate byte destruction. ## 0B.4 Folder deletion flow Deleting a folder is not equivalent to deleting one database row. The system must define recursive semantics: ``` text Folder A ├── file1 ├── file2 └── Folder B ├── file3 └── file4 ``` Deleting Folder A must atomically establish that the complete subtree is trashed from the user's perspective. Requirements: - Preserve the complete original hierarchy. - Preserve original parent relationships. - Preserve each object's original name. - Do not orphan children. - Do not expose trashed descendants in normal search/browse results. - Do not allow active files to remain inaccessible because a parent was trashed. - Large recursive deletes must be processed safely as a job where needed. - The API must return a deletion job/state for very large trees. - Repeated delete requests must be idempotent. ## 0B.5 Delete while upload is in progress Cases: ``` text Upload starts | User deletes file ``` The system must not create a corrupt "deleted but still uploading" object. Required behavior: 1. Identify upload session/chunks. 2. Mark upload as cancellation requested. 3. Stop accepting new chunks. 4. Clean up temporary chunks. 5. Never expose incomplete data as a valid file. 6. Audit the cancellation. 7. If the upload has already been committed, treat it as a normal deletion with Trash semantics. ## 0B.6 Delete while download is in progress If a user downloads a file and another operation deletes it: - An already authorized stream may be allowed to finish according to the configured consistency policy. - New download requests after deletion must respect the new state. - The server must not accidentally expose the file through a stale URL after permission is revoked. - A permanent purge must not delete an object still actively required by an open stream unless the storage layer can safely handle it. - If the platform cannot safely guarantee this, defer physical purge until active handles/leases finish or expire. ## 0B.7 Delete while file is shared Deleting a shared file must have deterministic behavior: ``` text ACTIVE + SHARE | v TRASHED | +--> share becomes inaccessible by default ``` Recommended policy: - A normal user delete invalidates active public share access. - The share record remains auditable but cannot retrieve the trashed object. - Restoring the file does NOT silently reactivate a revoked share unless policy explicitly says so. - If share restoration is supported, require an explicit reactivation. - Permanent purge permanently invalidates the share. ## 0B.8 Delete while another user has access For shared folders/files: - Authorization must be evaluated at request time. - A user's permission must not override an owner's deletion. - If an object is trashed, all inherited access must stop according to the share policy. - Existing cached listings must be invalidated. - Search indexes must remove or mark the object immediately enough to prevent stale exposure. - Audit who deleted it and under whose authority. ## 0B.9 Rename/move while trashed Default policy: ``` text TRASHED objects are not editable by normal users. ``` Only restore, permanent delete, or admin recovery operations should change them. If admin recovery permits editing: - record the action - preserve original metadata - never destroy the original recovery information ## 0B.10 Restore a single file ``` text User opens Trash | v Select file | v Restore | v Authenticate + authorize | v Verify object is TRASHED | v Read original parent/name metadata | v Check whether original parent still exists | +---+----------------------+ | | YES NO | | v v Use original parent Apply restore policy | +--> recreate parent +--> restore to safe fallback +--> ask user to choose location | v Check name collision | +---+-----------------------+ | | | none same object different object | | | v v v restore restore conflict resolution (rename / choose / cancel) | v Move/restore storage object | v Confirm checksum/size where applicable | v Change state to ACTIVE | v Invalidate caches/search indexes | v Write audit event | v Return restored object ``` ## 0B.11 Restore when original parent was deleted Example: ``` text Projects/ └── A/ └── report.pdf ``` Both `A` and `report.pdf` are trashed. If the user restores only `report.pdf`, the system must NOT silently create an arbitrary path. The configured policy must be one of: 1. Restore the required parent hierarchy. 2. Ask the user to choose a destination. 3. Restore to a designated "Recovered" folder. The policy must be explicit and tested. ## 0B.12 Restore when original parent was permanently purged The original parent no longer exists. Required behavior: - Never fail with an unexplained server error. - Never write outside the user's authorized root. - Never recreate deleted unrelated content. - Offer the configured fallback destination or an explicit user-selected destination. - Record that the original location was unavailable. ## 0B.13 Restore with filename collision Example: ``` text Before deletion: Projects/report.pdf After deletion: Projects/report.pdf <-- new file created User restores old report.pdf ``` Never overwrite automatically. Supported options should be: ``` text Keep both: report.pdf report (1).pdf Choose another destination Replace existing ``` "Replace existing" must require explicit confirmation and a second authorization check. If implemented, the replaced object itself should follow the platform's version/trash policy rather than disappearing without recovery. ## 0B.14 Restore a folder Folder restore must preserve the complete subtree. ``` text Trash | v Restore Folder A | +--> restore hierarchy +--> resolve parent conflicts +--> resolve child name conflicts +--> restore files/folders in dependency order +--> verify storage results +--> finalize ACTIVE state ``` For large trees: - use a durable background job - make the job resumable - expose progress - avoid partial "success" messaging - reconcile failures - allow retry without duplicating objects ## 0B.15 Partial restore failure Example: ``` text 100 files selected 98 restored 2 failed ``` The system must not say "100 restored". Instead: ``` text Restore complete with errors 98 restored 2 failed ``` Each failed object must have: - object ID - reason - current state - retry capability where safe The restore job must be idempotent. ## 0B.16 Storage offline during restore ``` text Restore requested | v Pi offline | v Do not mark ACTIVE | v Queue/reject according to policy | v Show "Storage unavailable" | v Retry when storage returns ``` Database state must never claim that bytes were restored when the Storage Agent did not confirm the operation. ## 0B.17 Storage offline during delete Same principle: ``` text Delete requested | v Pi offline | +--> logical deletion pending | v Do not claim physical move completed ``` A durable operation record must distinguish: - requested - accepted - storage move pending - storage move confirmed - reconciliation required - failed ## 0B.18 Duplicate/replayed delete or restore request Network retries are normal. For destructive or restoring operations: - accept an idempotency key where appropriate - identify already-completed operations - return the existing result instead of repeating the action - never create duplicate restored files because a client retried ## 0B.19 Trash retention and automatic purge Trash must support a configurable retention policy. Example: ``` text Deleted: 2026-08-01 Retention: 30 days Purge after: 2026-08-31 ``` A scheduled job must: 1. Find objects past retention. 2. Re-check ownership/state. 3. Ensure no active recovery operation exists. 4. Ensure required backup policy is satisfied if configured. 5. Request physical purge from Storage Agent. 6. Verify purge. 7. Mark metadata PURGED. 8. Invalidate caches/indexes. 9. Write audit record. Automatic purge must be restart-safe and idempotent. ## 0B.20 Permanent delete by user Permanent delete must be an explicit action. Before execution: - confirm the object is already in Trash - warn that normal user restore is no longer possible - require authorization - record an audit event After successful purge: - metadata must no longer appear in normal Trash - public shares must remain invalid - search indexes must be updated - storage bytes must be physically reclaimed or queued for verified reclamation If the platform supports backup recovery, the file may still exist in backup snapshots. The UI must distinguish "not restorable from Trash" from "not present in any backup". ## 0B.21 Admin recovery after permanent deletion Administrators need a separate recovery path when backups contain the object. ``` text Admin | v Recovery Center | v Select backup snapshot | v Search object/version | v Preview metadata | v Select restore destination | v Restore through StorageService | v Verify checksum/size | v Create ACTIVE object or recovered copy | v Audit ``` Never let an admin recovery operation silently overwrite an active file. ## 0B.22 Backup consistency Backups must cover both: **Database** - users - permissions - file metadata - share records - audit records - version metadata - operation records **Storage** - actual file bytes - required object/version mapping A database-only backup is NOT a complete NAS backup. A storage-only backup is also insufficient because file ownership and permissions may be missing. ## 0B.23 Backup restore ordering Recommended recovery sequence: ``` text 1. Provision healthy VPS/application 2. Restore MySQL 3. Restore application secrets/configuration 4. Restore storage objects 5. Reconnect Storage Agent 6. Run storage/database consistency scan 7. Rebuild search/index metadata 8. Validate permissions 9. Validate share invalidation 10. Run smoke tests 11. Open service to users ``` Do not open the service before integrity checks complete. ## 0B.24 Disaster scenarios that must be tested Claude Code must document and test recovery for at least: 1. User deletes a file and restores it. 2. User deletes a folder recursively and restores it. 3. Original parent was also deleted. 4. Original parent was permanently purged. 5. Restore collides with a newly created file. 6. Delete is retried because of network timeout. 7. Restore is retried because of network timeout. 8. Pi goes offline during delete. 9. Pi goes offline during restore. 10. Pi reboots during a storage operation. 11. VPS application restarts during a storage operation. 12. Queue worker crashes during purge. 13. Storage Agent crashes during write. 14. Upload is cancelled during chunking. 15. File is deleted while an upload is finishing. 16. File is deleted while being downloaded. 17. Shared file is deleted. 18. Shared folder is deleted. 19. User loses access while a share exists. 20. Trash retention job runs twice. 21. Permanent purge job runs twice. 22. Database restore occurs before storage restore. 23. Storage restore occurs with missing metadata. 24. Backup contains an older version of a file. 25. Backup contains a file that was later renamed. 26. Backup contains a file that was later moved. 27. Backup contains a file whose parent no longer exists. 28. Checksum mismatch occurs during restore. 29. Storage reports success but database transaction fails. 30. Database reports pending operation and storage already completed. 31. Two admins attempt the same recovery. 32. User attempts to restore another user's trashed file. 33. User attempts to access a trashed file through an old direct URL. 34. User attempts to access a purged file through an old share URL. 35. Path traversal or symlink tricks target Trash/recovery locations. ## 0B.25 Consistency reconciler Implement a periodic reconciliation job. It must compare: - database object state - storage-agent object existence - checksums where feasible - pending operation records - backup metadata where configured It must identify states such as: ``` text DB ACTIVE + storage MISSING DB TRASHED + storage ACTIVE DB PURGED + storage EXISTS DB PENDING + storage COMPLETED DB ACTIVE + checksum mismatch ORPHAN STORAGE OBJECT ORPHAN DB RECORD ``` The reconciler must NOT blindly delete or overwrite data. It should: - classify the inconsistency - create an alert/audit event - apply only explicitly defined safe repair rules - send ambiguous cases to admin recovery ## 0B.26 Recovery Center The admin interface should expose: - Trash - Permanently deleted objects available in backups - Pending storage operations - Failed storage operations - Inconsistency findings - Backup snapshots - Restore jobs - Storage health - Checksum/integrity results Every recovery action must display: - who initiated it - when - source - destination - object/version - result - errors - audit ID ------------------------------------------------------------------------ # 53. File Operation State Machine (v2 §0C) All important file operations should have durable state. Example: ``` text REQUESTED | v AUTHORIZED | v QUEUED (if asynchronous) | v STORAGE_PENDING | +------> FAILED_RETRYABLE | | | v | RETRYING | v STORAGE_CONFIRMED | v DB_FINALIZED | v COMPLETED ``` Never expose a user-visible "success" state before the authoritative operation has reached its required completion point. For every operation define: - idempotency strategy - timeout - retry policy - maximum retry count - compensation/reconciliation behavior - user-visible status - audit event ------------------------------------------------------------------------ # 54. Failure Matrix (v2 §0D) ----------------------------------------------------------------------- Failure Expected behavior ----------------------------------- ----------------------------------- Pi offline File operations fail gracefully or queue; auth remains available Pi reboot Agent reconnects; unfinished jobs reconcile WireGuard down Storage unavailable state; no public exposure Storage disk full Uploads stop safely; existing data remains readable Storage disk read-only Writes fail; alert admin; no false success Storage checksum mismatch Operation fails; object marked suspect VPS restart Docker services recover; queued operations resume MySQL restart API retries controlled DB errors; no corrupt writes Redis restart Durable critical state remains in DB Worker crash Job retry without duplicate file Client disconnect during upload Resume from last confirmed chunk Client disconnect during restore Server-side job continues or safely rolls back Browser retry Idempotent operation returns existing result Mobile retry Same idempotency semantics Duplicate upload Defined collision/deduplication policy Duplicate restore Return current state; don't create duplicates Duplicate purge Safe no-op Share revoked mid-download New requests denied; active-stream policy enforced User disabled Sessions/tokens revoked or denied according to policy Backup unavailable Recovery Center reports unavailable source Database/storage mismatch Reconciler flags it; no destructive automatic guess ----------------------------------------------------------------------- ------------------------------------------------------------------------ # 55. §52-54 Implementation Status §52-54 arrived as an update to this document and describe a considerably more rigorous file-lifecycle/recovery model than what was built before that update landed. This section records, honestly, what already satisfied it, what was added to close a real gap, what's a deliberate and documented simplification, and what's still open — rather than treating §52-54 as fully implemented just because it's now in this document. ## Already satisfied before this update - **0B.7** (delete invalidates shares): `ShareService::findUsableOrFail()` already checks `$share->file->trashed()` — a trashed file's share links immediately stop resolving. - **0B.9** (trashed objects aren't editable): enforced at the routing layer, not the policy layer — the rename (`PATCH /files/{file}`) and move (`POST /files/{file}/move`) routes don't use `withTrashed()`, so Laravel's route-model binding 404s before the controller ever runs for a trashed file's `public_id`. Only `restore` and the permanent- delete route opt into `withTrashed()`. - **0B.4** (folder delete/restore atomicity): `FileService::delete()`/ `restore()` update a whole subtree in one bulk query each; descendants never end up orphaned or half-trashed. - **0B.16/0B.17** (storage-agent-offline during restore/delete): trivially satisfied by construction — `delete()`/`restore()` are metadata-only and never call the Storage Agent at all (see the next section), so there's no storage confirmation for them to falsely claim. The operations that do touch storage (upload completion, download, permanent delete, copy) already fail fast with a clear error instead of hanging or silently succeeding — verified end-to-end against a real Storage Agent container, including with the container stopped mid-request. - **Storage Agent crash during write (0B.24 #13)**: `writeObject()` streams to a `.part--