changed to incubator

added new install script
updated security
updated ETA

docs/deployment.md

@@ -19,7 +19,8 @@ curl -fsSL https://forgejo.digital-droplets.de/philschlo/incus-backup-ui/raw/bra
 
 Useful installer environment variables:
 
-- `INSTALL_DIR`: install location, default `/opt/incus-backup-ui`.
+- `INSTALL_DIR`: install location, default `/opt/incubator`.
+- `LEGACY_INSTALL_DIR`: old install location used for `.env` migration, default `/opt/incus-backup-ui`.
 - `INCUS_BACKUP_REPO_URL`: Git repository URL.
 - `INCUS_BACKUP_BRANCH`: Git branch, default `main`.
 - `API_TOKEN`: explicit agent token. If omitted, the installer generates one and prints it once.
@@ -71,6 +72,22 @@ sudo systemctl enable --now incus-backup-agent
 sudo journalctl -u incus-backup-agent -f
 ```
 
+The installer uses Git sparse-checkout and only checks out agent-related paths: `agent/` and `deploy/`.
+When upgrading from the old default path, it copies an existing `/opt/incus-backup-ui/agent/.env` or `/opt/incus-backup-ui/backend/.env` into `/opt/incubator/agent/.env` if the new env file does not exist yet.
+
+The installer also installs a small operational CLI:
+
+```bash
+sudo incubator update
+sudo incubator status
+sudo incubator logs
+sudo incubator doctor
+sudo incubator restart
+sudo incubator env
+```
+
+Use `update` for normal agent updates. It pulls the configured branch, installs dependencies, refreshes the systemd unit, and restarts the agent.
+
 ## Management API
 
 Run this on the management server:

docs/issues.md

@@ -26,11 +26,12 @@ Known caveat: active agent jobs are persisted, but deeper cleanup recovery for p
 
 1. Harden backup verification and add tests for stream/pipeline failure cases.
 2. Validate the staged restore workflow on a disposable Incus VM, including rollback scenarios.
-3. Add cleanup and visibility for pre-restore/failed-restore ZVOLs.
-4. Improve agent crash cleanup for partially changed ZFS/Incus resources.
-5. Surface detailed node health diagnostics in the UI.
-6. Keep the root-running agent tightly network-restricted.
-7. Add automated tests and CI.
+3. Evaluate true incremental ZFS-send based backups for large VM disks.
+4. Add cleanup and visibility for pre-restore/failed-restore ZVOLs.
+5. Improve agent crash cleanup for partially changed ZFS/Incus resources.
+6. Surface detailed node health diagnostics in the UI.
+7. Keep the root-running agent tightly network-restricted.
+8. Add automated tests and CI.
 
 ## P0 - Production Blockers
 
@@ -149,7 +150,31 @@ Acceptance criteria:
 
 ## P1 - High Priority
 
-### 6. Add Automated Tests and CI
+### 6. Evaluate True Incremental VM Backups
+
+Status: open.
+
+Current behavior: VM backups stream the full ZVOL as `vm.raw` through `restic backup --stdin`. Restic deduplicates storage, but the agent still has to read, chunk, and hash the complete virtual disk every run. A VM with no changed files can therefore still take a long time.
+
+Goal: reduce backup duration for large mostly-unchanged VMs by reading only changed ZFS blocks after the first full backup.
+
+Tasks:
+
+- [ ] Evaluate ZFS snapshot-chain based incremental backups with `zfs send -i previous current`.
+- [ ] Design snapshot naming and retention so required incremental bases are not deleted too early.
+- [ ] Define restore behavior for full plus incremental send chains.
+- [ ] Decide whether incremental streams should be stored in Restic, object storage directly, or another repository layout.
+- [ ] Define compatibility behavior for existing `.raw` Restic snapshots.
+- [ ] Compare operational tradeoffs: faster backups versus more complex restore and retention.
+- [ ] Add UI wording that distinguishes `Processed` bytes from data actually uploaded/stored.
+
+Acceptance criteria:
+
+- [ ] A second backup of an unchanged large VM does not need to read the full ZVOL.
+- [ ] Restore can reconstruct a VM from the chosen full/incremental chain.
+- [ ] Retention cannot delete an incremental base required for restore.
+
+### 7. Add Automated Tests and CI
 
 Status: open.
 
@@ -170,7 +195,7 @@ Acceptance criteria:
 - [ ] Simulated restore failures leave the original ZVOL name restored in the command sequence.
 - [ ] Agent restart behavior is covered by tests.
 
-### 7. Improve Agent Crash Cleanup and Resource Recovery
+### 8. Improve Agent Crash Cleanup and Resource Recovery
 
 Status: open.
 
@@ -191,7 +216,7 @@ Acceptance criteria:
 - [ ] Crashing after restore swap does not automatically destroy rollback copies.
 - [ ] Management can show cleanup-required states.
 
-### 8. Improve Health Diagnostics in the UI
+### 9. Improve Health Diagnostics in the UI
 
 Status: partially done on API, open in UI.
 
@@ -211,7 +236,7 @@ Acceptance criteria:
 - [ ] Wrong Restic credentials are visible as a Restic health failure.
 - [ ] TLS or connectivity failures are distinguishable from degraded agent health.
 
-### 9. Fix Snapshot Device Visibility Race
+### 10. Fix Snapshot Device Visibility Race
 
 Status: open.
 
@@ -227,7 +252,7 @@ Acceptance criteria:
 
 - [ ] Slow device-node creation does not fail randomly.
 
-### 10. Make Snapshot-ID Prefix Matching Unambiguous
+### 11. Make Snapshot-ID Prefix Matching Unambiguous
 
 Status: open.
 
@@ -243,7 +268,7 @@ Acceptance criteria:
 
 - [ ] Ambiguous snapshot prefixes cannot restore the wrong snapshot.
 
-### 11. Settings and Environment Hardening
+### 12. Settings and Environment Hardening
 
 Status: partially done.
 
@@ -262,7 +287,7 @@ Acceptance criteria:
 - [ ] Secret settings cannot be exfiltrated through the UI/API.
 - [ ] Writing `.env` cannot create shell-expansion surprises.
 
-### 12. Add RBAC
+### 13. Add RBAC
 
 Status: open.
 
@@ -280,7 +305,7 @@ Acceptance criteria:
 - [ ] Restore is admin-only.
 - [ ] Viewer cannot trigger backup, restore, node edits, or settings writes.
 
-### 13. Session and Auth Cleanup
+### 14. Session and Auth Cleanup
 
 Status: partially done.
 
@@ -298,7 +323,7 @@ Acceptance criteria:
 - [ ] Expired sessions do not accumulate unbounded in SQLite.
 - [ ] Session IDs are rotated after login.
 
-### 14. Container Restore Decision
+### 15. Container Restore Decision
 
 Status: open.
 
@@ -316,7 +341,7 @@ Acceptance criteria:
 
 ## P2 - Product and Operations
 
-### 15. Add Per-VM Backup Policy
+### 16. Add Per-VM Backup Policy
 
 Status: partially done.
 
@@ -337,7 +362,7 @@ Acceptance criteria:
 - [ ] Two VMs on the same node can have different retention policies.
 - [ ] Disabled policies do not trigger backups.
 
-### 16. Add Failure Notifications
+### 17. Add Failure Notifications
 
 Status: open.
 
@@ -356,7 +381,7 @@ Acceptance criteria:
 - [ ] A test notification can be triggered from the UI.
 - [ ] Notification failures are visible in Operations or audit logs.
 
-### 17. Implement Real Snapshot File Browsing
+### 18. Implement Real Snapshot File Browsing
 
 Status: open.
 
@@ -377,7 +402,7 @@ Acceptance criteria:
 - [ ] Mounted/temporary resources are cleaned up after use.
 - [ ] Unsupported or unsafe disk images fail with a clear error.
 
-### 18. Add Version Reporting
+### 19. Add Version Reporting
 
 Status: open.
 
@@ -395,7 +420,7 @@ Acceptance criteria:
 - [ ] Management can identify incompatible agents.
 - [ ] Health output includes version information.
 
-### 19. Containerized Management and UI Deployment
+### 20. Containerized Management and UI Deployment
 
 Status: open.
 
@@ -418,7 +443,7 @@ Acceptance criteria:
 - [ ] Cookie login works behind HTTPS.
 - [ ] Management can connect to HTTPS node-agents using the configured CA file.
 
-### 20. Backup Scheduling and Quotas
+### 21. Backup Scheduling and Quotas
 
 Status: open.
 
@@ -436,7 +461,7 @@ Acceptance criteria:
 
 ## P3 - Cleanup and Refactoring
 
-### 21. Systemd and Deployment Cleanup
+### 22. Systemd and Deployment Cleanup
 
 Tasks:
 
@@ -445,7 +470,7 @@ Tasks:
 - [ ] Make `SCHEDULES_PATH` explicitly configurable.
 - [ ] Installer should warn on insecure agent exposure.
 
-### 22. Data Integrity and Schema Cleanup
+### 23. Data Integrity and Schema Cleanup
 
 Tasks:
 
@@ -454,7 +479,7 @@ Tasks:
 - [ ] Replace ad-hoc `addColumnIfMissing` with schema versioning and migrations.
 - [ ] Stream Restic `ls` instead of loading all output in RAM.
 
-### 23. Code Cleanup
+### 24. Code Cleanup
 
 Tasks:
 
@@ -481,6 +506,7 @@ Tasks:
 - [x] Staged VM restore implemented.
 - [x] Node-agent renamed from `backend/` to `agent/`.
 - [x] Agent installer script added.
+- [x] Agent update/diagnostic CLI added as `incubator`.
 - [x] Backup UI ETA/rate/bytes display added.
 
 ## OSS Release Requirements