added security settings

changed fixes and issues

docs/issues.md

@@ -14,6 +14,14 @@ Several earlier backlog items have already landed in the codebase:
 
 Known caveat: agent-side jobs are persisted, but subprocesses cannot survive an agent restart. Active jobs are marked `failed` on startup; deeper cleanup recovery for partially changed host resources is still future work.
 
+Current pre-production priorities:
+
+1. Harden backup verification and add tests for stream/pipeline failure cases.
+2. Validate the staged restore workflow on a disposable Incus VM, including rollback scenarios.
+3. Improve agent crash cleanup for partially changed ZFS/Incus resources.
+4. Surface detailed node health diagnostics in the UI.
+5. Keep the root-running agent tightly network-restricted.
+
 ## 1. Persist final agent job status in management
 
 Status: mostly done.
@@ -85,6 +93,8 @@ Ensure a backup is marked `success` only when the expected source data was fully
 - [x] Remove failed Restic snapshots with `forget` and `prune` on verification errors.
 - [ ] Make stream/pipeline failure handling explicit for all sources.
 - [ ] Avoid marking success if source stream closes early but Restic exits successfully.
+- [ ] Add a post-backup Restic integrity check strategy, such as targeted `restic check`/`restic stats` usage that is safe for large repositories.
+- [ ] Define an optional restore-probe workflow for critical VMs.
 - [ ] Add tests with mocked command failures and short reads.
 
 ### Acceptance Criteria
@@ -92,6 +102,7 @@ Ensure a backup is marked `success` only when the expected source data was fully
 - [x] Successful backup jobs include a verifiable Restic snapshot ID in management history.
 - [x] Size mismatches fail the job.
 - [ ] Simulated source stream errors cannot produce a successful job.
+- [ ] A backup can be independently verified without trusting only the successful process exit.
 - [ ] Verification behavior is covered by automated tests.
 
 ## 4. Add per-VM backup policy
@@ -139,6 +150,8 @@ Reduce the risk of accidental or unsafe restores.
 - Optionally offer “create backup before restore” when the VM is accessible.
 - [x] Log restore intent in audit log before dispatch.
 - [x] Replace direct `dd` to production ZVOL with a staged restore workflow.
+- [ ] Test staged restore on a disposable VM: backup, restore, boot, and confirm data.
+- [ ] Test failed staged restore paths: Restic dump failure before swap, rename failure after old volume rename, and VM start failure.
 - [ ] Define a safe container restore workflow with `zfs receive` or disable container restore surfaces entirely.
 
 ### Acceptance Criteria
@@ -147,6 +160,7 @@ Reduce the risk of accidental or unsafe restores.
 - Restore is blocked when required preflight checks fail.
 - Audit log records restore attempts and results.
 - [x] Restore stream failures happen on the staged volume, not the production disk.
+- [ ] Staged restore behavior is validated on a real Incus/ZFS test node.
 
 ## 6. Add failure notifications
 
@@ -240,3 +254,96 @@ Provide a production-ready Docker/Compose deployment for the management API and
 - Management database survives container recreation.
 - Cookie login works behind HTTPS.
 - Management can connect to HTTPS node-agents using the configured CA file.
+
+## 10. Add automated tests and CI
+
+The project currently has no automated tests. This is the biggest engineering gap for a backup system because most dangerous failures happen in error paths, not in the happy path.
+
+### Goal
+
+Catch regressions in validation, locking, backup verification, restore orchestration, authentication, and management polling before deployment.
+
+### Tasks
+
+- Add a test runner for agent and management code.
+- Test validators for instance names, snapshot IDs, and ambiguous snapshot prefixes.
+- Test job locking, persistence, trimming, and restart behavior.
+- Test backup pipeline failure handling with mocked command/process failures.
+- Test restore staged-volume command sequencing with mocked ZFS/Incus/Restic commands.
+- Test management job polling updates `job_history` correctly for success, failed, timeout, and missing-agent cases.
+- Add CI for install, tests, frontend build, and syntax checks.
+
+### Acceptance Criteria
+
+- Pull requests run tests automatically.
+- Simulated backup stream failures fail the job.
+- Simulated restore failures leave the original ZVOL name restored in the command sequence.
+- Agent restart behavior is covered by tests.
+
+## 11. Harden root-running agent exposure
+
+The agent runs with root-level host access because it needs Incus, ZFS, `/dev/zvol`, Restic, and device operations. A compromised agent is therefore a host-level incident.
+
+### Goal
+
+Reduce the network and systemd blast radius of the root-running node-agent.
+
+### Tasks
+
+- Require private management-to-agent connectivity in production documentation, such as NetBird or a dedicated management network.
+- Prefer HTTPS agents with a private CA; document local HTTP only as development mode.
+- Enable and document `ALLOWED_MANAGEMENT_IPS` for production.
+- Add systemd hardening where compatible with Incus/ZFS access.
+- Add explicit installer warnings when `HTTPS_ENABLED=false` and no `ALLOWED_MANAGEMENT_IPS` is configured.
+- Consider replacing `npm start` in systemd with direct `node src/index.js`.
+
+### Acceptance Criteria
+
+- Production install docs do not expose the agent publicly by default.
+- Installer warns on insecure network exposure.
+- systemd unit has a documented minimum hardening baseline.
+
+## 12. Improve agent crash cleanup and resource recovery
+
+Agent jobs are persisted and active jobs are marked `failed` on restart, but a process crash can still leave host resources behind, such as temporary Incus snapshots, staged restore volumes, visible snapshot devices, or changed ZFS properties.
+
+### Goal
+
+Make agent startup detect and clean up known leftover resources from interrupted jobs where doing so is safe.
+
+### Tasks
+
+- Persist cleanup metadata for backup jobs: temporary Incus snapshot name, ZVOL, snapdev state, and backup type.
+- Persist cleanup metadata for restore jobs: staged ZVOL, backup ZVOL, failed ZVOL, and swap state.
+- On startup, scan failed active jobs and run safe cleanup actions.
+- Log cleanup results into the persisted job logs.
+- Avoid destructive cleanup when state is ambiguous; surface manual action instead.
+
+### Acceptance Criteria
+
+- Crashing during backup streaming does not leave `snapdev=visible` or temporary snapshots unnoticed.
+- Crashing before restore swap removes staged restore volumes when safe.
+- Crashing after restore swap does not automatically destroy rollback copies.
+- Management can show cleanup-required states.
+
+## 13. Improve health diagnostics in the UI
+
+The agent health endpoint returns useful structured details, but the management UI currently summarizes this too aggressively. Operators need actionable health reasons without opening logs.
+
+### Goal
+
+Show per-node health detail in the UI with concrete failed checks and messages.
+
+### Tasks
+
+- Add a detailed health drawer or modal on the Nodes page.
+- Show command, ZFS pool, `/dev/zvol`, Restic repository, and credential check results.
+- Show last health timestamp and the management-side error if the agent is unreachable.
+- Distinguish unreachable, unauthorized, TLS failure, degraded health, and healthy states.
+- Avoid global UI timeout banners when only one enabled node is slow or unreachable.
+
+### Acceptance Criteria
+
+- A missing command is visible by name in the UI.
+- Wrong Restic credentials are visible as a Restic health failure.
+- TLS or connectivity failures are distinguishable from degraded agent health.