Compare commits

...
Author SHA1 Message Date
Pasha Sviderski 5a1e61ccff fix: broken links to completions docs in CLI reference 2025-08-28 20:15:54 +10:00
Pasha Sviderski db60a81b2d chore: lint 2025-08-28 20:09:48 +10:00
Pasha Sviderski 7c19323ea1 docs: 'uc docs' command to generate CLI reference docs 2025-08-28 20:09:18 +10:00
Pasha Sviderski 290e6db98e docs(ingress): Publishing service and Managing Caddy 2025-08-28 17:40:39 +10:00
Pasha Sviderski df29d9ba43 docs: Ingress & HTTP overview 2025-08-27 22:26:50 +10:00
Pasha Sviderski 64769081d9 fix: format for --publish flag 2025-08-27 22:25:32 +10:00
Pasha Sviderski 0107363d41 fix: allow host mode x-ports with x-caddy in compose 2025-08-26 19:15:09 +10:00
Pasha Sviderski 48dc1dd624 chore: delete unused image digest resolver 2025-08-25 20:36:56 +10:00
Pasha Sviderski 3a6eef410a feat: add --caddyfile flag to 'uc caddy deploy' and 'uc run' commands 2025-08-22 20:52:01 +10:00
Pasha Sviderski 9f5ca9a33d feat: add 'uc caddy config' command to show the current Caddyfile 2025-08-22 20:20:02 +10:00
Pasha Sviderski b046b78398 feat: migrate Caddy to generated Caddyfile, mount persistent data volume 2025-08-22 19:51:15 +10:00
Pasha Sviderski ff213e71d3 fix: unmarshaling of ServiceSpec in ServiceContainer struct 2025-08-22 17:25:28 +10:00
Pasha Sviderski 3cda5cc564 chore: caddy client to get caddy config 2025-08-22 15:58:21 +10:00
Pasha Sviderski 75fdbaf2f4 chore: relax ports+Caddy spec validation to allow host mode ports 2025-08-22 15:34:59 +10:00
Pasha Sviderski 81f4e3a67a chore: fix mockery for linux in .mise.lock 2025-08-21 17:31:43 +10:00
Pasha Sviderski e99e769455 chore: add Caddy gRPC service to retrieve Caddyfile config from machines 2025-08-21 17:04:42 +10:00
Pasha Sviderski 11949eeb3b chore: include validation errors for user-defined Caddy configs as a comment in Caddyfile 2025-08-21 14:01:24 +10:00
Pasha Sviderski b437659678 chore: update comments in generated Caddyfile 2025-08-21 13:14:54 +10:00
Pasha Sviderski c01365b416 chore: add header to generated Caddyfile that it's autogenerated 2025-08-21 08:54:06 +10:00
Pasha Sviderski 813c397644 chore: change reverse_proxy upstreams from 'to' to the directive arguments 2025-08-21 08:32:45 +10:00
Pasha Sviderski 5cc005a423 feat: validate and append custom per-service Caddy configs to generated Caddyfile 2025-08-21 08:18:59 +10:00
Pasha Sviderski 066d411367 chore: parse Created time on container with CreatedTime 2025-08-20 22:54:51 +10:00
Pasha Sviderski 03970862ab feat: concatenate custom Caddy configs for services into final Caddyfile (no upstream interpolation) 2025-08-19 22:01:48 +10:00
Pasha Sviderski 8bf9fc0c9c chore: introduce mockery for generating mocks, generate for CaddyfileValidator 2025-08-18 18:50:47 +10:00
Pasha Sviderski 93fef88fac chore: refactor Caddyfile generator to accept a validator 2025-08-18 18:48:32 +10:00
Pasha Sviderski 455174ccb0 chore: generate sites in Caddyfile from x-ports alongside caddy.json 2025-08-15 18:29:23 +10:00
Pasha Sviderski 1ce3e62dbb fix: use local and remote Docker credentials to pull image from private registry 2025-08-15 14:09:40 +10:00
Pasha Sviderski 4be8339c51 chore: generate a minimal Caddyfile with verify handler alongside caddy.json 2025-08-14 19:43:40 +10:00
Pasha Sviderski 8dd69b46da chore: refactor docker gRPC server to use docker service for inspecting and listing containers 2025-08-14 15:34:59 +10:00
Pasha Sviderski 4cc1e556dd chore: store ServiceContainer (includes service spec) instead of Container in Corrosion store 2025-08-14 14:53:33 +10:00
Pasha Sviderski 9186d31d12 chore: go mod tidy 2025-08-13 19:31:13 +10:00
Pasha Sviderski dd7bc6c982 chore: trim spaces for x-caddy, diff Caddy configs when comparing service specs 2025-08-13 19:27:40 +10:00
Pasha Sviderski 12c07812a2 chore: add Caddy config to ServiceSpec, load x-caddy to it 2025-08-13 18:44:38 +10:00
Pasha Sviderski ec73f9ecd8 chore: handle x-caddy: path/to/Caddyfile to read Caddy config in compose from file 2025-08-13 13:45:39 +10:00
Pasha Sviderski 879c7c1876 test: x-caddy extension parsing 2025-08-13 13:45:39 +10:00
Anton Ovchinnikov c67127f83f feat: Add basic LLM instruction files 2025-08-11 23:57:13 +02:00
Pasha Sviderski 5d3f1fe225 chore: x-caddy extension type in compose 2025-08-11 21:11:40 +10:00
Pasha Sviderski 2e585d0183 feat: add --recreate flag for deploy command to force container recreation 2025-08-07 18:09:35 +10:00
Pasha Sviderski ae9f943404 chore: change default restart policy for service containers always -> unless-stopped 2025-08-07 18:09:35 +10:00
Anton Ovchinnikov 8805178a58 doc: Add Sentry to sponsors 2025-08-07 00:03:38 +02:00
Pasha Sviderski ec3de3a099 feat: ask whether to reset already initialised machine on 'machine init/add' 2025-08-06 16:57:09 +10:00
Pasha Sviderski 2c02139369 fix: add ssh_key_path for connections in uncloud config only when using SSH key explicitly (not SSH agent) 2025-08-06 16:49:57 +10:00
Pasha Sviderski 6c244bb8f9 fix: do not try to reset machine when removing unreachable machine 2025-08-06 15:11:27 +10:00
Pasha Sviderski fc0bf4a91b chore: lint 2025-08-05 19:37:12 +10:00
Pasha Sviderski bc577fe405 docs: emphasize passwordless sudo in requirements 2025-08-05 19:32:11 +10:00
Pasha Sviderski 6cc0611d75 chore: meaningful error message when passwordless sudo required on machine provisioning 2025-08-05 18:56:14 +10:00
Pasha Sviderski 8beb9e2679 chore: lint 2025-08-04 19:20:52 +10:00
Pasha Sviderski 0c2ff499c6 fix: log message deleting iptables chain 2025-08-04 19:16:54 +10:00
Pasha Sviderski b39585df4c chore: do not masquerade container IPs when communicating via WG mesh 2025-08-04 19:09:38 +10:00
Pasha Sviderski 35d0a90125 fix: add machine with UFW firewall (allow ipv6 management traffic) fixes #65 2025-08-04 16:50:18 +10:00
Pasha Sviderski a7273c8c96 chore: fix wide svg rendering on landing 2025-08-04 15:22:05 +10:00
Pasha Sviderski 9b48003a3f blog: set diagram as og image for wireguard overlay post 2025-08-04 15:19:27 +10:00
Pasha Sviderski 1e750cc7e1 chore: add Blog links on landing page, fix header on small screens 2025-08-04 14:50:56 +10:00
Pasha Sviderski 2d73b541c5 blog: proof read WireGuard overlay post, update blog description tag 2025-08-04 14:21:48 +10:00
Pasha Sviderski cd9c0a1d2c docs: enable blog in docusaurus, add first wireguard overlay blog post 2025-08-01 21:45:50 +10:00
Pasha Sviderski 37a76809ec docs: wwizzle breadcrumbs and overwrite base url to /docs 2025-07-31 19:07:12 +10:00
Pasha Sviderski f6062cf343 docs: fix docs urls for local dev 2025-07-31 19:04:24 +10:00
Pasha Sviderski 4fe360c573 docs: combine landing page and docusaurus together and deploy as one image 2025-07-31 18:43:06 +10:00
Pasha Sviderski cac1550b7d docs: enable blog in docusaurus and change domain to uncloud.run 2025-07-30 21:28:06 +10:00
Pasha Sviderski 0bd42b2fca chore: do not allow to remove a machine the client is connected to, add --no-reset flag 2025-07-29 19:54:16 +10:00
Pasha Sviderski ad83d942e1 docs: manual cli upgrade 2025-07-28 21:45:33 +10:00
Pasha Sviderski 65f5a714dd feat: machine rm command to remove a machine from the cluster and reset it 2025-07-25 19:32:02 +10:00
Pasha Sviderski 2714587ec5 feat: add RemoveMachine API endpoint to remove a machine from the cluster 2025-07-25 19:30:59 +10:00
Pasha Sviderski 4166474ee8 fix: race on cluster init by waiting for corrosion service to become ready with schema applied 2025-07-25 16:28:32 +10:00
Pasha Sviderski 3faaac4da7 chore: run machine tests in parallel 2025-07-25 13:56:35 +10:00
Pasha Sviderski 1b6130447f chore: use a new uncancelled context for stopping corrosion service 2025-07-25 13:03:20 +10:00
Pasha Sviderski 614212a24c chore: clean up custom iptables chains on machine reset 2025-07-25 12:37:54 +10:00
Pasha Sviderski 3ab708a437 fix: graceful machine shutdown when not initialised 2025-07-25 10:50:23 +10:00
Pasha Sviderski 7a6c5bf6d7 feat: machine Reset endpoint with asynchronous resource and data cleanup 2025-07-24 19:50:25 +10:00
Pasha Sviderski 2acfafe218 chore: regenerate proto 2025-07-24 19:34:05 +10:00
Pasha Sviderski b24b55c4c7 chore: update api.MachineClient interface 2025-07-24 14:25:09 +10:00
Pasha Sviderski 9a88e914f4 chore: simplify cluster controller initialisation, prepare for reset 2025-07-24 14:23:33 +10:00
Pasha Sviderski f613d3ce6d chore: reformat code 2025-07-22 18:53:05 +10:00
Evgenii OrlovandGitHub 2c3bea64e5 feat: impl of functionality for renaming and updating machines (#91) 2025-07-22 18:45:19 +10:00
da3634b690 E2E tests use repeated strings for test data where constants add no value (#97)
---------

Co-authored-by: Pasha Sviderski <me@psviderski.name>
Co-authored-by: Anton Ovchinnikov <anton@tonyo.info>
2025-07-21 19:30:02 +02:00
Pasha Sviderski 6fb07db4b2 feat: follow-up compose 'ports' support: use ingress mode by default (closes #81) 2025-07-21 16:56:52 +10:00
Evgenii OrlovandGitHub fea7edcbc5 feat: add support for standard compose ports directive (#95) 2025-07-21 15:24:37 +10:00
Anton OvchinnikovandGitHub a54555cd13 feat(deploy): Recreate container when pull_policy set to Always (#100) 2025-07-21 14:47:38 +10:00
Pasha Sviderski 87c7889c2e docs: format debian installation section 2025-07-18 21:59:54 +10:00
Anton Ovchinnikov 05d0078451 chore: Enable more linters 2025-07-14 00:32:26 +02:00
Anton Ovchinnikov 053d73048c feat: Basic linting config 2025-07-13 23:50:27 +02:00
Matías García IsaíaandGitHub 4ec3e97a54 docs: add documentation regarding --no-dns flag (#93) 2025-07-11 11:59:25 +10:00
Evgenii OrlovandGitHub 10bbe9fbc5 fix: prevent race condition with Docker network creation (#89) 2025-07-10 17:51:08 +10:00
Evgenii OrlovandGitHub 31cd4c77e9 feat: support x-machines placement constraints in compose files (#90) 2025-07-10 16:46:29 +10:00
Pasha Sviderski 492a0af2b2 fix: migration script for uncloud Docker network to be compatible with Docker 28.2.0+ 2025-07-09 14:04:21 +10:00
Pasha Sviderski ff7bb25f2c fix: custom version handling in install.sh script 2025-07-09 13:22:21 +10:00
Pasha Sviderski a73cbfd691 chore: hide incomplete machine rm command from help output 2025-07-09 11:28:14 +10:00
Pasha Sviderski 55773e92ce fix: allow direct routing from WireGuard mesh to containers for Docker 28.2.0+ 2025-07-08 21:20:49 +10:00
243 changed files with 29407 additions and 20615 deletions
+1
View File
@@ -0,0 +1 @@
../AI.md
+3 -1
View File
@@ -37,7 +37,9 @@ jobs:
(echo "go.mod or go.sum has changed. Please run 'go mod tidy' and commit the changes." && exit 1) (echo "go.mod or go.sum has changed. Please run 'go mod tidy' and commit the changes." && exit 1)
- name: Run tests - name: Run tests
run: make test run: |
make ucind-image
make test
timeout-minutes: 10 timeout-minutes: 10
check-protobuf: check-protobuf:
+39
View File
@@ -0,0 +1,39 @@
name: Lint
on:
push:
branches:
- "main"
- "test/**"
- "release/**"
pull_request:
branches:
- main
paths:
- "**.go"
- "go.*"
permissions:
contents: read
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Set up Go
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with:
go-version: "1.23.2"
- name: golangci-lint
uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0
with:
version: v2.2.2
- name: Format code
run: |
make format
git diff --exit-code ||
(echo "Code is not formatted. Please run 'make format' and commit the changes." && exit 1)
timeout-minutes: 10
+42
View File
@@ -0,0 +1,42 @@
version: "2"
run:
concurrency: 4
tests: true
timeout: 5m
linters:
default: none
enable:
- bodyclose
# - dogsled
- dupl
# - errcheck
- gochecknoinits
- goconst
# - gocritic
# - gocyclo
# - godot
# - gosec
# - govet
- ineffassign
- misspell
- nakedret
# - prealloc
# - revive
# - staticcheck
- unconvert
# - unparam
# - unused
- whitespace
exclusions:
rules:
- path: ^test/e2e
linters:
- goconst # constants here add no value, so we skip goconst only for test/e2e.
formatters:
enable:
- gofumpt
- goimports
+16
View File
@@ -1,3 +1,11 @@
[tools."aqua:vektra/mockery"]
version = "3.5.3"
backend = "aqua:vektra/mockery"
[tools."aqua:vektra/mockery".checksums]
"mockery_3.5.3_Darwin_arm64.tar.gz" = "sha256:a3a94b14c7414e148f2252199ffc4a0108d311358f3d336cbe05bb73cb203704"
"mockery_3.5.3_Linux_x86_64.tar.gz" = "sha256:ebce416b0175338525246c376885a1579ca6cd4d4015140ba0c70e6b5339a39c"
[tools.go] [tools.go]
version = "1.23.10" version = "1.23.10"
backend = "core:go" backend = "core:go"
@@ -6,6 +14,14 @@ backend = "core:go"
"go1.23.10.darwin-arm64.tar.gz" = "sha256:25c64bfa8a8fd8e7f62fb54afa4354af8409a4bb2358c2699a1003b733e6fce5" "go1.23.10.darwin-arm64.tar.gz" = "sha256:25c64bfa8a8fd8e7f62fb54afa4354af8409a4bb2358c2699a1003b733e6fce5"
"go1.23.10.linux-amd64.tar.gz" = "sha256:535f9f81802499f2a7dbfa70abb8fda3793725fcc29460f719815f6e10b5fd60" "go1.23.10.linux-amd64.tar.gz" = "sha256:535f9f81802499f2a7dbfa70abb8fda3793725fcc29460f719815f6e10b5fd60"
[tools.golangci-lint]
version = "2.2.2"
backend = "aqua:golangci/golangci-lint"
[tools.golangci-lint.checksums]
"golangci-lint-2.2.2-darwin-arm64.tar.gz" = "sha256:d84d94d042c0d495fd1746f3d18948a75de163b17a14e8de3ef840928dd2df74"
"golangci-lint-2.2.2-linux-amd64.tar.gz" = "sha256:c27fbde948a87d326feacd21df2f61a9c54dbd2e3bfa185c0a1cd6917a6f964f"
[tools.protoc] [tools.protoc]
version = "27.3" version = "27.3"
backend = "aqua:protocolbuffers/protobuf/protoc" backend = "aqua:protocolbuffers/protobuf/protoc"
+2
View File
@@ -2,7 +2,9 @@
experimental = true experimental = true
[tools] [tools]
"aqua:vektra/mockery" = "3.5.3"
go = "1.23" go = "1.23"
golangci-lint = "2.2.2"
protoc = "27.3" protoc = "27.3"
protoc-gen-go = "1.34.2" protoc-gen-go = "1.34.2"
protoc-gen-go-grpc = "1.5.1" protoc-gen-go-grpc = "1.5.1"
+4
View File
@@ -0,0 +1,4 @@
packages:
github.com/psviderski/uncloud/internal/machine/caddyconfig:
interfaces:
CaddyfileValidator:
+244
View File
@@ -0,0 +1,244 @@
# AI.md - Uncloud Project Guide
This document provides comprehensive information about the Uncloud project for AI assistants to understand the codebase, architecture, and development practices.
## Project Overview
**Uncloud** is a lightweight clustering and container orchestration tool that enables deployment and management of web applications across cloud VMs and bare metal servers. It creates a secure WireGuard mesh network between Docker hosts and provides automatic service discovery, load balancing, HTTPS ingress, and simple CLI commands for application management.
### Key Characteristics
- **Language**: Go
- **Architecture**: Decentralized, no control plane
- **Target**: Self-hosted infrastructure without Kubernetes complexity
- **License**: View LICENSE file for details
- **Status**: Active development, not yet ready for production
## Core Features
### 🏗️ Infrastructure
- **Multi-machine deployment**: Combine cloud VMs, dedicated servers, and bare metal
- **Zero-config networking**: Automatic WireGuard mesh with NAT traversal
- **Decentralized design**: No central control plane, all machines are equal
- **Service discovery**: Built-in DNS server resolves service names to container IPs
### 🚀 Application Management
- **Docker Compose compatibility**: Uses familiar Docker Compose format
- **Zero-downtime deployments**: Rolling updates without service interruption
- **Automatic HTTPS**: Caddy reverse proxy with Let's Encrypt integration
- **Managed DNS**: Free `*.cluster.uncloud.run` subdomains via Uncloud DNS service
- **Cross-machine scaling**: Run containers across multiple machines
### 🔧 Developer Experience
- **Docker-like CLI**: Familiar commands (`uc` binary)
- **Imperative operations**: Direct commands vs. declarative state reconciliation
- **Remote management**: Control entire infrastructure via SSH to any machine
- **Minimal overhead**: ~150MB RAM footprint per machine
## Architecture
### Core Components
1. **CLI (`uc`)** - Main user interface for cluster management
2. **Daemon (`uncloudd`)** - Machine daemon running on each node
3. **Corrosion** - Distributed SQLite database for cluster state (Fly.io project)
4. **Caddy** - Reverse proxy for HTTPS termination and routing
5. **WireGuard** - Secure mesh networking between machines
### Network Architecture
- Each machine gets unique subnet (e.g., `10.210.0.0/24`, `10.210.1.0/24`)
- Containers get cluster-unique IPs for direct communication
- Automatic peer discovery and key management
- NAT traversal for machines behind firewalls
### State Management
- **CRDT-based distributed storage** using Corrosion
- **Eventually consistent** state across all machines
- **Gossip protocol** (Serf) for state propagation
- **No quorum requirements** - partial network splits remain functional
## Project Structure
### Key Directories
- **`cmd/`**: Contains main applications
- `uncloud/`: CLI tool with subcommands for machine, service, volume management
- `uncloudd/`: Daemon that runs on each machine
- `ucind/`: Development cluster management for testing
- **`internal/`**: Internal implementation packages
- `cli/`: Command-line interface logic
- `machine/`: Machine lifecycle and state management
- `daemon/`: Daemon implementation and gRPC services
- `dns/`: Internal DNS server for service discovery
- **`pkg/`**: Public API packages for external use
- `api/`: Core API types and definitions
- `client/`: Client libraries for interacting with Uncloud
- **`experiment/`**: Experimental features and prototypes
- **`scripts/`**: Installation and utility scripts
- **`test/`**: Test suites and test infrastructure
- **`website/`**: Documentation website (Docusaurus)
- **`misc/`**: Design documents and guides
## Key Technologies
### Core Dependencies
```go
// Networking and orchestration
github.com/docker/docker // Docker API client
github.com/docker/compose/v2 // Docker Compose integration
golang.zx2c4.com/wireguard // WireGuard implementation
github.com/hashicorp/serf // Gossip protocol
// State management
github.com/ipfs/go-ds-crdt // CRDT distributed storage
github.com/dgraph-io/badger/v3 // Embedded database
// Web proxy
github.com/caddyserver/caddy/v2 // HTTP server and reverse proxy
// CLI and UX
github.com/spf13/cobra // CLI framework
github.com/charmbracelet/huh // Interactive forms
// gRPC and networking
google.golang.org/grpc // gRPC framework
github.com/siderolabs/grpc-proxy // gRPC proxy for forwarding
```
## Development Workflow
### Build and Development
```bash
# Build binaries
go build -o uncloud ./cmd/uncloud
go build -o uncloudd ./cmd/uncloudd
```
### Key Make Targets
- `proto`: Generate protobuf code
- `ucind-cluster`: Create development cluster
- `update-dev`: Deploy to development machines
- `demo-reset`: Reset demo environment
- `fmt`: Format code
- `test`: Run all tests
- `lint`: Lint the code using golangci-lint
- `lint-and-fix`: Lint the code and fix issues whenever possible
## CLI Commands Structure
The `uc` CLI provides these main command groups:
### Machine Management
```bash
uc machine init <user@host> # Initialize new cluster
uc machine add <user@host> # Add machine to cluster
uc machine ls # List machines
uc machine rm <name> # Remove machine
```
### Service Management
```bash
uc run <image> # Run container from image
uc deploy # Deploy from compose.yaml
uc scale <service> <count> # Scale service replicas
uc ls # List services
uc rm <service> # Remove service
```
### Context and Connectivity
```bash
uc context ls # List available contexts
uc context use <name> # Switch context
```
### Global Flags
- `--connect`: Connect to remote machine directly, without a config file
- `--uncloud-config`: Override config file path
## Development Guidelines
### Code Organization
- **Package naming**: Use clear, descriptive names
- **Error handling**: Wrap errors with context using `fmt.Errorf`
- **Logging**: Use structured logging with levels
- **gRPC**: Services defined in `internal/machine/api/pb/`
### Testing
- Unit tests alongside source files (`*_test.go`)
- Integration tests in `test/e2e/`
- Test fixtures in `test/fixtures/`
### Dependencies
- Prefer standard library when possible
- Pin versions in `go.mod`
- Document rationale for external dependencies
### Configuration
- Support environment variables for key settings
- Validate configuration early
- Provide sensible defaults
## Troubleshooting and Debugging
### Common Issues
- **Networking**: Check WireGuard status, iptables rules
- **DNS**: Verify service discovery resolution
- **Containers**: Use standard Docker debugging tools
- **State sync**: Check Corrosion logs for replication issues
### Debugging Tools
- Standard Linux networking tools (`ping`, `traceroute`, `wireshark`)
- Docker commands (`docker ps`, `docker logs`)
- SSH access to machines for direct inspection
- gRPC debugging tools
### Logs and Monitoring
- Systemd services (getting logs via `journalctl -u SERVICE_NAME`)
- `uncloud` -- Uncloud daemon
- `uncloud-corrosion` -- Corrosion process
- Machine daemon logs
- Container logs via Docker
## File Patterns and Conventions
### Important Files to Understand
- `cmd/uncloud/main.go`: CLI entry point and command structure
- `internal/cli/cli.go`: CLI implementation and configuration
- `internal/machine/machine.go`: Core machine management
- `pkg/api/`: Public API definitions
- `misc/design.md`: Architecture and design philosophy
- `README.md`: User-facing documentation
### Configuration Files
- `go.mod/go.sum`: Go dependency management
- `Makefile`: Build and development tasks
- `Dockerfile`: Container build instructions forUncloud-in-Docker (used for testing)
This document should help AI assistants understand the project structure, make informed suggestions, and contribute effectively to the Uncloud codebase.
Symlink
+1
View File
@@ -0,0 +1 @@
./AI.md
+31 -5
View File
@@ -1,6 +1,5 @@
CORROSION_IMAGE ?= ghcr.io/psviderski/corrosion:latest CORROSION_IMAGE ?= ghcr.io/psviderski/corrosion:latest
UCIND_IMAGE ?= ghcr.io/psviderski/ucind:latest UCIND_IMAGE ?= ghcr.io/psviderski/ucind:latest
DOCS_IMAGE ?= ghcr.io/psviderski/uncloud-docs:latest
update-dev: update-dev:
GOOS=linux GOARCH=amd64 go build -o uncloudd-linux-amd64 ./cmd/uncloudd && \ GOOS=linux GOARCH=amd64 go build -o uncloudd-linux-amd64 ./cmd/uncloudd && \
@@ -46,6 +45,11 @@ proto:
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative \ protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative \
--proto_path=. --proto_path=internal/machine/api/vendor internal/machine/api/pb/*.proto --proto_path=. --proto_path=internal/machine/api/vendor internal/machine/api/pb/*.proto
.PHONY: proto-mise
proto-mise:
mise exec -- protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative \
--proto_path=. --proto_path=internal/machine/api/vendor internal/machine/api/pb/*.proto
.PHONY: corrosion-image .PHONY: corrosion-image
corrosion-image: corrosion-image:
docker build -t "$(CORROSION_IMAGE)" --target corrosion . docker build -t "$(CORROSION_IMAGE)" --target corrosion .
@@ -62,14 +66,22 @@ ucind-image:
ucind-multiarch-image-push: ucind-multiarch-image-push:
docker buildx build --push --platform linux/amd64,linux/arm64 -t "$(UCIND_IMAGE)" --target ucind . docker buildx build --push --platform linux/amd64,linux/arm64 -t "$(UCIND_IMAGE)" --target ucind .
.PHONY: mocks
mocks:
@mockery
.PHONY: test .PHONY: test
test: test:
ifeq ($(TEST_NAME),) ifeq ($(TEST_NAME),)
go test -count=1 -v ./... go test -count=1 -v ./...
else else
go test -count=1 -v -run ^$(TEST_NAME)$$ ./... go test -race -count=1 -v -run ^$(TEST_NAME)$$ ./...
endif endif
.PHONY: test-e2e
test-e2e:
go test -race -count=1 -v ./test/e2e
.PHONY: test-clean .PHONY: test-clean
test-clean: test-clean:
@CONTAINERS=$$(docker ps --filter "name=ucind-test" -q); \ @CONTAINERS=$$(docker ps --filter "name=ucind-test" -q); \
@@ -89,6 +101,20 @@ test-clean:
vet: vet:
go vet ./... go vet ./...
.PHONY: docs-image-push .PHONY: format fmt
docs-image: format fmt:
docker buildx build --push --platform linux/amd64,linux/arm64 -t "$(DOCS_IMAGE)" ./docs GOOS=linux golangci-lint fmt
LINT_TARGETS := lint lint-and-fix
.PHONY: $(LINT_TARGETS) _lint
$(LINT_TARGETS): _lint
lint: ARGS=
lint-and-fix: ARGS=--fix
_lint:
# Explicitly set OS to Linux to not skip *_linux.go files when running on macOS.
# Uncloud daemon won't likely support OS other than Linux anytime soon, so for now we can rely on that.
GOOS=linux golangci-lint run $(ARGS)
.PHONY: cli-docs
cli-docs:
go run ./cmd/uncloud docs
+20 -11
View File
@@ -1,10 +1,10 @@
<div align="center"> <div align="center">
<img src="./website/images/logo.svg" height="100" width="100" alt="Uncloud logo"/> <img src="./website/landing/images/logo.svg" height="100" width="100" alt="Uncloud logo"/>
<h1>Uncloud</h1> <h1>Uncloud</h1>
<p><strong>Docker simplicity. Multi-machine power.</strong></p> <p><strong>Docker simplicity. Multi-machine power.</strong></p>
<p> <p>
<a href="https://docs.uncloud.run"><img src="https://img.shields.io/badge/Docs-blue.svg?style=for-the-badge&logo=gitbook&logoColor=white" alt="Documentation"></a> <a href="https://uncloud.run/docs"><img src="https://img.shields.io/badge/Docs-blue.svg?style=for-the-badge&logo=gitbook&logoColor=white" alt="Documentation"></a>
<a href="https://discord.gg/eR35KQJhPu"><img src="https://img.shields.io/badge/discord-5865F2.svg?style=for-the-badge&logo=discord&logoColor=white" alt="Join Discord"></a> <a href="https://discord.gg/eR35KQJhPu"><img src="https://img.shields.io/badge/discord-5865F2.svg?style=for-the-badge&logo=discord&logoColor=white" alt="Join Discord"></a>
<a href="https://x.com/psviderski"><img src="https://img.shields.io/badge/follow-black?style=for-the-badge&logo=X&logoColor=while" alt="Follow on X"></a> <a href="https://x.com/psviderski"><img src="https://img.shields.io/badge/follow-black?style=for-the-badge&logo=X&logoColor=while" alt="Follow on X"></a>
<a href="https://github.com/sponsors/psviderski"><img src="https://img.shields.io/badge/Donate-EA4AAA.svg?style=for-the-badge&logo=githubsponsors&logoColor=white" alt="Donate"></a> <a href="https://github.com/sponsors/psviderski"><img src="https://img.shields.io/badge/Donate-EA4AAA.svg?style=for-the-badge&logo=githubsponsors&logoColor=white" alt="Donate"></a>
@@ -52,11 +52,11 @@ complexity of Kubernetes.
## 🎬 Quick demo ## 🎬 Quick demo
The screenshot below demonstrates how I use Uncloud to deploy the [Uncloud Documentation](https://docs.uncloud.run) The screenshot below demonstrates how I use Uncloud to deploy https://uncloud.run website to 2 remote machines from
website to 2 remote machines (why not?) from the [`compose.yaml`](docs/compose.yaml) file on my local machine. the [`compose.yaml`](website/compose.yaml) file on my local machine.
It exposes the container port `8000/tcp` as HTTPS on the domain `docs.uncloud.run`, served by the Caddy reverse proxy on It exposes the container port `8000/tcp` as HTTPS on the domain `uncloud.run`, served by the Caddy reverse proxy on the
the remote machines. All managed by Uncloud. remote machines. All managed by Uncloud.
![Uncloud compose deployment demo](.github/images/compose-deploy.jpg) ![Uncloud compose deployment demo](.github/images/compose-deploy.jpg)
@@ -96,8 +96,8 @@ platform, whether you're running on a $5 VPS, a spare Mac mini, or a rack of bar
# or using curl (macOS/Linux) # or using curl (macOS/Linux)
curl -fsS https://get.uncloud.run/install.sh | sh curl -fsS https://get.uncloud.run/install.sh | sh
``` ```
See [Installation](https://docs.uncloud.run/getting-started/install-cli) for more options. See [Installation](https://uncloud.run/docs/getting-started/install-cli) for more options.
2. Initialise your first machine: 2. Initialise your first machine:
@@ -130,15 +130,15 @@ platform, whether you're running on a $5 VPS, a spare Mac mini, or a rack of bar
uncloud-uninstall uncloud-uninstall
``` ```
View the [Documentation](https://docs.uncloud.run) for more information. View the [Documentation](https://uncloud.run/docs) for more information.
## ⚙️ How it works ## ⚙️ How it works
Check out the [design document](docs/design.md) to understand Uncloud's design philosophy and goals. Check out the [design document](misc/design.md) to understand Uncloud's design philosophy and goals.
Here is a diagram of an Uncloud multi-provider cluster of 3 machines: Here is a diagram of an Uncloud multi-provider cluster of 3 machines:
![Diagram: multi-provider cluster of 3 machines](website/images/diagram.webp) ![Diagram: multi-provider cluster of 3 machines](website/landing/images/diagram.webp)
<details> <details>
<summary>Peek under the hood to see what happens when you run certain commands.</summary> <summary>Peek under the hood to see what happens when you run certain commands.</summary>
@@ -323,6 +323,15 @@ SQLite database used to share Uncloud's cluster state.
features, and be the first to know when it's ready for production use. features, and be the first to know when it's ready for production use.
* Watch this repository for releases. * Watch this repository for releases.
## 💖 Sponsors
These companies and projects are helping Uncloud with their generous sponsorship and/or services:
<!-- Sentry -->
<a href="https://sentry.io/welcome/">
<img height="100" alt="Sentry" src="https://github.com/user-attachments/assets/6c1439c0-d20d-40dc-a669-c9aa94651dfa" />
</a>
## ❤️ Contributors ## ❤️ Contributors
Thank you [@cedws](https://github.com/cedws) for being the first contributor to Uncloud! 🎉 Thank you [@cedws](https://github.com/cedws) for being the first contributor to Uncloud! 🎉
+2 -1
View File
@@ -2,8 +2,9 @@ package cluster
import ( import (
"fmt" "fmt"
"github.com/spf13/cobra"
"github.com/psviderski/uncloud/internal/ucind" "github.com/psviderski/uncloud/internal/ucind"
"github.com/spf13/cobra"
) )
func NewCreateCommand() *cobra.Command { func NewCreateCommand() *cobra.Command {
+2 -1
View File
@@ -2,8 +2,9 @@ package cluster
import ( import (
"fmt" "fmt"
"github.com/spf13/cobra"
"github.com/psviderski/uncloud/internal/ucind" "github.com/psviderski/uncloud/internal/ucind"
"github.com/spf13/cobra"
) )
func NewRemoveCommand() *cobra.Command { func NewRemoveCommand() *cobra.Command {
+1 -1
View File
@@ -13,7 +13,7 @@ func NewRootCommand() *cobra.Command {
} }
cmd.AddCommand( cmd.AddCommand(
NewCreateCommand(), NewCreateCommand(),
//NewListCommand(), // NewListCommand(),
NewRemoveCommand(), NewRemoveCommand(),
) )
return cmd return cmd
+76
View File
@@ -0,0 +1,76 @@
package caddy
import (
"context"
"fmt"
"os"
"github.com/alecthomas/chroma/v2/quick"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type configOptions struct {
machine string
noColor bool
context string
}
func NewConfigCommand() *cobra.Command {
opts := configOptions{}
cmd := &cobra.Command{
Use: "config",
Short: "Show the current Caddy configuration (Caddyfile).",
Long: "Display the current Caddy configuration (Caddyfile) from the connected machine or a specified one.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return runConfig(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVarP(&opts.machine, "machine", "m", "",
"Name or ID of the machine to get the configuration from. (default is connected machine)")
cmd.Flags().BoolVar(&opts.noColor, "no-color", false,
"Disable syntax highlighting for the output.")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func runConfig(ctx context.Context, uncli *cli.CLI, opts configOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
if opts.machine != "" {
// If a specific machine is requested, use it to get the Caddy configuration.
ctx, _, err = api.ProxyMachinesContext(ctx, clusterClient, []string{opts.machine})
if err != nil {
return err
}
}
config, err := clusterClient.Caddy.GetConfig(ctx, nil)
if err != nil {
return fmt.Errorf("get Caddy config: %w", err)
}
// Print the Caddyfile with syntax highlighting.
if opts.noColor {
fmt.Print(config.Caddyfile)
} else {
if err = quick.Highlight(os.Stdout, config.Caddyfile, "caddy", "terminal256", "monokai"); err != nil {
// If highlighting fails, fall back to plain output.
fmt.Print(config.Caddyfile)
}
}
return nil
}
+17 -4
View File
@@ -5,6 +5,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"maps" "maps"
"os"
"slices" "slices"
"strings" "strings"
@@ -18,9 +19,10 @@ import (
) )
type deployOptions struct { type deployOptions struct {
image string caddyfile string
machines []string image string
context string machines []string
context string
} }
func NewDeployCommand() *cobra.Command { func NewDeployCommand() *cobra.Command {
@@ -37,6 +39,8 @@ func NewDeployCommand() *cobra.Command {
}, },
} }
cmd.Flags().StringVar(&opts.caddyfile, "caddyfile", "",
"Path to a custom global Caddy config (Caddyfile) that will be prepended to the auto-generated Caddy config.")
cmd.Flags().StringVar(&opts.image, "image", "", cmd.Flags().StringVar(&opts.image, "image", "",
"Caddy Docker image to deploy. (default caddy:LATEST_VERSION)") "Caddy Docker image to deploy. (default caddy:LATEST_VERSION)")
cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil, cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil,
@@ -51,6 +55,15 @@ func NewDeployCommand() *cobra.Command {
} }
func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error { func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
caddyfile := ""
if opts.caddyfile != "" {
data, err := os.ReadFile(opts.caddyfile)
if err != nil {
return fmt.Errorf("read Caddyfile: %w", err)
}
caddyfile = strings.TrimSpace(string(data))
}
clusterClient, err := uncli.ConnectCluster(ctx, opts.context) clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
@@ -91,7 +104,7 @@ func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
placement := api.Placement{ placement := api.Placement{
Machines: cli.ExpandCommaSeparatedValues(opts.machines), Machines: cli.ExpandCommaSeparatedValues(opts.machines),
} }
d, err := clusterClient.NewCaddyDeployment(opts.image, placement) d, err := clusterClient.NewCaddyDeployment(opts.image, caddyfile, placement)
if err != nil { if err != nil {
return fmt.Errorf("create caddy deployment: %w", err) return fmt.Errorf("create caddy deployment: %w", err)
} }
+1
View File
@@ -10,6 +10,7 @@ func NewRootCommand() *cobra.Command {
Short: "Manage Caddy reverse proxy service.", Short: "Manage Caddy reverse proxy service.",
} }
cmd.AddCommand( cmd.AddCommand(
NewConfigCommand(),
NewDeployCommand(), NewDeployCommand(),
) )
return cmd return cmd
+8 -1
View File
@@ -21,6 +21,7 @@ type deployOptions struct {
profiles []string profiles []string
services []string services []string
noBuild bool noBuild bool
recreate bool
context string context string
} }
@@ -50,6 +51,8 @@ func NewDeployCommand() *cobra.Command {
"Name of the cluster context to deploy to (default is the current context)") "Name of the cluster context to deploy to (default is the current context)")
cmd.Flags().BoolVarP(&opts.noBuild, "no-build", "n", false, cmd.Flags().BoolVarP(&opts.noBuild, "no-build", "n", false,
"Do not build images before deploying services. (default false)") "Do not build images before deploying services. (default false)")
cmd.Flags().BoolVar(&opts.recreate, "recreate", false,
"Recreate containers even if their configuration and image haven't changed.")
// TODO: Consider adding a filter flag to specify which machines to deploy to but keep the rest running. // TODO: Consider adding a filter flag to specify which machines to deploy to but keep the rest running.
// Could be useful to test a new version on a subset of machines before rolling out to all. // Could be useful to test a new version on a subset of machines before rolling out to all.
@@ -108,7 +111,11 @@ func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
} }
defer clusterClient.Close() defer clusterClient.Close()
composeDeploy, err := compose.NewDeployment(ctx, clusterClient, project) var strategy deploy.Strategy
if opts.recreate {
strategy = &deploy.RollingStrategy{ForceRecreate: true}
}
composeDeploy, err := compose.NewDeploymentWithStrategy(ctx, clusterClient, project, strategy)
if err != nil { if err != nil {
return fmt.Errorf("create compose deployment: %w", err) return fmt.Errorf("create compose deployment: %w", err)
} }
+122
View File
@@ -0,0 +1,122 @@
package main
import (
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/spf13/cobra"
"github.com/spf13/cobra/doc"
)
const docsDir = "website/docs/9-cli-reference"
type cmdWrapper struct {
cmd *cobra.Command
}
// NewDocsCommand creates a new hidden command to generate CLI reference docs.
func NewDocsCommand() *cobra.Command {
wrapper := &cmdWrapper{}
cmd := &cobra.Command{
Use: "docs",
Short: "Generate Uncloud CLI reference docs",
SilenceUsage: true,
DisableFlagsInUseLine: true,
Hidden: true,
Args: cobra.NoArgs,
ValidArgsFunction: cobra.NoFileCompletions,
RunE: func(cmd *cobra.Command, _ []string) error {
// Remove existing markdown files.
mdFiles, err := filepath.Glob(filepath.Join(docsDir, "*.md"))
if err != nil {
return fmt.Errorf("list existing CLI docs: %w", err)
}
for _, f := range mdFiles {
if err = os.Remove(f); err != nil {
return fmt.Errorf("remove '%s': %w", f, err)
}
}
// Generate new CLI reference docs.
wrapper.cmd.Root().DisableAutoGenTag = true
if err := doc.GenMarkdownTree(cmd.Root(), docsDir); err != nil {
return fmt.Errorf("generate CLI docs: %w", err)
}
// Remove *completion*.md files that contain malformatted code blocks that break Docusaurus.
mdFiles, err = filepath.Glob(filepath.Join(docsDir, "*completion*.md"))
if err != nil {
return fmt.Errorf("list generated CLI docs: %w", err)
}
for _, f := range mdFiles {
if err = os.Remove(f); err != nil {
return fmt.Errorf("remove '%s': %w", f, err)
}
}
// Post-process generated markdown files.
mdFiles, err = filepath.Glob(filepath.Join(docsDir, "*.md"))
if err != nil {
return fmt.Errorf("list generated CLI docs: %w", err)
}
for _, f := range mdFiles {
if err = postProcessMarkdown(f); err != nil {
return fmt.Errorf("post-process '%s': %w", f, err)
}
}
return nil
},
}
wrapper.cmd = cmd
return cmd
}
// postProcessMarkdown applies transformations to generated markdown files.
func postProcessMarkdown(filename string) error {
data, err := os.ReadFile(filename)
if err != nil {
return err
}
content := string(data)
// Replace "SEE ALSO" with "See also".
content = strings.ReplaceAll(content, "SEE ALSO", "See also")
// Escape <id> to avoid Docusaurus treating it as an HTML tag.
content = strings.ReplaceAll(content, "<id>", "\\<id>")
// Remove broken links to completion docs.
if strings.Contains(content, "[uc completion") {
lines := strings.Split(content, "\n")
var filteredLines []string
for _, line := range lines {
if !strings.Contains(line, "[uc completion") {
filteredLines = append(filteredLines, line)
}
}
content = strings.Join(filteredLines, "\n")
}
// Adjust heading levels. Process from shortest to longest to avoid double replacements.
replacements := []struct {
old, new string
}{
{`(?m)^## `, `# `},
{`(?m)^### `, `## `},
{`(?m)^#### `, `### `},
{`(?m)^##### `, `#### `},
}
for _, r := range replacements {
re := regexp.MustCompile(r.old)
content = re.ReplaceAllString(content, r.new)
}
return os.WriteFile(filename, []byte(content), 0o644)
}
+8 -7
View File
@@ -41,7 +41,7 @@ func NewAddCommand() *cobra.Command {
if err != nil { if err != nil {
return fmt.Errorf("parse remote machine: %w", err) return fmt.Errorf("parse remote machine: %w", err)
} }
remoteMachine := cli.RemoteMachine{ remoteMachine := &cli.RemoteMachine{
User: user, User: user,
Host: host, Host: host,
Port: port, Port: port,
@@ -59,11 +59,12 @@ func NewAddCommand() *cobra.Command {
cmd.Flags().StringVar( cmd.Flags().StringVar(
&opts.publicIP, "public-ip", "auto", &opts.publicIP, "public-ip", "auto",
"Public IP address of the machine for ingress configuration. Use 'auto' for automatic detection, "+ "Public IP address of the machine for ingress configuration. Use 'auto' for automatic detection, "+
"blank '' or 'none' to disable ingress on this machine, or specify an IP address.", fmt.Sprintf("blank '' or '%s' to disable ingress on this machine, or specify an IP address.", PublicIPNone),
) )
cmd.Flags().StringVarP( cmd.Flags().StringVarP(
&opts.sshKey, "ssh-key", "i", "~/.ssh/id_ed25519", &opts.sshKey, "ssh-key", "i", "",
"Path to SSH private key for remote login (if not already added to SSH agent).", fmt.Sprintf("Path to SSH private key for remote login (if not already added to SSH agent). (default %q)",
cli.DefaultSSHKeyPath),
) )
cmd.Flags().StringVar( cmd.Flags().StringVar(
&opts.version, "version", "latest", &opts.version, "version", "latest",
@@ -77,12 +78,12 @@ func NewAddCommand() *cobra.Command {
return cmd return cmd
} }
func add(ctx context.Context, uncli *cli.CLI, remoteMachine cli.RemoteMachine, opts addOptions) error { func add(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine, opts addOptions) error {
var publicIP *netip.Addr var publicIP *netip.Addr
switch opts.publicIP { switch opts.publicIP {
case "auto": case "auto":
publicIP = &netip.Addr{} publicIP = &netip.Addr{}
case "", "none": case "", PublicIPNone:
publicIP = nil publicIP = nil
default: default:
ip, err := netip.ParseAddr(opts.publicIP) ip, err := netip.ParseAddr(opts.publicIP)
@@ -145,7 +146,7 @@ func add(ctx context.Context, uncli *cli.CLI, remoteMachine cli.RemoteMachine, o
// TODO: scale the existing Caddy service to the new machine instead of running a new deployment // TODO: scale the existing Caddy service to the new machine instead of running a new deployment
// that may cause a small downtime. // that may cause a small downtime.
d, err := clusterClient.NewCaddyDeployment(caddyImage, api.Placement{}) d, err := clusterClient.NewCaddyDeployment(caddyImage, "", api.Placement{})
if err != nil { if err != nil {
return fmt.Errorf("create caddy deployment: %w", err) return fmt.Errorf("create caddy deployment: %w", err)
} }
+6
View File
@@ -0,0 +1,6 @@
package machine
const (
// PublicIPNone is the value used to indicate removal of public IP
PublicIPNone = "none"
)
+6 -5
View File
@@ -77,11 +77,12 @@ func NewInitCommand() *cobra.Command {
cmd.Flags().StringVar( cmd.Flags().StringVar(
&opts.publicIP, "public-ip", "auto", &opts.publicIP, "public-ip", "auto",
"Public IP address of the machine for ingress configuration. Use 'auto' for automatic detection, "+ "Public IP address of the machine for ingress configuration. Use 'auto' for automatic detection, "+
"blank '' or 'none' to disable ingress on this machine, or specify an IP address.", fmt.Sprintf("blank '' or '%s' to disable ingress on this machine, or specify an IP address.", PublicIPNone),
) )
cmd.Flags().StringVarP( cmd.Flags().StringVarP(
&opts.sshKey, "ssh-key", "i", "~/.ssh/id_ed25519", &opts.sshKey, "ssh-key", "i", "",
"Path to SSH private key for remote login (if not already added to SSH agent).", fmt.Sprintf("Path to SSH private key for remote login (if not already added to SSH agent). (default %q)",
cli.DefaultSSHKeyPath),
) )
cmd.Flags().StringVar( cmd.Flags().StringVar(
&opts.version, "version", "latest", &opts.version, "version", "latest",
@@ -105,7 +106,7 @@ func initCluster(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteM
switch opts.publicIP { switch opts.publicIP {
case "auto": case "auto":
publicIP = &netip.Addr{} publicIP = &netip.Addr{}
case "", "none": case "", PublicIPNone:
publicIP = nil publicIP = nil
default: default:
ip, err := netip.ParseAddr(opts.publicIP) ip, err := netip.ParseAddr(opts.publicIP)
@@ -146,7 +147,7 @@ func initCluster(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteM
} }
if !opts.noCaddy { if !opts.noCaddy {
d, err := client.NewCaddyDeployment("", api.Placement{}) d, err := client.NewCaddyDeployment("", "", api.Placement{})
if err != nil { if err != nil {
return fmt.Errorf("create caddy deployment: %w", err) return fmt.Errorf("create caddy deployment: %w", err)
} }
+47
View File
@@ -0,0 +1,47 @@
package machine
import (
"context"
"fmt"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
func NewRenameCommand() *cobra.Command {
var contextName string
cmd := &cobra.Command{
Use: "rename OLD_NAME NEW_NAME",
Short: "Rename a machine in the cluster.",
Long: `Rename a machine in the cluster.
This command changes the name of an existing machine while preserving all other
configuration including network settings, public IP, and cluster membership.`,
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return rename(cmd.Context(), uncli, contextName, args[0], args[1])
},
}
cmd.Flags().StringVarP(
&contextName, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func rename(ctx context.Context, uncli *cli.CLI, contextName, oldName, newName string) error {
client, err := uncli.ConnectCluster(ctx, contextName)
if err != nil {
return err
}
defer client.Close()
machine, err := client.RenameMachine(ctx, oldName, newName)
if err != nil {
return fmt.Errorf("rename machine: %w", err)
}
fmt.Printf("Machine %q renamed to %q (ID: %s)\n", oldName, machine.Name, machine.Id)
return nil
}
+76 -27
View File
@@ -14,12 +14,13 @@ import (
"github.com/docker/compose/v2/pkg/progress" "github.com/docker/compose/v2/pkg/progress"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/psviderski/uncloud/internal/cli" "github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
type removeOptions struct { type removeOptions struct {
force bool noReset bool
yes bool yes bool
context string context string
} }
@@ -30,7 +31,7 @@ func NewRmCommand() *cobra.Command {
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "rm MACHINE", Use: "rm MACHINE",
Aliases: []string{"remove", "delete"}, Aliases: []string{"remove", "delete"},
Short: "Remove a machine from a cluster.", Short: "Remove a machine from a cluster and reset it.",
Args: cobra.ExactArgs(1), Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
@@ -42,11 +43,14 @@ func NewRmCommand() *cobra.Command {
"Name of the cluster context. (default is the current context)") "Name of the cluster context. (default is the current context)")
cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false, cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
"Do not prompt for confirmation before removing the machine.") "Do not prompt for confirmation before removing the machine.")
cmd.Flags().BoolVar(&opts.noReset, "no-reset", false,
"Do not reset the machine after removing it from the cluster. This will leave all containers and data intact.")
return cmd return cmd
} }
func remove(ctx context.Context, uncli *cli.CLI, machineName string, opts removeOptions) error { func remove(ctx context.Context, uncli *cli.CLI, nameOrID string, opts removeOptions) error {
// TODO: automatically choose a connection to the machine that is not being removed.
client, err := uncli.ConnectCluster(ctx, opts.context) client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
@@ -54,35 +58,66 @@ func remove(ctx context.Context, uncli *cli.CLI, machineName string, opts remove
defer client.Close() defer client.Close()
// Verify the machine exists and list all service containers on it including stopped ones. // Verify the machine exists and list all service containers on it including stopped ones.
listCtx, machines, err := api.ProxyMachinesContext(ctx, client, []string{machineName}) mctx, machines, err := api.ProxyMachinesContext(ctx, client, []string{nameOrID})
if err != nil { if err != nil {
return err return err
} }
if len(machines) == 0 { if len(machines) == 0 {
return fmt.Errorf("machine '%s' not found in the cluster", machineName) return fmt.Errorf("machine '%s' not found in the cluster", nameOrID)
} }
m := machines[0].Machine m := machines[0].Machine
listOpts := container.ListOptions{All: true} // Verify if the machine being removed is the proxy machine we're connected to.
machineContainers, err := client.Docker.ListServiceContainers(listCtx, "", listOpts) proxyMachine, err := client.MachineClient.Inspect(ctx, nil)
if err != nil { if err != nil {
return fmt.Errorf("list containers: %w", err) return fmt.Errorf("inspect proxy machine: %w", err)
} }
containers := machineContainers[0].Containers if proxyMachine.Id == m.Id {
allMachines, err := client.ListMachines(ctx, nil)
if err != nil {
return fmt.Errorf("list machines: %w", err)
}
if len(allMachines) > 1 {
return errors.New("cannot remove the machine you are currently connected to. " +
"Please connect to another machine in the cluster and try again. " +
"Use --connect flag or update 'connections' for the cluster context in your Uncloud config")
// It's ok to remove the proxy machine if it's the last one in the cluster.
}
}
if len(containers) > 0 { // TODO: mark the machine as being removed and unschedulable when this is possible to prevent new containers
plural := "" // from being scheduled on it while the removal is in progress.
if len(containers) > 1 {
plural = "s" reset := !opts.noReset
var containers []api.ServiceContainer
reachable := false
if reset {
// Check if the machine is up and has service containers.
listOpts := container.ListOptions{All: true}
machineContainers, err := client.Docker.ListServiceContainers(mctx, "", listOpts)
if err == nil {
reachable = true
containers = machineContainers[0].Containers
if len(containers) > 0 {
plural := ""
if len(containers) > 1 {
plural = "s"
}
fmt.Printf("Found %d service container%s on machine '%s':\n", len(containers), plural, m.Name)
fmt.Println(formatContainerTree(containers))
fmt.Println()
fmt.Println("This will remove all service containers from the machine, remove it from the cluster, " +
"and reset it to the uninitialised state.")
} else {
fmt.Printf("No service containers found on machine '%s'.\n", m.Name)
fmt.Println("This will remove the machine from the cluster and reset it to the uninitialised state.")
}
} else {
fmt.Printf("This will remove machine '%s' from the cluster without resetting it as it's unreachable.\n",
m.Name)
} }
fmt.Printf("Found %d service container%s on machine '%s':\n", len(containers), plural, m.Name)
fmt.Println(formatContainerTree(containers))
fmt.Println()
fmt.Println("This will remove all service containers on the machine, reset it to the uninitialised state, " +
"and remove it from the cluster.")
} else { } else {
fmt.Printf("No service containers found on machine '%s'.\n", m.Name) fmt.Printf("This will remove machine '%s' from the cluster without resetting it.\n", m.Name)
fmt.Println("This will reset the machine to the uninitialised state and remove it from the cluster.")
} }
if !opts.yes { if !opts.yes {
@@ -96,23 +131,37 @@ func remove(ctx context.Context, uncli *cli.CLI, machineName string, opts remove
} }
} }
if len(containers) > 0 { if reset && len(containers) > 0 {
err = progress.RunWithTitle(ctx, func(ctx context.Context) error { err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
return removeContainers(ctx, client, containers) return removeContainers(ctx, client, containers)
}, uncli.ProgressOut(), "Removing containers") }, uncli.ProgressOut(), "Removing containers")
if err != nil { if err != nil {
return fmt.Errorf("remove containers: %w", err) return fmt.Errorf("remove containers: %w", err)
} }
fmt.Println() fmt.Println()
} }
// TODO: 4. Implement and call Reset via Machine API to reset the machine state to uninitialised. if _, err = client.RemoveMachine(ctx, &pb.RemoveMachineRequest{Id: m.Id}); err != nil {
// TODO: 5. Remove the machine from the cluster store. return fmt.Errorf("remove machine from cluster: %w", err)
}
fmt.Printf("Machine '%s' removed from the cluster.\n", m.Name)
return fmt.Errorf("resetting machine is not fully implemented yet") if reset && reachable {
//fmt.Printf("Machine '%s' removed from the cluster.\n", m.Name) _, err = client.MachineClient.Reset(mctx, &pb.ResetRequest{})
//return nil if err != nil {
fmt.Printf("WARNING: Failed to reset machine: %v\n", err)
} else {
fmt.Println("Machine reset initiated and will complete in the background.")
}
}
// TODO: remove the connection to the machine from the uncloud config if it exists. We need a way to associate
// the machine with its connection in the config, e.g. by storing the machine name in the connection metadata.
// TODO: If Caddy was running on this machine and a cluster domain is reserved,
// let the user know that the DNS records should be updated.
return nil
} }
// formatContainerTree formats a list of containers grouped by service as a tree structure. // formatContainerTree formats a list of containers grouped by service as a tree structure.
+2
View File
@@ -14,7 +14,9 @@ func NewRootCommand() *cobra.Command {
NewAddCommand(), NewAddCommand(),
NewInitCommand(), NewInitCommand(),
NewListCommand(), NewListCommand(),
NewRenameCommand(),
NewRmCommand(), NewRmCommand(),
NewUpdateCommand(),
NewTokenCommand(), NewTokenCommand(),
) )
return cmd return cmd
+2 -1
View File
@@ -2,9 +2,10 @@ package machine
import ( import (
"fmt" "fmt"
"github.com/spf13/cobra"
"github.com/psviderski/uncloud/internal/daemon" "github.com/psviderski/uncloud/internal/daemon"
"github.com/psviderski/uncloud/internal/machine" "github.com/psviderski/uncloud/internal/machine"
"github.com/spf13/cobra"
) )
type tokenOptions struct { type tokenOptions struct {
+128
View File
@@ -0,0 +1,128 @@
package machine
import (
"context"
"fmt"
"net/netip"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/spf13/cobra"
)
type updateOptions struct {
name string
publicIP string
context string
}
func NewUpdateCommand() *cobra.Command {
opts := updateOptions{}
cmd := &cobra.Command{
Use: "update",
Short: "Update machine configuration in the cluster.",
Long: `Update machine configuration in the cluster.
This command allows setting various machine properties including:
- Machine name (--name)
- Public IP address (--public-ip)
At least one flag must be specified to perform an update operation.`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return update(cmd.Context(), uncli, cmd, opts, args[0])
},
}
cmd.Flags().StringVar(
&opts.name, "name", "",
"New name for the machine",
)
cmd.Flags().StringVar(
&opts.publicIP, "public-ip", "",
fmt.Sprintf("Public IP address of the machine for ingress configuration. Use '%s' or '' to remove the public IP.", PublicIPNone),
)
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func update(ctx context.Context, uncli *cli.CLI, cmd *cobra.Command, opts updateOptions, machineNameOrID string) error {
// Check if at least one flag was explicitly set
if !cmd.Flags().Changed("name") && !cmd.Flags().Changed("public-ip") {
return fmt.Errorf("at least one update flag must be specified (--name, --public-ip)")
}
client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return err
}
defer client.Close()
// First, resolve the machine to get its ID
machine, err := client.InspectMachine(ctx, machineNameOrID)
if err != nil {
return fmt.Errorf("find machine: %w", err)
}
// Build the update request
req := &pb.UpdateMachineRequest{
MachineId: machine.Machine.Id,
}
if opts.name != "" {
req.Name = &opts.name
}
// Check if --public-ip flag was explicitly provided
if cmd.Flags().Changed("public-ip") {
if opts.publicIP == "" || opts.publicIP == PublicIPNone {
req.PublicIp = &pb.IP{} // Empty IP to signal removal
} else {
// Parse and validate the public IP
ip, err := netip.ParseAddr(opts.publicIP)
if err != nil {
return fmt.Errorf("invalid public IP address %q: %w", opts.publicIP, err)
}
req.PublicIp = pb.NewIP(ip)
}
}
// Perform the update operation
updatedMachine, err := client.UpdateMachine(ctx, req)
if err != nil {
return fmt.Errorf("update machine: %w", err)
}
// Report what was changed
changes := make([]string, 0)
if opts.name != "" {
changes = append(changes, fmt.Sprintf("name: %q -> %q", machine.Machine.Name, updatedMachine.Name))
}
if cmd.Flags().Changed("public-ip") {
oldIP := PublicIPNone
if machine.Machine.PublicIp != nil {
if addr, err := machine.Machine.PublicIp.ToAddr(); err == nil {
oldIP = addr.String()
}
}
newIP := PublicIPNone
if updatedMachine.PublicIp != nil {
if addr, err := updatedMachine.PublicIp.ToAddr(); err == nil {
newIP = addr.String()
}
}
changes = append(changes, fmt.Sprintf("public IP: %s -> %s", oldIP, newIP))
}
fmt.Printf("Machine %q (ID: %s) configuration updated:\n", updatedMachine.Name, updatedMachine.Id)
for _, change := range changes {
fmt.Printf(" %s\n", change)
}
return nil
}
+2 -1
View File
@@ -27,7 +27,7 @@ type globalOptions struct {
func main() { func main() {
opts := globalOptions{} opts := globalOptions{}
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "uncloud", Use: "uc",
Short: "A CLI tool for managing Uncloud resources such as clusters, machines, and services.", Short: "A CLI tool for managing Uncloud resources such as clusters, machines, and services.",
Version: version.String(), Version: version.String(),
SilenceUsage: true, SilenceUsage: true,
@@ -75,6 +75,7 @@ func main() {
cmd.AddCommand( cmd.AddCommand(
NewDeployCommand(), NewDeployCommand(),
NewDocsCommand(),
NewBuildCommand(), NewBuildCommand(),
caddy.NewRootCommand(), caddy.NewRootCommand(),
cmdcontext.NewRootCommand(), cmdcontext.NewRootCommand(),
-1
View File
@@ -9,7 +9,6 @@ import (
"github.com/docker/docker/pkg/stringid" "github.com/docker/docker/pkg/stringid"
"github.com/docker/go-units" "github.com/docker/go-units"
"github.com/psviderski/uncloud/internal/cli" "github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
+20 -1
View File
@@ -17,6 +17,7 @@ import (
) )
type runOptions struct { type runOptions struct {
caddyfile string
command []string command []string
cpu dockeropts.NanoCPUs cpu dockeropts.NanoCPUs
entrypoint string entrypoint string
@@ -57,6 +58,9 @@ func NewRunCommand() *cobra.Command {
}, },
} }
cmd.Flags().StringVar(&opts.caddyfile, "caddyfile", "",
"Path to a custom Caddy config (Caddyfile) for the service. "+
"Cannot be used together with non-@host published ports.")
cmd.Flags().VarP(&opts.cpu, "cpu", "", cmd.Flags().VarP(&opts.cpu, "cpu", "",
"Maximum number of CPU cores a service container can use. Fractional values are allowed: "+ "Maximum number of CPU cores a service container can use. Fractional values are allowed: "+
"0.5 for half a core or 2.25 for two and a quarter cores.") "0.5 for half a core or 2.25 for two and a quarter cores.")
@@ -82,7 +86,7 @@ func NewRunCommand() *cobra.Command {
"Give extended privileges to service containers. This is a security risk and should be used with caution.") "Give extended privileges to service containers. This is a security risk and should be used with caution.")
cmd.Flags().StringSliceVarP(&opts.publish, "publish", "p", nil, cmd.Flags().StringSliceVarP(&opts.publish, "publish", "p", nil,
"Publish a service port to make it accessible outside the cluster. Can be specified multiple times.\n"+ "Publish a service port to make it accessible outside the cluster. Can be specified multiple times.\n"+
"Format: [hostname:][load_balancer_port:]container_port[/protocol] or [host_ip:]:host_port:container_port[/protocol]@host\n"+ "Format: [hostname:]container_port[/protocol] or [host_ip:]host_port:container_port[/protocol]@host\n"+
"Supported protocols: tcp, udp, http, https (default is tcp). If a hostname for http(s) port is not specified\n"+ "Supported protocols: tcp, udp, http, https (default is tcp). If a hostname for http(s) port is not specified\n"+
"and a cluster domain is reserved, service-name.cluster-domain will be used as the hostname.\n"+ "and a cluster domain is reserved, service-name.cluster-domain will be used as the hostname.\n"+
"Examples:\n"+ "Examples:\n"+
@@ -161,6 +165,15 @@ func run(ctx context.Context, uncli *cli.CLI, opts runOptions) error {
func prepareServiceSpec(opts runOptions) (api.ServiceSpec, error) { func prepareServiceSpec(opts runOptions) (api.ServiceSpec, error) {
var spec api.ServiceSpec var spec api.ServiceSpec
caddyfile := ""
if opts.caddyfile != "" {
data, err := os.ReadFile(opts.caddyfile)
if err != nil {
return spec, fmt.Errorf("read Caddyfile: %w", err)
}
caddyfile = strings.TrimSpace(string(data))
}
env, err := parseEnv(opts.env) env, err := parseEnv(opts.env)
if err != nil { if err != nil {
return spec, err return spec, err
@@ -218,6 +231,12 @@ func prepareServiceSpec(opts runOptions) (api.ServiceSpec, error) {
Volumes: volumes, Volumes: volumes,
} }
if caddyfile != "" {
spec.Caddy = &api.CaddySpec{
Config: caddyfile,
}
}
// Overwrite the default ENTRYPOINT of the image or reset it if an empty string is passed. // Overwrite the default ENTRYPOINT of the image or reset it if an empty string is passed.
if opts.entrypoint != "" { if opts.entrypoint != "" {
spec.Container.Entrypoint = []string{opts.entrypoint} spec.Container.Entrypoint = []string{opts.entrypoint}
-41
View File
@@ -1,41 +0,0 @@
# Website
This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
### Installation
```
$ yarn
```
### Local Development
```
$ yarn start
```
This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
### Build
```
$ yarn build
```
This command generates static content into the `build` directory and can be served using any static contents hosting service.
### Deployment
Using SSH:
```
$ USE_SSH=true yarn deploy
```
Not using SSH:
```
$ GIT_USER=<Your GitHub username> yarn deploy
```
If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
-8
View File
@@ -1,8 +0,0 @@
services:
uncloud-docs:
image: ghcr.io/psviderski/uncloud-docs:latest
pull_policy: always
user: nobody
x-ports:
- docs.uncloud.run:8000/https
scale: 2
-18434
View File
File diff suppressed because it is too large Load Diff
-45
View File
@@ -1,45 +0,0 @@
{
"name": "docs",
"version": "0.0.0",
"private": true,
"scripts": {
"docusaurus": "docusaurus",
"start": "docusaurus start",
"build": "docusaurus build",
"swizzle": "docusaurus swizzle",
"deploy": "docusaurus deploy",
"clear": "docusaurus clear",
"serve": "docusaurus serve",
"write-translations": "docusaurus write-translations",
"write-heading-ids": "docusaurus write-heading-ids"
},
"dependencies": {
"@docusaurus/core": "3.7.0",
"@docusaurus/preset-classic": "3.7.0",
"@easyops-cn/docusaurus-search-local": "^0.49.2",
"@mdx-js/react": "^3.0.0",
"clsx": "^2.0.0",
"prism-react-renderer": "^2.3.0",
"react": "^19.0.0",
"react-dom": "^19.0.0"
},
"devDependencies": {
"@docusaurus/module-type-aliases": "3.7.0",
"@docusaurus/types": "3.7.0"
},
"browserslist": {
"production": [
">0.5%",
"not dead",
"not op_mini all"
],
"development": [
"last 3 chrome version",
"last 3 firefox version",
"last 5 safari version"
]
},
"engines": {
"node": ">=18.0"
}
}
+3 -2
View File
@@ -3,10 +3,11 @@ package main
import ( import (
"context" "context"
"fmt" "fmt"
"github.com/hashicorp/serf/serf"
crdt "github.com/ipfs/go-ds-crdt"
"log/slog" "log/slog"
"time" "time"
"github.com/hashicorp/serf/serf"
crdt "github.com/ipfs/go-ds-crdt"
) )
// Implements the Broadcaster interface. // Implements the Broadcaster interface.
+2 -1
View File
@@ -2,9 +2,10 @@ package main
import ( import (
"fmt" "fmt"
"github.com/dgraph-io/badger/v3"
"log" "log"
"time" "time"
"github.com/dgraph-io/badger/v3"
) )
func customTimeEncoder(t time.Time) string { func customTimeEncoder(t time.Time) string {
+2 -1
View File
@@ -2,9 +2,10 @@ package main
import ( import (
"fmt" "fmt"
"github.com/ipfs/go-log/v2"
"log/slog" "log/slog"
"os" "os"
"github.com/ipfs/go-log/v2"
) )
// ipfsLogger is an slog logger that implements the IPFS go-log StandardLogger interface. // ipfsLogger is an slog logger that implements the IPFS go-log StandardLogger interface.
+2 -1
View File
@@ -2,12 +2,13 @@ package main
import ( import (
"fmt" "fmt"
"github.com/docker/docker/libnetwork/networkdb"
"log/slog" "log/slog"
"os" "os"
"os/signal" "os/signal"
"syscall" "syscall"
"time" "time"
"github.com/docker/docker/libnetwork/networkdb"
) )
func main() { func main() {
+13 -12
View File
@@ -4,6 +4,13 @@ import (
"context" "context"
"flag" "flag"
"fmt" "fmt"
"log/slog"
"net"
"os"
"os/signal"
"syscall"
"time"
"github.com/hashicorp/memberlist" "github.com/hashicorp/memberlist"
"github.com/hashicorp/serf/cmd/serf/command/agent" "github.com/hashicorp/serf/cmd/serf/command/agent"
"github.com/hashicorp/serf/serf" "github.com/hashicorp/serf/serf"
@@ -11,12 +18,6 @@ import (
badger "github.com/ipfs/go-ds-badger3" badger "github.com/ipfs/go-ds-badger3"
crdt "github.com/ipfs/go-ds-crdt" crdt "github.com/ipfs/go-ds-crdt"
"github.com/lmittmann/tint" "github.com/lmittmann/tint"
"log/slog"
"net"
"os"
"os/signal"
"syscall"
"time"
) )
func createSerfAgentConfig(name, bindAddr, rpcAddr, profile string) *agent.Config { func createSerfAgentConfig(name, bindAddr, rpcAddr, profile string) *agent.Config {
@@ -49,9 +50,9 @@ func createSerfAgent(config *agent.Config) (*agent.Agent, error) {
serfConfig.MemberlistConfig.BindAddr = bindIP serfConfig.MemberlistConfig.BindAddr = bindIP
serfConfig.MemberlistConfig.BindPort = bindPort serfConfig.MemberlistConfig.BindPort = bindPort
//serfConfig.MemberlistConfig.AdvertiseAddr = advertiseIP // serfConfig.MemberlistConfig.AdvertiseAddr = advertiseIP
//serfConfig.MemberlistConfig.AdvertisePort = advertisePort // serfConfig.MemberlistConfig.AdvertisePort = advertisePort
//serfConfig.MemberlistConfig.SecretKey = encryptKey // serfConfig.MemberlistConfig.SecretKey = encryptKey
serfConfig.NodeName = config.NodeName serfConfig.NodeName = config.NodeName
serfConfig.Tags = config.Tags serfConfig.Tags = config.Tags
serfConfig.SnapshotPath = config.SnapshotPath serfConfig.SnapshotPath = config.SnapshotPath
@@ -129,7 +130,7 @@ func main() {
logger := slog.New(tint.NewHandler(os.Stdout, &tint.Options{ logger := slog.New(tint.NewHandler(os.Stdout, &tint.Options{
AddSource: true, AddSource: true,
Level: slog.LevelDebug, Level: slog.LevelDebug,
//Level: slog.LevelInfo, // Level: slog.LevelInfo,
TimeFormat: time.RFC3339Nano, TimeFormat: time.RFC3339Nano,
})) }))
slog.SetDefault(logger) slog.SetDefault(logger)
@@ -164,7 +165,7 @@ func main() {
opts := crdt.DefaultOptions() opts := crdt.DefaultOptions()
opts.Logger = newIPFSLogger(logger) opts.Logger = newIPFSLogger(logger)
//opts.MultiHeadProcessing = true // opts.MultiHeadProcessing = true
// TODO: debug why the heads count may grow on the receiving side if the event backlog is huge and the processing // TODO: debug why the heads count may grow on the receiving side if the event backlog is huge and the processing
// is slow. // is slow.
store, err := crdt.New(localStore, ds.NewKey("/"), syncer, broadcaster, opts) store, err := crdt.New(localStore, ds.NewKey("/"), syncer, broadcaster, opts)
@@ -172,7 +173,7 @@ func main() {
panic(err) panic(err)
} }
//ticker := time.NewTicker(10 * time.Millisecond) // ticker := time.NewTicker(10 * time.Millisecond)
ticker := time.NewTicker(3 * time.Second) ticker := time.NewTicker(3 * time.Second)
go func() { go func() {
for { for {
+4 -5
View File
@@ -4,18 +4,19 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"log/slog"
"github.com/hashicorp/serf/serf" "github.com/hashicorp/serf/serf"
"github.com/ipfs/boxo/datastore/dshelp" "github.com/ipfs/boxo/datastore/dshelp"
dag "github.com/ipfs/boxo/ipld/merkledag" dag "github.com/ipfs/boxo/ipld/merkledag"
"github.com/ipfs/go-cid" "github.com/ipfs/go-cid"
ds "github.com/ipfs/go-datastore" ds "github.com/ipfs/go-datastore"
ipld "github.com/ipfs/go-ipld-format" ipld "github.com/ipfs/go-ipld-format"
"log/slog"
) )
// Implements the DAGService interface. // Implements the DAGService interface.
// TODO: implement SessionDAGService to optimize node fetching. // TODO: implement SessionDAGService to optimize node fetching.
// TOOD: persistentSerfDAG? // TODO: persistentSerfDAG?
type dagSyncer struct { type dagSyncer struct {
// Persistent storage for the nodes. // Persistent storage for the nodes.
store ds.Datastore store ds.Datastore
@@ -50,8 +51,7 @@ func (d *dagSyncer) Get(ctx context.Context, cid cid.Cid) (ipld.Node, error) {
} }
slog.Debug("Queried node from peers", "cid", cid, "deadline", query.Deadline()) slog.Debug("Queried node from peers", "cid", cid, "deadline", query.Deadline())
responded := false for {
for !responded {
select { select {
case resp, ok := <-query.ResponseCh(): case resp, ok := <-query.ResponseCh():
if !ok { if !ok {
@@ -63,7 +63,6 @@ func (d *dagSyncer) Get(ctx context.Context, cid cid.Cid) (ipld.Node, error) {
continue continue
} }
slog.Debug("Received node from peer", "cid", cid, "peer", resp.From) slog.Debug("Received node from peer", "cid", cid, "peer", resp.From)
responded = true
query.Close() query.Close()
node, err = nodeFromBytes(resp.Payload) node, err = nodeFromBytes(resp.Payload)
+4 -3
View File
@@ -6,12 +6,13 @@ import (
"crypto/cipher" "crypto/cipher"
"encoding/hex" "encoding/hex"
"fmt" "fmt"
"net/netip"
"time"
"github.com/psviderski/uncloud/internal/machine/network"
"github.com/siderolabs/discovery-api/api/v1alpha1/client/pb" "github.com/siderolabs/discovery-api/api/v1alpha1/client/pb"
discovery "github.com/siderolabs/discovery-client/pkg/client" discovery "github.com/siderolabs/discovery-client/pkg/client"
"go.uber.org/zap" "go.uber.org/zap"
"net/netip"
"time"
"github.com/psviderski/uncloud/internal/machine/network"
) )
const ( const (
+4 -1
View File
@@ -6,6 +6,7 @@ require (
github.com/BurntSushi/toml v1.4.0 github.com/BurntSushi/toml v1.4.0
github.com/Masterminds/semver v1.5.0 github.com/Masterminds/semver v1.5.0
github.com/Masterminds/squirrel v1.5.4 github.com/Masterminds/squirrel v1.5.4
github.com/alecthomas/chroma/v2 v2.20.0
github.com/caddyserver/caddy/v2 v2.8.4 github.com/caddyserver/caddy/v2 v2.8.4
github.com/cenkalti/backoff/v4 v4.3.0 github.com/cenkalti/backoff/v4 v4.3.0
github.com/charmbracelet/huh v0.6.0 github.com/charmbracelet/huh v0.6.0
@@ -35,6 +36,7 @@ require (
github.com/jmoiron/sqlx v1.4.0 github.com/jmoiron/sqlx v1.4.0
github.com/lmittmann/tint v1.0.5 github.com/lmittmann/tint v1.0.5
github.com/miekg/dns v1.1.65 github.com/miekg/dns v1.1.65
github.com/mitchellh/mapstructure v1.5.0
github.com/moby/term v0.5.0 github.com/moby/term v0.5.0
github.com/opencontainers/go-digest v1.0.0 github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.1.0 github.com/opencontainers/image-spec v1.1.0
@@ -107,6 +109,7 @@ require (
github.com/dgraph-io/badger/v2 v2.2007.4 // indirect github.com/dgraph-io/badger/v2 v2.2007.4 // indirect
github.com/dgraph-io/ristretto v0.1.1 // indirect github.com/dgraph-io/ristretto v0.1.1 // indirect
github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect
github.com/dlclark/regexp2 v1.11.5 // indirect
github.com/docker/buildx v0.18.0 // indirect github.com/docker/buildx v0.18.0 // indirect
github.com/docker/distribution v2.8.3+incompatible // indirect github.com/docker/distribution v2.8.3+incompatible // indirect
github.com/docker/docker-credential-helpers v0.8.2 // indirect github.com/docker/docker-credential-helpers v0.8.2 // indirect
@@ -204,7 +207,6 @@ require (
github.com/mitchellh/go-homedir v1.1.0 // indirect github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-ps v1.0.0 // indirect github.com/mitchellh/go-ps v1.0.0 // indirect
github.com/mitchellh/hashstructure/v2 v2.0.2 // indirect github.com/mitchellh/hashstructure/v2 v2.0.2 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/moby/buildkit v0.17.2 // indirect github.com/moby/buildkit v0.17.2 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect github.com/moby/docker-image-spec v1.3.1 // indirect
@@ -265,6 +267,7 @@ require (
github.com/spf13/cast v1.7.0 // indirect github.com/spf13/cast v1.7.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect github.com/spf13/pflag v1.0.5 // indirect
github.com/stoewer/go-strcase v1.2.0 // indirect github.com/stoewer/go-strcase v1.2.0 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/tailscale/tscert v0.0.0-20240517230440-bbccfbf48933 // indirect github.com/tailscale/tscert v0.0.0-20240517230440-bbccfbf48933 // indirect
github.com/theupdateframework/notary v0.7.0 // indirect github.com/theupdateframework/notary v0.7.0 // indirect
github.com/tonistiigi/dchapes-mode v0.0.0-20241001053921-ca0759fec205 // indirect github.com/tonistiigi/dchapes-mode v0.0.0-20241001053921-ca0759fec205 // indirect
+11
View File
@@ -53,6 +53,12 @@ github.com/OneOfOne/xxhash v1.2.8/go.mod h1:eZbhyaAYD41SGSSsnmcpxVoRiQ/MPUTjUdII
github.com/Shopify/logrus-bugsnag v0.0.0-20170309145241-6dbc35f2c30d/go.mod h1:HI8ITrYtUY+O+ZhtlqUnD8+KwNPOyugEhfP9fdUIaEQ= github.com/Shopify/logrus-bugsnag v0.0.0-20170309145241-6dbc35f2c30d/go.mod h1:HI8ITrYtUY+O+ZhtlqUnD8+KwNPOyugEhfP9fdUIaEQ=
github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d h1:UrqY+r/OJnIp5u0s1SbQ8dVfLCZJsnvazdBP5hS4iRs= github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d h1:UrqY+r/OJnIp5u0s1SbQ8dVfLCZJsnvazdBP5hS4iRs=
github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d/go.mod h1:HI8ITrYtUY+O+ZhtlqUnD8+KwNPOyugEhfP9fdUIaEQ= github.com/Shopify/logrus-bugsnag v0.0.0-20171204204709-577dee27f20d/go.mod h1:HI8ITrYtUY+O+ZhtlqUnD8+KwNPOyugEhfP9fdUIaEQ=
github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/chroma/v2 v2.20.0 h1:sfIHpxPyR07/Oylvmcai3X/exDlE8+FA820NTz+9sGw=
github.com/alecthomas/chroma/v2 v2.20.0/go.mod h1:e7tViK0xh/Nf4BYHl00ycY6rV7b8iXBksI9E359yNmA=
github.com/alecthomas/repr v0.5.1 h1:E3G4t2QbHTSNpPKBgMTln5KLkZHLOcU7r37J4pXBuIg=
github.com/alecthomas/repr v0.5.1/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
@@ -259,6 +265,8 @@ github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 h1:fAjc9m62+UWV/WA
github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw= github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13/go.mod h1:SqUrOPUnsFjfmXRMNPybcSiG0BgUW2AuFH8PAnS2iTw=
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk= github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E= github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/dlclark/regexp2 v1.11.5 h1:Q/sSnsKerHeCkc/jSTNq1oCm7KiVgUMZRDUoRu0JQZQ=
github.com/dlclark/regexp2 v1.11.5/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/docker/buildx v0.18.0 h1:rSauXHeJt90NvtXrLK5J992Eb0UPJZs2vV3u1zTf1nE= github.com/docker/buildx v0.18.0 h1:rSauXHeJt90NvtXrLK5J992Eb0UPJZs2vV3u1zTf1nE=
github.com/docker/buildx v0.18.0/go.mod h1:JGNSshOhHs5FhG3u51jXUf4lLOeD2QBIlJ2vaRB67p4= github.com/docker/buildx v0.18.0/go.mod h1:JGNSshOhHs5FhG3u51jXUf4lLOeD2QBIlJ2vaRB67p4=
github.com/docker/cli v27.5.0+incompatible h1:aMphQkcGtpHixwwhAXJT1rrK/detk2JIvDaFkLctbGM= github.com/docker/cli v27.5.0+incompatible h1:aMphQkcGtpHixwwhAXJT1rrK/detk2JIvDaFkLctbGM=
@@ -487,6 +495,8 @@ github.com/hashicorp/memberlist v0.5.1 h1:mk5dRuzeDNis2bi6LLoQIXfMH7JQvAzt3mQD0v
github.com/hashicorp/memberlist v0.5.1/go.mod h1:zGDXV6AqbDTKTM6yxW0I4+JtFzZAJVoIPvss4hV8F24= github.com/hashicorp/memberlist v0.5.1/go.mod h1:zGDXV6AqbDTKTM6yxW0I4+JtFzZAJVoIPvss4hV8F24=
github.com/hashicorp/serf v0.10.1 h1:Z1H2J60yRKvfDYAOZLd2MU0ND4AH/WDz7xYHDWQsIPY= github.com/hashicorp/serf v0.10.1 h1:Z1H2J60yRKvfDYAOZLd2MU0ND4AH/WDz7xYHDWQsIPY=
github.com/hashicorp/serf v0.10.1/go.mod h1:yL2t6BqATOLGc5HF7qbFkTfXoPIY0WZdWHfEvMqbG+4= github.com/hashicorp/serf v0.10.1/go.mod h1:yL2t6BqATOLGc5HF7qbFkTfXoPIY0WZdWHfEvMqbG+4=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU= github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/huandu/xstrings v1.3.1/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE= github.com/huandu/xstrings v1.3.1/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE=
github.com/huandu/xstrings v1.3.2/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE= github.com/huandu/xstrings v1.3.2/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE=
@@ -1039,6 +1049,7 @@ github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+
github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE= github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA= github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
+35 -35
View File
@@ -6,8 +6,8 @@ import (
"fmt" "fmt"
"net/netip" "net/netip"
"os" "os"
"slices"
"github.com/charmbracelet/huh"
"github.com/docker/cli/cli/streams" "github.com/docker/cli/cli/streams"
"github.com/psviderski/uncloud/internal/cli/config" "github.com/psviderski/uncloud/internal/cli/config"
"github.com/psviderski/uncloud/internal/fs" "github.com/psviderski/uncloud/internal/fs"
@@ -22,7 +22,12 @@ import (
"google.golang.org/protobuf/types/known/emptypb" "google.golang.org/protobuf/types/known/emptypb"
) )
const defaultContextName = "default" const (
// DefaultSSHKeyPath is the fallback location for the SSH private key when provisioning remote machines.
// Used when no key is explicitly provided and SSH agent authentication fails.
DefaultSSHKeyPath = "~/.ssh/id_ed25519"
defaultContextName = "default"
)
type CLI struct { type CLI struct {
Config *config.Config Config *config.Config
@@ -173,7 +178,7 @@ func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions)
return nil, fmt.Errorf("cluster context '%s' already exists", contextName) return nil, fmt.Errorf("cluster context '%s' already exists", contextName)
} }
machineClient, err := cli.provisionRemoteMachine(ctx, *opts.RemoteMachine, opts.Version) machineClient, err := provisionRemoteMachine(ctx, opts.RemoteMachine, opts.Version)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -190,7 +195,7 @@ func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions)
return nil, fmt.Errorf("inspect machine: %w", err) return nil, fmt.Errorf("inspect machine: %w", err)
} }
if minfo.Id != "" { if minfo.Id != "" {
if err = cli.promptResetMachine(); err != nil { if err = promptResetMachine(ctx, machineClient.MachineClient); err != nil {
return nil, err return nil, err
} }
} }
@@ -249,7 +254,7 @@ type AddMachineOptions struct {
Context string Context string
MachineName string MachineName string
PublicIP *netip.Addr PublicIP *netip.Addr
RemoteMachine RemoteMachine RemoteMachine *RemoteMachine
Version string Version string
} }
@@ -272,7 +277,7 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
} }
}() }()
machineClient, err := cli.provisionRemoteMachine(ctx, opts.RemoteMachine, opts.Version) machineClient, err := provisionRemoteMachine(ctx, opts.RemoteMachine, opts.Version)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
@@ -288,7 +293,18 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
return nil, nil, fmt.Errorf("inspect machine: %w", err) return nil, nil, fmt.Errorf("inspect machine: %w", err)
} }
if minfo.Id != "" { if minfo.Id != "" {
if err = cli.promptResetMachine(); err != nil { // Check if the machine is already a member of this cluster.
machines, err := c.ListMachines(ctx, nil)
if err != nil {
return nil, nil, fmt.Errorf("list cluster machines: %w", err)
}
if slices.ContainsFunc(machines, func(m *pb.MachineMember) bool {
return m.Machine.Id == minfo.Id
}) {
return nil, nil, fmt.Errorf("machine is already a member of this cluster (%s)", minfo.Name)
}
if err = promptResetMachine(ctx, machineClient.MachineClient); err != nil {
return nil, nil, err return nil, nil, err
} }
} }
@@ -339,7 +355,7 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
return nil, nil, fmt.Errorf("add machine to cluster (context '%s'): %w", contextName, err) return nil, nil, fmt.Errorf("add machine to cluster (context '%s'): %w", contextName, err)
} }
// List other machines in the cluster to include them in the join request. // Get the most up-to-date list of other machines in the cluster to include them in the join request.
machines, err := c.ListMachines(ctx, nil) machines, err := c.ListMachines(ctx, nil)
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("list cluster machines: %w", err) return nil, nil, fmt.Errorf("list cluster machines: %w", err)
@@ -382,11 +398,20 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
// provisionRemoteMachine installs the Uncloud daemon and dependencies on the remote machine over SSH and returns // provisionRemoteMachine installs the Uncloud daemon and dependencies on the remote machine over SSH and returns
// a machine API client to interact with the machine. The client should be closed after use by the caller. // a machine API client to interact with the machine. The client should be closed after use by the caller.
// The version parameter specifies the version of the Uncloud daemon to install. If empty, the latest version is used. // The version parameter specifies the version of the Uncloud daemon to install. If empty, the latest version is used.
func (cli *CLI) provisionRemoteMachine( // The remoteMachine.SSHKeyPath could be updated to the default SSH key path if it is not set and the SSH agent
ctx context.Context, remoteMachine RemoteMachine, version string, // authentication fails.
func provisionRemoteMachine(
ctx context.Context, remoteMachine *RemoteMachine, version string,
) (*client.Client, error) { ) (*client.Client, error) {
// Provision the remote machine by installing the Uncloud daemon and dependencies over SSH. // Provision the remote machine by installing the Uncloud daemon and dependencies over SSH.
sshClient, err := sshexec.Connect(remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath) sshClient, err := sshexec.Connect(remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath)
// If the SSH connection using SSH agent fails and no key path is provided, try to use the default SSH key.
if err != nil && remoteMachine.KeyPath == "" {
remoteMachine.KeyPath = DefaultSSHKeyPath
sshClient, err = sshexec.Connect(
remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath,
)
}
if err != nil { if err != nil {
return nil, fmt.Errorf( return nil, fmt.Errorf(
"SSH login to remote machine %s: %w", "SSH login to remote machine %s: %w",
@@ -420,31 +445,6 @@ func (cli *CLI) provisionRemoteMachine(
return machineClient, nil return machineClient, nil
} }
func (cli *CLI) promptResetMachine() error {
var confirm bool
form := huh.NewForm(
huh.NewGroup(
huh.NewConfirm().
Title(
"The remote machine is already initialised as a cluster member. Do you want to reset it first?",
).
Affirmative("Yes!").
Negative("No").
Value(&confirm),
),
).WithAccessible(true)
if err := form.Run(); err != nil {
return fmt.Errorf("prompt user to confirm: %w", err)
}
if !confirm {
return fmt.Errorf("remote machine is already initialised as a cluster member")
}
// TODO: implement resetting the remote machine.
return fmt.Errorf("resetting the remote machine is not implemented yet. " +
"Please manually run 'uncloud-uninstall' on the remote machine to fully uninstall Uncloud from it")
}
// ProgressOut returns an output stream for progress writer. // ProgressOut returns an output stream for progress writer.
func (cli *CLI) ProgressOut() *streams.Out { func (cli *CLI) ProgressOut() *streams.Out {
return streams.NewOut(os.Stdout) return streams.NewOut(os.Stdout)
+2 -2
View File
@@ -53,11 +53,11 @@ func (c *Config) Read() error {
func (c *Config) Save() error { func (c *Config) Save() error {
dir, _ := filepath.Split(c.path) dir, _ := filepath.Split(c.path)
if err := os.MkdirAll(dir, 0700); err != nil { if err := os.MkdirAll(dir, 0o700); err != nil {
return fmt.Errorf("create config directory '%s': %w", dir, err) return fmt.Errorf("create config directory '%s': %w", dir, err)
} }
f, err := os.OpenFile(c.path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600) f, err := os.OpenFile(c.path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o600)
if err != nil { if err != nil {
return fmt.Errorf("write config file '%s': %w", c.path, err) return fmt.Errorf("write config file '%s': %w", c.path, err)
} }
+84 -3
View File
@@ -5,12 +5,20 @@ import (
"fmt" "fmt"
"os" "os"
"strings" "strings"
"time"
"github.com/cenkalti/backoff/v4"
"github.com/charmbracelet/huh"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/sshexec" "github.com/psviderski/uncloud/internal/sshexec"
"google.golang.org/protobuf/types/known/emptypb"
) )
// TODO: support pinning the script version to the CLI version. const (
const installScriptURL = "https://raw.githubusercontent.com/psviderski/uncloud/refs/heads/main/scripts/install.sh" // TODO: support pinning the script version to the CLI version.
installScriptURL = "https://raw.githubusercontent.com/psviderski/uncloud/refs/heads/main/scripts/install.sh"
rootUser = "root"
)
type RemoteMachine struct { type RemoteMachine struct {
User string User string
@@ -24,7 +32,7 @@ func installCmd(user string, version string) string {
var env []string var env []string
// Add the SSH user (non-root) to the uncloud group to allow access to the Uncloud daemon unix socket. // Add the SSH user (non-root) to the uncloud group to allow access to the Uncloud daemon unix socket.
if user != "root" { if user != rootUser {
sudoPrefix = "sudo" sudoPrefix = "sudo"
env = append(env, "UNCLOUD_GROUP_ADD_USER="+sshexec.Quote(user)) env = append(env, "UNCLOUD_GROUP_ADD_USER="+sshexec.Quote(user))
} }
@@ -46,6 +54,26 @@ func provisionMachine(ctx context.Context, exec sshexec.Executor, version string
return fmt.Errorf("run whoami: %w", err) return fmt.Errorf("run whoami: %w", err)
} }
if user != rootUser {
// 'sudo -n' is not used because it fails with 'sudo: a password is required' when the user has no password
// in /etc/shadow even though it may have valid sudo access.
out, err := exec.Run(ctx, "sudo true")
if err != nil {
if strings.Contains(out, "password is required") {
return fmt.Errorf(
"user '%[1]s' requires a password for sudo, but Uncloud needs passwordless sudo or root access "+
"to install and configure the uncloudd daemon on the remote machine.\n\n"+
"Possible solutions:\n"+
"1. Use root user or a user with passwordless sudo instead.\n"+
"2. Configure passwordless sudo for the user '%[1]s' by running on the remote machine:\n"+
" echo '%[1]s ALL=(ALL) NOPASSWD:ALL' | sudo tee /etc/sudoers.d/%[1]s",
user)
}
return fmt.Errorf("sudo command failed for user '%s': %w. "+
"Please ensure the user has sudo privileges or use root user instead", user, err)
}
}
cmd := installCmd(user, version) cmd := installCmd(user, version)
fmt.Println("Downloading Uncloud install script:", installScriptURL) fmt.Println("Downloading Uncloud install script:", installScriptURL)
@@ -56,3 +84,56 @@ func provisionMachine(ctx context.Context, exec sshexec.Executor, version string
} }
return nil return nil
} }
func promptResetMachine(ctx context.Context, machineClient pb.MachineClient) error {
var confirm bool
form := huh.NewForm(
huh.NewGroup(
huh.NewConfirm().
Title(
"The remote machine is already initialised as a cluster member. Do you want to reset it first?\n" +
"This will:\n" +
"- Remove all service containers from the machine\n" +
"- Reset the machine to the uninitialised state",
).
Affirmative("Yes!").
Negative("No").
Value(&confirm),
),
).WithAccessible(true)
if err := form.Run(); err != nil {
return fmt.Errorf("prompt user to confirm: %w", err)
}
if !confirm {
return fmt.Errorf("remote machine is already initialised as a cluster member")
}
if _, err := machineClient.Reset(ctx, &pb.ResetRequest{}); err != nil {
return fmt.Errorf("reset remote machine: %w. You can also manually run 'uncloud-uninstall' "+
"on the remote machine to fully uninstall Uncloud from it", err)
}
fmt.Println("Resetting the remote machine...")
if err := waitMachineReady(ctx, machineClient, 1*time.Minute); err != nil {
return fmt.Errorf("wait for machine to be ready after reset: %w", err)
}
return nil
}
// waitMachineReady waits for the machine to be ready to serve requests.
func waitMachineReady(ctx context.Context, machineClient pb.MachineClient, timeout time.Duration) error {
boff := backoff.WithContext(backoff.NewExponentialBackOff(
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(timeout),
), ctx)
inspect := func() error {
_, err := machineClient.Inspect(ctx, &emptypb.Empty{})
if err != nil {
return fmt.Errorf("inspect machine: %w", err)
}
return nil
}
return backoff.Retry(inspect, boff)
}
+3 -2
View File
@@ -5,14 +5,15 @@ import (
"crypto/tls" "crypto/tls"
"errors" "errors"
"fmt" "fmt"
"github.com/cenkalti/backoff/v4"
"golang.org/x/net/http2"
"log/slog" "log/slog"
"net" "net"
"net/http" "net/http"
"net/netip" "net/netip"
"net/url" "net/url"
"time" "time"
"github.com/cenkalti/backoff/v4"
"golang.org/x/net/http2"
) )
const ( const (
+2 -1
View File
@@ -6,11 +6,12 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"github.com/cenkalti/backoff/v4"
"io" "io"
"log/slog" "log/slog"
"net/http" "net/http"
"strconv" "strconv"
"github.com/cenkalti/backoff/v4"
) )
type ChangeType string type ChangeType string
+2 -1
View File
@@ -3,8 +3,9 @@ package daemon
import ( import (
"context" "context"
"fmt" "fmt"
systemd "github.com/coreos/go-systemd/daemon"
"log/slog" "log/slog"
systemd "github.com/coreos/go-systemd/daemon"
"github.com/psviderski/uncloud/internal/machine" "github.com/psviderski/uncloud/internal/machine"
) )
+3 -2
View File
@@ -3,10 +3,11 @@ package daemon
import ( import (
"errors" "errors"
"fmt" "fmt"
"github.com/psviderski/uncloud/internal/machine"
"github.com/psviderski/uncloud/internal/machine/network"
"net/netip" "net/netip"
"os" "os"
"github.com/psviderski/uncloud/internal/machine"
"github.com/psviderski/uncloud/internal/machine/network"
) )
// MachineToken returns the local machine's token that can be used for adding the machine to a cluster. // MachineToken returns the local machine's token that can be used for adding the machine to a cluster.
+3 -2
View File
@@ -4,10 +4,11 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"github.com/cenkalti/backoff/v4"
"github.com/docker/docker/client"
"log/slog" "log/slog"
"time" "time"
"github.com/cenkalti/backoff/v4"
"github.com/docker/docker/client"
) )
// WaitDaemonReady waits for the Docker daemon to start and be ready to serve requests. // WaitDaemonReady waits for the Docker daemon to start and be ready to serve requests.
+176
View File
@@ -0,0 +1,176 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.34.2
// protoc v5.27.3
// source: internal/machine/api/pb/caddy.proto
package pb
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
emptypb "google.golang.org/protobuf/types/known/emptypb"
timestamppb "google.golang.org/protobuf/types/known/timestamppb"
reflect "reflect"
sync "sync"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type GetCaddyConfigResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
// The generated Caddyfile content.
Caddyfile string `protobuf:"bytes,1,opt,name=caddyfile,proto3" json:"caddyfile,omitempty"`
// Timestamp when the config was last modified.
ModifiedAt *timestamppb.Timestamp `protobuf:"bytes,2,opt,name=modified_at,json=modifiedAt,proto3" json:"modified_at,omitempty"`
}
func (x *GetCaddyConfigResponse) Reset() {
*x = GetCaddyConfigResponse{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_caddy_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *GetCaddyConfigResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*GetCaddyConfigResponse) ProtoMessage() {}
func (x *GetCaddyConfigResponse) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_caddy_proto_msgTypes[0]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use GetCaddyConfigResponse.ProtoReflect.Descriptor instead.
func (*GetCaddyConfigResponse) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_caddy_proto_rawDescGZIP(), []int{0}
}
func (x *GetCaddyConfigResponse) GetCaddyfile() string {
if x != nil {
return x.Caddyfile
}
return ""
}
func (x *GetCaddyConfigResponse) GetModifiedAt() *timestamppb.Timestamp {
if x != nil {
return x.ModifiedAt
}
return nil
}
var File_internal_machine_api_pb_caddy_proto protoreflect.FileDescriptor
var file_internal_machine_api_pb_caddy_proto_rawDesc = []byte{
0x0a, 0x23, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69,
0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x70, 0x62, 0x2f, 0x63, 0x61, 0x64, 0x64, 0x79, 0x2e,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x03, 0x61, 0x70, 0x69, 0x1a, 0x1b, 0x67, 0x6f, 0x6f, 0x67,
0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x65, 0x6d, 0x70, 0x74,
0x79, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x1f, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61,
0x6d, 0x70, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x73, 0x0a, 0x16, 0x47, 0x65, 0x74, 0x43,
0x61, 0x64, 0x64, 0x79, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e,
0x73, 0x65, 0x12, 0x1c, 0x0a, 0x09, 0x63, 0x61, 0x64, 0x64, 0x79, 0x66, 0x69, 0x6c, 0x65, 0x18,
0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x63, 0x61, 0x64, 0x64, 0x79, 0x66, 0x69, 0x6c, 0x65,
0x12, 0x3b, 0x0a, 0x0b, 0x6d, 0x6f, 0x64, 0x69, 0x66, 0x69, 0x65, 0x64, 0x5f, 0x61, 0x74, 0x18,
0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d,
0x70, 0x52, 0x0a, 0x6d, 0x6f, 0x64, 0x69, 0x66, 0x69, 0x65, 0x64, 0x41, 0x74, 0x32, 0x49, 0x0a,
0x05, 0x43, 0x61, 0x64, 0x64, 0x79, 0x12, 0x40, 0x0a, 0x09, 0x47, 0x65, 0x74, 0x43, 0x6f, 0x6e,
0x66, 0x69, 0x67, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f,
0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x1b, 0x2e, 0x61, 0x70,
0x69, 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x64, 0x64, 0x79, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67,
0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x37, 0x5a, 0x35, 0x67, 0x69, 0x74, 0x68,
0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x70, 0x73, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73, 0x6b,
0x69, 0x2f, 0x75, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e,
0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x70,
0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
}
var (
file_internal_machine_api_pb_caddy_proto_rawDescOnce sync.Once
file_internal_machine_api_pb_caddy_proto_rawDescData = file_internal_machine_api_pb_caddy_proto_rawDesc
)
func file_internal_machine_api_pb_caddy_proto_rawDescGZIP() []byte {
file_internal_machine_api_pb_caddy_proto_rawDescOnce.Do(func() {
file_internal_machine_api_pb_caddy_proto_rawDescData = protoimpl.X.CompressGZIP(file_internal_machine_api_pb_caddy_proto_rawDescData)
})
return file_internal_machine_api_pb_caddy_proto_rawDescData
}
var file_internal_machine_api_pb_caddy_proto_msgTypes = make([]protoimpl.MessageInfo, 1)
var file_internal_machine_api_pb_caddy_proto_goTypes = []any{
(*GetCaddyConfigResponse)(nil), // 0: api.GetCaddyConfigResponse
(*timestamppb.Timestamp)(nil), // 1: google.protobuf.Timestamp
(*emptypb.Empty)(nil), // 2: google.protobuf.Empty
}
var file_internal_machine_api_pb_caddy_proto_depIdxs = []int32{
1, // 0: api.GetCaddyConfigResponse.modified_at:type_name -> google.protobuf.Timestamp
2, // 1: api.Caddy.GetConfig:input_type -> google.protobuf.Empty
0, // 2: api.Caddy.GetConfig:output_type -> api.GetCaddyConfigResponse
2, // [2:3] is the sub-list for method output_type
1, // [1:2] is the sub-list for method input_type
1, // [1:1] is the sub-list for extension type_name
1, // [1:1] is the sub-list for extension extendee
0, // [0:1] is the sub-list for field type_name
}
func init() { file_internal_machine_api_pb_caddy_proto_init() }
func file_internal_machine_api_pb_caddy_proto_init() {
if File_internal_machine_api_pb_caddy_proto != nil {
return
}
if !protoimpl.UnsafeEnabled {
file_internal_machine_api_pb_caddy_proto_msgTypes[0].Exporter = func(v any, i int) any {
switch v := v.(*GetCaddyConfigResponse); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_internal_machine_api_pb_caddy_proto_rawDesc,
NumEnums: 0,
NumMessages: 1,
NumExtensions: 0,
NumServices: 1,
},
GoTypes: file_internal_machine_api_pb_caddy_proto_goTypes,
DependencyIndexes: file_internal_machine_api_pb_caddy_proto_depIdxs,
MessageInfos: file_internal_machine_api_pb_caddy_proto_msgTypes,
}.Build()
File_internal_machine_api_pb_caddy_proto = out.File
file_internal_machine_api_pb_caddy_proto_rawDesc = nil
file_internal_machine_api_pb_caddy_proto_goTypes = nil
file_internal_machine_api_pb_caddy_proto_depIdxs = nil
}
+20
View File
@@ -0,0 +1,20 @@
syntax = "proto3";
package api;
option go_package = "github.com/psviderski/uncloud/internal/machine/api/pb";
import "google/protobuf/empty.proto";
import "google/protobuf/timestamp.proto";
service Caddy {
// GetConfig retrieves the current Caddy configuration from the machine.
rpc GetConfig(google.protobuf.Empty) returns (GetCaddyConfigResponse);
}
message GetCaddyConfigResponse {
// The generated Caddyfile content.
string caddyfile = 1;
// Timestamp when the config was last modified.
google.protobuf.Timestamp modified_at = 2;
}
+124
View File
@@ -0,0 +1,124 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.5.1
// - protoc v5.27.3
// source: internal/machine/api/pb/caddy.proto
package pb
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
emptypb "google.golang.org/protobuf/types/known/emptypb"
)
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
// Requires gRPC-Go v1.64.0 or later.
const _ = grpc.SupportPackageIsVersion9
const (
Caddy_GetConfig_FullMethodName = "/api.Caddy/GetConfig"
)
// CaddyClient is the client API for Caddy service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type CaddyClient interface {
// GetConfig retrieves the current Caddy configuration from the machine.
GetConfig(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*GetCaddyConfigResponse, error)
}
type caddyClient struct {
cc grpc.ClientConnInterface
}
func NewCaddyClient(cc grpc.ClientConnInterface) CaddyClient {
return &caddyClient{cc}
}
func (c *caddyClient) GetConfig(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*GetCaddyConfigResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(GetCaddyConfigResponse)
err := c.cc.Invoke(ctx, Caddy_GetConfig_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
// CaddyServer is the server API for Caddy service.
// All implementations must embed UnimplementedCaddyServer
// for forward compatibility.
type CaddyServer interface {
// GetConfig retrieves the current Caddy configuration from the machine.
GetConfig(context.Context, *emptypb.Empty) (*GetCaddyConfigResponse, error)
mustEmbedUnimplementedCaddyServer()
}
// UnimplementedCaddyServer must be embedded to have
// forward compatible implementations.
//
// NOTE: this should be embedded by value instead of pointer to avoid a nil
// pointer dereference when methods are called.
type UnimplementedCaddyServer struct{}
func (UnimplementedCaddyServer) GetConfig(context.Context, *emptypb.Empty) (*GetCaddyConfigResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method GetConfig not implemented")
}
func (UnimplementedCaddyServer) mustEmbedUnimplementedCaddyServer() {}
func (UnimplementedCaddyServer) testEmbeddedByValue() {}
// UnsafeCaddyServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to CaddyServer will
// result in compilation errors.
type UnsafeCaddyServer interface {
mustEmbedUnimplementedCaddyServer()
}
func RegisterCaddyServer(s grpc.ServiceRegistrar, srv CaddyServer) {
// If the following call pancis, it indicates UnimplementedCaddyServer was
// embedded by pointer and is nil. This will cause panics if an
// unimplemented method is ever invoked, so we test this at initialization
// time to prevent it from happening at runtime later due to I/O.
if t, ok := srv.(interface{ testEmbeddedByValue() }); ok {
t.testEmbeddedByValue()
}
s.RegisterService(&Caddy_ServiceDesc, srv)
}
func _Caddy_GetConfig_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(emptypb.Empty)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(CaddyServer).GetConfig(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Caddy_GetConfig_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(CaddyServer).GetConfig(ctx, req.(*emptypb.Empty))
}
return interceptor(ctx, in, info, handler)
}
// Caddy_ServiceDesc is the grpc.ServiceDesc for Caddy service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
var Caddy_ServiceDesc = grpc.ServiceDesc{
ServiceName: "api.Caddy",
HandlerType: (*CaddyServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "GetConfig",
Handler: _Caddy_GetConfig_Handler,
},
},
Streams: []grpc.StreamDesc{},
Metadata: "internal/machine/api/pb/caddy.proto",
}
+350 -107
View File
@@ -124,7 +124,7 @@ func (x DNSRecord_RecordType) Number() protoreflect.EnumNumber {
// Deprecated: Use DNSRecord_RecordType.Descriptor instead. // Deprecated: Use DNSRecord_RecordType.Descriptor instead.
func (DNSRecord_RecordType) EnumDescriptor() ([]byte, []int) { func (DNSRecord_RecordType) EnumDescriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{8, 0} return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{11, 0}
} }
type AddMachineRequest struct { type AddMachineRequest struct {
@@ -339,6 +339,173 @@ func (x *ListMachinesResponse) GetMachines() []*MachineMember {
return nil return nil
} }
type UpdateMachineRequest struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
// Machine to update
MachineId string `protobuf:"bytes,1,opt,name=machine_id,json=machineId,proto3" json:"machine_id,omitempty"`
// Updated machine information
Name *string `protobuf:"bytes,2,opt,name=name,proto3,oneof" json:"name,omitempty"`
PublicIp *IP `protobuf:"bytes,3,opt,name=public_ip,json=publicIp,proto3,oneof" json:"public_ip,omitempty"`
Endpoints []*IPPort `protobuf:"bytes,4,rep,name=endpoints,proto3" json:"endpoints,omitempty"`
}
func (x *UpdateMachineRequest) Reset() {
*x = UpdateMachineRequest{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[4]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *UpdateMachineRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*UpdateMachineRequest) ProtoMessage() {}
func (x *UpdateMachineRequest) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[4]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use UpdateMachineRequest.ProtoReflect.Descriptor instead.
func (*UpdateMachineRequest) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{4}
}
func (x *UpdateMachineRequest) GetMachineId() string {
if x != nil {
return x.MachineId
}
return ""
}
func (x *UpdateMachineRequest) GetName() string {
if x != nil && x.Name != nil {
return *x.Name
}
return ""
}
func (x *UpdateMachineRequest) GetPublicIp() *IP {
if x != nil {
return x.PublicIp
}
return nil
}
func (x *UpdateMachineRequest) GetEndpoints() []*IPPort {
if x != nil {
return x.Endpoints
}
return nil
}
type UpdateMachineResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Machine *MachineInfo `protobuf:"bytes,1,opt,name=machine,proto3" json:"machine,omitempty"`
}
func (x *UpdateMachineResponse) Reset() {
*x = UpdateMachineResponse{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[5]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *UpdateMachineResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*UpdateMachineResponse) ProtoMessage() {}
func (x *UpdateMachineResponse) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[5]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use UpdateMachineResponse.ProtoReflect.Descriptor instead.
func (*UpdateMachineResponse) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{5}
}
func (x *UpdateMachineResponse) GetMachine() *MachineInfo {
if x != nil {
return x.Machine
}
return nil
}
type RemoveMachineRequest struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
}
func (x *RemoveMachineRequest) Reset() {
*x = RemoveMachineRequest{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[6]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *RemoveMachineRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*RemoveMachineRequest) ProtoMessage() {}
func (x *RemoveMachineRequest) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[6]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use RemoveMachineRequest.ProtoReflect.Descriptor instead.
func (*RemoveMachineRequest) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{6}
}
func (x *RemoveMachineRequest) GetId() string {
if x != nil {
return x.Id
}
return ""
}
type Domain struct { type Domain struct {
state protoimpl.MessageState state protoimpl.MessageState
sizeCache protoimpl.SizeCache sizeCache protoimpl.SizeCache
@@ -350,7 +517,7 @@ type Domain struct {
func (x *Domain) Reset() { func (x *Domain) Reset() {
*x = Domain{} *x = Domain{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[4] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[7]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -363,7 +530,7 @@ func (x *Domain) String() string {
func (*Domain) ProtoMessage() {} func (*Domain) ProtoMessage() {}
func (x *Domain) ProtoReflect() protoreflect.Message { func (x *Domain) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[4] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[7]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -376,7 +543,7 @@ func (x *Domain) ProtoReflect() protoreflect.Message {
// Deprecated: Use Domain.ProtoReflect.Descriptor instead. // Deprecated: Use Domain.ProtoReflect.Descriptor instead.
func (*Domain) Descriptor() ([]byte, []int) { func (*Domain) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{4} return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{7}
} }
func (x *Domain) GetName() string { func (x *Domain) GetName() string {
@@ -397,7 +564,7 @@ type ReserveDomainRequest struct {
func (x *ReserveDomainRequest) Reset() { func (x *ReserveDomainRequest) Reset() {
*x = ReserveDomainRequest{} *x = ReserveDomainRequest{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[5] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[8]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -410,7 +577,7 @@ func (x *ReserveDomainRequest) String() string {
func (*ReserveDomainRequest) ProtoMessage() {} func (*ReserveDomainRequest) ProtoMessage() {}
func (x *ReserveDomainRequest) ProtoReflect() protoreflect.Message { func (x *ReserveDomainRequest) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[5] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[8]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -423,7 +590,7 @@ func (x *ReserveDomainRequest) ProtoReflect() protoreflect.Message {
// Deprecated: Use ReserveDomainRequest.ProtoReflect.Descriptor instead. // Deprecated: Use ReserveDomainRequest.ProtoReflect.Descriptor instead.
func (*ReserveDomainRequest) Descriptor() ([]byte, []int) { func (*ReserveDomainRequest) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{5} return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{8}
} }
func (x *ReserveDomainRequest) GetEndpoint() string { func (x *ReserveDomainRequest) GetEndpoint() string {
@@ -444,7 +611,7 @@ type CreateDomainRecordsRequest struct {
func (x *CreateDomainRecordsRequest) Reset() { func (x *CreateDomainRecordsRequest) Reset() {
*x = CreateDomainRecordsRequest{} *x = CreateDomainRecordsRequest{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[6] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[9]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -457,7 +624,7 @@ func (x *CreateDomainRecordsRequest) String() string {
func (*CreateDomainRecordsRequest) ProtoMessage() {} func (*CreateDomainRecordsRequest) ProtoMessage() {}
func (x *CreateDomainRecordsRequest) ProtoReflect() protoreflect.Message { func (x *CreateDomainRecordsRequest) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[6] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[9]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -470,7 +637,7 @@ func (x *CreateDomainRecordsRequest) ProtoReflect() protoreflect.Message {
// Deprecated: Use CreateDomainRecordsRequest.ProtoReflect.Descriptor instead. // Deprecated: Use CreateDomainRecordsRequest.ProtoReflect.Descriptor instead.
func (*CreateDomainRecordsRequest) Descriptor() ([]byte, []int) { func (*CreateDomainRecordsRequest) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{6} return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{9}
} }
func (x *CreateDomainRecordsRequest) GetRecords() []*DNSRecord { func (x *CreateDomainRecordsRequest) GetRecords() []*DNSRecord {
@@ -491,7 +658,7 @@ type CreateDomainRecordsResponse struct {
func (x *CreateDomainRecordsResponse) Reset() { func (x *CreateDomainRecordsResponse) Reset() {
*x = CreateDomainRecordsResponse{} *x = CreateDomainRecordsResponse{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[7] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[10]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -504,7 +671,7 @@ func (x *CreateDomainRecordsResponse) String() string {
func (*CreateDomainRecordsResponse) ProtoMessage() {} func (*CreateDomainRecordsResponse) ProtoMessage() {}
func (x *CreateDomainRecordsResponse) ProtoReflect() protoreflect.Message { func (x *CreateDomainRecordsResponse) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[7] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[10]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -517,7 +684,7 @@ func (x *CreateDomainRecordsResponse) ProtoReflect() protoreflect.Message {
// Deprecated: Use CreateDomainRecordsResponse.ProtoReflect.Descriptor instead. // Deprecated: Use CreateDomainRecordsResponse.ProtoReflect.Descriptor instead.
func (*CreateDomainRecordsResponse) Descriptor() ([]byte, []int) { func (*CreateDomainRecordsResponse) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{7} return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{10}
} }
func (x *CreateDomainRecordsResponse) GetRecords() []*DNSRecord { func (x *CreateDomainRecordsResponse) GetRecords() []*DNSRecord {
@@ -540,7 +707,7 @@ type DNSRecord struct {
func (x *DNSRecord) Reset() { func (x *DNSRecord) Reset() {
*x = DNSRecord{} *x = DNSRecord{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[8] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[11]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -553,7 +720,7 @@ func (x *DNSRecord) String() string {
func (*DNSRecord) ProtoMessage() {} func (*DNSRecord) ProtoMessage() {}
func (x *DNSRecord) ProtoReflect() protoreflect.Message { func (x *DNSRecord) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[8] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[11]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -566,7 +733,7 @@ func (x *DNSRecord) ProtoReflect() protoreflect.Message {
// Deprecated: Use DNSRecord.ProtoReflect.Descriptor instead. // Deprecated: Use DNSRecord.ProtoReflect.Descriptor instead.
func (*DNSRecord) Descriptor() ([]byte, []int) { func (*DNSRecord) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{8} return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{11}
} }
func (x *DNSRecord) GetName() string { func (x *DNSRecord) GetName() string {
@@ -630,59 +797,87 @@ var file_internal_machine_api_pb_cluster_proto_rawDesc = []byte{
0x6f, 0x6e, 0x73, 0x65, 0x12, 0x2e, 0x0a, 0x08, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x2e, 0x0a, 0x08, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x73,
0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x12, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4d, 0x61, 0x63, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x12, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4d, 0x61, 0x63,
0x68, 0x69, 0x6e, 0x65, 0x4d, 0x65, 0x6d, 0x62, 0x65, 0x72, 0x52, 0x08, 0x6d, 0x61, 0x63, 0x68, 0x68, 0x69, 0x6e, 0x65, 0x4d, 0x65, 0x6d, 0x62, 0x65, 0x72, 0x52, 0x08, 0x6d, 0x61, 0x63, 0x68,
0x69, 0x6e, 0x65, 0x73, 0x22, 0x1c, 0x0a, 0x06, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x12, 0x69, 0x6e, 0x65, 0x73, 0x22, 0xbb, 0x01, 0x0a, 0x14, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x4d,
0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1d, 0x0a,
0x6d, 0x65, 0x22, 0x32, 0x0a, 0x14, 0x52, 0x65, 0x73, 0x65, 0x72, 0x76, 0x65, 0x44, 0x6f, 0x6d, 0x0a, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28,
0x61, 0x69, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1a, 0x0a, 0x08, 0x65, 0x6e, 0x09, 0x52, 0x09, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x64, 0x12, 0x17, 0x0a, 0x04,
0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x65, 0x6e, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x48, 0x00, 0x52, 0x04, 0x6e, 0x61,
0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x22, 0x46, 0x0a, 0x1a, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x6d, 0x65, 0x88, 0x01, 0x01, 0x12, 0x29, 0x0a, 0x09, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f,
0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x52, 0x65, 0x71, 0x69, 0x70, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x07, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49,
0x75, 0x65, 0x73, 0x74, 0x12, 0x28, 0x0a, 0x07, 0x72, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x18, 0x50, 0x48, 0x01, 0x52, 0x08, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x49, 0x70, 0x88, 0x01, 0x01,
0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x4e, 0x53, 0x52, 0x12, 0x29, 0x0a, 0x09, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x73, 0x18, 0x04, 0x20,
0x65, 0x63, 0x6f, 0x72, 0x64, 0x52, 0x07, 0x72, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x22, 0x47, 0x03, 0x28, 0x0b, 0x32, 0x0b, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x50, 0x50, 0x6f, 0x72, 0x74,
0x0a, 0x1b, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x52, 0x09, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x73, 0x42, 0x07, 0x0a, 0x05, 0x5f,
0x63, 0x6f, 0x72, 0x64, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x28, 0x0a, 0x6e, 0x61, 0x6d, 0x65, 0x42, 0x0c, 0x0a, 0x0a, 0x5f, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f,
0x07, 0x72, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x0e, 0x69, 0x70, 0x22, 0x43, 0x0a, 0x15, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x4d, 0x61, 0x63, 0x68,
0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x4e, 0x53, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x52, 0x07, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x2a, 0x0a, 0x07, 0x6d,
0x72, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x22, 0x96, 0x01, 0x0a, 0x09, 0x44, 0x4e, 0x53, 0x52, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x61,
0x65, 0x63, 0x6f, 0x72, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x70, 0x69, 0x2e, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x07,
0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x2d, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x22, 0x26, 0x0a, 0x14, 0x52, 0x65, 0x6d, 0x6f, 0x76,
0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x19, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x4e, 0x65, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12,
0x53, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x79, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x02, 0x69, 0x64, 0x22,
0x70, 0x65, 0x52, 0x04, 0x74, 0x79, 0x70, 0x65, 0x12, 0x16, 0x0a, 0x06, 0x76, 0x61, 0x6c, 0x75, 0x1c, 0x0a, 0x06, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d,
0x65, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x09, 0x52, 0x06, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x22, 0x32, 0x0a,
0x22, 0x2e, 0x0a, 0x0a, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x79, 0x70, 0x65, 0x12, 0x0f, 0x14, 0x52, 0x65, 0x73, 0x65, 0x72, 0x76, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65,
0x0a, 0x0b, 0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, 0x12, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1a, 0x0a, 0x08, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e,
0x05, 0x0a, 0x01, 0x41, 0x10, 0x01, 0x12, 0x08, 0x0a, 0x04, 0x41, 0x41, 0x41, 0x41, 0x10, 0x02, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e,
0x32, 0x86, 0x03, 0x0a, 0x07, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x12, 0x3d, 0x0a, 0x0a, 0x74, 0x22, 0x46, 0x0a, 0x1a, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69,
0x41, 0x64, 0x64, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x12, 0x16, 0x2e, 0x61, 0x70, 0x69, 0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12,
0x2e, 0x41, 0x64, 0x64, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x28, 0x0a, 0x07, 0x72, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b,
0x73, 0x74, 0x1a, 0x17, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x41, 0x64, 0x64, 0x4d, 0x61, 0x63, 0x68, 0x32, 0x0e, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x4e, 0x53, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64,
0x69, 0x6e, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x41, 0x0a, 0x0c, 0x4c, 0x52, 0x07, 0x72, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x22, 0x47, 0x0a, 0x1b, 0x43, 0x72, 0x65,
0x69, 0x73, 0x74, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x12, 0x16, 0x2e, 0x67, 0x6f,
0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d,
0x70, 0x74, 0x79, 0x1a, 0x19, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x4d, 0x61,
0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x37,
0x0a, 0x0d, 0x52, 0x65, 0x73, 0x65, 0x72, 0x76, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12,
0x19, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x52, 0x65, 0x73, 0x65, 0x72, 0x76, 0x65, 0x44, 0x6f, 0x6d,
0x61, 0x69, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x0b, 0x2e, 0x61, 0x70, 0x69,
0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x30, 0x0a, 0x09, 0x47, 0x65, 0x74, 0x44, 0x6f,
0x6d, 0x61, 0x69, 0x6e, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72,
0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x0b, 0x2e, 0x61,
0x70, 0x69, 0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x34, 0x0a, 0x0d, 0x52, 0x65, 0x6c,
0x65, 0x61, 0x73, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f,
0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70,
0x74, 0x79, 0x1a, 0x0b, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12,
0x58, 0x0a, 0x13, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52,
0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x12, 0x1f, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x43, 0x72, 0x65,
0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73,
0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x20, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x43, 0x72, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x28, 0x0a, 0x07, 0x72, 0x65, 0x63, 0x6f,
0x65, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x72, 0x64, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x70, 0x69, 0x2e,
0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x37, 0x5a, 0x35, 0x67, 0x69, 0x74, 0x44, 0x4e, 0x53, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x52, 0x07, 0x72, 0x65, 0x63, 0x6f, 0x72,
0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x70, 0x73, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73, 0x64, 0x73, 0x22, 0x96, 0x01, 0x0a, 0x09, 0x44, 0x4e, 0x53, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64,
0x6b, 0x69, 0x2f, 0x75, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04,
0x6e, 0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x2d, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, 0x02, 0x20, 0x01,
0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, 0x28, 0x0e, 0x32, 0x19, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x4e, 0x53, 0x52, 0x65, 0x63, 0x6f,
0x72, 0x64, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x79, 0x70, 0x65, 0x52, 0x04, 0x74,
0x79, 0x70, 0x65, 0x12, 0x16, 0x0a, 0x06, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x18, 0x03, 0x20,
0x03, 0x28, 0x09, 0x52, 0x06, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x22, 0x2e, 0x0a, 0x0a, 0x52,
0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x79, 0x70, 0x65, 0x12, 0x0f, 0x0a, 0x0b, 0x55, 0x4e, 0x53,
0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, 0x12, 0x05, 0x0a, 0x01, 0x41, 0x10,
0x01, 0x12, 0x08, 0x0a, 0x04, 0x41, 0x41, 0x41, 0x41, 0x10, 0x02, 0x32, 0x92, 0x04, 0x0a, 0x07,
0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x12, 0x3d, 0x0a, 0x0a, 0x41, 0x64, 0x64, 0x4d, 0x61,
0x63, 0x68, 0x69, 0x6e, 0x65, 0x12, 0x16, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x41, 0x64, 0x64, 0x4d,
0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x17, 0x2e,
0x61, 0x70, 0x69, 0x2e, 0x41, 0x64, 0x64, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65,
0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x41, 0x0a, 0x0c, 0x4c, 0x69, 0x73, 0x74, 0x4d, 0x61,
0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x19,
0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65,
0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x46, 0x0a, 0x0d, 0x55, 0x70, 0x64,
0x61, 0x74, 0x65, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x12, 0x19, 0x2e, 0x61, 0x70, 0x69,
0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65,
0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1a, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x55, 0x70, 0x64, 0x61,
0x74, 0x65, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73,
0x65, 0x12, 0x42, 0x0a, 0x0d, 0x52, 0x65, 0x6d, 0x6f, 0x76, 0x65, 0x4d, 0x61, 0x63, 0x68, 0x69,
0x6e, 0x65, 0x12, 0x19, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x52, 0x65, 0x6d, 0x6f, 0x76, 0x65, 0x4d,
0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x16, 0x2e,
0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e,
0x45, 0x6d, 0x70, 0x74, 0x79, 0x12, 0x37, 0x0a, 0x0d, 0x52, 0x65, 0x73, 0x65, 0x72, 0x76, 0x65,
0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x19, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x52, 0x65, 0x73,
0x65, 0x72, 0x76, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
0x74, 0x1a, 0x0b, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x30,
0x0a, 0x09, 0x47, 0x65, 0x74, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x16, 0x2e, 0x67, 0x6f,
0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d,
0x70, 0x74, 0x79, 0x1a, 0x0b, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e,
0x12, 0x34, 0x0a, 0x0d, 0x52, 0x65, 0x6c, 0x65, 0x61, 0x73, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69,
0x6e, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f,
0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x0b, 0x2e, 0x61, 0x70, 0x69, 0x2e,
0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x58, 0x0a, 0x13, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65,
0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x12, 0x1f, 0x2e,
0x61, 0x70, 0x69, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e,
0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x20,
0x2e, 0x61, 0x70, 0x69, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69,
0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x42, 0x37, 0x5a, 0x35, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x70,
0x73, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73, 0x6b, 0x69, 0x2f, 0x75, 0x6e, 0x63, 0x6c, 0x6f, 0x75,
0x64, 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69,
0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f,
0x33,
} }
var ( var (
@@ -698,7 +893,7 @@ func file_internal_machine_api_pb_cluster_proto_rawDescGZIP() []byte {
} }
var file_internal_machine_api_pb_cluster_proto_enumTypes = make([]protoimpl.EnumInfo, 2) var file_internal_machine_api_pb_cluster_proto_enumTypes = make([]protoimpl.EnumInfo, 2)
var file_internal_machine_api_pb_cluster_proto_msgTypes = make([]protoimpl.MessageInfo, 9) var file_internal_machine_api_pb_cluster_proto_msgTypes = make([]protoimpl.MessageInfo, 12)
var file_internal_machine_api_pb_cluster_proto_goTypes = []any{ var file_internal_machine_api_pb_cluster_proto_goTypes = []any{
(MachineMember_MembershipState)(0), // 0: api.MachineMember.MembershipState (MachineMember_MembershipState)(0), // 0: api.MachineMember.MembershipState
(DNSRecord_RecordType)(0), // 1: api.DNSRecord.RecordType (DNSRecord_RecordType)(0), // 1: api.DNSRecord.RecordType
@@ -706,43 +901,54 @@ var file_internal_machine_api_pb_cluster_proto_goTypes = []any{
(*AddMachineResponse)(nil), // 3: api.AddMachineResponse (*AddMachineResponse)(nil), // 3: api.AddMachineResponse
(*MachineMember)(nil), // 4: api.MachineMember (*MachineMember)(nil), // 4: api.MachineMember
(*ListMachinesResponse)(nil), // 5: api.ListMachinesResponse (*ListMachinesResponse)(nil), // 5: api.ListMachinesResponse
(*Domain)(nil), // 6: api.Domain (*UpdateMachineRequest)(nil), // 6: api.UpdateMachineRequest
(*ReserveDomainRequest)(nil), // 7: api.ReserveDomainRequest (*UpdateMachineResponse)(nil), // 7: api.UpdateMachineResponse
(*CreateDomainRecordsRequest)(nil), // 8: api.CreateDomainRecordsRequest (*RemoveMachineRequest)(nil), // 8: api.RemoveMachineRequest
(*CreateDomainRecordsResponse)(nil), // 9: api.CreateDomainRecordsResponse (*Domain)(nil), // 9: api.Domain
(*DNSRecord)(nil), // 10: api.DNSRecord (*ReserveDomainRequest)(nil), // 10: api.ReserveDomainRequest
(*NetworkConfig)(nil), // 11: api.NetworkConfig (*CreateDomainRecordsRequest)(nil), // 11: api.CreateDomainRecordsRequest
(*IP)(nil), // 12: api.IP (*CreateDomainRecordsResponse)(nil), // 12: api.CreateDomainRecordsResponse
(*MachineInfo)(nil), // 13: api.MachineInfo (*DNSRecord)(nil), // 13: api.DNSRecord
(*emptypb.Empty)(nil), // 14: google.protobuf.Empty (*NetworkConfig)(nil), // 14: api.NetworkConfig
(*IP)(nil), // 15: api.IP
(*MachineInfo)(nil), // 16: api.MachineInfo
(*IPPort)(nil), // 17: api.IPPort
(*emptypb.Empty)(nil), // 18: google.protobuf.Empty
} }
var file_internal_machine_api_pb_cluster_proto_depIdxs = []int32{ var file_internal_machine_api_pb_cluster_proto_depIdxs = []int32{
11, // 0: api.AddMachineRequest.network:type_name -> api.NetworkConfig 14, // 0: api.AddMachineRequest.network:type_name -> api.NetworkConfig
12, // 1: api.AddMachineRequest.public_ip:type_name -> api.IP 15, // 1: api.AddMachineRequest.public_ip:type_name -> api.IP
13, // 2: api.AddMachineResponse.machine:type_name -> api.MachineInfo 16, // 2: api.AddMachineResponse.machine:type_name -> api.MachineInfo
13, // 3: api.MachineMember.machine:type_name -> api.MachineInfo 16, // 3: api.MachineMember.machine:type_name -> api.MachineInfo
0, // 4: api.MachineMember.state:type_name -> api.MachineMember.MembershipState 0, // 4: api.MachineMember.state:type_name -> api.MachineMember.MembershipState
4, // 5: api.ListMachinesResponse.machines:type_name -> api.MachineMember 4, // 5: api.ListMachinesResponse.machines:type_name -> api.MachineMember
10, // 6: api.CreateDomainRecordsRequest.records:type_name -> api.DNSRecord 15, // 6: api.UpdateMachineRequest.public_ip:type_name -> api.IP
10, // 7: api.CreateDomainRecordsResponse.records:type_name -> api.DNSRecord 17, // 7: api.UpdateMachineRequest.endpoints:type_name -> api.IPPort
1, // 8: api.DNSRecord.type:type_name -> api.DNSRecord.RecordType 16, // 8: api.UpdateMachineResponse.machine:type_name -> api.MachineInfo
2, // 9: api.Cluster.AddMachine:input_type -> api.AddMachineRequest 13, // 9: api.CreateDomainRecordsRequest.records:type_name -> api.DNSRecord
14, // 10: api.Cluster.ListMachines:input_type -> google.protobuf.Empty 13, // 10: api.CreateDomainRecordsResponse.records:type_name -> api.DNSRecord
7, // 11: api.Cluster.ReserveDomain:input_type -> api.ReserveDomainRequest 1, // 11: api.DNSRecord.type:type_name -> api.DNSRecord.RecordType
14, // 12: api.Cluster.GetDomain:input_type -> google.protobuf.Empty 2, // 12: api.Cluster.AddMachine:input_type -> api.AddMachineRequest
14, // 13: api.Cluster.ReleaseDomain:input_type -> google.protobuf.Empty 18, // 13: api.Cluster.ListMachines:input_type -> google.protobuf.Empty
8, // 14: api.Cluster.CreateDomainRecords:input_type -> api.CreateDomainRecordsRequest 6, // 14: api.Cluster.UpdateMachine:input_type -> api.UpdateMachineRequest
3, // 15: api.Cluster.AddMachine:output_type -> api.AddMachineResponse 8, // 15: api.Cluster.RemoveMachine:input_type -> api.RemoveMachineRequest
5, // 16: api.Cluster.ListMachines:output_type -> api.ListMachinesResponse 10, // 16: api.Cluster.ReserveDomain:input_type -> api.ReserveDomainRequest
6, // 17: api.Cluster.ReserveDomain:output_type -> api.Domain 18, // 17: api.Cluster.GetDomain:input_type -> google.protobuf.Empty
6, // 18: api.Cluster.GetDomain:output_type -> api.Domain 18, // 18: api.Cluster.ReleaseDomain:input_type -> google.protobuf.Empty
6, // 19: api.Cluster.ReleaseDomain:output_type -> api.Domain 11, // 19: api.Cluster.CreateDomainRecords:input_type -> api.CreateDomainRecordsRequest
9, // 20: api.Cluster.CreateDomainRecords:output_type -> api.CreateDomainRecordsResponse 3, // 20: api.Cluster.AddMachine:output_type -> api.AddMachineResponse
15, // [15:21] is the sub-list for method output_type 5, // 21: api.Cluster.ListMachines:output_type -> api.ListMachinesResponse
9, // [9:15] is the sub-list for method input_type 7, // 22: api.Cluster.UpdateMachine:output_type -> api.UpdateMachineResponse
9, // [9:9] is the sub-list for extension type_name 18, // 23: api.Cluster.RemoveMachine:output_type -> google.protobuf.Empty
9, // [9:9] is the sub-list for extension extendee 9, // 24: api.Cluster.ReserveDomain:output_type -> api.Domain
0, // [0:9] is the sub-list for field type_name 9, // 25: api.Cluster.GetDomain:output_type -> api.Domain
9, // 26: api.Cluster.ReleaseDomain:output_type -> api.Domain
12, // 27: api.Cluster.CreateDomainRecords:output_type -> api.CreateDomainRecordsResponse
20, // [20:28] is the sub-list for method output_type
12, // [12:20] is the sub-list for method input_type
12, // [12:12] is the sub-list for extension type_name
12, // [12:12] is the sub-list for extension extendee
0, // [0:12] is the sub-list for field type_name
} }
func init() { file_internal_machine_api_pb_cluster_proto_init() } func init() { file_internal_machine_api_pb_cluster_proto_init() }
@@ -802,7 +1008,7 @@ func file_internal_machine_api_pb_cluster_proto_init() {
} }
} }
file_internal_machine_api_pb_cluster_proto_msgTypes[4].Exporter = func(v any, i int) any { file_internal_machine_api_pb_cluster_proto_msgTypes[4].Exporter = func(v any, i int) any {
switch v := v.(*Domain); i { switch v := v.(*UpdateMachineRequest); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -814,7 +1020,7 @@ func file_internal_machine_api_pb_cluster_proto_init() {
} }
} }
file_internal_machine_api_pb_cluster_proto_msgTypes[5].Exporter = func(v any, i int) any { file_internal_machine_api_pb_cluster_proto_msgTypes[5].Exporter = func(v any, i int) any {
switch v := v.(*ReserveDomainRequest); i { switch v := v.(*UpdateMachineResponse); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -826,7 +1032,7 @@ func file_internal_machine_api_pb_cluster_proto_init() {
} }
} }
file_internal_machine_api_pb_cluster_proto_msgTypes[6].Exporter = func(v any, i int) any { file_internal_machine_api_pb_cluster_proto_msgTypes[6].Exporter = func(v any, i int) any {
switch v := v.(*CreateDomainRecordsRequest); i { switch v := v.(*RemoveMachineRequest); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -838,7 +1044,7 @@ func file_internal_machine_api_pb_cluster_proto_init() {
} }
} }
file_internal_machine_api_pb_cluster_proto_msgTypes[7].Exporter = func(v any, i int) any { file_internal_machine_api_pb_cluster_proto_msgTypes[7].Exporter = func(v any, i int) any {
switch v := v.(*CreateDomainRecordsResponse); i { switch v := v.(*Domain); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -850,6 +1056,42 @@ func file_internal_machine_api_pb_cluster_proto_init() {
} }
} }
file_internal_machine_api_pb_cluster_proto_msgTypes[8].Exporter = func(v any, i int) any { file_internal_machine_api_pb_cluster_proto_msgTypes[8].Exporter = func(v any, i int) any {
switch v := v.(*ReserveDomainRequest); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_internal_machine_api_pb_cluster_proto_msgTypes[9].Exporter = func(v any, i int) any {
switch v := v.(*CreateDomainRecordsRequest); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_internal_machine_api_pb_cluster_proto_msgTypes[10].Exporter = func(v any, i int) any {
switch v := v.(*CreateDomainRecordsResponse); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_internal_machine_api_pb_cluster_proto_msgTypes[11].Exporter = func(v any, i int) any {
switch v := v.(*DNSRecord); i { switch v := v.(*DNSRecord); i {
case 0: case 0:
return &v.state return &v.state
@@ -862,13 +1104,14 @@ func file_internal_machine_api_pb_cluster_proto_init() {
} }
} }
} }
file_internal_machine_api_pb_cluster_proto_msgTypes[4].OneofWrappers = []any{}
type x struct{} type x struct{}
out := protoimpl.TypeBuilder{ out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{ File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(), GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_internal_machine_api_pb_cluster_proto_rawDesc, RawDescriptor: file_internal_machine_api_pb_cluster_proto_rawDesc,
NumEnums: 2, NumEnums: 2,
NumMessages: 9, NumMessages: 12,
NumExtensions: 0, NumExtensions: 0,
NumServices: 1, NumServices: 1,
}, },
+20
View File
@@ -11,6 +11,8 @@ import "internal/machine/api/pb/machine.proto";
service Cluster { service Cluster {
rpc AddMachine(AddMachineRequest) returns (AddMachineResponse); rpc AddMachine(AddMachineRequest) returns (AddMachineResponse);
rpc ListMachines(google.protobuf.Empty) returns (ListMachinesResponse); rpc ListMachines(google.protobuf.Empty) returns (ListMachinesResponse);
rpc UpdateMachine(UpdateMachineRequest) returns (UpdateMachineResponse);
rpc RemoveMachine(RemoveMachineRequest) returns (google.protobuf.Empty);
rpc ReserveDomain(ReserveDomainRequest) returns (Domain); rpc ReserveDomain(ReserveDomainRequest) returns (Domain);
rpc GetDomain(google.protobuf.Empty) returns (Domain); rpc GetDomain(google.protobuf.Empty) returns (Domain);
@@ -49,6 +51,24 @@ message ListMachinesResponse {
repeated MachineMember machines = 1; repeated MachineMember machines = 1;
} }
message UpdateMachineRequest {
// Machine to update
string machine_id = 1;
// Updated machine information
optional string name = 2;
optional IP public_ip = 3;
repeated IPPort endpoints = 4;
}
message UpdateMachineResponse {
MachineInfo machine = 1;
}
message RemoveMachineRequest {
string id = 1;
}
message Domain { message Domain {
string name = 1; string name = 1;
} }
@@ -22,6 +22,8 @@ const _ = grpc.SupportPackageIsVersion9
const ( const (
Cluster_AddMachine_FullMethodName = "/api.Cluster/AddMachine" Cluster_AddMachine_FullMethodName = "/api.Cluster/AddMachine"
Cluster_ListMachines_FullMethodName = "/api.Cluster/ListMachines" Cluster_ListMachines_FullMethodName = "/api.Cluster/ListMachines"
Cluster_UpdateMachine_FullMethodName = "/api.Cluster/UpdateMachine"
Cluster_RemoveMachine_FullMethodName = "/api.Cluster/RemoveMachine"
Cluster_ReserveDomain_FullMethodName = "/api.Cluster/ReserveDomain" Cluster_ReserveDomain_FullMethodName = "/api.Cluster/ReserveDomain"
Cluster_GetDomain_FullMethodName = "/api.Cluster/GetDomain" Cluster_GetDomain_FullMethodName = "/api.Cluster/GetDomain"
Cluster_ReleaseDomain_FullMethodName = "/api.Cluster/ReleaseDomain" Cluster_ReleaseDomain_FullMethodName = "/api.Cluster/ReleaseDomain"
@@ -34,6 +36,8 @@ const (
type ClusterClient interface { type ClusterClient interface {
AddMachine(ctx context.Context, in *AddMachineRequest, opts ...grpc.CallOption) (*AddMachineResponse, error) AddMachine(ctx context.Context, in *AddMachineRequest, opts ...grpc.CallOption) (*AddMachineResponse, error)
ListMachines(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*ListMachinesResponse, error) ListMachines(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*ListMachinesResponse, error)
UpdateMachine(ctx context.Context, in *UpdateMachineRequest, opts ...grpc.CallOption) (*UpdateMachineResponse, error)
RemoveMachine(ctx context.Context, in *RemoveMachineRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
ReserveDomain(ctx context.Context, in *ReserveDomainRequest, opts ...grpc.CallOption) (*Domain, error) ReserveDomain(ctx context.Context, in *ReserveDomainRequest, opts ...grpc.CallOption) (*Domain, error)
GetDomain(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*Domain, error) GetDomain(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*Domain, error)
ReleaseDomain(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*Domain, error) ReleaseDomain(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*Domain, error)
@@ -68,6 +72,26 @@ func (c *clusterClient) ListMachines(ctx context.Context, in *emptypb.Empty, opt
return out, nil return out, nil
} }
func (c *clusterClient) UpdateMachine(ctx context.Context, in *UpdateMachineRequest, opts ...grpc.CallOption) (*UpdateMachineResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(UpdateMachineResponse)
err := c.cc.Invoke(ctx, Cluster_UpdateMachine_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *clusterClient) RemoveMachine(ctx context.Context, in *RemoveMachineRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(emptypb.Empty)
err := c.cc.Invoke(ctx, Cluster_RemoveMachine_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *clusterClient) ReserveDomain(ctx context.Context, in *ReserveDomainRequest, opts ...grpc.CallOption) (*Domain, error) { func (c *clusterClient) ReserveDomain(ctx context.Context, in *ReserveDomainRequest, opts ...grpc.CallOption) (*Domain, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...) cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(Domain) out := new(Domain)
@@ -114,6 +138,8 @@ func (c *clusterClient) CreateDomainRecords(ctx context.Context, in *CreateDomai
type ClusterServer interface { type ClusterServer interface {
AddMachine(context.Context, *AddMachineRequest) (*AddMachineResponse, error) AddMachine(context.Context, *AddMachineRequest) (*AddMachineResponse, error)
ListMachines(context.Context, *emptypb.Empty) (*ListMachinesResponse, error) ListMachines(context.Context, *emptypb.Empty) (*ListMachinesResponse, error)
UpdateMachine(context.Context, *UpdateMachineRequest) (*UpdateMachineResponse, error)
RemoveMachine(context.Context, *RemoveMachineRequest) (*emptypb.Empty, error)
ReserveDomain(context.Context, *ReserveDomainRequest) (*Domain, error) ReserveDomain(context.Context, *ReserveDomainRequest) (*Domain, error)
GetDomain(context.Context, *emptypb.Empty) (*Domain, error) GetDomain(context.Context, *emptypb.Empty) (*Domain, error)
ReleaseDomain(context.Context, *emptypb.Empty) (*Domain, error) ReleaseDomain(context.Context, *emptypb.Empty) (*Domain, error)
@@ -134,6 +160,12 @@ func (UnimplementedClusterServer) AddMachine(context.Context, *AddMachineRequest
func (UnimplementedClusterServer) ListMachines(context.Context, *emptypb.Empty) (*ListMachinesResponse, error) { func (UnimplementedClusterServer) ListMachines(context.Context, *emptypb.Empty) (*ListMachinesResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method ListMachines not implemented") return nil, status.Errorf(codes.Unimplemented, "method ListMachines not implemented")
} }
func (UnimplementedClusterServer) UpdateMachine(context.Context, *UpdateMachineRequest) (*UpdateMachineResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method UpdateMachine not implemented")
}
func (UnimplementedClusterServer) RemoveMachine(context.Context, *RemoveMachineRequest) (*emptypb.Empty, error) {
return nil, status.Errorf(codes.Unimplemented, "method RemoveMachine not implemented")
}
func (UnimplementedClusterServer) ReserveDomain(context.Context, *ReserveDomainRequest) (*Domain, error) { func (UnimplementedClusterServer) ReserveDomain(context.Context, *ReserveDomainRequest) (*Domain, error) {
return nil, status.Errorf(codes.Unimplemented, "method ReserveDomain not implemented") return nil, status.Errorf(codes.Unimplemented, "method ReserveDomain not implemented")
} }
@@ -203,6 +235,42 @@ func _Cluster_ListMachines_Handler(srv interface{}, ctx context.Context, dec fun
return interceptor(ctx, in, info, handler) return interceptor(ctx, in, info, handler)
} }
func _Cluster_UpdateMachine_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(UpdateMachineRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ClusterServer).UpdateMachine(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Cluster_UpdateMachine_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ClusterServer).UpdateMachine(ctx, req.(*UpdateMachineRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Cluster_RemoveMachine_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(RemoveMachineRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ClusterServer).RemoveMachine(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Cluster_RemoveMachine_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ClusterServer).RemoveMachine(ctx, req.(*RemoveMachineRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Cluster_ReserveDomain_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { func _Cluster_ReserveDomain_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ReserveDomainRequest) in := new(ReserveDomainRequest)
if err := dec(in); err != nil { if err := dec(in); err != nil {
@@ -290,6 +358,14 @@ var Cluster_ServiceDesc = grpc.ServiceDesc{
MethodName: "ListMachines", MethodName: "ListMachines",
Handler: _Cluster_ListMachines_Handler, Handler: _Cluster_ListMachines_Handler,
}, },
{
MethodName: "UpdateMachine",
Handler: _Cluster_UpdateMachine_Handler,
},
{
MethodName: "RemoveMachine",
Handler: _Cluster_RemoveMachine_Handler,
},
{ {
MethodName: "ReserveDomain", MethodName: "ReserveDomain",
Handler: _Cluster_ReserveDomain_Handler, Handler: _Cluster_ReserveDomain_Handler,
+1 -1
View File
@@ -14,7 +14,7 @@ service Machine {
rpc JoinCluster(JoinClusterRequest) returns (google.protobuf.Empty); rpc JoinCluster(JoinClusterRequest) returns (google.protobuf.Empty);
rpc Token(google.protobuf.Empty) returns (TokenResponse); rpc Token(google.protobuf.Empty) returns (TokenResponse);
rpc Inspect(google.protobuf.Empty) returns (MachineInfo); rpc Inspect(google.protobuf.Empty) returns (MachineInfo);
// Reset restores the machine to a clean state, removing all cluster-related сonfiguration and data. // Reset restores the machine to a clean state, removing all cluster-related configuration and data.
rpc Reset(ResetRequest) returns (google.protobuf.Empty); rpc Reset(ResetRequest) returns (google.protobuf.Empty);
rpc InspectService(InspectServiceRequest) returns (InspectServiceResponse); rpc InspectService(InspectServiceRequest) returns (InspectServiceResponse);
+2 -2
View File
@@ -39,7 +39,7 @@ type MachineClient interface {
JoinCluster(ctx context.Context, in *JoinClusterRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) JoinCluster(ctx context.Context, in *JoinClusterRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
Token(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*TokenResponse, error) Token(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*TokenResponse, error)
Inspect(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*MachineInfo, error) Inspect(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*MachineInfo, error)
// Reset restores the machine to a clean state, removing all cluster-related сonfiguration and data. // Reset restores the machine to a clean state, removing all cluster-related configuration and data.
Reset(ctx context.Context, in *ResetRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) Reset(ctx context.Context, in *ResetRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
InspectService(ctx context.Context, in *InspectServiceRequest, opts ...grpc.CallOption) (*InspectServiceResponse, error) InspectService(ctx context.Context, in *InspectServiceRequest, opts ...grpc.CallOption) (*InspectServiceResponse, error)
} }
@@ -132,7 +132,7 @@ type MachineServer interface {
JoinCluster(context.Context, *JoinClusterRequest) (*emptypb.Empty, error) JoinCluster(context.Context, *JoinClusterRequest) (*emptypb.Empty, error)
Token(context.Context, *emptypb.Empty) (*TokenResponse, error) Token(context.Context, *emptypb.Empty) (*TokenResponse, error)
Inspect(context.Context, *emptypb.Empty) (*MachineInfo, error) Inspect(context.Context, *emptypb.Empty) (*MachineInfo, error)
// Reset restores the machine to a clean state, removing all cluster-related сonfiguration and data. // Reset restores the machine to a clean state, removing all cluster-related configuration and data.
Reset(context.Context, *ResetRequest) (*emptypb.Empty, error) Reset(context.Context, *ResetRequest) (*emptypb.Empty, error)
InspectService(context.Context, *InspectServiceRequest) (*InspectServiceResponse, error) InspectService(context.Context, *InspectServiceRequest) (*InspectServiceResponse, error)
mustEmbedUnimplementedMachineServer() mustEmbedUnimplementedMachineServer()
+2 -1
View File
@@ -2,10 +2,11 @@ package proxy
import ( import (
"fmt" "fmt"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"google.golang.org/grpc/status" "google.golang.org/grpc/status"
"google.golang.org/protobuf/encoding/protowire" "google.golang.org/protobuf/encoding/protowire"
"google.golang.org/protobuf/proto" "google.golang.org/protobuf/proto"
"github.com/psviderski/uncloud/internal/machine/api/pb"
) )
// One2ManyResponder converts upstream responses into messages from upstreams, so that multiple // One2ManyResponder converts upstream responses into messages from upstreams, so that multiple
+2 -1
View File
@@ -2,11 +2,12 @@ package proxy
import ( import (
"context" "context"
"sync"
"github.com/siderolabs/grpc-proxy/proxy" "github.com/siderolabs/grpc-proxy/proxy"
"google.golang.org/grpc/codes" "google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata" "google.golang.org/grpc/metadata"
"google.golang.org/grpc/status" "google.golang.org/grpc/status"
"sync"
) )
// Director manages routing of gRPC requests between local and remote backends. // Director manages routing of gRPC requests between local and remote backends.
+2 -1
View File
@@ -2,11 +2,12 @@ package proxy
import ( import (
"context" "context"
"sync"
"github.com/siderolabs/grpc-proxy/proxy" "github.com/siderolabs/grpc-proxy/proxy"
"google.golang.org/grpc" "google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure" "google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/metadata" "google.golang.org/grpc/metadata"
"sync"
) )
// LocalBackend is a proxy.One2ManyResponder implementation that proxies to a local gRPC server listening on a Unix socket. // LocalBackend is a proxy.One2ManyResponder implementation that proxies to a local gRPC server listening on a Unix socket.
+4 -3
View File
@@ -3,14 +3,15 @@ package proxy
import ( import (
"context" "context"
"fmt" "fmt"
"net/netip"
"sync"
"time"
"github.com/siderolabs/grpc-proxy/proxy" "github.com/siderolabs/grpc-proxy/proxy"
"google.golang.org/grpc" "google.golang.org/grpc"
"google.golang.org/grpc/backoff" "google.golang.org/grpc/backoff"
"google.golang.org/grpc/credentials/insecure" "google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/metadata" "google.golang.org/grpc/metadata"
"net/netip"
"sync"
"time"
) )
// RemoteBackend is a proxy.One2ManyResponder implementation that proxies to a remote gRPC server, injecting machine metadata // RemoteBackend is a proxy.One2ManyResponder implementation that proxies to a remote gRPC server, injecting machine metadata
+331
View File
@@ -0,0 +1,331 @@
package caddyconfig
import (
"bytes"
"cmp"
"context"
"fmt"
"log/slog"
"maps"
"net"
"slices"
"strconv"
"strings"
"text/template"
"github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/uncloud/pkg/api"
)
const (
caddyfileHeader = `# This file is autogenerated by Uncloud based on the configuration of running services.
# Do not edit manually. Any manual changes will be overwritten on the next update.
`
caddyfileTemplate = `# Health check endpoint to verify Caddy reachability on this machine.
http:// {
handle {{.VerifyPath}} {
respond "{{.VerifyResponse}}" 200
}
log
}
(common_proxy) {
# Retry failed requests up to lb_retries times against other available upstreams.
lb_retries 3
# Upstreams are marked unhealthy for fail_duration after a failed request (passive health checking).
fail_duration 30s
}
{{- if or .HTTPHostUpstreams .HTTPSHostUpstreams }}
# Sites generated from service ports.{{end}}
{{- range $hostname, $upstreams := .HTTPHostUpstreams}}
http://{{$hostname}} {
reverse_proxy {{join $upstreams " "}} {
import common_proxy
}
log
}{{end}}
{{- range $hostname, $upstreams := .HTTPSHostUpstreams}}
https://{{$hostname}} {
reverse_proxy {{join $upstreams " "}} {
import common_proxy
}
log
}{{end}}
`
)
// CaddyfileGenerator generates a Caddyfile configuration for the Caddy reverse proxy.
type CaddyfileGenerator struct {
// machineID is the unique identifier of the machine where the controller is running.
machineID string
validator CaddyfileValidator
log *slog.Logger
}
// CaddyfileValidator is an interface for validating Caddyfile configurations.
type CaddyfileValidator interface {
Validate(ctx context.Context, caddyfile string) error
}
func NewCaddyfileGenerator(machineID string, validator CaddyfileValidator, log *slog.Logger) *CaddyfileGenerator {
if log == nil {
log = slog.Default()
}
return &CaddyfileGenerator{
machineID: machineID,
validator: validator,
log: log,
}
}
// Generate creates a Caddyfile configuration based on the provided service containers.
// The Caddyfile is generated from the service ports of the healthy containers.
// If a 'caddy' service container is running on this machine and defines a custom Caddy config (x-caddy) in its service
// spec, it will be validated and prepended to the generated Caddyfile. Custom Caddy configs (x-caddy) defined in other
// service specs are validated and appended to the generated Caddyfile. Invalid configs are logged and skipped to ensure
// the generated Caddyfile remains valid.
//
// The final Caddyfile structure includes:
//
// [caddy x-caddy (global config)]
// [generated Caddyfile from all service ports]
// [service-a x-caddy]
// ...
// [service-z x-caddy]
func (g *CaddyfileGenerator) Generate(ctx context.Context, records []store.ContainerRecord) (string, error) {
containers := make([]api.ServiceContainer, len(records))
for i, cr := range records {
containers[i] = cr.Container
}
// Sort containers by service name and creation time to generate a stable Caddyfile.
slices.SortStableFunc(containers, func(a, b api.ServiceContainer) int {
return cmp.Or(
strings.Compare(a.ServiceName(), b.ServiceName()),
a.CreatedTime().Compare(b.CreatedTime()),
)
})
caddyfile, err := g.generateBaseFromPorts(containers)
if err != nil {
return "", fmt.Errorf("generate base Caddyfile from service ports: %w", err)
}
upstreams := serviceUpstreams(containers)
// Track validation errors for reporting.
var configErrors []string
// Find the 'caddy' service container on this machine. Use the most recent one if multiple exist.
var caddyCtr *api.ServiceContainer
for _, cr := range records {
if cr.MachineID == g.machineID && cr.Container.ServiceName() == CaddyServiceName &&
(caddyCtr == nil || cr.Container.CreatedTime().Compare(caddyCtr.CreatedTime()) > 0) {
caddyCtr = &cr.Container
}
}
// If the caddy container is running on this machine and has a custom Caddy config (global),
// prepend it to the generated Caddyfile and validate it.
if caddyCtr != nil && caddyCtr.ServiceSpec.CaddyConfig() != "" {
// Render the custom global Caddy config as a Go template with the upstreams.
tmplCtx := templateContext{
Name: caddyCtr.ServiceName(),
Upstreams: upstreams,
}
renderedConfig, err := renderCaddyfile(tmplCtx, caddyCtr.ServiceSpec.CaddyConfig())
if err != nil {
g.log.Error("Failed to render template directives in user-defined global Caddy config, skipping it.",
"service", caddyCtr.ServiceName(), "container", caddyCtr.ID, "err", err)
configErrors = append(configErrors,
fmt.Sprintf("service '%s': failed to render template: %v", caddyCtr.ServiceName(), err))
} else {
caddyfileCandidate := fmt.Sprintf("# User-defined global config from service '%s'.\n%s\n\n%s",
caddyCtr.ServiceName(), renderedConfig, caddyfile)
if err = g.validator.Validate(ctx, caddyfileCandidate); err != nil {
g.log.Error("User-defined global Caddy config is invalid, skipping it.",
"service", caddyCtr.ServiceName(), "container", caddyCtr.ID, "err", err)
configErrors = append(configErrors,
fmt.Sprintf("service '%s': validation failed: %v", caddyCtr.ServiceName(), err))
} else {
caddyfile = caddyfileCandidate
}
}
}
// There could be multiple service containers for the same service with different custom Caddy configs, for example,
// if the service has been partially updated. The most recent container for each service defines the current custom
// Caddy config for that service.
latestServiceContainers := make(map[string]api.ServiceContainer, len(containers))
for _, ctr := range containers {
if latest, ok := latestServiceContainers[ctr.ServiceName()]; ok {
if ctr.CreatedTime().Compare(latest.CreatedTime()) > 0 {
latestServiceContainers[ctr.ServiceName()] = ctr
}
} else {
latestServiceContainers[ctr.ServiceName()] = ctr
}
}
sortedServiceNames := slices.Sorted(maps.Keys(latestServiceContainers))
// Append a custom Caddy config for each service to the Caddyfile and validate it. If the config for a service
// is invalid, skip it but continue processing other services to ensure the Caddyfile remains valid.
for _, serviceName := range sortedServiceNames {
// Skip the caddy container as we already processed it.
if serviceName == CaddyServiceName {
continue
}
ctr := latestServiceContainers[serviceName]
if ctr.ServiceSpec.CaddyConfig() == "" {
continue
}
// Render the template actions in the service's Caddy config.
tmplCtx := templateContext{
Name: serviceName,
Upstreams: upstreams,
}
renderedConfig, err := renderCaddyfile(tmplCtx, ctr.ServiceSpec.CaddyConfig())
if err != nil {
g.log.Error("Failed to render template directives in user-defined Caddy config for service, skipping it.",
"service", serviceName, "err", err)
configErrors = append(configErrors,
fmt.Sprintf("service '%s': failed to render template: %v", serviceName, err))
continue
}
caddyfileCandidate := fmt.Sprintf("%s\n# User-defined config for service '%s'.\n%s\n",
caddyfile, serviceName, renderedConfig)
if err = g.validator.Validate(ctx, caddyfileCandidate); err != nil {
g.log.Error("User-defined Caddy config for service is invalid, skipping it.",
"service", serviceName, "err", err)
configErrors = append(configErrors, fmt.Sprintf("service '%s': validation failed: %v", serviceName, err))
} else {
caddyfile = caddyfileCandidate
}
}
// Append error summary as comment if there were any invalid configs.
if len(configErrors) > 0 {
errorsComment := "# Skipped invalid user-defined configs:\n"
for _, e := range configErrors {
errorsComment += fmt.Sprintf("# - %s\n", e)
}
caddyfile += "\n" + errorsComment
}
return caddyfileHeader + "\n" + caddyfile, nil
}
func (g *CaddyfileGenerator) generateBaseFromPorts(containers []api.ServiceContainer) (string, error) {
httpHostUpstreams, httpsHostUpstreams := httpUpstreamsFromPorts(containers)
funcs := template.FuncMap{"join": strings.Join}
tmpl, err := template.New("Caddyfile").Funcs(funcs).Parse(caddyfileTemplate)
if err != nil {
return "", fmt.Errorf("parse Caddyfile template: %w", err)
}
data := struct {
VerifyPath string
VerifyResponse string
HTTPHostUpstreams map[string][]string
HTTPSHostUpstreams map[string][]string
}{
VerifyPath: VerifyPath,
VerifyResponse: g.machineID,
HTTPHostUpstreams: httpHostUpstreams,
HTTPSHostUpstreams: httpsHostUpstreams,
}
var buf bytes.Buffer
if err = tmpl.Execute(&buf, data); err != nil {
return "", fmt.Errorf("execute Caddyfile template: %w", err)
}
return buf.String(), nil
}
// httpUpstreamsFromPorts extracts upstreams for HTTP and HTTPS protocols from the published ports of the provided
// service containers. It's expected that all containers are healthy.
func httpUpstreamsFromPorts(containers []api.ServiceContainer) (map[string][]string, map[string][]string) {
// Maps hostnames to lists of upstreams (container IP:port pairs).
httpHostUpstreams := make(map[string][]string)
httpsHostUpstreams := make(map[string][]string)
for _, ctr := range containers {
ip := ctr.UncloudNetworkIP()
if !ip.IsValid() {
// Container is not connected to the uncloud Docker network (could be host network).
continue
}
log := slog.With("container", ctr.ID)
ports, err := ctr.ServicePorts()
if err != nil {
log.Error("Failed to parse service ports for container.", "err", err)
continue
}
for _, port := range ports {
if port.Mode != api.PortModeIngress {
continue
}
switch port.Protocol {
case api.ProtocolHTTP:
upstream := net.JoinHostPort(ip.String(), strconv.Itoa(int(port.ContainerPort)))
httpHostUpstreams[port.Hostname] = append(httpHostUpstreams[port.Hostname], upstream)
case api.ProtocolHTTPS:
upstream := net.JoinHostPort(ip.String(), strconv.Itoa(int(port.ContainerPort)))
httpsHostUpstreams[port.Hostname] = append(httpsHostUpstreams[port.Hostname], upstream)
default:
// TODO: implement L4 ingress routing for TCP and UDP.
log.Error("Unsupported protocol for ingress port.", "port", port)
continue
}
}
}
return httpHostUpstreams, httpsHostUpstreams
}
// serviceUpstreams creates a map of service names to their container IPs.
// Only includes containers connected to the uncloud Docker network.
func serviceUpstreams(containers []api.ServiceContainer) map[string][]string {
upstreams := make(map[string][]string)
for _, ctr := range containers {
ip := ctr.UncloudNetworkIP()
if !ip.IsValid() {
// Container is not connected to the uncloud Docker network (could be host network).
continue
}
serviceName := ctr.ServiceName()
upstreams[serviceName] = append(upstreams[serviceName], ip.String())
}
return upstreams
}
// renderCaddyfile renders a Caddyfile template with the upstreams function and data.
func renderCaddyfile(tmplCtx templateContext, caddyfile string) (string, error) {
funcs := template.FuncMap{
"upstreams": upstreamsTemplateFn(tmplCtx),
}
tmpl, err := template.New("Caddyfile").Funcs(funcs).Parse(caddyfile)
if err != nil {
return "", fmt.Errorf("parse config as Go template: %w", err)
}
var buf bytes.Buffer
if err = tmpl.Execute(&buf, tmplCtx); err != nil {
return "", fmt.Errorf("execute template: %w", err)
}
return buf.String(), nil
}
@@ -0,0 +1,933 @@
package caddyconfig
import (
"context"
"errors"
"strings"
"testing"
"time"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/network"
"github.com/psviderski/uncloud/internal/machine/docker"
"github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/uncloud/pkg/api"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
)
func TestCaddyfileGenerator(t *testing.T) {
caddyfileHeader := `# This file is autogenerated by Uncloud based on the configuration of running services.
# Do not edit manually. Any manual changes will be overwritten on the next update.
# Health check endpoint to verify Caddy reachability on this machine.
http:// {
handle /.uncloud-verify {
respond "test-machine-id" 200
}
log
}
(common_proxy) {
# Retry failed requests up to lb_retries times against other available upstreams.
lb_retries 3
# Upstreams are marked unhealthy for fail_duration after a failed request (passive health checking).
fail_duration 30s
}
`
tests := []struct {
name string
containers []store.ContainerRecord
want string
wantErr bool
}{
{
name: "empty containers",
containers: []store.ContainerRecord{},
want: caddyfileHeader,
},
{
name: "HTTP container",
containers: []store.ContainerRecord{
newContainerRecord(newContainer("10.210.0.2", "app.example.com:8080/http"), "mach1"),
},
want: caddyfileHeader + `
# Sites generated from service ports.
http://app.example.com {
reverse_proxy 10.210.0.2:8080 {
import common_proxy
}
log
}
`,
},
{
name: "load balancing multiple containers",
containers: []store.ContainerRecord{
newContainerRecord(newContainer("10.210.0.2", "app.example.com:8080/http"), "mach1"),
newContainerRecord(newContainer("10.210.0.3", "app.example.com:8080/http"), "mach1"),
},
want: caddyfileHeader + `
# Sites generated from service ports.
http://app.example.com {
reverse_proxy 10.210.0.2:8080 10.210.0.3:8080 {
import common_proxy
}
log
}
`,
},
{
name: "HTTPS container",
containers: []store.ContainerRecord{
newContainerRecord(newContainer("10.210.0.2", "secure.example.com:8000/https"), "mach1"),
},
want: caddyfileHeader + `
# Sites generated from service ports.
https://secure.example.com {
reverse_proxy 10.210.0.2:8000 {
import common_proxy
}
log
}
`,
},
{
name: "mixed HTTP and HTTPS",
containers: []store.ContainerRecord{
newContainerRecord(
newContainer("10.210.0.2",
"app.example.com:8080/http",
"web.example.com:8000/http"),
"mach1",
),
newContainerRecord(
newContainer("10.210.0.3",
"app.example.com:8080/http",
"secure.example.com:8888/https"),
"mach1",
),
newContainerRecord(
newContainer("10.210.0.4",
"web.example.com:8000/http",
"secure.example.com:8888/https"),
"mach1",
),
newContainerRecord(
newContainer("10.210.0.5",
"app.example.com:8080/http",
"web.example.com:8000/http",
"secure.example.com:8888/https"),
"mach1",
),
},
want: caddyfileHeader + `
# Sites generated from service ports.
http://app.example.com {
reverse_proxy 10.210.0.2:8080 10.210.0.3:8080 10.210.0.5:8080 {
import common_proxy
}
log
}
http://web.example.com {
reverse_proxy 10.210.0.2:8000 10.210.0.4:8000 10.210.0.5:8000 {
import common_proxy
}
log
}
https://secure.example.com {
reverse_proxy 10.210.0.3:8888 10.210.0.4:8888 10.210.0.5:8888 {
import common_proxy
}
log
}
`,
},
{
name: "container without uncloud network ignored",
containers: []store.ContainerRecord{
newContainerRecord(newContainerWithoutNetwork("ignored.example.com:8080/http"), "mach1"),
},
want: caddyfileHeader,
},
{
name: "container with invalid port ignored",
containers: []store.ContainerRecord{
newContainerRecord(newContainer("10.210.0.2", "invalid-port"), "mach1"),
},
want: caddyfileHeader,
},
{
name: "containers with unsupported protocols and host mode ignored",
containers: []store.ContainerRecord{
newContainerRecord(newContainer("10.210.0.2", "5000/tcp"), "mach1"),
newContainerRecord(newContainer("10.210.0.3", "5000/udp"), "mach1"),
newContainerRecord(newContainer("10.210.0.4", "80:8080/tcp@host"), "mach1"),
},
want: caddyfileHeader,
},
}
ctx := context.Background()
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
// Validator is not expected to be called in these tests.
generator := NewCaddyfileGenerator("test-machine-id", nil, nil)
config, err := generator.Generate(ctx, tt.containers)
if tt.wantErr {
assert.Error(t, err)
return
}
require.NoError(t, err)
assert.Equal(t, tt.want, config, "Generated Caddyfile doesn't match")
})
}
}
func TestCaddyfileGeneratorWithCustomConfigs(t *testing.T) {
caddyfileBase := `# This file is autogenerated by Uncloud based on the configuration of running services.
# Do not edit manually. Any manual changes will be overwritten on the next update.
# Health check endpoint to verify Caddy reachability on this machine.
http:// {
handle /.uncloud-verify {
respond "test-machine-id" 200
}
log
}
(common_proxy) {
# Retry failed requests up to lb_retries times against other available upstreams.
lb_retries 3
# Upstreams are marked unhealthy for fail_duration after a failed request (passive health checking).
fail_duration 30s
}
`
tests := []struct {
name string
containers []store.ContainerRecord
want string
wantErr bool
}{
{
name: "caddy service with valid global config",
containers: []store.ContainerRecord{
newContainerRecordWithCaddyConfig(
"caddy",
"10.210.0.2",
`# Global Caddy configuration
{
global directive
}`,
"test-machine-id",
time.Now(),
),
},
want: `# This file is autogenerated by Uncloud based on the configuration of running services.
# Do not edit manually. Any manual changes will be overwritten on the next update.
# User-defined global config from service 'caddy'.
# Global Caddy configuration
{
global directive
}
# Health check endpoint to verify Caddy reachability on this machine.
http:// {
handle /.uncloud-verify {
respond "test-machine-id" 200
}
log
}
(common_proxy) {
# Retry failed requests up to lb_retries times against other available upstreams.
lb_retries 3
# Upstreams are marked unhealthy for fail_duration after a failed request (passive health checking).
fail_duration 30s
}
`,
},
{
name: "regular service with valid custom config",
containers: []store.ContainerRecord{
newContainerRecordWithCaddyConfig(
"web",
"10.210.0.2",
`# Custom config for web service
web.example.com {
reverse_proxy web:3000
}`,
"test-machine-id",
time.Now(),
),
},
want: caddyfileBase + `
# User-defined config for service 'web'.
# Custom config for web service
web.example.com {
reverse_proxy web:3000
}
`,
},
{
name: "service with invalid config is skipped",
containers: []store.ContainerRecord{
newContainerRecordWithCaddyConfig(
"bad-service",
"10.210.0.2",
`# test:invalid
bad.config.com {
respond "This config is invalid"
}`,
"test-machine-id",
time.Now(),
),
},
want: caddyfileBase + `
# Skipped invalid user-defined configs:
# - service 'bad-service': validation failed: invalid config detected
`,
},
{
name: "service with invalid config template is skipped",
containers: []store.ContainerRecord{
newContainerRecordWithCaddyConfig(
"bad-template",
"10.210.0.2",
`
bad.template.com {
reverse_proxy {{upstreams
}`,
"test-machine-id",
time.Now(),
),
},
want: caddyfileBase + `
# Skipped invalid user-defined configs:
# - service 'bad-template': failed to render template: parse config as Go template: template: Caddyfile:3: unexpected "}" in operand
`,
},
{
name: "caddy service with invalid global config is skipped",
containers: []store.ContainerRecord{
newContainerRecordWithCaddyConfig(
"caddy",
"10.210.0.2",
`# test:invalid
localhost {
respond "Invalid global config"
}`,
"test-machine-id",
time.Now(),
),
},
want: caddyfileBase + `
# Skipped invalid user-defined configs:
# - service 'caddy': validation failed: invalid config detected
`,
},
{
name: "caddy service on different machine is ignored",
containers: []store.ContainerRecord{
newContainerRecordWithCaddyConfig(
"caddy",
"10.210.0.2",
`# Global config from other machine
{
global directive
}`,
"other-machine-id",
time.Now(),
),
},
want: caddyfileBase,
},
{
name: "multiple services with mixed valid and invalid configs",
containers: []store.ContainerRecord{
newContainerRecordWithCaddyConfig(
"api",
"10.210.0.2",
`api.example.com {
reverse_proxy api:8080
}`,
"test-machine-id",
time.Now(),
),
newContainerRecordWithCaddyConfig(
"invalid-svc",
"10.210.0.3",
`# test:invalid
bad.example.com {
respond "Invalid"
}`,
"test-machine-id",
time.Now(),
),
newContainerRecordWithCaddyConfig(
"web",
"10.210.0.4",
`web.example.com {
reverse_proxy web:3000
}`,
"test-machine-id",
time.Now(),
),
},
want: caddyfileBase + `
# User-defined config for service 'api'.
api.example.com {
reverse_proxy api:8080
}
# User-defined config for service 'web'.
web.example.com {
reverse_proxy web:3000
}
# Skipped invalid user-defined configs:
# - service 'invalid-svc': validation failed: invalid config detected
`,
},
{
name: "combined: caddy global config + service configs + ports",
containers: []store.ContainerRecord{
newContainerRecordWithCaddyConfig(
"caddy",
"10.210.0.1",
`# Global config
{
global directive
}`,
"test-machine-id",
time.Now(),
),
newContainerRecordWithPorts(
"app",
"10.210.0.2",
[]string{"app.example.com:8080/http"},
"test-machine-id",
),
newContainerRecordWithCaddyConfig(
"api",
"10.210.0.3",
`api.example.com {
reverse_proxy api:8000
}`,
"other-machine-id",
time.Now(),
),
},
want: `# This file is autogenerated by Uncloud based on the configuration of running services.
# Do not edit manually. Any manual changes will be overwritten on the next update.
# User-defined global config from service 'caddy'.
# Global config
{
global directive
}
# Health check endpoint to verify Caddy reachability on this machine.
http:// {
handle /.uncloud-verify {
respond "test-machine-id" 200
}
log
}
(common_proxy) {
# Retry failed requests up to lb_retries times against other available upstreams.
lb_retries 3
# Upstreams are marked unhealthy for fail_duration after a failed request (passive health checking).
fail_duration 30s
}
# Sites generated from service ports.
http://app.example.com {
reverse_proxy 10.210.0.2:8080 {
import common_proxy
}
log
}
# User-defined config for service 'api'.
api.example.com {
reverse_proxy api:8000
}
`,
},
{
name: "service with template directives using upstreams",
containers: []store.ContainerRecord{
newContainerRecordWithCaddyConfig(
"web",
"10.210.0.2",
`web.example.com {
reverse_proxy {{upstreams}}
}`,
"test-machine-id",
time.Now(),
),
newContainerRecordWithPorts(
"api",
"10.210.0.3",
[]string{"api.example.com:8080/http"},
"test-machine-id",
),
},
want: caddyfileBase + `
# Sites generated from service ports.
http://api.example.com {
reverse_proxy 10.210.0.3:8080 {
import common_proxy
}
log
}
# User-defined config for service 'web'.
web.example.com {
reverse_proxy 10.210.0.2
}
`,
},
{
name: "only most recent container config is used per service",
containers: []store.ContainerRecord{
newContainerRecordWithCaddyConfig(
"web",
"10.210.0.2",
`# Old config
old.example.com {
respond "Old"
}`,
"test-machine-id",
time.Now().Add(-1*time.Hour),
),
newContainerRecordWithCaddyConfig(
"web",
"10.210.0.3",
`# New config
new.example.com {
respond "New"
}`,
"test-machine-id",
time.Now(),
),
},
want: caddyfileBase + `
# User-defined config for service 'web'.
# New config
new.example.com {
respond "New"
}
`,
},
{
name: "compound test: upstreams variants, global caddy, and multi-machine services",
containers: []store.ContainerRecord{
// Global Caddy service on test-machine-id
newContainerRecordWithCaddyConfig(
"caddy",
"10.210.1.1",
`# Global config from test machine
{
admin off
}
localhost:8080 {
respond "Admin panel"
}`,
"test-machine-id",
time.Now(),
),
// Another caddy on different machine (should be ignored)
newContainerRecordWithCaddyConfig(
"caddy",
"10.210.2.1",
`# Should be ignored
{
debug
}`,
"machine-2",
time.Now(),
),
// API service containers across different machines
newContainerRecordWithPorts("api", "10.210.1.2", []string{"api.example.com:8080/http"},
"test-machine-id"),
newContainerRecordWithPorts("api", "10.210.2.2", []string{"api.example.com:8080/http"}, "machine-2"),
newContainerRecordWithPorts("api", "10.210.3.2", []string{"api.example.com:8080/http"}, "machine-3"),
// Web service with different versions on different machines
newContainerRecordWithCaddyConfig(
"web",
"10.210.1.3",
`# Web service config v1 (older)
web-v1.example.com {
reverse_proxy web:3000
}`,
"test-machine-id",
time.Now().Add(-2*time.Hour),
),
newContainerRecordWithPorts("web", "10.210.3.3", []string{"web.example.com:3000/http"}, "machine-3"),
newContainerRecordWithCaddyConfig(
"web",
"10.210.2.3",
`# Web service config v2 (most recent)
web-v2.example.com {
reverse_proxy {{upstreams 8080}}
}`,
"machine-2",
time.Now().Add(1*time.Second),
),
// DB service with custom config
newContainerRecordWithCaddyConfig(
"db",
"10.210.1.4",
`# DB admin panel
dbadmin.example.com {
basicauth {
admin $2a$14$Zkx19XLiW6VYouLHR5NmfOFU0z2GTNmpkT/5qqR7hx4IjWJPDhjvG
}
reverse_proxy {{upstreams 5432}}
}`,
"test-machine-id",
time.Now(),
),
// Gateway service with various upstream template usages
newContainerRecordWithCaddyConfig(
"gateway",
"10.210.1.5",
`# Testing different upstream template functions
gateway.example.com {
# Current service upstreams (gateway)
handle /self {
reverse_proxy {{upstreams}}
}
# Named service upstreams without port
handle /api {
reverse_proxy {{upstreams "api"}}
}
# Named service upstreams with port
handle /api-custom {
reverse_proxy {{upstreams "api" 9000}}
}
# Current service with name and port
handle /self-port {
reverse_proxy {{upstreams .Name 8888}}
}
# Service with mixed containers (web) and advanced template
handle /web {
reverse_proxy {{- range $ip := index .Upstreams "web"}} https://{{$ip}}{{end}}
}
# Non-existent service
handle /missing {
reverse_proxy {{upstreams "nonexistent"}}
}
}`,
"test-machine-id",
time.Now(),
),
// App service with just ports (no custom config)
newContainerRecordWithPorts("app", "10.210.1.6", []string{"app.example.com:3000/http"},
"test-machine-id"),
newContainerRecordWithPorts("app", "10.210.2.6", []string{"app.example.com:3000/http"}, "machine-2"),
// Service with invalid config (should be ignored)
newContainerRecordWithCaddyConfig(
"invalid",
"10.210.1.7",
`# test:invalid
badconfig.com {
respond "This config is invalid"
}`,
"test-machine-id",
time.Now(),
),
},
want: `# This file is autogenerated by Uncloud based on the configuration of running services.
# Do not edit manually. Any manual changes will be overwritten on the next update.
# User-defined global config from service 'caddy'.
# Global config from test machine
{
admin off
}
localhost:8080 {
respond "Admin panel"
}
# Health check endpoint to verify Caddy reachability on this machine.
http:// {
handle /.uncloud-verify {
respond "test-machine-id" 200
}
log
}
(common_proxy) {
# Retry failed requests up to lb_retries times against other available upstreams.
lb_retries 3
# Upstreams are marked unhealthy for fail_duration after a failed request (passive health checking).
fail_duration 30s
}
# Sites generated from service ports.
http://api.example.com {
reverse_proxy 10.210.1.2:8080 10.210.2.2:8080 10.210.3.2:8080 {
import common_proxy
}
log
}
http://app.example.com {
reverse_proxy 10.210.1.6:3000 10.210.2.6:3000 {
import common_proxy
}
log
}
http://web.example.com {
reverse_proxy 10.210.3.3:3000 {
import common_proxy
}
log
}
# User-defined config for service 'db'.
# DB admin panel
dbadmin.example.com {
basicauth {
admin $2a$14$Zkx19XLiW6VYouLHR5NmfOFU0z2GTNmpkT/5qqR7hx4IjWJPDhjvG
}
reverse_proxy 10.210.1.4:5432
}
# User-defined config for service 'gateway'.
# Testing different upstream template functions
gateway.example.com {
# Current service upstreams (gateway)
handle /self {
reverse_proxy 10.210.1.5
}
# Named service upstreams without port
handle /api {
reverse_proxy 10.210.1.2 10.210.2.2 10.210.3.2
}
# Named service upstreams with port
handle /api-custom {
reverse_proxy 10.210.1.2:9000 10.210.2.2:9000 10.210.3.2:9000
}
# Current service with name and port
handle /self-port {
reverse_proxy 10.210.1.5:8888
}
# Service with mixed containers (web) and advanced template
handle /web {
reverse_proxy https://10.210.1.3 https://10.210.3.3 https://10.210.2.3
}
# Non-existent service
handle /missing {
` + "\t\treverse_proxy " + `
}
}
# User-defined config for service 'web'.
# Web service config v2 (most recent)
web-v2.example.com {
reverse_proxy 10.210.1.3:8080 10.210.3.3:8080 10.210.2.3:8080
}
# Skipped invalid user-defined configs:
# - service 'invalid': validation failed: invalid config detected
`,
},
{
name: "multiple errors: invalid global, template error, and validation error",
containers: []store.ContainerRecord{
newContainerRecordWithCaddyConfig(
"caddy",
"10.210.0.1",
`# test:invalid
{
invalid global
}`,
"test-machine-id",
time.Now(),
),
newContainerRecordWithCaddyConfig(
"broken-template",
"10.210.0.2",
`broken.example.com {
reverse_proxy {{upstreams "missing
}`,
"test-machine-id",
time.Now(),
),
newContainerRecordWithCaddyConfig(
"invalid",
"10.210.0.3",
`# test:invalid
invalid.example.com {
respond "Invalid config"
}`,
"test-machine-id",
time.Now(),
),
newContainerRecordWithCaddyConfig(
"valid",
"10.210.0.4",
`valid.example.com {
respond "Valid config"
}`,
"test-machine-id",
time.Now(),
),
},
want: caddyfileBase + `
# User-defined config for service 'valid'.
valid.example.com {
respond "Valid config"
}
# Skipped invalid user-defined configs:
# - service 'caddy': validation failed: invalid config detected
# - service 'broken-template': failed to render template: parse config as Go template: template: Caddyfile:2: unterminated quoted string
# - service 'invalid': validation failed: invalid config detected
`,
},
}
ctx := context.Background()
validator := NewMockCaddyfileValidator(t)
validator.EXPECT().Validate(mock.Anything, mock.Anything).RunAndReturn(
func(ctx context.Context, caddyfile string) error {
if strings.Contains(caddyfile, "# test:invalid") {
return errors.New("invalid config detected")
}
return nil
})
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
generator := NewCaddyfileGenerator("test-machine-id", validator, nil)
config, err := generator.Generate(ctx, tt.containers)
if tt.wantErr {
assert.Error(t, err)
return
}
require.NoError(t, err)
assert.Equal(t, tt.want, config, "Generated Caddyfile doesn't match")
})
}
}
func newContainerRecord(ctr api.ServiceContainer, machineID string) store.ContainerRecord {
return store.ContainerRecord{
Container: ctr,
MachineID: machineID,
}
}
func newContainerRecordWithCaddyConfig(serviceName, ip, caddyConfig, machineID string, created time.Time) store.ContainerRecord {
return store.ContainerRecord{
Container: api.ServiceContainer{
Container: api.Container{
ContainerJSON: types.ContainerJSON{
ContainerJSONBase: &types.ContainerJSONBase{
ID: serviceName + "-" + ip, // Add ID for stable sorting
State: &types.ContainerState{
Running: true,
},
Created: created.UTC().Format(time.RFC3339Nano),
},
NetworkSettings: &types.NetworkSettings{
Networks: map[string]*network.EndpointSettings{
docker.NetworkName: {
IPAddress: ip,
},
},
},
Config: &container.Config{
Labels: map[string]string{
api.LabelServiceName: serviceName,
},
},
},
},
ServiceSpec: api.ServiceSpec{
Caddy: &api.CaddySpec{
Config: caddyConfig,
},
},
},
MachineID: machineID,
}
}
func newContainerRecordWithPorts(serviceName, ip string, ports []string, machineID string) store.ContainerRecord {
portsLabel := strings.Join(ports, ",")
return store.ContainerRecord{
Container: api.ServiceContainer{
Container: api.Container{
ContainerJSON: types.ContainerJSON{
ContainerJSONBase: &types.ContainerJSONBase{
ID: serviceName + "-" + ip, // Add ID for stable sorting
State: &types.ContainerState{
Running: true,
},
Created: time.Now().UTC().Format(time.RFC3339Nano),
},
NetworkSettings: &types.NetworkSettings{
Networks: map[string]*network.EndpointSettings{
docker.NetworkName: {
IPAddress: ip,
},
},
},
Config: &container.Config{
Labels: map[string]string{
api.LabelServiceName: serviceName,
api.LabelServicePorts: portsLabel,
},
},
},
},
},
MachineID: machineID,
}
}
+79 -49
View File
@@ -14,48 +14,56 @@ import (
) )
const ( const (
CaddyGroup = "uncloud" CaddyServiceName = "caddy"
VerifyPath = "/.uncloud-verify" CaddyGroup = "uncloud"
VerifyPath = "/.uncloud-verify"
) )
// Controller monitors container changes in the cluster store and generates a configuration file for Caddy reverse // Controller monitors container changes in the cluster store and generates a configuration file for Caddy reverse
// proxy. The generated configuration allows Caddy to route external traffic to service containers across the internal // proxy. The generated configuration allows Caddy to route external traffic to service containers across the internal
// network. // network.
type Controller struct { type Controller struct {
store *store.Store machineID string
path string configDir string
verifyResponse string generator *CaddyfileGenerator
store *store.Store
log *slog.Logger
} }
func NewController(store *store.Store, path string, verifyResponse string) (*Controller, error) { func NewController(machineID, configDir, adminSock string, store *store.Store) (*Controller, error) {
dir := filepath.Dir(path) if err := os.MkdirAll(configDir, 0o750); err != nil {
if err := os.MkdirAll(dir, 0750); err != nil { return nil, fmt.Errorf("create directory for Caddy configuration '%s': %w", configDir, err)
return nil, fmt.Errorf("create parent directory for Caddy configuration '%s': %w", dir, err)
} }
if err := fs.Chown(dir, "", CaddyGroup); err != nil { if err := fs.Chown(configDir, "", CaddyGroup); err != nil {
return nil, fmt.Errorf("change owner of parent directory for Caddy configuration '%s': %w", dir, err) return nil, fmt.Errorf("change owner of directory for Caddy configuration '%s': %w", configDir, err)
} }
log := slog.With("component", "caddy-controller")
validator := NewCaddyAdminValidator(adminSock)
generator := NewCaddyfileGenerator(machineID, validator, log)
return &Controller{ return &Controller{
store: store, machineID: machineID,
path: path, configDir: configDir,
verifyResponse: verifyResponse, generator: generator,
store: store,
log: log,
}, nil }, nil
} }
func (c *Controller) Run(ctx context.Context) error { func (c *Controller) Run(ctx context.Context) error {
containerRecords, changes, err := c.store.SubscribeContainers(ctx) containers, changes, err := c.store.SubscribeContainers(ctx)
if err != nil { if err != nil {
return fmt.Errorf("subscribe to container changes: %w", err) return fmt.Errorf("subscribe to container changes: %w", err)
} }
slog.Info("Subscribed to container changes in the cluster to generate Caddy configuration.") c.log.Info("Subscribed to container changes in the cluster to generate Caddy configuration.")
containers, err := c.filterAvailableContainers(containerRecords) containers = filterHealthyContainers(containers)
if err != nil { if err = c.generateCaddyfile(ctx, containers); err != nil {
return fmt.Errorf("filter available containers: %w", err) return fmt.Errorf("generate Caddyfile configuration: %w", err)
} }
if err = c.generateConfig(containers); err != nil { if err = c.generateJSONConfig(containers); err != nil {
return fmt.Errorf("generate Caddy configuration: %w", err) return fmt.Errorf("generate Caddy JSON configuration: %w", err)
} }
for { for {
@@ -64,47 +72,68 @@ func (c *Controller) Run(ctx context.Context) error {
if !ok { if !ok {
return fmt.Errorf("containers subscription failed") return fmt.Errorf("containers subscription failed")
} }
slog.Debug("Cluster containers changed, updating Caddy configuration.") c.log.Info("Cluster containers changed, updating Caddy configuration.")
containerRecords, err = c.store.ListContainers(ctx, store.ListOptions{}) containers, err = c.store.ListContainers(ctx, store.ListOptions{})
if err != nil { if err != nil {
slog.Error("Failed to list containers.", "err", err) c.log.Error("Failed to list containers.", "err", err)
continue continue
} }
containers, err = c.filterAvailableContainers(containerRecords) containers = filterHealthyContainers(containers)
if err != nil {
slog.Error("Failed to filter available containers.", "err", err) if err = c.generateCaddyfile(ctx, containers); err != nil {
continue c.log.Error("Failed to generate Caddyfile configuration.", "err", err)
} }
if err = c.generateConfig(containers); err != nil { if err = c.generateJSONConfig(containers); err != nil {
slog.Error("Failed to generate Caddy configuration.", "err", err) c.log.Error("Failed to generate Caddy JSON configuration.", "err", err)
} }
slog.Debug("Updated Caddy configuration.", "path", c.path) c.log.Info("Updated Caddy configuration.", "dir", c.configDir)
case <-ctx.Done(): case <-ctx.Done():
return nil return nil
} }
} }
} }
// filterAvailableContainers filters out containers from this machine that are likely unavailable. The availability // filterHealthyContainers filters out containers that are not healthy.
// is determined by the cluster membership state of the machine that the container is running on. // TODO: Filters out containers from this machine that are likely unavailable. The availability can be determined
// TODO: implement machine membership check using Corrossion Admin client. // by the cluster membership state of the machine that the container is running on. Implement machine membership
func (c *Controller) filterAvailableContainers( // check using Corrossion Admin client.
containerRecords []store.ContainerRecord, func filterHealthyContainers(containers []store.ContainerRecord) []store.ContainerRecord {
) ([]api.ServiceContainer, error) { healthy := make([]store.ContainerRecord, 0, len(containers))
containers := make([]api.ServiceContainer, len(containerRecords)) for _, cr := range containers {
for i, cr := range containerRecords { if cr.Container.Healthy() {
containers[i] = api.ServiceContainer{ healthy = append(healthy, cr)
Container: cr.Container,
// TODO: restore ServiceSpec from the container record once it's saved in the store.
} }
} }
return containers, nil return healthy
} }
func (c *Controller) generateConfig(containers []api.ServiceContainer) error { func (c *Controller) generateCaddyfile(ctx context.Context, containers []store.ContainerRecord) error {
config, err := GenerateConfig(containers, c.verifyResponse) caddyfile, err := c.generator.Generate(ctx, containers)
if err != nil {
return fmt.Errorf("generate Caddyfile: %w", err)
}
caddyfilePath := filepath.Join(c.configDir, "Caddyfile")
// TODO: use atomic file write to avoid partial loads on Caddy watch reload.
if err = os.WriteFile(caddyfilePath, []byte(caddyfile), 0o640); err != nil {
return fmt.Errorf("write Caddyfile to file '%s': %w", caddyfilePath, err)
}
if err = fs.Chown(caddyfilePath, "", CaddyGroup); err != nil {
return fmt.Errorf("change owner of Caddyfile '%s': %w", caddyfilePath, err)
}
return nil
}
func (c *Controller) generateJSONConfig(containers []store.ContainerRecord) error {
serviceContainers := make([]api.ServiceContainer, len(containers))
for i, cr := range containers {
serviceContainers[i] = cr.Container
}
config, err := GenerateJSONConfig(serviceContainers, c.machineID)
if err != nil { if err != nil {
return err return err
} }
@@ -113,12 +142,13 @@ func (c *Controller) generateConfig(containers []api.ServiceContainer) error {
if err != nil { if err != nil {
return fmt.Errorf("marshal Caddy configuration: %w", err) return fmt.Errorf("marshal Caddy configuration: %w", err)
} }
configPath := filepath.Join(c.configDir, "caddy.json")
if err = os.WriteFile(c.path, configBytes, 0640); err != nil { if err = os.WriteFile(configPath, configBytes, 0o640); err != nil {
return fmt.Errorf("write Caddy configuration to file '%s': %w", c.path, err) return fmt.Errorf("write Caddy configuration to file '%s': %w", configPath, err)
} }
if err = fs.Chown(c.path, "", CaddyGroup); err != nil { if err = fs.Chown(configPath, "", CaddyGroup); err != nil {
return fmt.Errorf("change owner of Caddy configuration file '%s': %w", c.path, err) return fmt.Errorf("change owner of Caddy configuration file '%s': %w", configPath, err)
} }
return nil return nil
@@ -4,9 +4,7 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"log/slog"
"maps" "maps"
"net"
"net/http" "net/http"
"slices" "slices"
"strconv" "strconv"
@@ -19,47 +17,8 @@ import (
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
) )
func GenerateConfig(containers []api.ServiceContainer, verifyResponse string) (*caddy.Config, error) { func GenerateJSONConfig(containers []api.ServiceContainer, verifyResponse string) (*caddy.Config, error) {
// Maps hostnames to lists of upstreams (container IP:port pairs). httpHostUpstreams, httpsHostUpstreams := httpUpstreamsFromPorts(containers)
httpHostUpstreams := make(map[string][]string)
httpsHostUpstreams := make(map[string][]string)
for _, ctr := range containers {
if !ctr.Healthy() {
continue
}
ip := ctr.UncloudNetworkIP()
if !ip.IsValid() {
// Container is not connected to the uncloud Docker network (could be host network).
continue
}
log := slog.With("container", ctr.ID)
ports, err := ctr.ServicePorts()
if err != nil {
log.Error("Failed to parse service ports for container.", "err", err)
continue
}
for _, port := range ports {
if port.Mode != api.PortModeIngress {
continue
}
switch port.Protocol {
case api.ProtocolHTTP:
upstream := net.JoinHostPort(ip.String(), strconv.Itoa(int(port.ContainerPort)))
httpHostUpstreams[port.Hostname] = append(httpHostUpstreams[port.Hostname], upstream)
case api.ProtocolHTTPS:
upstream := net.JoinHostPort(ip.String(), strconv.Itoa(int(port.ContainerPort)))
httpsHostUpstreams[port.Hostname] = append(httpsHostUpstreams[port.Hostname], upstream)
default:
// TODO: implement L4 ingress routing for TCP and UDP.
log.Error("Unsupported protocol for ingress port.", "port", port)
continue
}
}
}
var warnings []caddyconfig.Warning var warnings []caddyconfig.Warning
servers := make(map[string]*caddyhttp.Server) servers := make(map[string]*caddyhttp.Server)
@@ -13,7 +13,7 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
func TestGenerateConfig(t *testing.T) { func TestGenerateJSONConfig(t *testing.T) {
configWithoutServices := `{ configWithoutServices := `{
"servers": { "servers": {
"http": { "http": {
@@ -312,73 +312,11 @@ func TestGenerateConfig(t *testing.T) {
want: configWithoutServices, want: configWithoutServices,
wantErr: false, wantErr: false,
}, },
{
name: "restarting container ignored",
containers: []api.ServiceContainer{
newRestartingContainer("10.210.0.2", "app.example.com:8080/http"),
},
want: configWithoutServices,
wantErr: false,
},
{
name: "stopped container ignored",
containers: []api.ServiceContainer{
newStoppedContainer("10.210.0.2", "app.example.com:8080/http"),
},
want: configWithoutServices,
wantErr: false,
},
{
name: "mix of running, restarting, and stopped containers",
containers: []api.ServiceContainer{
newContainer("10.210.0.2", "app.example.com:8080/http"),
newRestartingContainer("10.210.0.3", "app.example.com:8080/http"),
newStoppedContainer("10.210.0.4", "app.example.com:8080/http"),
},
want: `{
"servers": {
"http": {
"listen": [":80"],
"routes": [
{
"match": [{"host": ["app.example.com"]}],
"handle": [{
"handler": "reverse_proxy",
"health_checks": {
"passive": {
"fail_duration": 30000000000
}
},
"load_balancing": {
"retries": 3
},
"upstreams": [{"dial": "10.210.0.2:8080"}]
}]
},
{
"match": [{"path": ["/.uncloud-verify"]}],
"handle": [{
"body": "verification-response-body",
"handler": "static_response",
"status_code": 200
}]
}
],
"logs": {}
},
"https": {
"listen": [":443"],
"logs": {}
}
}
}`,
wantErr: false,
},
} }
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
config, err := GenerateConfig(tt.containers, "verification-response-body") config, err := GenerateJSONConfig(tt.containers, "verification-response-body")
if tt.wantErr { if tt.wantErr {
assert.Error(t, err) assert.Error(t, err)
@@ -439,15 +377,3 @@ func newContainerWithoutNetwork(ports ...string) api.ServiceContainer {
}, },
}}} }}}
} }
func newRestartingContainer(ip string, ports ...string) api.ServiceContainer {
ctr := newContainer(ip, ports...)
ctr.Container.State.Restarting = true
return ctr
}
func newStoppedContainer(ip string, ports ...string) api.ServiceContainer {
ctr := newContainer(ip, ports...)
ctr.Container.State.Running = false
return ctr
}
@@ -0,0 +1,95 @@
// Code generated by mockery; DO NOT EDIT.
// github.com/vektra/mockery
// template: testify
package caddyconfig
import (
"context"
mock "github.com/stretchr/testify/mock"
)
// NewMockCaddyfileValidator creates a new instance of MockCaddyfileValidator. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewMockCaddyfileValidator(t interface {
mock.TestingT
Cleanup(func())
}) *MockCaddyfileValidator {
mock := &MockCaddyfileValidator{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
// MockCaddyfileValidator is an autogenerated mock type for the CaddyfileValidator type
type MockCaddyfileValidator struct {
mock.Mock
}
type MockCaddyfileValidator_Expecter struct {
mock *mock.Mock
}
func (_m *MockCaddyfileValidator) EXPECT() *MockCaddyfileValidator_Expecter {
return &MockCaddyfileValidator_Expecter{mock: &_m.Mock}
}
// Validate provides a mock function for the type MockCaddyfileValidator
func (_mock *MockCaddyfileValidator) Validate(ctx context.Context, caddyfile string) error {
ret := _mock.Called(ctx, caddyfile)
if len(ret) == 0 {
panic("no return value specified for Validate")
}
var r0 error
if returnFunc, ok := ret.Get(0).(func(context.Context, string) error); ok {
r0 = returnFunc(ctx, caddyfile)
} else {
r0 = ret.Error(0)
}
return r0
}
// MockCaddyfileValidator_Validate_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Validate'
type MockCaddyfileValidator_Validate_Call struct {
*mock.Call
}
// Validate is a helper method to define mock.On call
// - ctx context.Context
// - caddyfile string
func (_e *MockCaddyfileValidator_Expecter) Validate(ctx interface{}, caddyfile interface{}) *MockCaddyfileValidator_Validate_Call {
return &MockCaddyfileValidator_Validate_Call{Call: _e.mock.On("Validate", ctx, caddyfile)}
}
func (_c *MockCaddyfileValidator_Validate_Call) Run(run func(ctx context.Context, caddyfile string)) *MockCaddyfileValidator_Validate_Call {
_c.Call.Run(func(args mock.Arguments) {
var arg0 context.Context
if args[0] != nil {
arg0 = args[0].(context.Context)
}
var arg1 string
if args[1] != nil {
arg1 = args[1].(string)
}
run(
arg0,
arg1,
)
})
return _c
}
func (_c *MockCaddyfileValidator_Validate_Call) Return(err error) *MockCaddyfileValidator_Validate_Call {
_c.Call.Return(err)
return _c
}
func (_c *MockCaddyfileValidator_Validate_Call) RunAndReturn(run func(ctx context.Context, caddyfile string) error) *MockCaddyfileValidator_Validate_Call {
_c.Call.Return(run)
return _c
}
+39
View File
@@ -0,0 +1,39 @@
package caddyconfig
import (
"context"
"os"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb"
"google.golang.org/protobuf/types/known/timestamppb"
"github.com/psviderski/uncloud/internal/machine/api/pb"
)
// Server implements the gRPC Caddy service.
type Server struct {
pb.UnimplementedCaddyServer
service *Service
}
func NewServer(service *Service) *Server {
return &Server{service: service}
}
// GetConfig retrieves the current Caddy configuration from the machine.
func (s *Server) GetConfig(ctx context.Context, _ *emptypb.Empty) (*pb.GetCaddyConfigResponse, error) {
caddyfile, modifiedAt, err := s.service.Caddyfile()
if err != nil {
if os.IsNotExist(err) {
return nil, status.Errorf(codes.NotFound, err.Error())
}
return nil, status.Errorf(codes.Internal, err.Error())
}
return &pb.GetCaddyConfigResponse{
Caddyfile: caddyfile,
ModifiedAt: timestamppb.New(modifiedAt),
}, nil
}
+35
View File
@@ -0,0 +1,35 @@
package caddyconfig
import (
"fmt"
"os"
"path/filepath"
"time"
)
// Service provides methods to interact with the Caddy configuration on the machine.
type Service struct {
configDir string
}
// NewService creates a new Service instance with the specified Caddy configuration directory.
func NewService(configDir string) *Service {
return &Service{configDir: configDir}
}
// Caddyfile retrieves the current Caddy configuration (Caddyfile) from the machine's config directory.
func (s *Service) Caddyfile() (string, time.Time, error) {
path := filepath.Join(s.configDir, "Caddyfile")
content, err := os.ReadFile(path)
if err != nil {
return "", time.Time{}, fmt.Errorf("read Caddyfile from file '%s': %w", path, err)
}
// Get the file modification time.
fileInfo, err := os.Stat(path)
if err != nil {
return "", time.Time{}, fmt.Errorf("get Caddyfile file info '%s': %w", path, err)
}
return string(content), fileInfo.ModTime(), nil
}
+77
View File
@@ -0,0 +1,77 @@
package caddyconfig
import (
"fmt"
"net"
"strconv"
"strings"
)
// templateContext holds the data available to Caddyfile templates.
type templateContext struct {
// Name is the current service name.
Name string
// Upstreams maps service names to their container IPs.
Upstreams map[string][]string
}
// upstreamsTemplateFn returns a template function that generates a space separated string of upstreams for the service.
// It optionally accepts a service name and a port number: {{upstreams [service-name] [port]}}.
func upstreamsTemplateFn(tmplCtx templateContext) func(args ...any) (string, error) {
return func(args ...any) (string, error) {
var serviceName string
var port int
// Parse arguments.
switch len(args) {
case 0:
// Current service, default port.
serviceName = tmplCtx.Name
case 1:
// Either port (int) for current service or service name (string).
switch arg := args[0].(type) {
case int:
serviceName = tmplCtx.Name
port = arg
case string:
serviceName = arg
port = 0
default:
return "", fmt.Errorf("upstreams function: invalid argument type: %T", arg)
}
case 2:
// Service name and port.
name, ok := args[0].(string)
if !ok {
return "", fmt.Errorf("upstreams function: first argument must be service name (string)")
}
serviceName = name
p, ok := args[1].(int)
if !ok {
return "", fmt.Errorf("upstreams function: second argument must be port (int)")
}
port = p
default:
return "", fmt.Errorf("upstreams function: too many arguments; expected 0-2, got %d", len(args))
}
ips, ok := tmplCtx.Upstreams[serviceName]
if !ok || len(ips) == 0 {
// No upstreams available.
return "", nil
}
// Build the space separated upstreams string.
var upstreams []string
for _, ip := range ips {
if port > 0 {
upstreams = append(upstreams, net.JoinHostPort(ip, strconv.Itoa(port)))
} else {
upstreams = append(upstreams, ip)
}
}
return strings.Join(upstreams, " "), nil
}
}
+71
View File
@@ -0,0 +1,71 @@
package caddyconfig
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"net/http"
"strings"
"time"
"github.com/caddyserver/caddy/v2"
)
// CaddyAdminValidator validates Caddyfile via the Caddy admin API.
type CaddyAdminValidator struct {
socketPath string
client *http.Client
}
func NewCaddyAdminValidator(socketPath string) *CaddyAdminValidator {
return &CaddyAdminValidator{
socketPath: socketPath,
client: &http.Client{
Timeout: 5 * time.Second,
Transport: &http.Transport{
DialContext: func(_ context.Context, _, _ string) (net.Conn, error) {
return net.Dial("unix", socketPath)
},
},
},
}
}
// Validate checks if the provided Caddyfile can be adapted to Caddy JSON config using the running Caddy instance via
// its admin API. It doesn't guarantee that the Caddyfile is actually valid and can be loaded. For example, a tls
// directive with a missing certificate will pass the adaptation but will fail when Caddy tries to load it.
// But this is the best we can do over the admin API.
// TODO: run 'docker exec caddy-container caddy validate' to do proper validation or implement a Caddy module that
// exposes a validation endpoint.
func (c *CaddyAdminValidator) Validate(ctx context.Context, caddyfile string) error {
// Bogus host is used so that http.NewRequest is happy but it doesn't matter since we're using a Unix socket.
req, err := http.NewRequestWithContext(ctx, "POST", "http://localhost/adapt", strings.NewReader(caddyfile))
if err != nil {
return fmt.Errorf("create adapt request: %w", err)
}
req.Header.Set("Content-Type", "text/caddyfile")
resp, err := c.client.Do(req)
if err != nil {
return fmt.Errorf("send adapt request: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusOK {
return nil
}
body, _ := io.ReadAll(resp.Body)
// If the response is a 400 Bad Request, try to parse the error message from it.
if resp.StatusCode == http.StatusBadRequest {
var apiError caddy.APIError
if err = json.Unmarshal(body, &apiError); err == nil {
return errors.New(apiError.Message)
}
}
return errors.New(string(body))
}
@@ -12,9 +12,9 @@ import (
"time" "time"
"github.com/cenkalti/backoff/v4" "github.com/cenkalti/backoff/v4"
"github.com/docker/docker/client"
"github.com/psviderski/uncloud/internal/machine/api/pb" "github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/machine/caddyconfig" "github.com/psviderski/uncloud/internal/machine/caddyconfig"
"github.com/psviderski/uncloud/internal/machine/constants"
"github.com/psviderski/uncloud/internal/machine/corroservice" "github.com/psviderski/uncloud/internal/machine/corroservice"
"github.com/psviderski/uncloud/internal/machine/dns" "github.com/psviderski/uncloud/internal/machine/dns"
"github.com/psviderski/uncloud/internal/machine/docker" "github.com/psviderski/uncloud/internal/machine/docker"
@@ -25,39 +25,42 @@ import (
"google.golang.org/grpc" "google.golang.org/grpc"
) )
const ( // clusterController is the main controller for the machine that is a cluster member. It manages components such as
APIPort = 51000 // the WireGuard network, API server listening the WireGuard network, Corrosion service, Docker network and containers,
) // and others.
type clusterController struct {
type networkController struct {
state *State state *State
store *store.Store store *store.Store
wgnet *network.WireGuardNetwork wgnet *network.WireGuardNetwork
endpointChanges <-chan network.EndpointChangeEvent endpointChanges <-chan network.EndpointChangeEvent
server *grpc.Server server *grpc.Server
corroService corroservice.Service corroService corroservice.Service
dockerCli *client.Client dockerCtrl *docker.Controller
caddyfileCtrl *caddyconfig.Controller // dockerReady is signalled when Docker is configured and ready for containers.
dockerReady chan<- struct{}
caddyconfigCtrl *caddyconfig.Controller
// dnsServer is the embedded internal DNS server for the cluster listening on the machine IP. // dnsServer is the embedded internal DNS server for the cluster listening on the machine IP.
dnsServer *dns.Server dnsServer *dns.Server
dnsResolver *dns.ClusterResolver dnsResolver *dns.ClusterResolver
// stopped is a channel that is closed when the controller is stopped.
stopped chan struct{}
} }
func newNetworkController( func newClusterController(
state *State, state *State,
store *store.Store, store *store.Store,
server *grpc.Server, server *grpc.Server,
corroService corroservice.Service, corroService corroservice.Service,
dockerCli *client.Client, dockerService *docker.Service,
dockerReady chan<- struct{},
caddyfileCtrl *caddyconfig.Controller, caddyfileCtrl *caddyconfig.Controller,
dnsServer *dns.Server, dnsServer *dns.Server,
dnsResolver *dns.ClusterResolver, dnsResolver *dns.ClusterResolver,
) ( ) (*clusterController, error) {
*networkController, error,
) {
slog.Info("Starting WireGuard network.") slog.Info("Starting WireGuard network.")
wgnet, err := network.NewWireGuardNetwork() wgnet, err := network.NewWireGuardNetwork()
if err != nil { if err != nil {
@@ -65,88 +68,97 @@ func newNetworkController(
} }
endpointChanges := wgnet.WatchEndpoints() endpointChanges := wgnet.WatchEndpoints()
return &networkController{ return &clusterController{
state: state, state: state,
store: store, store: store,
wgnet: wgnet, wgnet: wgnet,
endpointChanges: endpointChanges, endpointChanges: endpointChanges,
server: server, server: server,
corroService: corroService, corroService: corroService,
dockerCli: dockerCli, dockerCtrl: docker.NewController(state.ID, dockerService, store),
caddyfileCtrl: caddyfileCtrl, dockerReady: dockerReady,
caddyconfigCtrl: caddyfileCtrl,
dnsServer: dnsServer, dnsServer: dnsServer,
dnsResolver: dnsResolver, dnsResolver: dnsResolver,
stopped: make(chan struct{}),
}, nil }, nil
} }
func (nc *networkController) Run(ctx context.Context) error { func (cc *clusterController) Run(ctx context.Context) error {
defer close(cc.stopped)
if err := firewall.ConfigureIptablesChains(); err != nil { if err := firewall.ConfigureIptablesChains(); err != nil {
return fmt.Errorf("configure iptables chains: %w", err) return fmt.Errorf("configure iptables chains: %w", err)
} }
if err := nc.wgnet.Configure(*nc.state.Network); err != nil { if err := cc.ensureDockerNetwork(ctx); err != nil {
return err
}
slog.Info("Docker network configured.")
if err := cc.wgnet.Configure(*cc.state.Network); err != nil {
return fmt.Errorf("configure WireGuard network: %w", err) return fmt.Errorf("configure WireGuard network: %w", err)
} }
slog.Info("WireGuard network configured.") slog.Info("WireGuard network configured.")
if nc.corroService.Running() { if cc.corroService.Running() {
// Corrosion service was running before the WireGuard network was configured so we need to restart it. // Corrosion service was running before the WireGuard network was configured so we need to restart it.
slog.Info("Restarting corrosion service to apply new configuration with WireGuard network.") slog.Info("Restarting corrosion service to apply new configuration with WireGuard network.")
if err := nc.corroService.Restart(ctx); err != nil { if err := cc.corroService.Restart(ctx); err != nil {
return fmt.Errorf("restart corrosion service: %w", err) return fmt.Errorf("restart corrosion service: %w", err)
} }
slog.Info("Corrosion service restarted.")
} else { } else {
slog.Info("Starting corrosion service.") slog.Info("Starting corrosion service.")
if err := nc.corroService.Start(ctx); err != nil { if err := cc.corroService.Start(ctx); err != nil {
return fmt.Errorf("start corrosion service: %w", err) return fmt.Errorf("start corrosion service: %w", err)
} }
slog.Info("Corrosion service started.")
} }
// TODO: Figure out if we need to manually stop the corrosion service when the context is done or just
// rely on systemd to handle service dependencies on its own.
errGroup, ctx := errgroup.WithContext(ctx) errGroup, ctx := errgroup.WithContext(ctx)
// Start the network API server. Assume the management IP can't be changed when the network is running. // Start the network API server. Assume the management IP can't be changed when the network is running.
apiAddr := net.JoinHostPort(nc.state.Network.ManagementIP.String(), strconv.Itoa(APIPort)) apiAddr := net.JoinHostPort(cc.state.Network.ManagementIP.String(), strconv.Itoa(constants.MachineAPIPort))
listener, err := net.Listen("tcp", apiAddr) listener, err := net.Listen("tcp", apiAddr)
if err != nil { if err != nil {
return fmt.Errorf("listen API port: %w", err) return fmt.Errorf("listen API port: %w", err)
} }
errGroup.Go( errGroup.Go(func() error {
func() error { slog.Info("Starting network API server.", "addr", apiAddr)
slog.Info("Starting network API server.", "addr", apiAddr) if err := cc.server.Serve(listener); err != nil {
if err := nc.server.Serve(listener); err != nil { return fmt.Errorf("network API server failed: %w", err)
return fmt.Errorf("network API server failed: %w", err) }
} return nil
return nil })
},
)
errGroup.Go(func() error { errGroup.Go(func() error {
slog.Info("Starting embedded DNS resolver.") slog.Info("Starting embedded DNS resolver.")
if err := nc.dnsResolver.Run(ctx); err != nil { if err := cc.dnsResolver.Run(ctx); err != nil {
return fmt.Errorf("embedded DNS resolver failed: %w", err) return fmt.Errorf("embedded DNS resolver failed: %w", err)
} }
return nil return nil
}) })
// The Docker network must be created before starting the DNS server because it listens on the machine IP.
errGroup.Go(func() error { errGroup.Go(func() error {
slog.Info("Starting embedded DNS server.") slog.Info("Starting embedded DNS server.")
if err := nc.dnsServer.Run(ctx); err != nil { if err := cc.dnsServer.Run(ctx); err != nil {
return fmt.Errorf("embedded DNS server failed: %w", err) return fmt.Errorf("embedded DNS server failed: %w", err)
} }
return nil return nil
}) })
// Setup Docker network and synchronise containers to the cluster store. // Synchronise Docker containers to the cluster store.
errGroup.Go(func() error { errGroup.Go(func() error {
return nc.prepareAndWatchDocker(ctx) slog.Info("Watching Docker containers and syncing them to cluster store.")
return cc.syncDockerContainers(ctx)
}) })
// Handle machine changes in the cluster. Handling machine and endpoint changes should be done // Handle machine changes in the cluster. Handling machine and endpoint changes should be done
// in separate goroutines to avoid a deadlock when reconfiguring the network. // in separate goroutines to avoid a deadlock when reconfiguring the network.
errGroup.Go(func() error { errGroup.Go(func() error {
if err := nc.handleMachineChanges(ctx); err != nil { if err := cc.handleMachineChanges(ctx); err != nil {
return fmt.Errorf("handle new machines: %w", err) return fmt.Errorf("handle new machines: %w", err)
} }
return nil return nil
@@ -156,24 +168,24 @@ func (nc *networkController) Run(ctx context.Context) error {
errGroup.Go(func() error { errGroup.Go(func() error {
for { for {
select { select {
case e, ok := <-nc.endpointChanges: case e, ok := <-cc.endpointChanges:
if !ok { if !ok {
// The channel was closed, stop watching for changes. // The channel was closed, stop watching for changes.
nc.endpointChanges = nil cc.endpointChanges = nil
return nil return nil
} }
nc.state.mu.Lock() cc.state.mu.Lock()
for i := range nc.state.Network.Peers { for i := range cc.state.Network.Peers {
if nc.state.Network.Peers[i].PublicKey.Equal(e.PublicKey) { if cc.state.Network.Peers[i].PublicKey.Equal(e.PublicKey) {
nc.state.Network.Peers[i].Endpoint = &e.Endpoint cc.state.Network.Peers[i].Endpoint = &e.Endpoint
break break
} }
} }
if err := nc.state.Save(); err != nil { if err := cc.state.Save(); err != nil {
slog.Error("Failed to save machine state.", "err", err) slog.Error("Failed to save machine state.", "err", err)
} }
nc.state.mu.Unlock() cc.state.mu.Unlock()
slog.Debug("Preserved endpoint change in the machine state.", slog.Debug("Preserved endpoint change in the machine state.",
"public_key", e.PublicKey, "endpoint", e.Endpoint) "public_key", e.PublicKey, "endpoint", e.Endpoint)
@@ -184,48 +196,66 @@ func (nc *networkController) Run(ctx context.Context) error {
}) })
errGroup.Go(func() error { errGroup.Go(func() error {
if err := nc.wgnet.Run(ctx); err != nil { if err := cc.wgnet.Run(ctx); err != nil {
return fmt.Errorf("WireGuard network failed: %w", err) return fmt.Errorf("WireGuard network failed: %w", err)
} }
return nil return nil
}) })
errGroup.Go(func() error { errGroup.Go(func() error {
slog.Info("Starting Caddyconfig controller.") slog.Info("Starting caddyconfig controller.")
if err := nc.caddyfileCtrl.Run(ctx); err != nil { if err := cc.caddyconfigCtrl.Run(ctx); err != nil {
//goland:noinspection GoErrorStringFormat return fmt.Errorf("caddyconfig controller failed: %w", err)
return fmt.Errorf("Caddyconfig controller failed: %w", err)
} }
return nil return nil
}) })
// Wait for the context to be done and stop the network API server. // Wait for the context to be done and stop the network API server.
errGroup.Go(func() error { <-ctx.Done()
<-ctx.Done() slog.Info("Stopping network API server.")
slog.Info("Stopping network API server.") // TODO: implement timeout for graceful shutdown.
// TODO: implement timeout for graceful shutdown. cc.server.GracefulStop()
nc.server.GracefulStop() slog.Info("Network API server stopped.")
slog.Info("Network API server stopped.")
return nil
})
return errGroup.Wait() // Wait for all controllers to finish.
err = errGroup.Wait()
// It's safe to stop the Corrosion service after the controllers depending on it and API server are stopped.
// Use a new context with a timeout as the current context is already canceled.
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if corroErr := cc.corroService.Stop(ctx); corroErr != nil {
err = errors.Join(err, fmt.Errorf("stop corrosion service: %w", corroErr))
} else {
slog.Info("Corrosion service stopped.")
}
return err
} }
// prepareAndWatchDocker configures the Docker network and watches local Docker containers to sync them // ensureDockerNetwork ensures that the Docker network is configured and ready for containers.
// to the cluster store. func (cc *clusterController) ensureDockerNetwork(ctx context.Context) error {
func (nc *networkController) prepareAndWatchDocker(ctx context.Context) error { if err := cc.dockerCtrl.WaitDaemonReady(ctx); err != nil {
manager := docker.NewManager(nc.dockerCli, nc.state.ID, nc.store)
if err := manager.WaitDaemonReady(ctx); err != nil {
return fmt.Errorf("wait for Docker daemon: %w", err) return fmt.Errorf("wait for Docker daemon: %w", err)
} }
if err := manager.EnsureUncloudNetwork(ctx, nc.state.Network.Subnet, nc.dnsServer.ListenAddr()); err != nil { if err := cc.dockerCtrl.EnsureUncloudNetwork(
ctx,
cc.state.Network.Subnet,
cc.dnsServer.ListenAddr(),
); err != nil {
return fmt.Errorf("ensure Docker network: %w", err) return fmt.Errorf("ensure Docker network: %w", err)
} }
slog.Info("Docker network configured.")
slog.Info("Watching Docker containers and syncing them to cluster store.") // Signal that Docker is ready for containers.
close(cc.dockerReady)
return nil
}
// syncDockerContainers watches local Docker containers and syncs them to the cluster store.
// TODO: move this to the Docker controller.
func (cc *clusterController) syncDockerContainers(ctx context.Context) error {
// Retry to watch and sync containers until the context is done. // Retry to watch and sync containers until the context is done.
boff := backoff.WithContext(backoff.NewExponentialBackOff( boff := backoff.WithContext(backoff.NewExponentialBackOff(
backoff.WithInitialInterval(100*time.Millisecond), backoff.WithInitialInterval(100*time.Millisecond),
@@ -233,7 +263,7 @@ func (nc *networkController) prepareAndWatchDocker(ctx context.Context) error {
backoff.WithMaxElapsedTime(0), backoff.WithMaxElapsedTime(0),
), ctx) ), ctx)
watchAndSync := func() error { watchAndSync := func() error {
if wErr := manager.WatchAndSyncContainers(ctx); wErr != nil { if wErr := cc.dockerCtrl.WatchAndSyncContainers(ctx); wErr != nil {
slog.Error("Failed to watch and sync containers to cluster store, retrying.", "err", wErr) slog.Error("Failed to watch and sync containers to cluster store, retrying.", "err", wErr)
return wErr return wErr
} }
@@ -251,7 +281,7 @@ func (nc *networkController) prepareAndWatchDocker(ctx context.Context) error {
// handleMachineChanges subscribes to machine changes in the cluster and reconfigures the network peers accordingly // handleMachineChanges subscribes to machine changes in the cluster and reconfigures the network peers accordingly
// when changes occur. // when changes occur.
func (nc *networkController) handleMachineChanges(ctx context.Context) error { func (cc *clusterController) handleMachineChanges(ctx context.Context) error {
for { for {
// Retry to subscribe to machine changes indefinitely until the context is done. // Retry to subscribe to machine changes indefinitely until the context is done.
boff := backoff.WithContext(backoff.NewExponentialBackOff( boff := backoff.WithContext(backoff.NewExponentialBackOff(
@@ -266,7 +296,7 @@ func (nc *networkController) handleMachineChanges(ctx context.Context) error {
err error err error
) )
subscribe := func() error { subscribe := func() error {
if machines, changes, err = nc.store.SubscribeMachines(ctx); err != nil { if machines, changes, err = cc.store.SubscribeMachines(ctx); err != nil {
slog.Info("Failed to subscribe to machine changes, retrying.", "err", err) slog.Info("Failed to subscribe to machine changes, retrying.", "err", err)
} }
return err return err
@@ -284,7 +314,7 @@ func (nc *networkController) handleMachineChanges(ctx context.Context) error {
// completes. Skip configuration now and apply it when the store changes are received. // completes. Skip configuration now and apply it when the store changes are received.
if len(machines) > 0 { if len(machines) > 0 {
slog.Info("Reconfiguring network peers with the current machines.", "machines", len(machines)) slog.Info("Reconfiguring network peers with the current machines.", "machines", len(machines))
if err = nc.configurePeers(machines); err != nil { if err = cc.configurePeers(machines); err != nil {
slog.Error("Failed to configure peers.", "err", err) slog.Error("Failed to configure peers.", "err", err)
} }
} }
@@ -296,11 +326,11 @@ func (nc *networkController) handleMachineChanges(ctx context.Context) error {
// be reworked as well. // be reworked as well.
case <-changes: case <-changes:
slog.Info("Cluster machines changed, reconfiguring network peers.") slog.Info("Cluster machines changed, reconfiguring network peers.")
if machines, err = nc.store.ListMachines(ctx); err != nil { if machines, err = cc.store.ListMachines(ctx); err != nil {
slog.Error("Failed to list machines.", "err", err) slog.Error("Failed to list machines.", "err", err)
continue continue
} }
if err = nc.configurePeers(machines); err != nil { if err = cc.configurePeers(machines); err != nil {
slog.Error("Failed to configure peers.", "err", err) slog.Error("Failed to configure peers.", "err", err)
} }
case <-ctx.Done(): case <-ctx.Done():
@@ -310,23 +340,23 @@ func (nc *networkController) handleMachineChanges(ctx context.Context) error {
} }
} }
func (nc *networkController) configurePeers(machines []*pb.MachineInfo) error { func (cc *clusterController) configurePeers(machines []*pb.MachineInfo) error {
if len(machines) == 0 { if len(machines) == 0 {
return fmt.Errorf("no machines to configure peers") return fmt.Errorf("no machines to configure peers")
} }
nc.state.mu.RLock() cc.state.mu.RLock()
currentPeerEndpoints := make(map[string]*netip.AddrPort, len(nc.state.Network.Peers)) currentPeerEndpoints := make(map[string]*netip.AddrPort, len(cc.state.Network.Peers))
for _, p := range nc.state.Network.Peers { for _, p := range cc.state.Network.Peers {
currentPeerEndpoints[p.PublicKey.String()] = p.Endpoint currentPeerEndpoints[p.PublicKey.String()] = p.Endpoint
} }
nc.state.mu.RUnlock() cc.state.mu.RUnlock()
// Construct the list of peers from the machine configurations ensuring that the current endpoint is preserved. // Construct the list of peers from the machine configurations ensuring that the current endpoint is preserved.
peers := make([]network.PeerConfig, 0, len(machines)-1) peers := make([]network.PeerConfig, 0, len(machines)-1)
for _, m := range machines { for _, m := range machines {
// Skip the current machine. // Skip the current machine.
if m.Id == nc.state.ID { if m.Id == cc.state.ID {
continue continue
} }
if err := m.Network.Validate(); err != nil { if err := m.Network.Validate(); err != nil {
@@ -359,20 +389,37 @@ func (nc *networkController) configurePeers(machines []*pb.MachineInfo) error {
} }
// Preserve the new list of peers in the machine state. // Preserve the new list of peers in the machine state.
nc.state.mu.Lock() cc.state.mu.Lock()
nc.state.Network.Peers = peers cc.state.Network.Peers = peers
err := nc.state.Save() err := cc.state.Save()
nc.state.mu.Unlock() cc.state.mu.Unlock()
if err != nil { if err != nil {
return fmt.Errorf("save machine state: %w", err) return fmt.Errorf("save machine state: %w", err)
} }
nc.state.mu.RLock() cc.state.mu.RLock()
defer nc.state.mu.RUnlock() defer cc.state.mu.RUnlock()
if err = nc.wgnet.Configure(*nc.state.Network); err != nil { if err = cc.wgnet.Configure(*cc.state.Network); err != nil {
return fmt.Errorf("configure network peers: %w", err) return fmt.Errorf("configure network peers: %w", err)
} }
return nil return nil
} }
// TODO: method to shutdown network when leaving a cluster. Regular context cancellation shouldn't bring it down. // Cleanup cleans up the cluster resources such as the WireGuard network, iptables rules, Docker network and containers.
func (cc *clusterController) Cleanup() error {
// Wait for the controller to stop before cleaning up.
<-cc.stopped
var errs []error
if err := cc.dockerCtrl.Cleanup(); err != nil {
errs = append(errs, fmt.Errorf("cleanup Docker resources: %w", err))
}
if err := cc.wgnet.Cleanup(); err != nil {
errs = append(errs, fmt.Errorf("cleanup WireGuard network: %w", err))
}
if err := firewall.CleanupIptablesChains(); err != nil {
errs = append(errs, fmt.Errorf("cleanup iptables chains: %w", err))
}
return errors.Join(errs...)
}
+108 -3
View File
@@ -5,17 +5,18 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb"
"log/slog" "log/slog"
"net/netip" "net/netip"
"time" "time"
"github.com/psviderski/uncloud/internal/corrosion" "github.com/psviderski/uncloud/internal/corrosion"
"github.com/psviderski/uncloud/internal/machine/api/pb" "github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/machine/network" "github.com/psviderski/uncloud/internal/machine/network"
"github.com/psviderski/uncloud/internal/machine/store" "github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/uncloud/internal/secret" "github.com/psviderski/uncloud/internal/secret"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb"
) )
type Cluster struct { type Cluster struct {
@@ -197,6 +198,89 @@ func (c *Cluster) AddMachine(ctx context.Context, req *pb.AddMachineRequest) (*p
return resp, nil return resp, nil
} }
// UpdateMachine updates machine configuration in the cluster.
func (c *Cluster) UpdateMachine(ctx context.Context, req *pb.UpdateMachineRequest) (*pb.UpdateMachineResponse, error) {
if err := c.checkInitialised(ctx); err != nil {
return nil, err
}
if req.MachineId == "" {
return nil, status.Error(codes.InvalidArgument, "machine_id not set")
}
// Get the current machine info
currentMachine, err := c.store.GetMachine(ctx, req.MachineId)
if err != nil {
if errors.Is(err, store.ErrMachineNotFound) {
return nil, status.Errorf(codes.NotFound, "machine not found: %s", req.MachineId)
}
return nil, status.Errorf(codes.Internal, "failed to get machine: %v", err)
}
// Create a copy of the current machine for updating
updatedMachine := &pb.MachineInfo{
Id: currentMachine.Id,
Name: currentMachine.Name,
Network: currentMachine.Network,
PublicIp: currentMachine.PublicIp,
}
// Apply updates from the request
if req.Name != nil {
// Check for empty name
if *req.Name == "" {
return nil, status.Error(codes.InvalidArgument, "machine name cannot be empty")
}
// Check for duplicate names (excluding the current machine)
if *req.Name != currentMachine.Name {
machines, err := c.store.ListMachines(ctx)
if err != nil {
return nil, status.Errorf(codes.Internal, "list machines: %v", err)
}
for _, m := range machines {
if m.Id != req.MachineId && m.Name == *req.Name {
return nil, status.Errorf(codes.AlreadyExists, "machine with name %q already exists", *req.Name)
}
}
}
updatedMachine.Name = *req.Name
}
if req.PublicIp != nil {
// Check if this is an empty IP (used to signal removal)
if len(req.PublicIp.Ip) == 0 {
// User wants to remove public IP
updatedMachine.PublicIp = nil
} else {
// Validate and set the new IP
ip, err := req.PublicIp.ToAddr()
if err != nil {
return nil, status.Errorf(codes.InvalidArgument, "invalid public IP: %v", err)
}
if !ip.IsValid() {
return nil, status.Error(codes.InvalidArgument, "invalid public IP")
}
updatedMachine.PublicIp = req.PublicIp
}
}
if req.Endpoints != nil {
updatedMachine.Network.Endpoints = req.Endpoints
}
// Update the machine in the store
if err = c.store.UpdateMachine(ctx, updatedMachine); err != nil {
if errors.Is(err, store.ErrMachineNotFound) {
return nil, status.Errorf(codes.NotFound, "machine not found: %s", req.MachineId)
}
return nil, status.Errorf(codes.Internal, "update machine: %v", err)
}
slog.Info("Machine configuration updated in the cluster.",
"id", updatedMachine.Id, "name", updatedMachine.Name)
resp := &pb.UpdateMachineResponse{Machine: updatedMachine}
return resp, nil
}
// ListMachines lists all machines in the cluster including their membership states. // ListMachines lists all machines in the cluster including their membership states.
func (c *Cluster) ListMachines(ctx context.Context, _ *emptypb.Empty) (*pb.ListMachinesResponse, error) { func (c *Cluster) ListMachines(ctx context.Context, _ *emptypb.Empty) (*pb.ListMachinesResponse, error) {
if err := c.checkInitialised(ctx); err != nil { if err := c.checkInitialised(ctx); err != nil {
@@ -242,3 +326,24 @@ func (c *Cluster) ListMachines(ctx context.Context, _ *emptypb.Empty) (*pb.ListM
return &pb.ListMachinesResponse{Machines: members}, nil return &pb.ListMachinesResponse{Machines: members}, nil
} }
// RemoveMachine removes a machine from the cluster.
func (c *Cluster) RemoveMachine(ctx context.Context, req *pb.RemoveMachineRequest) (*emptypb.Empty, error) {
if err := c.checkInitialised(ctx); err != nil {
return nil, err
}
if req.Id == "" {
return nil, status.Error(codes.InvalidArgument, "machine ID not set")
}
if err := c.store.DeleteMachine(ctx, req.Id); err != nil {
if errors.Is(err, store.ErrMachineNotFound) {
return nil, status.Errorf(codes.NotFound, "machine not found: %s", req.Id)
}
return nil, status.Errorf(codes.Internal, "delete machine from store: %v", err)
}
slog.Info("Machine removed from the cluster.", "id", req.Id)
return &emptypb.Empty{}, nil
}
+1
View File
@@ -4,6 +4,7 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
"github.com/psviderski/uncloud/internal/dns" "github.com/psviderski/uncloud/internal/dns"
"github.com/psviderski/uncloud/internal/machine/api/pb" "github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/machine/store" "github.com/psviderski/uncloud/internal/machine/store"
+2 -1
View File
@@ -3,8 +3,9 @@ package cluster
import ( import (
"errors" "errors"
"fmt" "fmt"
"go4.org/netipx"
"net/netip" "net/netip"
"go4.org/netipx"
) )
const DefaultSubnetBits = 24 const DefaultSubnetBits = 24
+1
View File
@@ -2,6 +2,7 @@ package cluster
import ( import (
"fmt" "fmt"
"github.com/psviderski/uncloud/internal/secret" "github.com/psviderski/uncloud/internal/secret"
) )
+6
View File
@@ -0,0 +1,6 @@
package constants
const (
// MachineAPIPort is the port for the Machine API service on the management WireGuard network.
MachineAPIPort = 51000
)
+5 -4
View File
@@ -3,10 +3,11 @@ package corroservice
import ( import (
"bytes" "bytes"
"fmt" "fmt"
"github.com/BurntSushi/toml"
"net/netip" "net/netip"
"os" "os"
"path/filepath" "path/filepath"
"github.com/BurntSushi/toml"
"github.com/psviderski/uncloud/internal/fs" "github.com/psviderski/uncloud/internal/fs"
) )
@@ -50,7 +51,7 @@ func (c *Config) Write(path, owner string) error {
if err := encoder.Encode(c); err != nil { if err := encoder.Encode(c); err != nil {
return fmt.Errorf("encode config: %w", err) return fmt.Errorf("encode config: %w", err)
} }
if err := os.WriteFile(path, data.Bytes(), 0600); err != nil { if err := os.WriteFile(path, data.Bytes(), 0o600); err != nil {
return err return err
} }
if err := fs.Chown(path, owner, owner); err != nil { if err := fs.Chown(path, owner, owner); err != nil {
@@ -62,10 +63,10 @@ func (c *Config) Write(path, owner string) error {
func MkDataDir(dir, owner string) error { func MkDataDir(dir, owner string) error {
parent, _ := filepath.Split(dir) parent, _ := filepath.Split(dir)
// Use 0711 for parent directories to allow `owner` to access its nested data directory. // Use 0711 for parent directories to allow `owner` to access its nested data directory.
if err := os.MkdirAll(parent, 0711); err != nil { if err := os.MkdirAll(parent, 0o711); err != nil {
return fmt.Errorf("create directory %q: %w", parent, err) return fmt.Errorf("create directory %q: %w", parent, err)
} }
if err := os.Mkdir(dir, 0700); err != nil { if err := os.Mkdir(dir, 0o700); err != nil {
if !os.IsExist(err) { if !os.IsExist(err) {
return fmt.Errorf("create directory %q: %w", dir, err) return fmt.Errorf("create directory %q: %w", dir, err)
} }
+40 -21
View File
@@ -3,15 +3,16 @@ package corroservice
import ( import (
"context" "context"
"fmt" "fmt"
"io"
"log/slog"
"path/filepath"
"time"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/image" "github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/mount" "github.com/docker/docker/api/types/mount"
"github.com/docker/docker/api/types/network" "github.com/docker/docker/api/types/network"
"github.com/docker/docker/client" "github.com/docker/docker/client"
"io"
"log/slog"
"path/filepath"
"time"
) )
const ( const (
@@ -26,29 +27,47 @@ type DockerService struct {
User string User string
} }
func NewDockerService(cli *client.Client, image, name, dataDir string) *DockerService {
return &DockerService{
Client: cli,
Image: image,
Name: name,
DataDir: dataDir,
}
}
func (s *DockerService) Start(ctx context.Context) error { func (s *DockerService) Start(ctx context.Context) error {
_, err := s.Client.ContainerInspect(ctx, s.Name) _, err := s.Client.ContainerInspect(ctx, s.Name)
if err != nil { if err != nil {
if client.IsErrNotFound(err) { if !client.IsErrNotFound(err) {
return s.startNewContainer(ctx) return fmt.Errorf("inspect container %q: %w", s.Name, err)
}
if err = s.startNewContainer(ctx); err != nil {
return err
}
} else {
// Container already exists.
// TODO: recreate only if the container configuration has to be changed.
if err = s.Client.ContainerRemove(ctx, s.Name, container.RemoveOptions{Force: true}); err != nil {
return fmt.Errorf("remove container %q: %w", s.Name, err)
}
if err = s.startNewContainer(ctx); err != nil {
return err
} }
return fmt.Errorf("inspect container %q: %w", s.Name, err)
}
// TODO: recreate only if the container configuration has to be changed.
if err = s.Client.ContainerRemove(ctx, s.Name, container.RemoveOptions{Force: true}); err != nil {
return fmt.Errorf("remove container %q: %w", s.Name, err)
} }
return s.startNewContainer(ctx) slog.Debug("Waiting for corrosion service to be ready.")
if err = WaitReady(ctx, s.DataDir); err != nil {
return err
}
slog.Debug("Corrosion service is ready.")
return nil
}
func (s *DockerService) Stop(ctx context.Context) error {
if err := s.Client.ContainerStop(ctx, s.Name, container.StopOptions{}); err != nil {
return fmt.Errorf("stop container %q: %w", s.Name, err)
}
slog.Debug("Corrosion Docker container stopped.", "name", s.Name)
if err := s.Client.ContainerRemove(ctx, s.Name, container.RemoveOptions{}); err != nil {
return fmt.Errorf("remove container %q: %w", s.Name, err)
}
slog.Debug("Corrosion Docker container removed.", "name", s.Name)
return nil
} }
func (s *DockerService) Restart(ctx context.Context) error { func (s *DockerService) Restart(ctx context.Context) error {
+54 -1
View File
@@ -1,9 +1,62 @@
package corroservice package corroservice
import "context" import (
"context"
"fmt"
"os"
"path/filepath"
"time"
"github.com/BurntSushi/toml"
"github.com/cenkalti/backoff/v4"
"github.com/psviderski/uncloud/internal/corrosion"
)
type Service interface { type Service interface {
Start(ctx context.Context) error Start(ctx context.Context) error
Stop(ctx context.Context) error
Restart(ctx context.Context) error Restart(ctx context.Context) error
Running() bool Running() bool
} }
// WaitReady waits for the Corrosion service to be ready with the uncloud schema applied.
func WaitReady(ctx context.Context, dataDir string) error {
// Read the config file to get the API address.
configPath := filepath.Join(dataDir, "config.toml")
configData, err := os.ReadFile(configPath)
if err != nil {
return fmt.Errorf("read config file: %w", err)
}
var config Config
if err = toml.Unmarshal(configData, &config); err != nil {
return fmt.Errorf("unmarshal config: %w", err)
}
corro, err := corrosion.NewAPIClient(config.API.Addr)
if err != nil {
return fmt.Errorf("create corrosion API client: %w", err)
}
// Corrosion starts serving the API before applying the schema. Query the cluster table with exponential backoff
// to check if the uncloud schema has been applied.
checkReady := func() error {
rows, err := corro.QueryContext(ctx, "SELECT 1 FROM cluster LIMIT 1")
if err != nil {
return fmt.Errorf("query cluster table: %w", err)
}
defer rows.Close()
return nil
}
b := backoff.NewExponentialBackOff(
backoff.WithInitialInterval(50*time.Millisecond),
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(15*time.Second),
)
if err = backoff.Retry(checkReady, backoff.WithContext(b, ctx)); err != nil {
return fmt.Errorf("corrosion service did not become ready: %w", err)
}
return nil
}
-157
View File
@@ -1,157 +0,0 @@
package corroservice
import (
"bufio"
"context"
"fmt"
"log/slog"
"os/exec"
"path/filepath"
"sync"
"syscall"
"time"
)
const (
DefaultCommand = "corrosion"
DefaultDataDir = "/var/lib/uncloud/corrosion"
)
// SubprocessService implements the Service interface by running the service as a subprocess.
type SubprocessService struct {
Command string
DataDir string
cmd *exec.Cmd
running bool
mu sync.Mutex
cancelWatch context.CancelFunc
}
func DefaultSubprocessService() *SubprocessService {
return &SubprocessService{
Command: DefaultCommand,
DataDir: DefaultDataDir,
}
}
// TODO: maybe stop the process if this ctx is cancelled.
func (s *SubprocessService) Start(ctx context.Context) error {
s.mu.Lock()
defer s.mu.Unlock()
if s.running {
return nil
}
return s.startProcess(ctx)
}
func (s *SubprocessService) Restart(ctx context.Context) error {
s.mu.Lock()
defer s.mu.Unlock()
if s.running {
if err := s.stopProcess(); err != nil {
return fmt.Errorf("stop process: %w", err)
}
}
return s.startProcess(ctx)
}
func (s *SubprocessService) Running() bool {
s.mu.Lock()
defer s.mu.Unlock()
return s.running
}
func (s *SubprocessService) startProcess(ctx context.Context) error {
s.cmd = exec.Command(s.Command, "agent", "-c", filepath.Join(s.DataDir, "config.toml"))
// Redirect stdout and stderr to the logger.
stdout, err := s.cmd.StdoutPipe()
if err != nil {
return fmt.Errorf("create stdout pipe: %w", err)
}
stderr, err := s.cmd.StderrPipe()
if err != nil {
return fmt.Errorf("create stderr pipe: %w", err)
}
go func() {
scanner := bufio.NewScanner(stdout)
for scanner.Scan() {
slog.Info("[corrosion]: " + scanner.Text())
}
// TODO: remove
slog.Info("######## corrosion redirect go routine end ########")
}()
go func() {
scanner := bufio.NewScanner(stderr)
for scanner.Scan() {
slog.Error("[corrosion]: " + scanner.Text())
}
}()
if err = s.cmd.Start(); err != nil {
return fmt.Errorf("start process: %w", err)
}
s.running = true
// Watch for process exit to update running status.
go func() {
if err := s.cmd.Wait(); err != nil {
slog.Error("corrosion process exited with error.", "code", s.cmd.ProcessState.ExitCode(), "err", err)
}
s.mu.Lock()
s.running = false
s.mu.Unlock()
}()
// TODO: figure out the waiting process
// Wait for initialization
//timer := time.NewTimer(2 * time.Second)
//defer timer.Stop()
//select {
////case <-timer.C:
//// s.running = true
//// return nil
//case <-watchCtx.Done():
// return fmt.Errorf("process failed to start")
//case <-ctx.Done():
// s.stopProcess()
// return ctx.Err()
//}
return nil
}
func (s *SubprocessService) stopProcess() error {
if s.cmd == nil || s.cmd.Process == nil {
return nil
}
if err := s.cmd.Process.Signal(syscall.SIGTERM); err != nil {
return fmt.Errorf("send SIGTERM: %w", err)
}
// Wait up to 5 seconds for graceful shutdown before killing the process.
done := make(chan error, 1)
go func() {
done <- s.cmd.Wait()
}()
select {
case <-time.After(5 * time.Second):
if err := s.cmd.Process.Kill(); err != nil {
return fmt.Errorf("kill process: %w", err)
}
case err := <-done:
if err != nil {
return fmt.Errorf("process exited with error: %w", err)
}
}
return nil
}
+15 -12
View File
@@ -5,7 +5,6 @@ import (
"fmt" "fmt"
"log/slog" "log/slog"
"os/exec" "os/exec"
"time"
) )
const DefaultSystemdUnit = "uncloud-corrosion.service" const DefaultSystemdUnit = "uncloud-corrosion.service"
@@ -27,6 +26,15 @@ func (s *SystemdService) Start(ctx context.Context) error {
return s.startOrRestart(ctx, "start") return s.startOrRestart(ctx, "start")
} }
func (s *SystemdService) Stop(_ context.Context) error {
if _, err := exec.Command("systemctl", "stop", s.Unit).Output(); err != nil {
return fmt.Errorf("systemctl stop %s: %w", s.Unit, err)
}
slog.Info("Corrosion systemd service stopped.", "unit", s.Unit)
return nil
}
func (s *SystemdService) Restart(ctx context.Context) error { func (s *SystemdService) Restart(ctx context.Context) error {
return s.startOrRestart(ctx, "restart") return s.startOrRestart(ctx, "restart")
} }
@@ -35,20 +43,15 @@ func (s *SystemdService) startOrRestart(ctx context.Context, cmd string) error {
if _, err := exec.Command("systemctl", cmd, s.Unit).Output(); err != nil { if _, err := exec.Command("systemctl", cmd, s.Unit).Output(); err != nil {
return fmt.Errorf("systemctl %s %s: %w", cmd, s.Unit, err) return fmt.Errorf("systemctl %s %s: %w", cmd, s.Unit, err)
} }
slog.Info(fmt.Sprintf("Corrosion systemd service %sed.", cmd), "unit", s.Unit) slog.Debug(fmt.Sprintf("Corrosion systemd service %sed.", cmd), "unit", s.Unit)
// Optimistically wait for the corrosion service to start and initialise the database schema before proceeding. slog.Debug("Waiting for corrosion service to be ready.")
timer := time.NewTimer(2 * time.Second) if err := WaitReady(ctx, s.DataDir); err != nil {
defer timer.Stop() return err
select {
case <-timer.C:
case <-ctx.Done():
return nil
} }
slog.Debug("Corrosion service is ready.")
// TODO: run a goroutine to check the status of the service and log any errors in the uncloud log.
s.running = true s.running = true
return nil return nil
} }
+2 -2
View File
@@ -14,13 +14,13 @@ const DBFileName = "machine.db"
func NewDB(path string) (*sqlx.DB, error) { func NewDB(path string) (*sqlx.DB, error) {
// Create the database file with 0600 permissions if it doesn't exist, or update permissions if exists. // Create the database file with 0600 permissions if it doesn't exist, or update permissions if exists.
if _, err := os.Stat(path); os.IsNotExist(err) { if _, err := os.Stat(path); os.IsNotExist(err) {
file, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR, 0600) file, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR, 0o600)
if err != nil { if err != nil {
return nil, fmt.Errorf("create SQLite database '%s': %w", path, err) return nil, fmt.Errorf("create SQLite database '%s': %w", path, err)
} }
file.Close() file.Close()
} else { } else {
if err = os.Chmod(path, 0600); err != nil { if err = os.Chmod(path, 0o600); err != nil {
return nil, fmt.Errorf("update SQLite database permissions '%s': %w", path, err) return nil, fmt.Errorf("update SQLite database permissions '%s': %w", path, err)
} }
} }
+2 -8
View File
@@ -5,12 +5,10 @@ import (
"fmt" "fmt"
"log/slog" "log/slog"
"net/netip" "net/netip"
"strings"
"sync" "sync"
"time" "time"
"github.com/psviderski/uncloud/internal/machine/store" "github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/uncloud/pkg/api"
) )
// ClusterResolver implements Resolver by tracking containers in the cluster and resolving service names // ClusterResolver implements Resolver by tracking containers in the cluster and resolving service names
@@ -84,17 +82,13 @@ func (r *ClusterResolver) updateServiceIPs(containers []store.ContainerRecord) {
continue continue
} }
ctr := api.ServiceContainer{Container: record.Container} ctr := record.Container
if ctr.ServiceID() == "" || ctr.ServiceName() == "" { if ctr.ServiceID() == "" || ctr.ServiceName() == "" {
// Container is not part of a service, skip it. // Container is not part of a service, skip it.
continue continue
} }
// TODO: remove normalisation after implementing service name validation: newServiceIPs[ctr.ServiceName()] = append(newServiceIPs[ctr.ServiceName()], ip)
//.https://github.com/psviderski/uncloud/issues/53
serviceName := strings.ToLower(ctr.ServiceName())
newServiceIPs[serviceName] = append(newServiceIPs[serviceName], ip)
// Also add the service ID as a valid lookup. // Also add the service ID as a valid lookup.
newServiceIPs[ctr.ServiceID()] = append(newServiceIPs[ctr.ServiceID()], ip) newServiceIPs[ctr.ServiceID()] = append(newServiceIPs[ctr.ServiceID()], ip)
containersCount++ containersCount++
+16 -2
View File
@@ -196,13 +196,27 @@ func (c *Client) RemoveContainer(ctx context.Context, id string, opts container.
return err return err
} }
// PullOptions defines the options for pulling an image from a remote registry.
// This is a copy of image.PullOptions from the Docker API without the PrivilegeFunc field that is non-serialisable.
type PullOptions struct {
All bool
// RegistryAuth is the base64 encoded credentials for the registry.
RegistryAuth string
Platform string
}
type PullImageMessage struct { type PullImageMessage struct {
Message jsonmessage.JSONMessage Message jsonmessage.JSONMessage
Err error Err error
} }
func (c *Client) PullImage(ctx context.Context, image string) (<-chan PullImageMessage, error) { func (c *Client) PullImage(ctx context.Context, image string, opts PullOptions) (<-chan PullImageMessage, error) {
stream, err := c.grpcClient.PullImage(ctx, &pb.PullImageRequest{Image: image}) optsBytes, err := json.Marshal(opts)
if err != nil {
return nil, fmt.Errorf("marshal options: %w", err)
}
stream, err := c.grpcClient.PullImage(ctx, &pb.PullImageRequest{Image: image, Options: optsBytes})
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -7,12 +7,11 @@ import (
"log/slog" "log/slog"
"time" "time"
dockercontainer "github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/events" "github.com/docker/docker/api/types/events"
"github.com/docker/docker/api/types/filters" "github.com/docker/docker/api/types/filters"
"github.com/docker/docker/client" "github.com/docker/docker/client"
"github.com/psviderski/uncloud/internal/machine/store" "github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/uncloud/pkg/api"
) )
const ( const (
@@ -24,23 +23,26 @@ const (
SyncInterval = 30 * time.Second SyncInterval = 30 * time.Second
) )
type Manager struct { // Controller monitors Docker events and synchronises service containers with the cluster store.
client *client.Client type Controller struct {
// machineID is the ID of the machine where the managed Docker daemon is running. // machineID is the ID of the machine where the managed Docker daemon is running.
machineID string machineID string
client *client.Client
service *Service
store *store.Store store *store.Store
} }
func NewManager(client *client.Client, machineID string, store *store.Store) *Manager { func NewController(machineID string, service *Service, store *store.Store) *Controller {
return &Manager{ return &Controller{
client: client,
machineID: machineID, machineID: machineID,
client: service.Client,
service: service,
store: store, store: store,
} }
} }
// WaitDaemonReady waits for the Docker daemon to start and be ready to serve requests. // WaitDaemonReady waits for the Docker daemon to start and be ready to serve requests.
func (m *Manager) WaitDaemonReady(ctx context.Context) error { func (c *Controller) WaitDaemonReady(ctx context.Context) error {
ticker := time.NewTicker(1 * time.Second) ticker := time.NewTicker(1 * time.Second)
defer ticker.Stop() defer ticker.Stop()
@@ -50,7 +52,7 @@ func (m *Manager) WaitDaemonReady(ctx context.Context) error {
case <-ctx.Done(): case <-ctx.Done():
return ctx.Err() return ctx.Err()
case <-ticker.C: case <-ticker.C:
_, err := m.client.Ping(ctx) _, err := c.client.Ping(ctx)
if err == nil { if err == nil {
ready = true ready = true
break break
@@ -67,7 +69,7 @@ func (m *Manager) WaitDaemonReady(ctx context.Context) error {
return nil return nil
} }
func (m *Manager) WatchAndSyncContainers(ctx context.Context) error { func (c *Controller) WatchAndSyncContainers(ctx context.Context) error {
ctx, cancel := context.WithCancel(ctx) ctx, cancel := context.WithCancel(ctx)
defer cancel() defer cancel()
// Filter only local container events. // Filter only local container events.
@@ -79,9 +81,9 @@ func (m *Manager) WatchAndSyncContainers(ctx context.Context) error {
} }
// Subscribe to Docker events before running the initial sync to avoid missing any events. // Subscribe to Docker events before running the initial sync to avoid missing any events.
eventCh, errCh := m.client.Events(ctx, opts) eventCh, errCh := c.service.Client.Events(ctx, opts)
slog.Debug("Syncing containers to cluster store before processing Docker events.") slog.Debug("Syncing containers to cluster store before processing Docker events.")
if err := m.syncContainersToStore(ctx); err != nil { if err := c.syncContainersToStore(ctx); err != nil {
// The deferred cancel will stop the event subscription. // The deferred cancel will stop the event subscription.
return fmt.Errorf("sync containers to cluster store: %w", err) return fmt.Errorf("sync containers to cluster store: %w", err)
} }
@@ -126,13 +128,13 @@ func (m *Manager) WatchAndSyncContainers(ctx context.Context) error {
"container_name", e.Actor.Attributes["name"], "container_name", e.Actor.Attributes["name"],
"action", e.Action) "action", e.Action)
if err := m.syncContainersToStore(ctx); err != nil { if err := c.syncContainersToStore(ctx); err != nil {
return fmt.Errorf("sync containers to cluster store: %w", err) return fmt.Errorf("sync containers to cluster store: %w", err)
} }
case <-ticker.C: case <-ticker.C:
slog.Debug("Syncing containers to cluster store triggered by a regular interval.", slog.Debug("Syncing containers to cluster store triggered by a regular interval.",
"interval", SyncInterval) "interval", SyncInterval)
if err := m.syncContainersToStore(ctx); err != nil { if err := c.syncContainersToStore(ctx); err != nil {
return fmt.Errorf("sync containers to cluster store: %w", err) return fmt.Errorf("sync containers to cluster store: %w", err)
} }
case err := <-errCh: case err := <-errCh:
@@ -144,32 +146,16 @@ func (m *Manager) WatchAndSyncContainers(ctx context.Context) error {
} }
} }
func (m *Manager) syncContainersToStore(ctx context.Context) error { func (c *Controller) syncContainersToStore(ctx context.Context) error {
storeContainers, err := m.store.ListContainers(ctx, store.ListOptions{MachineIDs: []string{m.machineID}}) storeContainers, err := c.store.ListContainers(ctx, store.ListOptions{MachineIDs: []string{c.machineID}})
if err != nil { if err != nil {
return fmt.Errorf("list containers from store: %w", err) return fmt.Errorf("list containers from store: %w", err)
} }
// List only Uncloud service containers identified by their labels. containers, err := c.service.ListServiceContainers(ctx, "", container.ListOptions{})
containerSummaries, err := m.client.ContainerList(ctx, dockercontainer.ListOptions{
Filters: filters.NewArgs(
filters.Arg("label", api.LabelServiceID),
filters.Arg("label", api.LabelServiceName),
),
})
if err != nil { if err != nil {
// TODO: mark all containers as outdated in the store. // TODO: mark all containers as outdated in the store.
return fmt.Errorf("list Docker containers: %w", err) return fmt.Errorf("list service containers: %w", err)
}
// Inspect each container to get the full container details.
containers := make([]api.Container, len(containerSummaries))
for i, cs := range containerSummaries {
ctr, err := m.client.ContainerInspect(ctx, cs.ID)
if err != nil {
return fmt.Errorf("inspect container '%s': %w", cs.ID, err)
}
containers[i] = api.Container{ContainerJSON: ctr}
} }
// Delete containers from the store that are no longer present in the Docker daemon. // Delete containers from the store that are no longer present in the Docker daemon.
@@ -189,15 +175,15 @@ func (m *Manager) syncContainersToStore(ctx context.Context) error {
var storeErr error var storeErr error
if len(deleteIDs) > 0 { if len(deleteIDs) > 0 {
if err = m.store.DeleteContainers(ctx, store.DeleteOptions{IDs: deleteIDs}); err != nil { if err = c.store.DeleteContainers(ctx, store.DeleteOptions{IDs: deleteIDs}); err != nil {
storeErr = fmt.Errorf("delete containers from store: %w", err) storeErr = fmt.Errorf("delete containers from store: %w", err)
} }
} }
// Create or update the current Docker containers in the store. // Create or update the current Docker containers in the store.
for _, c := range containers { for _, ctr := range containers {
if err = m.store.CreateOrUpdateContainer(ctx, c, m.machineID); err != nil { if err = c.store.CreateOrUpdateContainer(ctx, ctr, c.machineID); err != nil {
storeErr = errors.Join(storeErr, fmt.Errorf("create or update container %q: %w", c.ID, err)) storeErr = errors.Join(storeErr, fmt.Errorf("create or update container '%s': %w", ctr.ID, err))
} }
} }
return storeErr return storeErr
@@ -0,0 +1,19 @@
//go:build darwin
package docker
import (
"context"
"fmt"
"net/netip"
)
// EnsureUncloudNetwork is a stub for Darwin.
func (c *Controller) EnsureUncloudNetwork(ctx context.Context, subnet netip.Prefix, dnsServer netip.Addr) error {
return fmt.Errorf("not supported on Darwin")
}
// Cleanup is a stub for Darwin.
func (c *Controller) Cleanup() error {
return fmt.Errorf("not supported on Darwin")
}
+234
View File
@@ -0,0 +1,234 @@
package docker
import (
"context"
"errors"
"fmt"
"log/slog"
"net/netip"
"strconv"
dockercontainer "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters"
dnetwork "github.com/docker/docker/api/types/network"
"github.com/docker/docker/client"
"github.com/docker/docker/libnetwork/iptables"
"github.com/psviderski/uncloud/internal/machine/dns"
"github.com/psviderski/uncloud/internal/machine/firewall"
"github.com/psviderski/uncloud/internal/machine/network"
"github.com/psviderski/uncloud/pkg/api"
)
// EnsureUncloudNetwork creates the Docker bridge network NetworkName with the provided machine subnet
// if it doesn't exist. If the network exists but has a different subnet, it removes and recreates the network.
// It also configures iptables to allow container access from the WireGuard network.
func (c *Controller) EnsureUncloudNetwork(ctx context.Context, subnet netip.Prefix, dnsServer netip.Addr) error {
// Ensure the Docker network 'uncloud' is created with the correct subnet.
needsCreation := false
nw, err := c.client.NetworkInspect(ctx, NetworkName, dnetwork.InspectOptions{})
if err != nil {
if !client.IsErrNotFound(err) {
return fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err)
}
needsCreation = true
} else if nw.IPAM.Config[0].Subnet != subnet.String() {
// Remove the Docker network if the subnet is different.
// It could be a leftover from a previous incomplete cleanup.
slog.Info(
"Removing Docker network with old subnet.", "name", NetworkName, "subnet", nw.IPAM.Config[0].Subnet,
)
if err = c.client.NetworkRemove(ctx, NetworkName); err != nil {
// It can still fail if the network is in use by a container. Leave it to the user to resolve the issue.
return fmt.Errorf("remove Docker network '%s': %w", NetworkName, err)
}
needsCreation = true
}
if needsCreation {
if _, err = c.client.NetworkCreate(
ctx, NetworkName, dnetwork.CreateOptions{
Driver: "bridge",
Scope: "local",
IPAM: &dnetwork.IPAM{
Config: []dnetwork.IPAMConfig{
{
Subnet: subnet.String(),
},
},
},
Labels: map[string]string{
api.LabelManaged: "",
},
Options: map[string]string{
// Starting with Docker 28.2.0 (https://github.com/moby/moby/pull/49832), we have to explicitly
// allow direct routing from the WireGuard interface to the bridge network.
"com.docker.network.bridge.trusted_host_interfaces": network.WireGuardInterfaceName,
},
},
); err != nil {
return fmt.Errorf("create Docker network '%s': %w", NetworkName, err)
}
slog.Info("Docker network created.", "name", NetworkName, "subnet", subnet.String())
if nw, err = c.client.NetworkInspect(ctx, NetworkName, dnetwork.InspectOptions{}); err != nil {
return fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err)
}
}
// Configure iptables to allow WireGuard network to access containers. The Docker daemon should have already
// created the DOCKER-USER chain at this point.
// TODO: check if this works when firewalld used instead of raw iptables. The Docker daemon has a different
// code path for firewalld.
// Bridge name doesn't seem to be documented but this is the source code where it is generated:
// https://github.com/moby/moby/blob/v27.2.1/libnetwork/drivers/bridge/bridge_linux.go#L664
bridgeName := "br-" + nw.ID[:12]
if err = configureIptables(bridgeName, subnet, dnsServer); err != nil {
return fmt.Errorf("configure iptables for Docker network '%s': %w", NetworkName, err)
}
return nil
}
// configureIptables configures iptables rules for the uncloud Docker network.
func configureIptables(bridgeName string, subnet netip.Prefix, dnsServer netip.Addr) error {
ipt := iptables.GetIptable(iptables.IPv4)
// Allow traffic from other machines and their containers through the WG mesh to the Uncloud containers
// on the machine.
wgRule := []string{
"--in-interface", network.WireGuardInterfaceName,
"--out-interface", bridgeName,
"-j", "ACCEPT",
}
if err := ipt.ProgramRule(iptables.Filter, firewall.DockerUserChain, iptables.Insert, wgRule); err != nil {
return fmt.Errorf("insert iptables rule: %w", err)
}
// Allow DNS queries from Uncloud containers to the embedded DNS server.
for _, proto := range []string{"udp", "tcp"} {
dnsRule := []string{
"--in-interface", bridgeName,
"--dst", dnsServer.String(),
"--protocol", proto,
"--dport", strconv.Itoa(dns.Port),
"-j", "ACCEPT",
}
if err := ipt.ProgramRule(iptables.Filter, firewall.UncloudInputChain, iptables.Insert, dnsRule); err != nil {
return fmt.Errorf("insert iptables rule: %w", err)
}
}
// Skip masquerading for the container traffic going from the uncloud Docker network through the WG mesh.
// https://uncloud.run/blog/connect-docker-containers-across-hosts-wireguard#step-3-configure-ip-routing
skipMasqueradeRule := []string{
"--src", subnet.String(),
"--out-interface", network.WireGuardInterfaceName,
"-j", "RETURN",
}
// Delete and reinsert the rule to ensure it's at the top of the POSTROUTING chain before the MASQUERADE rule
// added by Docker: POSTROUTING -s 10.210.X.0/24 ! -o br-XXX -j MASQUERADE
if err := ipt.ProgramRule(iptables.Nat, "POSTROUTING", iptables.Delete, skipMasqueradeRule); err != nil {
return fmt.Errorf("delete iptables rule: %w", err)
}
if err := ipt.ProgramRule(iptables.Nat, "POSTROUTING", iptables.Insert, skipMasqueradeRule); err != nil {
return fmt.Errorf("insert iptables rule: %w", err)
}
return nil
}
// cleanupIptables deletes the iptables rules for the uncloud Docker network.
func cleanupIptables(bridgeName string, subnet netip.Prefix) error {
ipt := iptables.GetIptable(iptables.IPv4)
// Delete the rule allowing traffic from the WireGuard network to the Docker bridge.
wgRule := []string{
"--in-interface", network.WireGuardInterfaceName,
"--out-interface", bridgeName,
"-j", "ACCEPT",
}
if err := ipt.ProgramRule(iptables.Filter, firewall.DockerUserChain, iptables.Delete, wgRule); err != nil {
return fmt.Errorf("delete iptables rule: %w", err)
}
// Delete the rule that skips masquerading for the container traffic going from the uncloud Docker network
// through the WG mesh.
skipMasqueradeRule := []string{
"--src", subnet.String(),
"--out-interface", network.WireGuardInterfaceName,
"-j", "RETURN",
}
if err := ipt.ProgramRule(iptables.Nat, "POSTROUTING", iptables.Delete, skipMasqueradeRule); err != nil {
return fmt.Errorf("delete iptables rule: %w", err)
}
// Rules in uncloud-owned chains will be automatically cleaned up by the machine cleanup.
return nil
}
// Cleanup removes all uncloud-managed containers and the uncloud Docker network.
func (c *Controller) Cleanup() error {
ctx := context.Background()
var errs []error
// Remove uncloud-managed Docker containers.
containers, err := c.client.ContainerList(ctx, dockercontainer.ListOptions{
All: true, // Include stopped containers.
Filters: filters.NewArgs(
filters.Arg("label", api.LabelManaged),
),
})
if err != nil {
errs = append(errs, fmt.Errorf("list uncloud-managed Docker containers: %w", err))
} else if len(containers) > 0 {
slog.Info("Removing uncloud-managed Docker containers.", "count", len(containers))
removed := 0
for _, ctr := range containers {
err = c.client.ContainerStop(ctx, ctr.ID, dockercontainer.StopOptions{})
if err != nil && !client.IsErrNotFound(err) {
errs = append(errs, fmt.Errorf("stop container '%s': %w", ctr.ID, err))
}
err = c.client.ContainerRemove(ctx, ctr.ID, dockercontainer.RemoveOptions{
// Remove anonymous volumes created by the container.
RemoveVolumes: true,
})
if err == nil {
removed++
} else if !client.IsErrNotFound(err) {
errs = append(errs, fmt.Errorf("remove container '%s': %w", ctr.ID, err))
}
}
slog.Info("Removed uncloud-managed Docker containers.", "count", removed)
}
// Remove the uncloud Docker network and related iptables rules.
nw, err := c.client.NetworkInspect(ctx, NetworkName, dnetwork.InspectOptions{})
if err == nil {
bridgeName := "br-" + nw.ID[:12]
var subnet netip.Prefix
if len(nw.IPAM.Config) > 0 {
subnet, _ = netip.ParsePrefix(nw.IPAM.Config[0].Subnet)
}
if subnet.IsValid() {
if err = cleanupIptables(bridgeName, subnet); err != nil {
errs = append(errs, fmt.Errorf("cleanup iptables for Docker network '%s': %w", NetworkName, err))
} else {
slog.Info("Cleaned up iptables rules for Docker network.", "name", NetworkName, "bridge", bridgeName)
}
}
if err = c.client.NetworkRemove(ctx, NetworkName); err == nil {
slog.Info("Docker network removed.", "name", NetworkName)
} else if !client.IsErrNotFound(err) {
errs = append(errs, fmt.Errorf("remove Docker network '%s': %w", NetworkName, err))
}
} else if !client.IsErrNotFound(err) {
errs = append(errs, fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err))
}
return errors.Join(errs...)
}
-14
View File
@@ -1,14 +0,0 @@
//go:build darwin
package docker
import (
"context"
"fmt"
"net/netip"
)
// EnsureUncloudNetwork is a stub for Darwin.
func (m *Manager) EnsureUncloudNetwork(ctx context.Context, subnet netip.Prefix, dnsServer netip.Addr) error {
return fmt.Errorf("not supported on Darwin")
}
-107
View File
@@ -1,107 +0,0 @@
package docker
import (
"context"
"fmt"
"log/slog"
"net/netip"
"strconv"
dnetwork "github.com/docker/docker/api/types/network"
"github.com/docker/docker/client"
"github.com/docker/docker/libnetwork/iptables"
"github.com/psviderski/uncloud/internal/machine/dns"
"github.com/psviderski/uncloud/internal/machine/firewall"
"github.com/psviderski/uncloud/internal/machine/network"
)
// EnsureUncloudNetwork creates the Docker bridge network NetworkName with the provided machine subnet
// if it doesn't exist. If the network exists but has a different subnet, it removes and recreates the network.
// It also configures iptables to allow container access from the WireGuard network.
func (m *Manager) EnsureUncloudNetwork(ctx context.Context, subnet netip.Prefix, dnsServer netip.Addr) error {
// Ensure the Docker network 'uncloud' is created with the correct subnet.
needsCreation := false
nw, err := m.client.NetworkInspect(ctx, NetworkName, dnetwork.InspectOptions{})
if err != nil {
if !client.IsErrNotFound(err) {
return fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err)
}
needsCreation = true
} else if nw.IPAM.Config[0].Subnet != subnet.String() {
// Remove the Docker network if the subnet is different.
// It could be a leftover from a previous incomplete cleanup.
slog.Info(
"Removing Docker network with old subnet.", "name", NetworkName, "subnet", nw.IPAM.Config[0].Subnet,
)
if err = m.client.NetworkRemove(ctx, NetworkName); err != nil {
// It can still fail if the network is in use by a container. Leave it to the user to resolve the issue.
return fmt.Errorf("remove Docker network '%s': %w", NetworkName, err)
}
needsCreation = true
}
if needsCreation {
if _, err = m.client.NetworkCreate(
ctx, NetworkName, dnetwork.CreateOptions{
Driver: "bridge",
Scope: "local",
IPAM: &dnetwork.IPAM{
Config: []dnetwork.IPAMConfig{
{
Subnet: subnet.String(),
},
},
},
},
); err != nil {
return fmt.Errorf("create Docker network '%s': %w", NetworkName, err)
}
slog.Info("Docker network created.", "name", NetworkName, "subnet", subnet.String())
if nw, err = m.client.NetworkInspect(ctx, NetworkName, dnetwork.InspectOptions{}); err != nil {
return fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err)
}
}
// Configure iptables to allow WireGuard network to access containers. The Docker daemon should have already
// created the DOCKER-USER chain at this point.
// TODO: check if this works when firewalld used instead of raw iptables. The Docker daemon has a different
// code path for firewalld.
// Bridge name doesn't seem to be documented but this is the source code where it is generated:
// https://github.com/moby/moby/blob/v27.2.1/libnetwork/drivers/bridge/bridge_linux.go#L664
bridgeName := "br-" + nw.ID[:12]
if err = configureIptables(bridgeName, dnsServer); err != nil {
return fmt.Errorf("configure iptables for Docker network '%s': %w", NetworkName, err)
}
return nil
}
// configureIptables configures iptables rules for the uncloud Docker network.
func configureIptables(bridgeName string, dnsServer netip.Addr) error {
ipt := iptables.GetIptable(iptables.IPv4)
// Allow traffic from other machines and their containers through the WG mesh to the Uncloud containers
// on the machine.
wgRule := []string{"--in-interface", network.WireGuardInterfaceName, "--out-interface", bridgeName, "-j", "ACCEPT"}
if err := ipt.ProgramRule(iptables.Filter, firewall.DockerUserChain, iptables.Insert, wgRule); err != nil {
return fmt.Errorf("insert iptables rule: %w", err)
}
// Allow DNS queries from Uncloud containers to the embedded DNS server.
for _, proto := range []string{"udp", "tcp"} {
dnsRule := []string{
"--in-interface", bridgeName,
"--dst", dnsServer.String(),
"--protocol", proto,
"--dport", strconv.Itoa(dns.Port),
"-j", "ACCEPT",
}
if err := ipt.ProgramRule(iptables.Filter, firewall.UncloudInputChain, iptables.Insert, dnsRule); err != nil {
return fmt.Errorf("insert iptables rule: %w", err)
}
}
return nil
}
+72 -55
View File
@@ -2,19 +2,21 @@ package docker
import ( import (
"context" "context"
"database/sql"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"io" "io"
"log/slog" "log/slog"
"net/netip" "net/netip"
"os"
"regexp" "regexp"
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
"github.com/distribution/reference" "github.com/distribution/reference"
dockercommand "github.com/docker/cli/cli/command"
dockerconfig "github.com/docker/cli/cli/config"
"github.com/docker/docker/api/types" "github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters" "github.com/docker/docker/api/types/filters"
@@ -45,20 +47,49 @@ var fullDockerIDRegex = regexp.MustCompile(`^[a-f0-9]{64}$`)
// Server implements the gRPC Docker service that proxies requests to the Docker daemon. // Server implements the gRPC Docker service that proxies requests to the Docker daemon.
type Server struct { type Server struct {
pb.UnimplementedDockerServer pb.UnimplementedDockerServer
client *client.Client client *client.Client
db *sqlx.DB service *Service
db *sqlx.DB
// internalDNSIP is a function that returns the IP address of the internal DNS server. It may return an empty // internalDNSIP is a function that returns the IP address of the internal DNS server. It may return an empty
// address if the address is unknown (e.g. when the machine is not initialised yet). // address if the address is unknown (e.g. when the machine is not initialised yet).
internalDNSIP func() netip.Addr internalDNSIP func() netip.Addr
// networkReady is a function that returns true if the Docker network is ready for containers.
networkReady func() bool
// waitForNetworkReady is a function that waits for the Docker network to be ready for containers.
waitForNetworkReady func(ctx context.Context) error
} }
// NewServer creates a new Docker gRPC server with the provided Docker client. // ServerOption configures the Docker server.
func NewServer(cli *client.Client, db *sqlx.DB, internalDNSIP func() netip.Addr) *Server { type ServerOption func(*Server)
return &Server{
client: cli, // WithNetworkReady sets the network readiness check function.
func WithNetworkReady(networkReady func() bool) ServerOption {
return func(s *Server) {
s.networkReady = networkReady
}
}
// WithWaitForNetworkReady sets the network readiness wait function.
func WithWaitForNetworkReady(waitForNetworkReady func(ctx context.Context) error) ServerOption {
return func(s *Server) {
s.waitForNetworkReady = waitForNetworkReady
}
}
// NewServer creates a new Docker gRPC server with the provided Docker service.
func NewServer(service *Service, db *sqlx.DB, internalDNSIP func() netip.Addr, opts ...ServerOption) *Server {
s := &Server{
client: service.Client,
service: service,
db: db, db: db,
internalDNSIP: internalDNSIP, internalDNSIP: internalDNSIP,
} }
for _, opt := range opts {
opt(s)
}
return s
} }
// CreateContainer creates a new container based on the given configuration. // CreateContainer creates a new container based on the given configuration.
@@ -118,6 +149,15 @@ func (s *Server) InspectContainer(ctx context.Context, req *pb.InspectContainerR
// StartContainer starts a container with the given ID and options. // StartContainer starts a container with the given ID and options.
func (s *Server) StartContainer(ctx context.Context, req *pb.StartContainerRequest) (*emptypb.Empty, error) { func (s *Server) StartContainer(ctx context.Context, req *pb.StartContainerRequest) (*emptypb.Empty, error) {
// Wait for Docker network to be ready before starting the container
if s.waitForNetworkReady != nil {
if err := s.waitForNetworkReady(ctx); err != nil {
return nil, status.Errorf(codes.Unavailable, "Docker network not ready: %v", err)
}
} else if s.networkReady != nil && !s.networkReady() {
return nil, status.Errorf(codes.Unavailable, "Docker network not ready")
}
var opts container.StartOptions var opts container.StartOptions
if len(req.Options) > 0 { if len(req.Options) > 0 {
if err := json.Unmarshal(req.Options, &opts); err != nil { if err := json.Unmarshal(req.Options, &opts); err != nil {
@@ -229,7 +269,6 @@ func (s *Server) RemoveContainer(ctx context.Context, req *pb.RemoveContainerReq
func (s *Server) PullImage(req *pb.PullImageRequest, stream grpc.ServerStreamingServer[pb.JSONMessage]) error { func (s *Server) PullImage(req *pb.PullImageRequest, stream grpc.ServerStreamingServer[pb.JSONMessage]) error {
ctx := stream.Context() ctx := stream.Context()
// TODO: replace with another JSON serializable type (PullOptions.PrivilegeFunc is not serializable).
var opts image.PullOptions var opts image.PullOptions
if len(req.Options) > 0 { if len(req.Options) > 0 {
if err := json.Unmarshal(req.Options, &opts); err != nil { if err := json.Unmarshal(req.Options, &opts); err != nil {
@@ -237,6 +276,14 @@ func (s *Server) PullImage(req *pb.PullImageRequest, stream grpc.ServerStreaming
} }
} }
if opts.RegistryAuth == "" {
// Try to retrieve the authentication token for the image from the default local Docker config file.
dockerConfig := dockerconfig.LoadDefaultConfigFile(os.Stderr)
if encodedAuth, err := dockercommand.RetrieveAuthTokenFromImage(dockerConfig, req.Image); err == nil {
opts.RegistryAuth = encodedAuth
}
}
respBody, err := s.client.ImagePull(ctx, req.Image, opts) respBody, err := s.client.ImagePull(ctx, req.Image, opts)
if err != nil { if err != nil {
return status.Errorf(codes.Internal, err.Error()) return status.Errorf(codes.Internal, err.Error())
@@ -419,6 +466,7 @@ func (s *Server) RemoveVolume(ctx context.Context, req *pb.RemoveVolumeRequest)
} }
// CreateServiceContainer creates a new container for the service with the given specifications. // CreateServiceContainer creates a new container for the service with the given specifications.
// TODO: move the main logic to the Docker service and remove db dependency from the server.
func (s *Server) CreateServiceContainer( func (s *Server) CreateServiceContainer(
ctx context.Context, req *pb.CreateServiceContainerRequest, ctx context.Context, req *pb.CreateServiceContainerRequest,
) (*pb.CreateContainerResponse, error) { ) (*pb.CreateContainerResponse, error) {
@@ -510,10 +558,10 @@ func (s *Server) CreateServiceContainer(
Memory: spec.Container.Resources.Memory, Memory: spec.Container.Resources.Memory,
MemoryReservation: spec.Container.Resources.MemoryReservation, MemoryReservation: spec.Container.Resources.MemoryReservation,
}, },
// Always restart service containers if they exit or a machine restarts. // Restart service containers if they exit or a machine restarts unless they are explicitly stopped.
// For one-off containers and batch jobs we plan to use a different service type/mode. // For one-off containers and batch jobs we plan to use a different service type/mode.
RestartPolicy: container.RestartPolicy{ RestartPolicy: container.RestartPolicy{
Name: container.RestartPolicyAlways, Name: container.RestartPolicyUnlessStopped,
}, },
} }
@@ -673,7 +721,7 @@ func (s *Server) verifyDockerVolumesExist(ctx context.Context, mounts []mount.Mo
func (s *Server) InspectServiceContainer( func (s *Server) InspectServiceContainer(
ctx context.Context, req *pb.InspectContainerRequest, ctx context.Context, req *pb.InspectContainerRequest,
) (*pb.ServiceContainer, error) { ) (*pb.ServiceContainer, error) {
ctr, err := s.client.ContainerInspect(ctx, req.Id) serviceCtr, err := s.service.InspectServiceContainer(ctx, req.Id)
if err != nil { if err != nil {
if client.IsErrNotFound(err) { if client.IsErrNotFound(err) {
return nil, status.Errorf(codes.NotFound, err.Error()) return nil, status.Errorf(codes.NotFound, err.Error())
@@ -681,19 +729,14 @@ func (s *Server) InspectServiceContainer(
return nil, status.Errorf(codes.Internal, err.Error()) return nil, status.Errorf(codes.Internal, err.Error())
} }
ctrBytes, err := json.Marshal(ctr) ctrBytes, err := json.Marshal(serviceCtr.Container)
if err != nil { if err != nil {
return nil, status.Errorf(codes.Internal, "marshal response: %v", err) return nil, status.Errorf(codes.Internal, "marshal container: %v", err)
} }
var specBytes []byte specBytes, err := json.Marshal(serviceCtr.ServiceSpec)
err = s.db.QueryRowContext(ctx, `SELECT service_spec FROM containers WHERE id = $1`, ctr.ID).Scan(&specBytes)
if err != nil { if err != nil {
if errors.Is(err, sql.ErrNoRows) { return nil, status.Errorf(codes.Internal, "marshal service spec: %v", err)
return nil, status.Errorf(codes.NotFound, "service spec not found for container: '%s'", ctr.ID)
}
return nil, status.Errorf(codes.Internal, "get service spec for container '%s' from machine database: %v",
ctr.ID, err)
} }
return &pb.ServiceContainer{ return &pb.ServiceContainer{
@@ -725,54 +768,28 @@ func (s *Server) ListServiceContainers(
return nil, status.Errorf(codes.InvalidArgument, "unmarshal filters: %v", err) return nil, status.Errorf(codes.InvalidArgument, "unmarshal filters: %v", err)
} }
opts.Filters = args opts.Filters = args
} else {
opts.Filters = filters.NewArgs()
} }
} }
// Only uncloud-managed containers that belong to some service.
opts.Filters.Add("label", api.LabelServiceID)
opts.Filters.Add("label", api.LabelManaged)
containerSummaries, err := s.client.ContainerList(ctx, opts) containers, err := s.service.ListServiceContainers(ctx, req.ServiceId, opts)
if err != nil { if err != nil {
return nil, status.Error(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
} }
containers := make([]*pb.ServiceContainer, 0, len(containerSummaries)) // Convert to protobuf format.
for _, cs := range containerSummaries { pbContainers := make([]*pb.ServiceContainer, 0, len(containers))
if req.ServiceId != "" && for _, ctr := range containers {
cs.Labels[api.LabelServiceID] != req.ServiceId && cs.Labels[api.LabelServiceName] != req.ServiceId { ctrBytes, err := json.Marshal(ctr.Container)
continue
}
ctr, err := s.client.ContainerInspect(ctx, cs.ID)
if err != nil {
if client.IsErrNotFound(err) {
// The listed container may have been removed while we were inspecting other containers.
continue
}
return nil, status.Errorf(codes.Internal, "inspect container %s: %v", cs.ID, err)
}
ctrBytes, err := json.Marshal(ctr)
if err != nil { if err != nil {
return nil, status.Errorf(codes.Internal, "marshal container: %v", err) return nil, status.Errorf(codes.Internal, "marshal container: %v", err)
} }
var specBytes []byte specBytes, err := json.Marshal(ctr.ServiceSpec)
err = s.db.QueryRowContext(ctx, `SELECT service_spec FROM containers WHERE id = $1`, ctr.ID).Scan(&specBytes)
if err != nil { if err != nil {
if errors.Is(err, sql.ErrNoRows) { return nil, status.Errorf(codes.Internal, "marshal service spec: %v", err)
// If this happens, there is a bug in the code, or someone manually removed the container from the DB,
// or created a managed container out of band.
slog.Error("Service container not found in machine database.", "id", ctr.ID)
// Just ignore such a container to not fail the list operation as it's not easily recoverable.
continue
}
return nil, status.Errorf(codes.Internal, "get service spec for container '%s' from machine database: %v",
ctr.ID, err)
} }
containers = append(containers, &pb.ServiceContainer{ pbContainers = append(pbContainers, &pb.ServiceContainer{
Container: ctrBytes, Container: ctrBytes,
ServiceSpec: specBytes, ServiceSpec: specBytes,
}) })
@@ -781,7 +798,7 @@ func (s *Server) ListServiceContainers(
return &pb.ListServiceContainersResponse{ return &pb.ListServiceContainersResponse{
Messages: []*pb.MachineServiceContainers{ Messages: []*pb.MachineServiceContainers{
{ {
Containers: containers, Containers: pbContainers,
}, },
}, },
}, nil }, nil
+103
View File
@@ -0,0 +1,103 @@
package docker
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"log/slog"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/client"
"github.com/jmoiron/sqlx"
"github.com/psviderski/uncloud/pkg/api"
)
// Service provides higher-level Docker operations that extends Docker API with Uncloud-specific data
// from the machine database.
type Service struct {
Client *client.Client
db *sqlx.DB
}
// NewService creates a new Docker service instance.
func NewService(client *client.Client, db *sqlx.DB) *Service {
return &Service{
Client: client,
db: db,
}
}
// InspectServiceContainer inspects a Docker container and retrieves its associated ServiceSpec
// from the machine database, returning a complete ServiceContainer.
func (s *Service) InspectServiceContainer(ctx context.Context, nameOrID string) (api.ServiceContainer, error) {
var serviceCtr api.ServiceContainer
ctr, err := s.Client.ContainerInspect(ctx, nameOrID)
if err != nil {
return serviceCtr, err
}
if _, ok := ctr.Config.Labels[api.LabelManaged]; !ok {
return serviceCtr, fmt.Errorf("container '%s' is not managed by Uncloud", nameOrID)
}
serviceCtr.Container = api.Container{ContainerJSON: ctr}
// Retrieve ServiceSpec from the machine database.
var specBytes []byte
err = s.db.QueryRowContext(ctx, `SELECT service_spec FROM containers WHERE id = $1`, ctr.ID).Scan(&specBytes)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
// If this happens, there is a bug in the code, or someone manually removed the container from the DB,
// or created a managed container out of band or by previous uncloud installation.
return serviceCtr, fmt.Errorf("service spec not found for container '%s' in machine DB", ctr.ID)
}
return serviceCtr, fmt.Errorf("get service spec for container '%s' from machine DB: %w", ctr.ID, err)
}
if err = json.Unmarshal(specBytes, &serviceCtr.ServiceSpec); err != nil {
return serviceCtr, fmt.Errorf("unmarshal service spec for container '%s': %w", ctr.ID, err)
}
return serviceCtr, nil
}
// ListServiceContainers lists Docker containers that belong to the service with the given name or ID.
// If serviceIDOrName is empty, all service containers are returned. The opts parameter allows additional filtering.
func (s *Service) ListServiceContainers(
ctx context.Context, serviceNameOrID string, opts container.ListOptions,
) ([]api.ServiceContainer, error) {
if opts.Filters.Len() == 0 {
opts.Filters = filters.NewArgs()
}
// Add labels to existing filters to list only Uncloud-managed service containers.
opts.Filters.Add("label", api.LabelServiceID)
opts.Filters.Add("label", api.LabelManaged)
containerSummaries, err := s.Client.ContainerList(ctx, opts)
if err != nil {
return nil, err
}
var containers []api.ServiceContainer
for _, cs := range containerSummaries {
// Filter by service name or ID if provided.
if serviceNameOrID != "" &&
cs.Labels[api.LabelServiceID] != serviceNameOrID &&
cs.Labels[api.LabelServiceName] != serviceNameOrID {
continue
}
ctr, err := s.InspectServiceContainer(ctx, cs.ID)
if err != nil {
// Log error but continue with other containers.
slog.Error("Failed to inspect service container.", "service", serviceNameOrID, "id", cs.ID, "err", err)
continue
}
containers = append(containers, ctr)
}
return containers, nil
}
@@ -6,3 +6,8 @@ import "fmt"
func ConfigureIptablesChains() error { func ConfigureIptablesChains() error {
return fmt.Errorf("not supported on Darwin") return fmt.Errorf("not supported on Darwin")
} }
// CleanupIptablesChains is a stub for Darwin.
func CleanupIptablesChains() error {
return fmt.Errorf("not supported on Darwin")
}
+134 -45
View File
@@ -2,10 +2,13 @@ package firewall
import ( import (
"fmt" "fmt"
"log/slog"
"strconv" "strconv"
"strings" "strings"
"github.com/docker/docker/libnetwork/iptables" "github.com/docker/docker/libnetwork/iptables"
"github.com/psviderski/uncloud/internal/machine/constants"
"github.com/psviderski/uncloud/internal/machine/corroservice"
"github.com/psviderski/uncloud/internal/machine/network" "github.com/psviderski/uncloud/internal/machine/network"
) )
@@ -16,59 +19,145 @@ const (
// ConfigureIptablesChains sets up custom iptables chains and initial firewall rules for Uncloud networking. // ConfigureIptablesChains sets up custom iptables chains and initial firewall rules for Uncloud networking.
func ConfigureIptablesChains() error { func ConfigureIptablesChains() error {
// Ensure iptables UNCLOUD-INPUT chain with a RETURN rule exists. All existing rules are flushed. if err := createIptablesChains(); err != nil {
ipt := iptables.GetIptable(iptables.IPv4) return err
if _, err := ipt.NewChain(UncloudInputChain, iptables.Filter); err != nil {
return fmt.Errorf("create iptables chain '%s': %w", UncloudInputChain, err)
}
if err := ipt.RawCombinedOutput("-t", string(iptables.Filter), "-F", UncloudInputChain); err != nil {
return fmt.Errorf("flush iptables chain '%s': %w", UncloudInputChain, err)
}
if err := ipt.AddReturnRule(UncloudInputChain); err != nil {
return fmt.Errorf("add the RETURN rule for iptables chain '%s': %w", UncloudInputChain, err)
} }
// Ensure the main iptables INPUT chain has a jump rule to the UNCLOUD-INPUT chain before any DROP/REJECT rules. ipt4 := iptables.GetIptable(iptables.IPv4)
jumpRule := []string{"-m", "comment", "--comment", "Uncloud-managed", "-j", UncloudInputChain} ipt6 := iptables.GetIptable(iptables.IPv6)
if !ipt.Exists(iptables.Filter, "INPUT", jumpRule...) {
// Look for the first DROP/REJECT rule in the INPUT chain.
out, err := ipt.Raw("-t", string(iptables.Filter), "-L", "INPUT", "--line-numbers")
if err != nil {
return fmt.Errorf("get iptables rules for chain '%s': %w", UncloudInputChain, err)
}
firstRejectRuleNum := 0
for _, line := range strings.Split(string(out), "\n") {
fields := strings.Fields(line)
if len(fields) < 2 {
continue
}
if fields[1] == "DROP" || fields[1] == "REJECT" {
if ruleNum, err := strconv.Atoi(fields[0]); err == nil {
firstRejectRuleNum = ruleNum
break
}
}
}
var addJumpRule []string
if firstRejectRuleNum > 0 {
addJumpRule = append([]string{"-t", string(iptables.Filter), "-I", "INPUT", strconv.Itoa(firstRejectRuleNum)},
jumpRule...)
} else {
addJumpRule = append([]string{"-t", string(iptables.Filter), "-A", "INPUT"}, jumpRule...)
}
if err = ipt.RawCombinedOutput(addJumpRule...); err != nil {
return fmt.Errorf("add iptables rule '%s': %w", strings.Join(addJumpRule, " "), err)
}
}
// Allow WireGuard traffic to the machine. // Allow WireGuard traffic to the machine.
acceptWireGuardRule := []string{"-p", "udp", "--dport", strconv.Itoa(network.WireGuardPort), "-j", "ACCEPT"} acceptWireGuardRule := []string{"-p", "udp", "--dport", strconv.Itoa(network.WireGuardPort), "-j", "ACCEPT"}
err := ipt.ProgramRule(iptables.Filter, UncloudInputChain, iptables.Insert, acceptWireGuardRule) err := ipt4.ProgramRule(iptables.Filter, UncloudInputChain, iptables.Insert, acceptWireGuardRule)
if err != nil { if err != nil {
return fmt.Errorf("insert iptables rule '%s': %w", strings.Join(acceptWireGuardRule, " "), err) return fmt.Errorf("insert iptables rule '%s': %w", strings.Join(acceptWireGuardRule, " "), err)
} }
// Allow cluster machines to access Machine API via the management IPv6 WireGuard network.
acceptMachineAPIRule := []string{
"-i", network.WireGuardInterfaceName,
"-s", "fdcc::/16",
"-p", "tcp",
"--dport", strconv.Itoa(constants.MachineAPIPort),
"-j", "ACCEPT",
}
// Allow Corrosion gossip traffic from cluster machines via the management IPv6 WireGuard network.
acceptCorrosionGossipRule := []string{
"-i", network.WireGuardInterfaceName,
"-s", "fdcc::/16",
"-p", "udp",
"--dport", strconv.Itoa(corroservice.DefaultGossipPort),
"-j", "ACCEPT",
}
for _, rule := range [][]string{acceptMachineAPIRule, acceptCorrosionGossipRule} {
if err = ipt6.ProgramRule(iptables.Filter, UncloudInputChain, iptables.Insert, rule); err != nil {
return fmt.Errorf("insert ip6tables rule '%s': %w", strings.Join(rule, " "), err)
}
}
return nil
}
// createIptablesChains ensures UNCLOUD-INPUT iptables and ip6tables chains exist and
// there are jump rules from the main INPUT chains.
func createIptablesChains() error {
ipt4 := iptables.GetIptable(iptables.IPv4)
ipt6 := iptables.GetIptable(iptables.IPv6)
for i, ipt := range []*iptables.IPTable{ipt4, ipt6} {
iptBin := "iptables"
if i == 1 {
iptBin = "ip6tables"
}
// Ensure UNCLOUD-INPUT chain exists. All existing rules are flushed.
if _, err := ipt.NewChain(UncloudInputChain, iptables.Filter); err != nil {
return fmt.Errorf("create %s chain '%s': %w", iptBin, UncloudInputChain, err)
}
if err := ipt.RawCombinedOutput("-t", string(iptables.Filter), "-F", UncloudInputChain); err != nil {
return fmt.Errorf("flush %s chain '%s': %w", iptBin, UncloudInputChain, err)
}
// Ensure the main INPUT chain has a jump rule to the UNCLOUD-INPUT chain before any DROP/REJECT rules.
jumpRule := []string{"-m", "comment", "--comment", "Uncloud-managed", "-j", UncloudInputChain}
if !ipt.Exists(iptables.Filter, "INPUT", jumpRule...) {
// Look for the first DROP/REJECT rule in the INPUT chain.
out, err := ipt.Raw("-t", string(iptables.Filter), "-L", "INPUT", "--line-numbers")
if err != nil {
return fmt.Errorf("get %s rules for chain '%s': %w", iptBin, UncloudInputChain, err)
}
firstRejectRuleNum := 0
for _, line := range strings.Split(string(out), "\n") {
fields := strings.Fields(line)
if len(fields) < 2 {
continue
}
if fields[1] == "DROP" || fields[1] == "REJECT" {
if ruleNum, err := strconv.Atoi(fields[0]); err == nil {
firstRejectRuleNum = ruleNum
break
}
}
}
var addJumpRule []string
if firstRejectRuleNum > 0 {
addJumpRule = append([]string{
"-t", string(iptables.Filter),
"-I", "INPUT",
strconv.Itoa(firstRejectRuleNum),
}, jumpRule...)
} else {
addJumpRule = append([]string{
"-t", string(iptables.Filter),
"-A", "INPUT",
}, jumpRule...)
}
if err = ipt.RawCombinedOutput(addJumpRule...); err != nil {
return fmt.Errorf("add %s rule '%s': %w", iptBin, strings.Join(addJumpRule, " "), err)
}
}
}
return nil
}
// CleanupIptablesChains removes the custom iptables chains and rules created by ConfigureIptablesChains.
func CleanupIptablesChains() error {
ipt4 := iptables.GetIptable(iptables.IPv4)
ipt6 := iptables.GetIptable(iptables.IPv6)
for i, ipt := range []*iptables.IPTable{ipt4, ipt6} {
iptBin := "iptables"
if i == 1 {
iptBin = "ip6tables"
}
// First, remove the jump rule from INPUT chain to UNCLOUD-INPUT.
jumpRule := []string{"-m", "comment", "--comment", "Uncloud-managed", "-j", UncloudInputChain}
if err := ipt.ProgramRule(iptables.Filter, "INPUT", iptables.Delete, jumpRule); err != nil {
return fmt.Errorf("delete %s jump rule from INPUT: %w", iptBin, err)
}
// Flush all rules from UNCLOUD-INPUT chain as it must be empty before deletion.
if err := ipt.RawCombinedOutput("-t", string(iptables.Filter), "-F", UncloudInputChain); err != nil {
// Chain might not exist which is fine.
if !strings.Contains(err.Error(), "No chain") {
return fmt.Errorf("flush %s chain '%s': %w", iptBin, UncloudInputChain, err)
}
}
// Delete the UNCLOUD-INPUT chain.
if err := ipt.RawCombinedOutput("-t", string(iptables.Filter), "-X", UncloudInputChain); err != nil {
// Chain might not exist which is fine.
if !strings.Contains(err.Error(), "No chain") {
return fmt.Errorf("delete %s chain '%s': %w", iptBin, UncloudInputChain, err)
}
} else {
slog.Info(fmt.Sprintf("Deleted %s chain.", iptBin), "chain", UncloudInputChain)
}
}
return nil return nil
} }
+233 -158
View File
@@ -13,6 +13,7 @@ import (
"path/filepath" "path/filepath"
"slices" "slices"
"strconv" "strconv"
"sync"
"github.com/docker/docker/client" "github.com/docker/docker/client"
"github.com/docker/go-connections/sockets" "github.com/docker/go-connections/sockets"
@@ -23,12 +24,12 @@ import (
apiproxy "github.com/psviderski/uncloud/internal/machine/api/proxy" apiproxy "github.com/psviderski/uncloud/internal/machine/api/proxy"
"github.com/psviderski/uncloud/internal/machine/caddyconfig" "github.com/psviderski/uncloud/internal/machine/caddyconfig"
"github.com/psviderski/uncloud/internal/machine/cluster" "github.com/psviderski/uncloud/internal/machine/cluster"
"github.com/psviderski/uncloud/internal/machine/constants"
"github.com/psviderski/uncloud/internal/machine/corroservice" "github.com/psviderski/uncloud/internal/machine/corroservice"
"github.com/psviderski/uncloud/internal/machine/dns" "github.com/psviderski/uncloud/internal/machine/dns"
machinedocker "github.com/psviderski/uncloud/internal/machine/docker" machinedocker "github.com/psviderski/uncloud/internal/machine/docker"
"github.com/psviderski/uncloud/internal/machine/network" "github.com/psviderski/uncloud/internal/machine/network"
"github.com/psviderski/uncloud/internal/machine/store" "github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/uncloud/pkg/api"
"github.com/siderolabs/grpc-proxy/proxy" "github.com/siderolabs/grpc-proxy/proxy"
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
"google.golang.org/grpc" "google.golang.org/grpc"
@@ -41,6 +42,9 @@ const (
DefaultMachineSockPath = "/run/uncloud/machine.sock" DefaultMachineSockPath = "/run/uncloud/machine.sock"
DefaultUncloudSockPath = "/run/uncloud/uncloud.sock" DefaultUncloudSockPath = "/run/uncloud/uncloud.sock"
DefaultSockGroup = "uncloud" DefaultSockGroup = "uncloud"
// DefaultCaddyAdminSockPath is the default path to the Caddy admin socket for validating the generated Caddy
// reverse proxy configuration.
DefaultCaddyAdminSockPath = "/run/uncloud/caddy/admin.sock"
) )
type Config struct { type Config struct {
@@ -60,9 +64,9 @@ type Config struct {
// DockerClient manages system and user containers using the local Docker daemon. // DockerClient manages system and user containers using the local Docker daemon.
DockerClient *client.Client DockerClient *client.Client
// CaddyConfigPath specifies where the machine generates the Caddy reverse proxy configuration file for routing // CaddyConfigDir specifies the directory where the machine generates the Caddy reverse proxy configuration file
// external traffic to service containers across the internal network. Default is DataDir/caddy/caddy.json. // for routing external traffic to service containers across the internal network. Default is DataDir/caddy.
CaddyConfigPath string CaddyConfigDir string
// DNSUpstreams specifies the upstream DNS servers for the embedded internal DNS server. // DNSUpstreams specifies the upstream DNS servers for the embedded internal DNS server.
DNSUpstreams []netip.AddrPort DNSUpstreams []netip.AddrPort
} }
@@ -127,8 +131,8 @@ func (c *Config) SetDefaults() (*Config, error) {
} }
} }
if cfg.CaddyConfigPath == "" { if cfg.CaddyConfigDir == "" {
cfg.CaddyConfigPath = filepath.Join(cfg.DataDir, "caddy", "caddy.json") cfg.CaddyConfigDir = filepath.Join(cfg.DataDir, "caddy")
} }
return &cfg, nil return &cfg, nil
@@ -149,11 +153,20 @@ type Machine struct {
started chan struct{} started chan struct{}
// initialised is signalled when the machine is configured as a member of a cluster. // initialised is signalled when the machine is configured as a member of a cluster.
initialised chan struct{} initialised chan struct{}
// networkReady is signalled when the Docker network is configured and ready for containers.
networkReady chan struct{}
// resetting is true when the machine is being reset.
resetting bool
// stop cancels the Run method context to stop the machine gracefully.
stop func()
clusterCtrl *clusterController
// store is the cluster store backed by a distributed Corrosion database. // store is the cluster store backed by a distributed Corrosion database.
store *store.Store store *store.Store
cluster *cluster.Cluster cluster *cluster.Cluster
docker *machinedocker.Server // dockerService provides high-level operations for managing Docker containers.
dockerService *machinedocker.Service
dockerServer *machinedocker.Server
// localMachineServer is the gRPC server for the machine API listening on the local Unix socket. // localMachineServer is the gRPC server for the machine API listening on the local Unix socket.
localMachineServer *grpc.Server localMachineServer *grpc.Server
@@ -163,6 +176,9 @@ type Machine struct {
// It proxies requests to the local or remote machine API servers depending on the request targets // It proxies requests to the local or remote machine API servers depending on the request targets
// and aggregates responses. // and aggregates responses.
localProxyServer *grpc.Server localProxyServer *grpc.Server
// mu protects the Machine from concurrent reads and writes.
mu sync.RWMutex
} }
func NewMachine(config *Config) (*Machine, error) { func NewMachine(config *Config) (*Machine, error) {
@@ -210,19 +226,16 @@ func NewMachine(config *Config) (*Machine, error) {
c := cluster.NewCluster(corroStore, corroAdmin) c := cluster.NewCluster(corroStore, corroAdmin)
// Init dependencies for a gRPC Docker server that proxies requests to the local Docker daemon. // Init dependencies for a gRPC Docker server that proxies requests to the local Docker daemon.
dockerCli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
if err != nil {
return nil, fmt.Errorf("create Docker client: %w", err)
}
dbFilePath := filepath.Join(config.DataDir, DBFileName) dbFilePath := filepath.Join(config.DataDir, DBFileName)
db, err := NewDB(dbFilePath) db, err := NewDB(dbFilePath)
if err != nil { if err != nil {
return nil, fmt.Errorf("init machine database: %w", err) return nil, fmt.Errorf("init machine database: %w", err)
} }
dockerService := machinedocker.NewService(config.DockerClient, db)
// Init a local gRPC proxy server that proxies requests to the local or remote machine API servers. // Init a local gRPC proxy server that proxies requests to the local or remote machine API servers.
proxyDirector := apiproxy.NewDirector(config.MachineSockPath, APIPort) proxyDirector := apiproxy.NewDirector(config.MachineSockPath, constants.MachineAPIPort)
localProxyServer := grpc.NewServer( localProxyServer := grpc.NewServer(
grpc.ForceServerCodecV2(proxy.Codec()), grpc.ForceServerCodecV2(proxy.Codec()),
grpc.UnknownServiceHandler( grpc.UnknownServiceHandler(
@@ -235,8 +248,10 @@ func NewMachine(config *Config) (*Machine, error) {
state: state, state: state,
started: make(chan struct{}), started: make(chan struct{}),
initialised: make(chan struct{}, 1), initialised: make(chan struct{}, 1),
networkReady: make(chan struct{}),
store: corroStore, store: corroStore,
cluster: c, cluster: c,
dockerService: dockerService,
localProxyServer: localProxyServer, localProxyServer: localProxyServer,
proxyDirector: proxyDirector, proxyDirector: proxyDirector,
} }
@@ -245,8 +260,11 @@ func NewMachine(config *Config) (*Machine, error) {
internalDNSIP := func() netip.Addr { internalDNSIP := func() netip.Addr {
return m.IP() return m.IP()
} }
m.docker = machinedocker.NewServer(dockerCli, db, internalDNSIP) m.dockerServer = machinedocker.NewServer(dockerService, db, internalDNSIP,
m.localMachineServer = newGRPCServer(m, c, m.docker) machinedocker.WithNetworkReady(m.IsNetworkReady),
machinedocker.WithWaitForNetworkReady(m.WaitForNetworkReady))
caddyServer := caddyconfig.NewServer(caddyconfig.NewService(config.CaddyConfigDir))
m.localMachineServer = newGRPCServer(m, c, m.dockerServer, caddyServer)
if m.Initialised() { if m.Initialised() {
m.initialised <- struct{}{} m.initialised <- struct{}{}
@@ -255,11 +273,12 @@ func NewMachine(config *Config) (*Machine, error) {
return m, nil return m, nil
} }
func newGRPCServer(m pb.MachineServer, c pb.ClusterServer, d pb.DockerServer) *grpc.Server { func newGRPCServer(m pb.MachineServer, c pb.ClusterServer, d pb.DockerServer, caddy pb.CaddyServer) *grpc.Server {
s := grpc.NewServer() s := grpc.NewServer()
pb.RegisterMachineServer(s, m) pb.RegisterMachineServer(s, m)
pb.RegisterClusterServer(s, c) pb.RegisterClusterServer(s, c)
pb.RegisterDockerServer(s, d) pb.RegisterDockerServer(s, d)
pb.RegisterCaddyServer(s, caddy)
return s return s
} }
@@ -287,6 +306,9 @@ func (m *Machine) IP() netip.Addr {
} }
func (m *Machine) Run(ctx context.Context) error { func (m *Machine) Run(ctx context.Context) error {
// Create a cancellable context for the Run method to allow stopping the machine gracefully.
ctx, m.stop = context.WithCancel(ctx)
// Docker dependency is essential for the machine to function. Block until it's ready. // Docker dependency is essential for the machine to function. Block until it's ready.
if err := docker.WaitDaemonReady(ctx, m.config.DockerClient); err != nil { if err := docker.WaitDaemonReady(ctx, m.config.DockerClient); err != nil {
return fmt.Errorf("wait for Docker daemon: %w", err) return fmt.Errorf("wait for Docker daemon: %w", err)
@@ -294,7 +316,7 @@ func (m *Machine) Run(ctx context.Context) error {
// Configure and start the corrosion service on the loopback if the machine is not initialised as a cluster // Configure and start the corrosion service on the loopback if the machine is not initialised as a cluster
// member. This provides the store required for the machine to initialise a new cluster on it. Once the machine // member. This provides the store required for the machine to initialise a new cluster on it. Once the machine
// is initialised, the corrosion service is managed by the networkController. // is initialised, the corrosion service is managed by the clusterController.
if !m.Initialised() { if !m.Initialised() {
if err := m.configureCorrosion(); err != nil { if err := m.configureCorrosion(); err != nil {
return fmt.Errorf("configure corrosion service: %w", err) return fmt.Errorf("configure corrosion service: %w", err)
@@ -304,6 +326,7 @@ func (m *Machine) Run(ctx context.Context) error {
if err := m.config.CorrosionService.Start(ctx); err != nil { if err := m.config.CorrosionService.Start(ctx); err != nil {
return fmt.Errorf("start corrosion service: %w", err) return fmt.Errorf("start corrosion service: %w", err)
} }
slog.Info("Corrosion service started.")
} }
// Use an errgroup to coordinate error handling and graceful shutdown of multiple machine components. // Use an errgroup to coordinate error handling and graceful shutdown of multiple machine components.
@@ -314,147 +337,134 @@ func (m *Machine) Run(ctx context.Context) error {
if err != nil { if err != nil {
return fmt.Errorf("listen machine API unix socket %q: %w", m.config.MachineSockPath, err) return fmt.Errorf("listen machine API unix socket %q: %w", m.config.MachineSockPath, err)
} }
errGroup.Go( errGroup.Go(func() error {
func() error { slog.Info("Starting local machine API server.", "path", m.config.MachineSockPath)
slog.Info("Starting local machine API server.", "path", m.config.MachineSockPath) if err := m.localMachineServer.Serve(machineListener); err != nil {
if err := m.localMachineServer.Serve(machineListener); err != nil { return fmt.Errorf("local machine API server failed: %w", err)
return fmt.Errorf("local machine API server failed: %w", err) }
} return nil
return nil })
},
)
// Start the local API proxy server. // Start the local API proxy server.
proxyListener, err := listenUnixSocket(m.config.UncloudSockPath) proxyListener, err := listenUnixSocket(m.config.UncloudSockPath)
if err != nil { if err != nil {
return fmt.Errorf("listen API proxy unix socket %q: %w", m.config.UncloudSockPath, err) return fmt.Errorf("listen API proxy unix socket %q: %w", m.config.UncloudSockPath, err)
} }
errGroup.Go( errGroup.Go(func() error {
func() error { slog.Info("Starting local API proxy server.", "path", m.config.UncloudSockPath)
slog.Info("Starting local API proxy server.", "path", m.config.UncloudSockPath) if err := m.localProxyServer.Serve(proxyListener); err != nil {
if err := m.localProxyServer.Serve(proxyListener); err != nil { return fmt.Errorf("local API proxy server failed: %w", err)
return fmt.Errorf("local API proxy server failed: %w", err) }
} return nil
return nil })
},
)
// Signal that the machine is ready. // Signal that the machine is ready.
close(m.started) close(m.started)
// Control loop for managing components that depend on the machine being initialised as a cluster member. // Wait for the machine to be initialised as a member of a cluster and run the cluster controller.
errGroup.Go( errGroup.Go(func() error {
func() error { if !m.Initialised() {
if !m.Initialised() { slog.Info(
slog.Info( "Waiting for the machine to be initialised as a member of a cluster to start the cluster controller.",
"Waiting for the machine to be initialised as a member of a cluster " + )
"to start the network controller.", }
)
select {
case <-m.initialised:
m.cluster.UpdateMachineID(m.state.ID)
// Ensure the corrosion config is up to date, including a new gossip address if the machine
// has just joined a cluster.
if err := m.configureCorrosion(); err != nil {
return fmt.Errorf("configure corrosion service: %w", err)
}
slog.Info("Configured corrosion service.", "dir", m.config.CorrosionDir)
slog.Info("Starting cluster controller.")
// Update the proxy director's local address to the machine's management IP address, allowing
// the proxy to identify which requests should be proxied to the local machine API server.
m.proxyDirector.UpdateLocalAddress(m.state.Network.ManagementIP.String())
proxyServer := grpc.NewServer(
grpc.ForceServerCodecV2(proxy.Codec()),
grpc.UnknownServiceHandler(
proxy.TransparentHandler(m.proxyDirector.Director),
),
)
// Create a new caddyconfig controller for managing the Caddy reverse proxy configuration.
// It will also serve the current machine ID at /.uncloud-verify to verify Caddy reachability.
caddyconfigCtrl, err := caddyconfig.NewController(
m.state.ID,
m.config.CaddyConfigDir,
DefaultCaddyAdminSockPath,
m.store,
)
if err != nil {
return fmt.Errorf("create caddyconfig controller: %w", err)
} }
var ctrl *networkController dnsResolver := dns.NewClusterResolver(m.store)
// Error channel for communicating the termination of the network controller. dnsServer, err := dns.NewServer(m.IP(), dnsResolver, m.config.DNSUpstreams)
errCh := make(chan error) if err != nil {
return fmt.Errorf("create embedded DNS server: %w", err)
for {
select {
// Wait for the machine to be initialised as a member of a cluster to start the network controller.
// It can be reset when leaving the cluster and then re-initialised again with a new configuration.
case <-m.initialised:
var err error
m.cluster.UpdateMachineID(m.state.ID)
// Ensure the corrosion config is up to date, including a new gossip address if the machine
// has just joined a cluster.
if err = m.configureCorrosion(); err != nil {
return fmt.Errorf("configure corrosion service: %w", err)
}
slog.Info("Configured corrosion service.", "dir", m.config.CorrosionDir)
slog.Info("Starting network controller.")
// Update the proxy director's local address to the machine's management IP address, allowing
// the proxy to identify which requests should be proxied to the local machine API server.
m.proxyDirector.UpdateLocalAddress(m.state.Network.ManagementIP.String())
proxyServer := grpc.NewServer(
grpc.ForceServerCodecV2(proxy.Codec()),
grpc.UnknownServiceHandler(
proxy.TransparentHandler(m.proxyDirector.Director),
),
)
// Create a new Caddyfile controller for managing the Caddy reverse proxy configuration.
// It will also serve the current machine ID at /.uncloud-verify to verify Caddy reachability.
caddyfileCtrl, err := caddyconfig.NewController(m.store, m.config.CaddyConfigPath, m.state.ID)
if err != nil {
return fmt.Errorf("create Caddyfile controller: %w", err)
}
dnsResolver := dns.NewClusterResolver(m.store)
dnsServer, err := dns.NewServer(m.IP(), dnsResolver, m.config.DNSUpstreams)
if err != nil {
return fmt.Errorf("create embedded DNS server: %w", err)
}
ctrl, err = newNetworkController(
m.state,
m.store,
proxyServer,
m.config.CorrosionService,
m.config.DockerClient,
caddyfileCtrl,
dnsServer,
dnsResolver,
)
if err != nil {
return fmt.Errorf("initialise network controller: %w", err)
}
go func() {
if err = ctrl.Run(ctx); err != nil {
errCh <- fmt.Errorf("run network controller: %w", err)
} else {
slog.Info("Network controller stopped.")
errCh <- nil
}
}()
case err := <-errCh:
if err != nil {
return err
}
ctrl = nil
case <-ctx.Done():
// Wait for the network controller to stop before returning.
if ctrl != nil {
if err := <-errCh; err != nil {
return err
}
}
return nil
}
} }
},
) m.mu.Lock()
m.clusterCtrl, err = newClusterController(
m.state,
m.store,
proxyServer,
m.config.CorrosionService,
m.dockerService,
m.networkReady,
caddyconfigCtrl,
dnsServer,
dnsResolver,
)
m.mu.Unlock()
if err != nil {
return fmt.Errorf("initialise cluster controller: %w", err)
}
if err = m.clusterCtrl.Run(ctx); err != nil {
return fmt.Errorf("run cluster controller: %w", err)
}
slog.Info("Cluster controller stopped.")
case <-ctx.Done():
// The context was cancelled before the machine was initialised.
}
return nil
})
// Shutdown goroutine. // Shutdown goroutine.
errGroup.Go( errGroup.Go(func() error {
func() error { var err error
<-ctx.Done()
slog.Info("Stopping local machine API server.")
// TODO: implement timeout for graceful shutdown.
m.localMachineServer.GracefulStop()
slog.Info("Local machine API server stopped.")
slog.Info("Stopping local API proxy server.") <-ctx.Done()
// TODO: implement timeout for graceful shutdown. slog.Info("Stopping local machine API server.")
m.localProxyServer.GracefulStop() // TODO: implement timeout for graceful shutdown.
// Close the proxy director to close all backend connections. m.localMachineServer.GracefulStop()
m.proxyDirector.Close() slog.Info("Local machine API server stopped.")
slog.Info("Local API proxy server stopped.")
m.config.DockerClient.Close() slog.Info("Stopping local API proxy server.")
return nil // TODO: implement timeout for graceful shutdown.
}, m.localProxyServer.GracefulStop()
) // Close the proxy director to close all backend connections.
m.proxyDirector.Close()
slog.Info("Local API proxy server stopped.")
// Clean up the machine data and resources if the machine shutdown was initiated by a reset.
if m.resetting {
slog.Info("Cleaning up machine data and resources.")
if err = m.cleanup(); err != nil {
slog.Error("Failed to clean up machine data and resources.", "err", err)
}
}
m.config.DockerClient.Close()
return err
})
return errGroup.Wait() return errGroup.Wait()
} }
@@ -483,7 +493,7 @@ func listenUnixSocket(path string) (net.Listener, error) {
// Ensure the parent directory exists and has the correct group permissions. // Ensure the parent directory exists and has the correct group permissions.
parent, _ := filepath.Split(path) parent, _ := filepath.Split(path)
if err = os.MkdirAll(parent, 0750); err != nil { if err = os.MkdirAll(parent, 0o750); err != nil {
return nil, fmt.Errorf("create directory %q: %w", parent, err) return nil, fmt.Errorf("create directory %q: %w", parent, err)
} }
if err = os.Chown(parent, -1, gid); err != nil { if err = os.Chown(parent, -1, gid); err != nil {
@@ -537,15 +547,38 @@ func (m *Machine) configureCorrosion() error {
return fmt.Errorf("write corrosion config: %w", err) return fmt.Errorf("write corrosion config: %w", err)
} }
if err := os.WriteFile(schemaPath, []byte(store.Schema), 0644); err != nil { if err := os.WriteFile(schemaPath, []byte(store.Schema), 0o644); err != nil {
return fmt.Errorf("write corrosion schema: %w", err) return fmt.Errorf("write corrosion schema: %w", err)
} }
return nil return nil
} }
// cleanup removes the machine resources and persistent state.
func (m *Machine) cleanup() error {
var errs []error
m.mu.RLock()
clusterCtrl := m.clusterCtrl
m.mu.RUnlock()
if clusterCtrl != nil {
if err := clusterCtrl.Cleanup(); err != nil {
errs = append(errs, fmt.Errorf("cleanup cluster resources: %w", err))
}
}
if err := os.RemoveAll(m.config.DataDir); err != nil {
errs = append(errs,
fmt.Errorf("remove data directory with persistent machine state '%s': %w", m.config.DataDir, err))
} else {
slog.Info("Removed data directory storing persistent machine state.", "path", m.config.DataDir)
}
return errors.Join(errs...)
}
// CheckPrerequisites verifies if the machine meets all necessary system requirements to participate in the cluster. // CheckPrerequisites verifies if the machine meets all necessary system requirements to participate in the cluster.
func (m *Machine) CheckPrerequisites(ctx context.Context, _ *emptypb.Empty) (*pb.CheckPrerequisitesResponse, error) { func (m *Machine) CheckPrerequisites(_ context.Context, _ *emptypb.Empty) (*pb.CheckPrerequisitesResponse, error) {
// Check DNS port (UDP) availability. // Check DNS port (UDP) availability.
if err := checkDNSPortAvailable(); err != nil { if err := checkDNSPortAvailable(); err != nil {
return &pb.CheckPrerequisitesResponse{ return &pb.CheckPrerequisitesResponse{
@@ -776,20 +809,62 @@ func (m *Machine) Inspect(_ context.Context, _ *emptypb.Empty) (*pb.MachineInfo,
}, nil }, nil
} }
// Reset restores the machine to a clean state, removing all cluster-related сonfiguration and data and scheduling // IsNetworkReady returns true if the Docker network is ready for containers.
// a graceful shutdown. The uncloud daemon will restart the machine if managed by systemd. func (m *Machine) IsNetworkReady() bool {
func (m *Machine) Reset(ctx context.Context, _ *pb.ResetRequest) (*emptypb.Empty, error) { if !m.Initialised() {
// If machine is not initialized, there's no network to check
return false
}
// Check if network is ready by checking if the networkReady channel has been closed
select {
case <-m.networkReady:
return true
default:
return false
}
}
// WaitForNetworkReady waits for the Docker network to be ready for containers.
// It returns nil when the network is ready or an error if the context is cancelled.
func (m *Machine) WaitForNetworkReady(ctx context.Context) error {
if !m.Initialised() {
// If machine is not initialized, there's no network to wait for
return nil
}
// Wait for network to be ready or context to be cancelled
select {
case <-m.networkReady:
return nil
case <-ctx.Done():
return ctx.Err()
}
}
// Reset restores the machine to a clean state, scheduling a graceful shutdown and removing all cluster-related
// configuration and resource. The uncloud daemon will restart the machine if managed by systemd.
func (m *Machine) Reset(_ context.Context, _ *pb.ResetRequest) (*emptypb.Empty, error) {
if !m.Initialised() {
return nil, nil
}
// Check if the machine is already being reset to avoid concurrent resets.
m.mu.Lock()
if m.resetting {
m.mu.Unlock()
return nil, status.Error(codes.FailedPrecondition, "machine is already being reset")
}
m.resetting = true
m.mu.Unlock()
slog.Info("Resetting machine to a clean state.") slog.Info("Resetting machine to a clean state.")
// Trigger the machine shutdown. The resetting boolean informs the machine to clean up its resources on shutdown.
// We can't clean up the resources synchronously here because this is an RPC call that depends on the running
// gRPC server and network.
m.stop()
// TODO: stop and remove all managed service containers. return &emptypb.Empty{}, nil
// TODO: check if the request is coming from the unix or network socket. For the network socket, the reset should
// be called in a separate goroutine to avoid blocking the RPC response.
// TODO: stop the network controller
// TODO: implement and call Cleanup on the network controller to remove Docker network, WG interface, iptables
// rules, corrosion state, ?stop corrosion service.
// TODO: stop the machine and remove the machine.json state. The daemon should restart it to a clean state.
return &emptypb.Empty{}, status.Error(codes.Unimplemented, "reset machine is not implemented yet")
} }
// InspectService returns detailed information about a service and its containers stored in the cluster store. // InspectService returns detailed information about a service and its containers stored in the cluster store.
@@ -824,7 +899,7 @@ func (m *Machine) InspectService(
} }
} }
ctr := api.ServiceContainer{Container: records[0].Container} ctr := records[0].Container
svc := &pb.Service{ svc := &pb.Service{
Id: ctr.ServiceID(), Id: ctr.ServiceID(),
Name: ctr.ServiceName(), Name: ctr.ServiceName(),
+1 -1
View File
@@ -14,7 +14,7 @@ func MachineIP(subnet netip.Prefix) netip.Addr {
} }
// ManagementIP returns the IPv6 address of a peer derived from the first 14 bytes of its public key. // ManagementIP returns the IPv6 address of a peer derived from the first 14 bytes of its public key.
// This address is intended for cluster management traffic. // This address always starts with fdcc: and is intended for cluster management traffic.
func ManagementIP(publicKey secret.Secret) netip.Addr { func ManagementIP(publicKey secret.Secret) netip.Addr {
bytes := [16]byte{0xfd, 0xcc} bytes := [16]byte{0xfd, 0xcc}
copy(bytes[2:], publicKey[:14]) copy(bytes[2:], publicKey[:14])
+2 -1
View File
@@ -1,11 +1,12 @@
package network package network
import ( import (
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
"log/slog" "log/slog"
"net/netip" "net/netip"
"slices" "slices"
"time" "time"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
) )
const ( const (
+4 -3
View File
@@ -3,13 +3,14 @@ package tunnel
import ( import (
"context" "context"
"fmt" "fmt"
"golang.zx2c4.com/wireguard/conn"
"golang.zx2c4.com/wireguard/device"
"golang.zx2c4.com/wireguard/tun/netstack"
"net" "net"
"net/netip" "net/netip"
"time" "time"
"github.com/psviderski/uncloud/internal/secret" "github.com/psviderski/uncloud/internal/secret"
"golang.zx2c4.com/wireguard/conn"
"golang.zx2c4.com/wireguard/device"
"golang.zx2c4.com/wireguard/tun/netstack"
) )
const ( const (

Some files were not shown because too many files have changed in this diff Show More