Compare commits

...
61 Commits
Author SHA1 Message Date
Pasha Sviderski 1ce3e62dbb fix: use local and remote Docker credentials to pull image from private registry 2025-08-15 14:09:40 +10:00
Pasha Sviderski 4be8339c51 chore: generate a minimal Caddyfile with verify handler alongside caddy.json 2025-08-14 19:43:40 +10:00
Pasha Sviderski 8dd69b46da chore: refactor docker gRPC server to use docker service for inspecting and listing containers 2025-08-14 15:34:59 +10:00
Pasha Sviderski 4cc1e556dd chore: store ServiceContainer (includes service spec) instead of Container in Corrosion store 2025-08-14 14:53:33 +10:00
Pasha Sviderski 9186d31d12 chore: go mod tidy 2025-08-13 19:31:13 +10:00
Pasha Sviderski dd7bc6c982 chore: trim spaces for x-caddy, diff Caddy configs when comparing service specs 2025-08-13 19:27:40 +10:00
Pasha Sviderski 12c07812a2 chore: add Caddy config to ServiceSpec, load x-caddy to it 2025-08-13 18:44:38 +10:00
Pasha Sviderski ec73f9ecd8 chore: handle x-caddy: path/to/Caddyfile to read Caddy config in compose from file 2025-08-13 13:45:39 +10:00
Pasha Sviderski 879c7c1876 test: x-caddy extension parsing 2025-08-13 13:45:39 +10:00
Anton Ovchinnikov c67127f83f feat: Add basic LLM instruction files 2025-08-11 23:57:13 +02:00
Pasha Sviderski 5d3f1fe225 chore: x-caddy extension type in compose 2025-08-11 21:11:40 +10:00
Pasha Sviderski 2e585d0183 feat: add --recreate flag for deploy command to force container recreation 2025-08-07 18:09:35 +10:00
Pasha Sviderski ae9f943404 chore: change default restart policy for service containers always -> unless-stopped 2025-08-07 18:09:35 +10:00
Anton Ovchinnikov 8805178a58 doc: Add Sentry to sponsors 2025-08-07 00:03:38 +02:00
Pasha Sviderski ec3de3a099 feat: ask whether to reset already initialised machine on 'machine init/add' 2025-08-06 16:57:09 +10:00
Pasha Sviderski 2c02139369 fix: add ssh_key_path for connections in uncloud config only when using SSH key explicitly (not SSH agent) 2025-08-06 16:49:57 +10:00
Pasha Sviderski 6c244bb8f9 fix: do not try to reset machine when removing unreachable machine 2025-08-06 15:11:27 +10:00
Pasha Sviderski fc0bf4a91b chore: lint 2025-08-05 19:37:12 +10:00
Pasha Sviderski bc577fe405 docs: emphasize passwordless sudo in requirements 2025-08-05 19:32:11 +10:00
Pasha Sviderski 6cc0611d75 chore: meaningful error message when passwordless sudo required on machine provisioning 2025-08-05 18:56:14 +10:00
Pasha Sviderski 8beb9e2679 chore: lint 2025-08-04 19:20:52 +10:00
Pasha Sviderski 0c2ff499c6 fix: log message deleting iptables chain 2025-08-04 19:16:54 +10:00
Pasha Sviderski b39585df4c chore: do not masquerade container IPs when communicating via WG mesh 2025-08-04 19:09:38 +10:00
Pasha Sviderski 35d0a90125 fix: add machine with UFW firewall (allow ipv6 management traffic) fixes #65 2025-08-04 16:50:18 +10:00
Pasha Sviderski a7273c8c96 chore: fix wide svg rendering on landing 2025-08-04 15:22:05 +10:00
Pasha Sviderski 9b48003a3f blog: set diagram as og image for wireguard overlay post 2025-08-04 15:19:27 +10:00
Pasha Sviderski 1e750cc7e1 chore: add Blog links on landing page, fix header on small screens 2025-08-04 14:50:56 +10:00
Pasha Sviderski 2d73b541c5 blog: proof read WireGuard overlay post, update blog description tag 2025-08-04 14:21:48 +10:00
Pasha Sviderski cd9c0a1d2c docs: enable blog in docusaurus, add first wireguard overlay blog post 2025-08-01 21:45:50 +10:00
Pasha Sviderski 37a76809ec docs: wwizzle breadcrumbs and overwrite base url to /docs 2025-07-31 19:07:12 +10:00
Pasha Sviderski f6062cf343 docs: fix docs urls for local dev 2025-07-31 19:04:24 +10:00
Pasha Sviderski 4fe360c573 docs: combine landing page and docusaurus together and deploy as one image 2025-07-31 18:43:06 +10:00
Pasha Sviderski cac1550b7d docs: enable blog in docusaurus and change domain to uncloud.run 2025-07-30 21:28:06 +10:00
Pasha Sviderski 0bd42b2fca chore: do not allow to remove a machine the client is connected to, add --no-reset flag 2025-07-29 19:54:16 +10:00
Pasha Sviderski ad83d942e1 docs: manual cli upgrade 2025-07-28 21:45:33 +10:00
Pasha Sviderski 65f5a714dd feat: machine rm command to remove a machine from the cluster and reset it 2025-07-25 19:32:02 +10:00
Pasha Sviderski 2714587ec5 feat: add RemoveMachine API endpoint to remove a machine from the cluster 2025-07-25 19:30:59 +10:00
Pasha Sviderski 4166474ee8 fix: race on cluster init by waiting for corrosion service to become ready with schema applied 2025-07-25 16:28:32 +10:00
Pasha Sviderski 3faaac4da7 chore: run machine tests in parallel 2025-07-25 13:56:35 +10:00
Pasha Sviderski 1b6130447f chore: use a new uncancelled context for stopping corrosion service 2025-07-25 13:03:20 +10:00
Pasha Sviderski 614212a24c chore: clean up custom iptables chains on machine reset 2025-07-25 12:37:54 +10:00
Pasha Sviderski 3ab708a437 fix: graceful machine shutdown when not initialised 2025-07-25 10:50:23 +10:00
Pasha Sviderski 7a6c5bf6d7 feat: machine Reset endpoint with asynchronous resource and data cleanup 2025-07-24 19:50:25 +10:00
Pasha Sviderski 2acfafe218 chore: regenerate proto 2025-07-24 19:34:05 +10:00
Pasha Sviderski b24b55c4c7 chore: update api.MachineClient interface 2025-07-24 14:25:09 +10:00
Pasha Sviderski 9a88e914f4 chore: simplify cluster controller initialisation, prepare for reset 2025-07-24 14:23:33 +10:00
Pasha Sviderski f613d3ce6d chore: reformat code 2025-07-22 18:53:05 +10:00
Evgenii OrlovandGitHub 2c3bea64e5 feat: impl of functionality for renaming and updating machines (#91) 2025-07-22 18:45:19 +10:00
da3634b690 E2E tests use repeated strings for test data where constants add no value (#97)
---------

Co-authored-by: Pasha Sviderski <me@psviderski.name>
Co-authored-by: Anton Ovchinnikov <anton@tonyo.info>
2025-07-21 19:30:02 +02:00
Pasha Sviderski 6fb07db4b2 feat: follow-up compose 'ports' support: use ingress mode by default (closes #81) 2025-07-21 16:56:52 +10:00
Evgenii OrlovandGitHub fea7edcbc5 feat: add support for standard compose ports directive (#95) 2025-07-21 15:24:37 +10:00
Anton OvchinnikovandGitHub a54555cd13 feat(deploy): Recreate container when pull_policy set to Always (#100) 2025-07-21 14:47:38 +10:00
Pasha Sviderski 87c7889c2e docs: format debian installation section 2025-07-18 21:59:54 +10:00
Anton Ovchinnikov 05d0078451 chore: Enable more linters 2025-07-14 00:32:26 +02:00
Anton Ovchinnikov 053d73048c feat: Basic linting config 2025-07-13 23:50:27 +02:00
Matías García IsaíaandGitHub 4ec3e97a54 docs: add documentation regarding --no-dns flag (#93) 2025-07-11 11:59:25 +10:00
Evgenii OrlovandGitHub 10bbe9fbc5 fix: prevent race condition with Docker network creation (#89) 2025-07-10 17:51:08 +10:00
Evgenii OrlovandGitHub 31cd4c77e9 feat: support x-machines placement constraints in compose files (#90) 2025-07-10 16:46:29 +10:00
Pasha Sviderski 492a0af2b2 fix: migration script for uncloud Docker network to be compatible with Docker 28.2.0+ 2025-07-09 14:04:21 +10:00
Pasha Sviderski ff7bb25f2c fix: custom version handling in install.sh script 2025-07-09 13:22:21 +10:00
Pasha Sviderski a73cbfd691 chore: hide incomplete machine rm command from help output 2025-07-09 11:28:14 +10:00
180 changed files with 25120 additions and 20267 deletions
+1
View File
@@ -0,0 +1 @@
../AI.md
+3 -1
View File
@@ -37,7 +37,9 @@ jobs:
(echo "go.mod or go.sum has changed. Please run 'go mod tidy' and commit the changes." && exit 1) (echo "go.mod or go.sum has changed. Please run 'go mod tidy' and commit the changes." && exit 1)
- name: Run tests - name: Run tests
run: make test run: |
make ucind-image
make test
timeout-minutes: 10 timeout-minutes: 10
check-protobuf: check-protobuf:
+39
View File
@@ -0,0 +1,39 @@
name: Lint
on:
push:
branches:
- "main"
- "test/**"
- "release/**"
pull_request:
branches:
- main
paths:
- "**.go"
- "go.*"
permissions:
contents: read
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Set up Go
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with:
go-version: "1.23.2"
- name: golangci-lint
uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0
with:
version: v2.2.2
- name: Format code
run: |
make format
git diff --exit-code ||
(echo "Code is not formatted. Please run 'make format' and commit the changes." && exit 1)
timeout-minutes: 10
+42
View File
@@ -0,0 +1,42 @@
version: "2"
run:
concurrency: 4
tests: true
timeout: 5m
linters:
default: none
enable:
- bodyclose
# - dogsled
- dupl
# - errcheck
- gochecknoinits
- goconst
# - gocritic
# - gocyclo
# - godot
# - gosec
# - govet
- ineffassign
- misspell
- nakedret
# - prealloc
# - revive
# - staticcheck
- unconvert
# - unparam
# - unused
- whitespace
exclusions:
rules:
- path: ^test/e2e
linters:
- goconst # constants here add no value, so we skip goconst only for test/e2e.
formatters:
enable:
- gofumpt
- goimports
+8
View File
@@ -6,6 +6,14 @@ backend = "core:go"
"go1.23.10.darwin-arm64.tar.gz" = "sha256:25c64bfa8a8fd8e7f62fb54afa4354af8409a4bb2358c2699a1003b733e6fce5" "go1.23.10.darwin-arm64.tar.gz" = "sha256:25c64bfa8a8fd8e7f62fb54afa4354af8409a4bb2358c2699a1003b733e6fce5"
"go1.23.10.linux-amd64.tar.gz" = "sha256:535f9f81802499f2a7dbfa70abb8fda3793725fcc29460f719815f6e10b5fd60" "go1.23.10.linux-amd64.tar.gz" = "sha256:535f9f81802499f2a7dbfa70abb8fda3793725fcc29460f719815f6e10b5fd60"
[tools.golangci-lint]
version = "2.2.2"
backend = "aqua:golangci/golangci-lint"
[tools.golangci-lint.checksums]
"golangci-lint-2.2.2-darwin-arm64.tar.gz" = "sha256:d84d94d042c0d495fd1746f3d18948a75de163b17a14e8de3ef840928dd2df74"
"golangci-lint-2.2.2-linux-amd64.tar.gz" = "sha256:c27fbde948a87d326feacd21df2f61a9c54dbd2e3bfa185c0a1cd6917a6f964f"
[tools.protoc] [tools.protoc]
version = "27.3" version = "27.3"
backend = "aqua:protocolbuffers/protobuf/protoc" backend = "aqua:protocolbuffers/protobuf/protoc"
+1
View File
@@ -3,6 +3,7 @@ experimental = true
[tools] [tools]
go = "1.23" go = "1.23"
golangci-lint = "2.2.2"
protoc = "27.3" protoc = "27.3"
protoc-gen-go = "1.34.2" protoc-gen-go = "1.34.2"
protoc-gen-go-grpc = "1.5.1" protoc-gen-go-grpc = "1.5.1"
+244
View File
@@ -0,0 +1,244 @@
# AI.md - Uncloud Project Guide
This document provides comprehensive information about the Uncloud project for AI assistants to understand the codebase, architecture, and development practices.
## Project Overview
**Uncloud** is a lightweight clustering and container orchestration tool that enables deployment and management of web applications across cloud VMs and bare metal servers. It creates a secure WireGuard mesh network between Docker hosts and provides automatic service discovery, load balancing, HTTPS ingress, and simple CLI commands for application management.
### Key Characteristics
- **Language**: Go
- **Architecture**: Decentralized, no control plane
- **Target**: Self-hosted infrastructure without Kubernetes complexity
- **License**: View LICENSE file for details
- **Status**: Active development, not yet ready for production
## Core Features
### 🏗️ Infrastructure
- **Multi-machine deployment**: Combine cloud VMs, dedicated servers, and bare metal
- **Zero-config networking**: Automatic WireGuard mesh with NAT traversal
- **Decentralized design**: No central control plane, all machines are equal
- **Service discovery**: Built-in DNS server resolves service names to container IPs
### 🚀 Application Management
- **Docker Compose compatibility**: Uses familiar Docker Compose format
- **Zero-downtime deployments**: Rolling updates without service interruption
- **Automatic HTTPS**: Caddy reverse proxy with Let's Encrypt integration
- **Managed DNS**: Free `*.cluster.uncloud.run` subdomains via Uncloud DNS service
- **Cross-machine scaling**: Run containers across multiple machines
### 🔧 Developer Experience
- **Docker-like CLI**: Familiar commands (`uc` binary)
- **Imperative operations**: Direct commands vs. declarative state reconciliation
- **Remote management**: Control entire infrastructure via SSH to any machine
- **Minimal overhead**: ~150MB RAM footprint per machine
## Architecture
### Core Components
1. **CLI (`uc`)** - Main user interface for cluster management
2. **Daemon (`uncloudd`)** - Machine daemon running on each node
3. **Corrosion** - Distributed SQLite database for cluster state (Fly.io project)
4. **Caddy** - Reverse proxy for HTTPS termination and routing
5. **WireGuard** - Secure mesh networking between machines
### Network Architecture
- Each machine gets unique subnet (e.g., `10.210.0.0/24`, `10.210.1.0/24`)
- Containers get cluster-unique IPs for direct communication
- Automatic peer discovery and key management
- NAT traversal for machines behind firewalls
### State Management
- **CRDT-based distributed storage** using Corrosion
- **Eventually consistent** state across all machines
- **Gossip protocol** (Serf) for state propagation
- **No quorum requirements** - partial network splits remain functional
## Project Structure
### Key Directories
- **`cmd/`**: Contains main applications
- `uncloud/`: CLI tool with subcommands for machine, service, volume management
- `uncloudd/`: Daemon that runs on each machine
- `ucind/`: Development cluster management for testing
- **`internal/`**: Internal implementation packages
- `cli/`: Command-line interface logic
- `machine/`: Machine lifecycle and state management
- `daemon/`: Daemon implementation and gRPC services
- `dns/`: Internal DNS server for service discovery
- **`pkg/`**: Public API packages for external use
- `api/`: Core API types and definitions
- `client/`: Client libraries for interacting with Uncloud
- **`experiment/`**: Experimental features and prototypes
- **`scripts/`**: Installation and utility scripts
- **`test/`**: Test suites and test infrastructure
- **`website/`**: Documentation website (Docusaurus)
- **`misc/`**: Design documents and guides
## Key Technologies
### Core Dependencies
```go
// Networking and orchestration
github.com/docker/docker // Docker API client
github.com/docker/compose/v2 // Docker Compose integration
golang.zx2c4.com/wireguard // WireGuard implementation
github.com/hashicorp/serf // Gossip protocol
// State management
github.com/ipfs/go-ds-crdt // CRDT distributed storage
github.com/dgraph-io/badger/v3 // Embedded database
// Web proxy
github.com/caddyserver/caddy/v2 // HTTP server and reverse proxy
// CLI and UX
github.com/spf13/cobra // CLI framework
github.com/charmbracelet/huh // Interactive forms
// gRPC and networking
google.golang.org/grpc // gRPC framework
github.com/siderolabs/grpc-proxy // gRPC proxy for forwarding
```
## Development Workflow
### Build and Development
```bash
# Build binaries
go build -o uncloud ./cmd/uncloud
go build -o uncloudd ./cmd/uncloudd
```
### Key Make Targets
- `proto`: Generate protobuf code
- `ucind-cluster`: Create development cluster
- `update-dev`: Deploy to development machines
- `demo-reset`: Reset demo environment
- `fmt`: Format code
- `test`: Run all tests
- `lint`: Lint the code using golangci-lint
- `lint-and-fix`: Lint the code and fix issues whenever possible
## CLI Commands Structure
The `uc` CLI provides these main command groups:
### Machine Management
```bash
uc machine init <user@host> # Initialize new cluster
uc machine add <user@host> # Add machine to cluster
uc machine ls # List machines
uc machine rm <name> # Remove machine
```
### Service Management
```bash
uc run <image> # Run container from image
uc deploy # Deploy from compose.yaml
uc scale <service> <count> # Scale service replicas
uc ls # List services
uc rm <service> # Remove service
```
### Context and Connectivity
```bash
uc context ls # List available contexts
uc context use <name> # Switch context
```
### Global Flags
- `--connect`: Connect to remote machine directly, without a config file
- `--uncloud-config`: Override config file path
## Development Guidelines
### Code Organization
- **Package naming**: Use clear, descriptive names
- **Error handling**: Wrap errors with context using `fmt.Errorf`
- **Logging**: Use structured logging with levels
- **gRPC**: Services defined in `internal/machine/api/pb/`
### Testing
- Unit tests alongside source files (`*_test.go`)
- Integration tests in `test/e2e/`
- Test fixtures in `test/fixtures/`
### Dependencies
- Prefer standard library when possible
- Pin versions in `go.mod`
- Document rationale for external dependencies
### Configuration
- Support environment variables for key settings
- Validate configuration early
- Provide sensible defaults
## Troubleshooting and Debugging
### Common Issues
- **Networking**: Check WireGuard status, iptables rules
- **DNS**: Verify service discovery resolution
- **Containers**: Use standard Docker debugging tools
- **State sync**: Check Corrosion logs for replication issues
### Debugging Tools
- Standard Linux networking tools (`ping`, `traceroute`, `wireshark`)
- Docker commands (`docker ps`, `docker logs`)
- SSH access to machines for direct inspection
- gRPC debugging tools
### Logs and Monitoring
- Systemd services (getting logs via `journalctl -u SERVICE_NAME`)
- `uncloud` -- Uncloud daemon
- `uncloud-corrosion` -- Corrosion process
- Machine daemon logs
- Container logs via Docker
## File Patterns and Conventions
### Important Files to Understand
- `cmd/uncloud/main.go`: CLI entry point and command structure
- `internal/cli/cli.go`: CLI implementation and configuration
- `internal/machine/machine.go`: Core machine management
- `pkg/api/`: Public API definitions
- `misc/design.md`: Architecture and design philosophy
- `README.md`: User-facing documentation
### Configuration Files
- `go.mod/go.sum`: Go dependency management
- `Makefile`: Build and development tasks
- `Dockerfile`: Container build instructions forUncloud-in-Docker (used for testing)
This document should help AI assistants understand the project structure, make informed suggestions, and contribute effectively to the Uncloud codebase.
Symlink
+1
View File
@@ -0,0 +1 @@
./AI.md
+24 -5
View File
@@ -1,6 +1,5 @@
CORROSION_IMAGE ?= ghcr.io/psviderski/corrosion:latest CORROSION_IMAGE ?= ghcr.io/psviderski/corrosion:latest
UCIND_IMAGE ?= ghcr.io/psviderski/ucind:latest UCIND_IMAGE ?= ghcr.io/psviderski/ucind:latest
DOCS_IMAGE ?= ghcr.io/psviderski/uncloud-docs:latest
update-dev: update-dev:
GOOS=linux GOARCH=amd64 go build -o uncloudd-linux-amd64 ./cmd/uncloudd && \ GOOS=linux GOARCH=amd64 go build -o uncloudd-linux-amd64 ./cmd/uncloudd && \
@@ -46,6 +45,11 @@ proto:
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative \ protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative \
--proto_path=. --proto_path=internal/machine/api/vendor internal/machine/api/pb/*.proto --proto_path=. --proto_path=internal/machine/api/vendor internal/machine/api/pb/*.proto
.PHONY: proto-mise
proto-mise:
mise exec -- protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative \
--proto_path=. --proto_path=internal/machine/api/vendor internal/machine/api/pb/*.proto
.PHONY: corrosion-image .PHONY: corrosion-image
corrosion-image: corrosion-image:
docker build -t "$(CORROSION_IMAGE)" --target corrosion . docker build -t "$(CORROSION_IMAGE)" --target corrosion .
@@ -67,9 +71,13 @@ test:
ifeq ($(TEST_NAME),) ifeq ($(TEST_NAME),)
go test -count=1 -v ./... go test -count=1 -v ./...
else else
go test -count=1 -v -run ^$(TEST_NAME)$$ ./... go test -race -count=1 -v -run ^$(TEST_NAME)$$ ./...
endif endif
.PHONY: test-e2e
test-e2e:
go test -race -count=1 -v ./test/e2e
.PHONY: test-clean .PHONY: test-clean
test-clean: test-clean:
@CONTAINERS=$$(docker ps --filter "name=ucind-test" -q); \ @CONTAINERS=$$(docker ps --filter "name=ucind-test" -q); \
@@ -89,6 +97,17 @@ test-clean:
vet: vet:
go vet ./... go vet ./...
.PHONY: docs-image-push .PHONY: format fmt
docs-image: format fmt:
docker buildx build --push --platform linux/amd64,linux/arm64 -t "$(DOCS_IMAGE)" ./docs GOOS=linux golangci-lint fmt
LINT_TARGETS := lint lint-and-fix
.PHONY: $(LINT_TARGETS) _lint
$(LINT_TARGETS): _lint
lint: ARGS=
lint-and-fix: ARGS=--fix
_lint:
# Explicitly set OS to Linux to not skip *_linux.go files when running on macOS.
# Uncloud daemon won't likely support OS other than Linux anytime soon, so for now we can rely on that.
GOOS=linux golangci-lint run $(ARGS)
+20 -11
View File
@@ -1,10 +1,10 @@
<div align="center"> <div align="center">
<img src="./website/images/logo.svg" height="100" width="100" alt="Uncloud logo"/> <img src="./website/landing/images/logo.svg" height="100" width="100" alt="Uncloud logo"/>
<h1>Uncloud</h1> <h1>Uncloud</h1>
<p><strong>Docker simplicity. Multi-machine power.</strong></p> <p><strong>Docker simplicity. Multi-machine power.</strong></p>
<p> <p>
<a href="https://docs.uncloud.run"><img src="https://img.shields.io/badge/Docs-blue.svg?style=for-the-badge&logo=gitbook&logoColor=white" alt="Documentation"></a> <a href="https://uncloud.run/docs"><img src="https://img.shields.io/badge/Docs-blue.svg?style=for-the-badge&logo=gitbook&logoColor=white" alt="Documentation"></a>
<a href="https://discord.gg/eR35KQJhPu"><img src="https://img.shields.io/badge/discord-5865F2.svg?style=for-the-badge&logo=discord&logoColor=white" alt="Join Discord"></a> <a href="https://discord.gg/eR35KQJhPu"><img src="https://img.shields.io/badge/discord-5865F2.svg?style=for-the-badge&logo=discord&logoColor=white" alt="Join Discord"></a>
<a href="https://x.com/psviderski"><img src="https://img.shields.io/badge/follow-black?style=for-the-badge&logo=X&logoColor=while" alt="Follow on X"></a> <a href="https://x.com/psviderski"><img src="https://img.shields.io/badge/follow-black?style=for-the-badge&logo=X&logoColor=while" alt="Follow on X"></a>
<a href="https://github.com/sponsors/psviderski"><img src="https://img.shields.io/badge/Donate-EA4AAA.svg?style=for-the-badge&logo=githubsponsors&logoColor=white" alt="Donate"></a> <a href="https://github.com/sponsors/psviderski"><img src="https://img.shields.io/badge/Donate-EA4AAA.svg?style=for-the-badge&logo=githubsponsors&logoColor=white" alt="Donate"></a>
@@ -52,11 +52,11 @@ complexity of Kubernetes.
## 🎬 Quick demo ## 🎬 Quick demo
The screenshot below demonstrates how I use Uncloud to deploy the [Uncloud Documentation](https://docs.uncloud.run) The screenshot below demonstrates how I use Uncloud to deploy https://uncloud.run website to 2 remote machines from
website to 2 remote machines (why not?) from the [`compose.yaml`](docs/compose.yaml) file on my local machine. the [`compose.yaml`](website/compose.yaml) file on my local machine.
It exposes the container port `8000/tcp` as HTTPS on the domain `docs.uncloud.run`, served by the Caddy reverse proxy on It exposes the container port `8000/tcp` as HTTPS on the domain `uncloud.run`, served by the Caddy reverse proxy on the
the remote machines. All managed by Uncloud. remote machines. All managed by Uncloud.
![Uncloud compose deployment demo](.github/images/compose-deploy.jpg) ![Uncloud compose deployment demo](.github/images/compose-deploy.jpg)
@@ -96,8 +96,8 @@ platform, whether you're running on a $5 VPS, a spare Mac mini, or a rack of bar
# or using curl (macOS/Linux) # or using curl (macOS/Linux)
curl -fsS https://get.uncloud.run/install.sh | sh curl -fsS https://get.uncloud.run/install.sh | sh
``` ```
See [Installation](https://docs.uncloud.run/getting-started/install-cli) for more options. See [Installation](https://uncloud.run/docs/getting-started/install-cli) for more options.
2. Initialise your first machine: 2. Initialise your first machine:
@@ -130,15 +130,15 @@ platform, whether you're running on a $5 VPS, a spare Mac mini, or a rack of bar
uncloud-uninstall uncloud-uninstall
``` ```
View the [Documentation](https://docs.uncloud.run) for more information. View the [Documentation](https://uncloud.run/docs) for more information.
## ⚙️ How it works ## ⚙️ How it works
Check out the [design document](docs/design.md) to understand Uncloud's design philosophy and goals. Check out the [design document](misc/design.md) to understand Uncloud's design philosophy and goals.
Here is a diagram of an Uncloud multi-provider cluster of 3 machines: Here is a diagram of an Uncloud multi-provider cluster of 3 machines:
![Diagram: multi-provider cluster of 3 machines](website/images/diagram.webp) ![Diagram: multi-provider cluster of 3 machines](website/landing/images/diagram.webp)
<details> <details>
<summary>Peek under the hood to see what happens when you run certain commands.</summary> <summary>Peek under the hood to see what happens when you run certain commands.</summary>
@@ -323,6 +323,15 @@ SQLite database used to share Uncloud's cluster state.
features, and be the first to know when it's ready for production use. features, and be the first to know when it's ready for production use.
* Watch this repository for releases. * Watch this repository for releases.
## 💖 Sponsors
These companies and projects are helping Uncloud with their generous sponsorship and/or services:
<!-- Sentry -->
<a href="https://sentry.io/welcome/">
<img height="100" alt="Sentry" src="https://github.com/user-attachments/assets/6c1439c0-d20d-40dc-a669-c9aa94651dfa" />
</a>
## ❤️ Contributors ## ❤️ Contributors
Thank you [@cedws](https://github.com/cedws) for being the first contributor to Uncloud! 🎉 Thank you [@cedws](https://github.com/cedws) for being the first contributor to Uncloud! 🎉
+2 -1
View File
@@ -2,8 +2,9 @@ package cluster
import ( import (
"fmt" "fmt"
"github.com/spf13/cobra"
"github.com/psviderski/uncloud/internal/ucind" "github.com/psviderski/uncloud/internal/ucind"
"github.com/spf13/cobra"
) )
func NewCreateCommand() *cobra.Command { func NewCreateCommand() *cobra.Command {
+2 -1
View File
@@ -2,8 +2,9 @@ package cluster
import ( import (
"fmt" "fmt"
"github.com/spf13/cobra"
"github.com/psviderski/uncloud/internal/ucind" "github.com/psviderski/uncloud/internal/ucind"
"github.com/spf13/cobra"
) )
func NewRemoveCommand() *cobra.Command { func NewRemoveCommand() *cobra.Command {
+1 -1
View File
@@ -13,7 +13,7 @@ func NewRootCommand() *cobra.Command {
} }
cmd.AddCommand( cmd.AddCommand(
NewCreateCommand(), NewCreateCommand(),
//NewListCommand(), // NewListCommand(),
NewRemoveCommand(), NewRemoveCommand(),
) )
return cmd return cmd
+8 -1
View File
@@ -21,6 +21,7 @@ type deployOptions struct {
profiles []string profiles []string
services []string services []string
noBuild bool noBuild bool
recreate bool
context string context string
} }
@@ -50,6 +51,8 @@ func NewDeployCommand() *cobra.Command {
"Name of the cluster context to deploy to (default is the current context)") "Name of the cluster context to deploy to (default is the current context)")
cmd.Flags().BoolVarP(&opts.noBuild, "no-build", "n", false, cmd.Flags().BoolVarP(&opts.noBuild, "no-build", "n", false,
"Do not build images before deploying services. (default false)") "Do not build images before deploying services. (default false)")
cmd.Flags().BoolVar(&opts.recreate, "recreate", false,
"Recreate containers even if their configuration and image haven't changed.")
// TODO: Consider adding a filter flag to specify which machines to deploy to but keep the rest running. // TODO: Consider adding a filter flag to specify which machines to deploy to but keep the rest running.
// Could be useful to test a new version on a subset of machines before rolling out to all. // Could be useful to test a new version on a subset of machines before rolling out to all.
@@ -108,7 +111,11 @@ func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
} }
defer clusterClient.Close() defer clusterClient.Close()
composeDeploy, err := compose.NewDeployment(ctx, clusterClient, project) var strategy deploy.Strategy
if opts.recreate {
strategy = &deploy.RollingStrategy{ForceRecreate: true}
}
composeDeploy, err := compose.NewDeploymentWithStrategy(ctx, clusterClient, project, strategy)
if err != nil { if err != nil {
return fmt.Errorf("create compose deployment: %w", err) return fmt.Errorf("create compose deployment: %w", err)
} }
+7 -6
View File
@@ -41,7 +41,7 @@ func NewAddCommand() *cobra.Command {
if err != nil { if err != nil {
return fmt.Errorf("parse remote machine: %w", err) return fmt.Errorf("parse remote machine: %w", err)
} }
remoteMachine := cli.RemoteMachine{ remoteMachine := &cli.RemoteMachine{
User: user, User: user,
Host: host, Host: host,
Port: port, Port: port,
@@ -59,11 +59,12 @@ func NewAddCommand() *cobra.Command {
cmd.Flags().StringVar( cmd.Flags().StringVar(
&opts.publicIP, "public-ip", "auto", &opts.publicIP, "public-ip", "auto",
"Public IP address of the machine for ingress configuration. Use 'auto' for automatic detection, "+ "Public IP address of the machine for ingress configuration. Use 'auto' for automatic detection, "+
"blank '' or 'none' to disable ingress on this machine, or specify an IP address.", fmt.Sprintf("blank '' or '%s' to disable ingress on this machine, or specify an IP address.", PublicIPNone),
) )
cmd.Flags().StringVarP( cmd.Flags().StringVarP(
&opts.sshKey, "ssh-key", "i", "~/.ssh/id_ed25519", &opts.sshKey, "ssh-key", "i", "",
"Path to SSH private key for remote login (if not already added to SSH agent).", fmt.Sprintf("Path to SSH private key for remote login (if not already added to SSH agent). (default %q)",
cli.DefaultSSHKeyPath),
) )
cmd.Flags().StringVar( cmd.Flags().StringVar(
&opts.version, "version", "latest", &opts.version, "version", "latest",
@@ -77,12 +78,12 @@ func NewAddCommand() *cobra.Command {
return cmd return cmd
} }
func add(ctx context.Context, uncli *cli.CLI, remoteMachine cli.RemoteMachine, opts addOptions) error { func add(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine, opts addOptions) error {
var publicIP *netip.Addr var publicIP *netip.Addr
switch opts.publicIP { switch opts.publicIP {
case "auto": case "auto":
publicIP = &netip.Addr{} publicIP = &netip.Addr{}
case "", "none": case "", PublicIPNone:
publicIP = nil publicIP = nil
default: default:
ip, err := netip.ParseAddr(opts.publicIP) ip, err := netip.ParseAddr(opts.publicIP)
+6
View File
@@ -0,0 +1,6 @@
package machine
const (
// PublicIPNone is the value used to indicate removal of public IP
PublicIPNone = "none"
)
+5 -4
View File
@@ -77,11 +77,12 @@ func NewInitCommand() *cobra.Command {
cmd.Flags().StringVar( cmd.Flags().StringVar(
&opts.publicIP, "public-ip", "auto", &opts.publicIP, "public-ip", "auto",
"Public IP address of the machine for ingress configuration. Use 'auto' for automatic detection, "+ "Public IP address of the machine for ingress configuration. Use 'auto' for automatic detection, "+
"blank '' or 'none' to disable ingress on this machine, or specify an IP address.", fmt.Sprintf("blank '' or '%s' to disable ingress on this machine, or specify an IP address.", PublicIPNone),
) )
cmd.Flags().StringVarP( cmd.Flags().StringVarP(
&opts.sshKey, "ssh-key", "i", "~/.ssh/id_ed25519", &opts.sshKey, "ssh-key", "i", "",
"Path to SSH private key for remote login (if not already added to SSH agent).", fmt.Sprintf("Path to SSH private key for remote login (if not already added to SSH agent). (default %q)",
cli.DefaultSSHKeyPath),
) )
cmd.Flags().StringVar( cmd.Flags().StringVar(
&opts.version, "version", "latest", &opts.version, "version", "latest",
@@ -105,7 +106,7 @@ func initCluster(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteM
switch opts.publicIP { switch opts.publicIP {
case "auto": case "auto":
publicIP = &netip.Addr{} publicIP = &netip.Addr{}
case "", "none": case "", PublicIPNone:
publicIP = nil publicIP = nil
default: default:
ip, err := netip.ParseAddr(opts.publicIP) ip, err := netip.ParseAddr(opts.publicIP)
+47
View File
@@ -0,0 +1,47 @@
package machine
import (
"context"
"fmt"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
func NewRenameCommand() *cobra.Command {
var contextName string
cmd := &cobra.Command{
Use: "rename OLD_NAME NEW_NAME",
Short: "Rename a machine in the cluster.",
Long: `Rename a machine in the cluster.
This command changes the name of an existing machine while preserving all other
configuration including network settings, public IP, and cluster membership.`,
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return rename(cmd.Context(), uncli, contextName, args[0], args[1])
},
}
cmd.Flags().StringVarP(
&contextName, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func rename(ctx context.Context, uncli *cli.CLI, contextName, oldName, newName string) error {
client, err := uncli.ConnectCluster(ctx, contextName)
if err != nil {
return err
}
defer client.Close()
machine, err := client.RenameMachine(ctx, oldName, newName)
if err != nil {
return fmt.Errorf("rename machine: %w", err)
}
fmt.Printf("Machine %q renamed to %q (ID: %s)\n", oldName, machine.Name, machine.Id)
return nil
}
+76 -27
View File
@@ -14,12 +14,13 @@ import (
"github.com/docker/compose/v2/pkg/progress" "github.com/docker/compose/v2/pkg/progress"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/psviderski/uncloud/internal/cli" "github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
type removeOptions struct { type removeOptions struct {
force bool noReset bool
yes bool yes bool
context string context string
} }
@@ -30,7 +31,7 @@ func NewRmCommand() *cobra.Command {
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "rm MACHINE", Use: "rm MACHINE",
Aliases: []string{"remove", "delete"}, Aliases: []string{"remove", "delete"},
Short: "Remove a machine from a cluster.", Short: "Remove a machine from a cluster and reset it.",
Args: cobra.ExactArgs(1), Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
@@ -42,11 +43,14 @@ func NewRmCommand() *cobra.Command {
"Name of the cluster context. (default is the current context)") "Name of the cluster context. (default is the current context)")
cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false, cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
"Do not prompt for confirmation before removing the machine.") "Do not prompt for confirmation before removing the machine.")
cmd.Flags().BoolVar(&opts.noReset, "no-reset", false,
"Do not reset the machine after removing it from the cluster. This will leave all containers and data intact.")
return cmd return cmd
} }
func remove(ctx context.Context, uncli *cli.CLI, machineName string, opts removeOptions) error { func remove(ctx context.Context, uncli *cli.CLI, nameOrID string, opts removeOptions) error {
// TODO: automatically choose a connection to the machine that is not being removed.
client, err := uncli.ConnectCluster(ctx, opts.context) client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
@@ -54,35 +58,66 @@ func remove(ctx context.Context, uncli *cli.CLI, machineName string, opts remove
defer client.Close() defer client.Close()
// Verify the machine exists and list all service containers on it including stopped ones. // Verify the machine exists and list all service containers on it including stopped ones.
listCtx, machines, err := api.ProxyMachinesContext(ctx, client, []string{machineName}) mctx, machines, err := api.ProxyMachinesContext(ctx, client, []string{nameOrID})
if err != nil { if err != nil {
return err return err
} }
if len(machines) == 0 { if len(machines) == 0 {
return fmt.Errorf("machine '%s' not found in the cluster", machineName) return fmt.Errorf("machine '%s' not found in the cluster", nameOrID)
} }
m := machines[0].Machine m := machines[0].Machine
listOpts := container.ListOptions{All: true} // Verify if the machine being removed is the proxy machine we're connected to.
machineContainers, err := client.Docker.ListServiceContainers(listCtx, "", listOpts) proxyMachine, err := client.MachineClient.Inspect(ctx, nil)
if err != nil { if err != nil {
return fmt.Errorf("list containers: %w", err) return fmt.Errorf("inspect proxy machine: %w", err)
} }
containers := machineContainers[0].Containers if proxyMachine.Id == m.Id {
allMachines, err := client.ListMachines(ctx, nil)
if err != nil {
return fmt.Errorf("list machines: %w", err)
}
if len(allMachines) > 1 {
return errors.New("cannot remove the machine you are currently connected to. " +
"Please connect to another machine in the cluster and try again. " +
"Use --connect flag or update 'connections' for the cluster context in your Uncloud config")
// It's ok to remove the proxy machine if it's the last one in the cluster.
}
}
if len(containers) > 0 { // TODO: mark the machine as being removed and unschedulable when this is possible to prevent new containers
plural := "" // from being scheduled on it while the removal is in progress.
if len(containers) > 1 {
plural = "s" reset := !opts.noReset
var containers []api.ServiceContainer
reachable := false
if reset {
// Check if the machine is up and has service containers.
listOpts := container.ListOptions{All: true}
machineContainers, err := client.Docker.ListServiceContainers(mctx, "", listOpts)
if err == nil {
reachable = true
containers = machineContainers[0].Containers
if len(containers) > 0 {
plural := ""
if len(containers) > 1 {
plural = "s"
}
fmt.Printf("Found %d service container%s on machine '%s':\n", len(containers), plural, m.Name)
fmt.Println(formatContainerTree(containers))
fmt.Println()
fmt.Println("This will remove all service containers from the machine, remove it from the cluster, " +
"and reset it to the uninitialised state.")
} else {
fmt.Printf("No service containers found on machine '%s'.\n", m.Name)
fmt.Println("This will remove the machine from the cluster and reset it to the uninitialised state.")
}
} else {
fmt.Printf("This will remove machine '%s' from the cluster without resetting it as it's unreachable.\n",
m.Name)
} }
fmt.Printf("Found %d service container%s on machine '%s':\n", len(containers), plural, m.Name)
fmt.Println(formatContainerTree(containers))
fmt.Println()
fmt.Println("This will remove all service containers on the machine, reset it to the uninitialised state, " +
"and remove it from the cluster.")
} else { } else {
fmt.Printf("No service containers found on machine '%s'.\n", m.Name) fmt.Printf("This will remove machine '%s' from the cluster without resetting it.\n", m.Name)
fmt.Println("This will reset the machine to the uninitialised state and remove it from the cluster.")
} }
if !opts.yes { if !opts.yes {
@@ -96,23 +131,37 @@ func remove(ctx context.Context, uncli *cli.CLI, machineName string, opts remove
} }
} }
if len(containers) > 0 { if reset && len(containers) > 0 {
err = progress.RunWithTitle(ctx, func(ctx context.Context) error { err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
return removeContainers(ctx, client, containers) return removeContainers(ctx, client, containers)
}, uncli.ProgressOut(), "Removing containers") }, uncli.ProgressOut(), "Removing containers")
if err != nil { if err != nil {
return fmt.Errorf("remove containers: %w", err) return fmt.Errorf("remove containers: %w", err)
} }
fmt.Println() fmt.Println()
} }
// TODO: 4. Implement and call Reset via Machine API to reset the machine state to uninitialised. if _, err = client.RemoveMachine(ctx, &pb.RemoveMachineRequest{Id: m.Id}); err != nil {
// TODO: 5. Remove the machine from the cluster store. return fmt.Errorf("remove machine from cluster: %w", err)
}
fmt.Printf("Machine '%s' removed from the cluster.\n", m.Name)
return fmt.Errorf("resetting machine is not fully implemented yet") if reset && reachable {
//fmt.Printf("Machine '%s' removed from the cluster.\n", m.Name) _, err = client.MachineClient.Reset(mctx, &pb.ResetRequest{})
//return nil if err != nil {
fmt.Printf("WARNING: Failed to reset machine: %v\n", err)
} else {
fmt.Println("Machine reset initiated and will complete in the background.")
}
}
// TODO: remove the connection to the machine from the uncloud config if it exists. We need a way to associate
// the machine with its connection in the config, e.g. by storing the machine name in the connection metadata.
// TODO: If Caddy was running on this machine and a cluster domain is reserved,
// let the user know that the DNS records should be updated.
return nil
} }
// formatContainerTree formats a list of containers grouped by service as a tree structure. // formatContainerTree formats a list of containers grouped by service as a tree structure.
+2
View File
@@ -14,7 +14,9 @@ func NewRootCommand() *cobra.Command {
NewAddCommand(), NewAddCommand(),
NewInitCommand(), NewInitCommand(),
NewListCommand(), NewListCommand(),
NewRenameCommand(),
NewRmCommand(), NewRmCommand(),
NewUpdateCommand(),
NewTokenCommand(), NewTokenCommand(),
) )
return cmd return cmd
+2 -1
View File
@@ -2,9 +2,10 @@ package machine
import ( import (
"fmt" "fmt"
"github.com/spf13/cobra"
"github.com/psviderski/uncloud/internal/daemon" "github.com/psviderski/uncloud/internal/daemon"
"github.com/psviderski/uncloud/internal/machine" "github.com/psviderski/uncloud/internal/machine"
"github.com/spf13/cobra"
) )
type tokenOptions struct { type tokenOptions struct {
+128
View File
@@ -0,0 +1,128 @@
package machine
import (
"context"
"fmt"
"net/netip"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/spf13/cobra"
)
type updateOptions struct {
name string
publicIP string
context string
}
func NewUpdateCommand() *cobra.Command {
opts := updateOptions{}
cmd := &cobra.Command{
Use: "update",
Short: "Update machine configuration in the cluster.",
Long: `Update machine configuration in the cluster.
This command allows setting various machine properties including:
- Machine name (--name)
- Public IP address (--public-ip)
At least one flag must be specified to perform an update operation.`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return update(cmd.Context(), uncli, cmd, opts, args[0])
},
}
cmd.Flags().StringVar(
&opts.name, "name", "",
"New name for the machine",
)
cmd.Flags().StringVar(
&opts.publicIP, "public-ip", "",
fmt.Sprintf("Public IP address of the machine for ingress configuration. Use '%s' or '' to remove the public IP.", PublicIPNone),
)
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func update(ctx context.Context, uncli *cli.CLI, cmd *cobra.Command, opts updateOptions, machineNameOrID string) error {
// Check if at least one flag was explicitly set
if !cmd.Flags().Changed("name") && !cmd.Flags().Changed("public-ip") {
return fmt.Errorf("at least one update flag must be specified (--name, --public-ip)")
}
client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return err
}
defer client.Close()
// First, resolve the machine to get its ID
machine, err := client.InspectMachine(ctx, machineNameOrID)
if err != nil {
return fmt.Errorf("find machine: %w", err)
}
// Build the update request
req := &pb.UpdateMachineRequest{
MachineId: machine.Machine.Id,
}
if opts.name != "" {
req.Name = &opts.name
}
// Check if --public-ip flag was explicitly provided
if cmd.Flags().Changed("public-ip") {
if opts.publicIP == "" || opts.publicIP == PublicIPNone {
req.PublicIp = &pb.IP{} // Empty IP to signal removal
} else {
// Parse and validate the public IP
ip, err := netip.ParseAddr(opts.publicIP)
if err != nil {
return fmt.Errorf("invalid public IP address %q: %w", opts.publicIP, err)
}
req.PublicIp = pb.NewIP(ip)
}
}
// Perform the update operation
updatedMachine, err := client.UpdateMachine(ctx, req)
if err != nil {
return fmt.Errorf("update machine: %w", err)
}
// Report what was changed
changes := make([]string, 0)
if opts.name != "" {
changes = append(changes, fmt.Sprintf("name: %q -> %q", machine.Machine.Name, updatedMachine.Name))
}
if cmd.Flags().Changed("public-ip") {
oldIP := PublicIPNone
if machine.Machine.PublicIp != nil {
if addr, err := machine.Machine.PublicIp.ToAddr(); err == nil {
oldIP = addr.String()
}
}
newIP := PublicIPNone
if updatedMachine.PublicIp != nil {
if addr, err := updatedMachine.PublicIp.ToAddr(); err == nil {
newIP = addr.String()
}
}
changes = append(changes, fmt.Sprintf("public IP: %s -> %s", oldIP, newIP))
}
fmt.Printf("Machine %q (ID: %s) configuration updated:\n", updatedMachine.Name, updatedMachine.Id)
for _, change := range changes {
fmt.Printf(" %s\n", change)
}
return nil
}
-1
View File
@@ -9,7 +9,6 @@ import (
"github.com/docker/docker/pkg/stringid" "github.com/docker/docker/pkg/stringid"
"github.com/docker/go-units" "github.com/docker/go-units"
"github.com/psviderski/uncloud/internal/cli" "github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
-41
View File
@@ -1,41 +0,0 @@
# Website
This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator.
### Installation
```
$ yarn
```
### Local Development
```
$ yarn start
```
This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.
### Build
```
$ yarn build
```
This command generates static content into the `build` directory and can be served using any static contents hosting service.
### Deployment
Using SSH:
```
$ USE_SSH=true yarn deploy
```
Not using SSH:
```
$ GIT_USER=<Your GitHub username> yarn deploy
```
If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch.
-8
View File
@@ -1,8 +0,0 @@
services:
uncloud-docs:
image: ghcr.io/psviderski/uncloud-docs:latest
pull_policy: always
user: nobody
x-ports:
- docs.uncloud.run:8000/https
scale: 2
-18434
View File
File diff suppressed because it is too large Load Diff
-45
View File
@@ -1,45 +0,0 @@
{
"name": "docs",
"version": "0.0.0",
"private": true,
"scripts": {
"docusaurus": "docusaurus",
"start": "docusaurus start",
"build": "docusaurus build",
"swizzle": "docusaurus swizzle",
"deploy": "docusaurus deploy",
"clear": "docusaurus clear",
"serve": "docusaurus serve",
"write-translations": "docusaurus write-translations",
"write-heading-ids": "docusaurus write-heading-ids"
},
"dependencies": {
"@docusaurus/core": "3.7.0",
"@docusaurus/preset-classic": "3.7.0",
"@easyops-cn/docusaurus-search-local": "^0.49.2",
"@mdx-js/react": "^3.0.0",
"clsx": "^2.0.0",
"prism-react-renderer": "^2.3.0",
"react": "^19.0.0",
"react-dom": "^19.0.0"
},
"devDependencies": {
"@docusaurus/module-type-aliases": "3.7.0",
"@docusaurus/types": "3.7.0"
},
"browserslist": {
"production": [
">0.5%",
"not dead",
"not op_mini all"
],
"development": [
"last 3 chrome version",
"last 3 firefox version",
"last 5 safari version"
]
},
"engines": {
"node": ">=18.0"
}
}
+3 -2
View File
@@ -3,10 +3,11 @@ package main
import ( import (
"context" "context"
"fmt" "fmt"
"github.com/hashicorp/serf/serf"
crdt "github.com/ipfs/go-ds-crdt"
"log/slog" "log/slog"
"time" "time"
"github.com/hashicorp/serf/serf"
crdt "github.com/ipfs/go-ds-crdt"
) )
// Implements the Broadcaster interface. // Implements the Broadcaster interface.
+2 -1
View File
@@ -2,9 +2,10 @@ package main
import ( import (
"fmt" "fmt"
"github.com/dgraph-io/badger/v3"
"log" "log"
"time" "time"
"github.com/dgraph-io/badger/v3"
) )
func customTimeEncoder(t time.Time) string { func customTimeEncoder(t time.Time) string {
+2 -1
View File
@@ -2,9 +2,10 @@ package main
import ( import (
"fmt" "fmt"
"github.com/ipfs/go-log/v2"
"log/slog" "log/slog"
"os" "os"
"github.com/ipfs/go-log/v2"
) )
// ipfsLogger is an slog logger that implements the IPFS go-log StandardLogger interface. // ipfsLogger is an slog logger that implements the IPFS go-log StandardLogger interface.
+2 -1
View File
@@ -2,12 +2,13 @@ package main
import ( import (
"fmt" "fmt"
"github.com/docker/docker/libnetwork/networkdb"
"log/slog" "log/slog"
"os" "os"
"os/signal" "os/signal"
"syscall" "syscall"
"time" "time"
"github.com/docker/docker/libnetwork/networkdb"
) )
func main() { func main() {
+13 -12
View File
@@ -4,6 +4,13 @@ import (
"context" "context"
"flag" "flag"
"fmt" "fmt"
"log/slog"
"net"
"os"
"os/signal"
"syscall"
"time"
"github.com/hashicorp/memberlist" "github.com/hashicorp/memberlist"
"github.com/hashicorp/serf/cmd/serf/command/agent" "github.com/hashicorp/serf/cmd/serf/command/agent"
"github.com/hashicorp/serf/serf" "github.com/hashicorp/serf/serf"
@@ -11,12 +18,6 @@ import (
badger "github.com/ipfs/go-ds-badger3" badger "github.com/ipfs/go-ds-badger3"
crdt "github.com/ipfs/go-ds-crdt" crdt "github.com/ipfs/go-ds-crdt"
"github.com/lmittmann/tint" "github.com/lmittmann/tint"
"log/slog"
"net"
"os"
"os/signal"
"syscall"
"time"
) )
func createSerfAgentConfig(name, bindAddr, rpcAddr, profile string) *agent.Config { func createSerfAgentConfig(name, bindAddr, rpcAddr, profile string) *agent.Config {
@@ -49,9 +50,9 @@ func createSerfAgent(config *agent.Config) (*agent.Agent, error) {
serfConfig.MemberlistConfig.BindAddr = bindIP serfConfig.MemberlistConfig.BindAddr = bindIP
serfConfig.MemberlistConfig.BindPort = bindPort serfConfig.MemberlistConfig.BindPort = bindPort
//serfConfig.MemberlistConfig.AdvertiseAddr = advertiseIP // serfConfig.MemberlistConfig.AdvertiseAddr = advertiseIP
//serfConfig.MemberlistConfig.AdvertisePort = advertisePort // serfConfig.MemberlistConfig.AdvertisePort = advertisePort
//serfConfig.MemberlistConfig.SecretKey = encryptKey // serfConfig.MemberlistConfig.SecretKey = encryptKey
serfConfig.NodeName = config.NodeName serfConfig.NodeName = config.NodeName
serfConfig.Tags = config.Tags serfConfig.Tags = config.Tags
serfConfig.SnapshotPath = config.SnapshotPath serfConfig.SnapshotPath = config.SnapshotPath
@@ -129,7 +130,7 @@ func main() {
logger := slog.New(tint.NewHandler(os.Stdout, &tint.Options{ logger := slog.New(tint.NewHandler(os.Stdout, &tint.Options{
AddSource: true, AddSource: true,
Level: slog.LevelDebug, Level: slog.LevelDebug,
//Level: slog.LevelInfo, // Level: slog.LevelInfo,
TimeFormat: time.RFC3339Nano, TimeFormat: time.RFC3339Nano,
})) }))
slog.SetDefault(logger) slog.SetDefault(logger)
@@ -164,7 +165,7 @@ func main() {
opts := crdt.DefaultOptions() opts := crdt.DefaultOptions()
opts.Logger = newIPFSLogger(logger) opts.Logger = newIPFSLogger(logger)
//opts.MultiHeadProcessing = true // opts.MultiHeadProcessing = true
// TODO: debug why the heads count may grow on the receiving side if the event backlog is huge and the processing // TODO: debug why the heads count may grow on the receiving side if the event backlog is huge and the processing
// is slow. // is slow.
store, err := crdt.New(localStore, ds.NewKey("/"), syncer, broadcaster, opts) store, err := crdt.New(localStore, ds.NewKey("/"), syncer, broadcaster, opts)
@@ -172,7 +173,7 @@ func main() {
panic(err) panic(err)
} }
//ticker := time.NewTicker(10 * time.Millisecond) // ticker := time.NewTicker(10 * time.Millisecond)
ticker := time.NewTicker(3 * time.Second) ticker := time.NewTicker(3 * time.Second)
go func() { go func() {
for { for {
+4 -5
View File
@@ -4,18 +4,19 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"log/slog"
"github.com/hashicorp/serf/serf" "github.com/hashicorp/serf/serf"
"github.com/ipfs/boxo/datastore/dshelp" "github.com/ipfs/boxo/datastore/dshelp"
dag "github.com/ipfs/boxo/ipld/merkledag" dag "github.com/ipfs/boxo/ipld/merkledag"
"github.com/ipfs/go-cid" "github.com/ipfs/go-cid"
ds "github.com/ipfs/go-datastore" ds "github.com/ipfs/go-datastore"
ipld "github.com/ipfs/go-ipld-format" ipld "github.com/ipfs/go-ipld-format"
"log/slog"
) )
// Implements the DAGService interface. // Implements the DAGService interface.
// TODO: implement SessionDAGService to optimize node fetching. // TODO: implement SessionDAGService to optimize node fetching.
// TOOD: persistentSerfDAG? // TODO: persistentSerfDAG?
type dagSyncer struct { type dagSyncer struct {
// Persistent storage for the nodes. // Persistent storage for the nodes.
store ds.Datastore store ds.Datastore
@@ -50,8 +51,7 @@ func (d *dagSyncer) Get(ctx context.Context, cid cid.Cid) (ipld.Node, error) {
} }
slog.Debug("Queried node from peers", "cid", cid, "deadline", query.Deadline()) slog.Debug("Queried node from peers", "cid", cid, "deadline", query.Deadline())
responded := false for {
for !responded {
select { select {
case resp, ok := <-query.ResponseCh(): case resp, ok := <-query.ResponseCh():
if !ok { if !ok {
@@ -63,7 +63,6 @@ func (d *dagSyncer) Get(ctx context.Context, cid cid.Cid) (ipld.Node, error) {
continue continue
} }
slog.Debug("Received node from peer", "cid", cid, "peer", resp.From) slog.Debug("Received node from peer", "cid", cid, "peer", resp.From)
responded = true
query.Close() query.Close()
node, err = nodeFromBytes(resp.Payload) node, err = nodeFromBytes(resp.Payload)
+4 -3
View File
@@ -6,12 +6,13 @@ import (
"crypto/cipher" "crypto/cipher"
"encoding/hex" "encoding/hex"
"fmt" "fmt"
"net/netip"
"time"
"github.com/psviderski/uncloud/internal/machine/network"
"github.com/siderolabs/discovery-api/api/v1alpha1/client/pb" "github.com/siderolabs/discovery-api/api/v1alpha1/client/pb"
discovery "github.com/siderolabs/discovery-client/pkg/client" discovery "github.com/siderolabs/discovery-client/pkg/client"
"go.uber.org/zap" "go.uber.org/zap"
"net/netip"
"time"
"github.com/psviderski/uncloud/internal/machine/network"
) )
const ( const (
+1 -1
View File
@@ -35,6 +35,7 @@ require (
github.com/jmoiron/sqlx v1.4.0 github.com/jmoiron/sqlx v1.4.0
github.com/lmittmann/tint v1.0.5 github.com/lmittmann/tint v1.0.5
github.com/miekg/dns v1.1.65 github.com/miekg/dns v1.1.65
github.com/mitchellh/mapstructure v1.5.0
github.com/moby/term v0.5.0 github.com/moby/term v0.5.0
github.com/opencontainers/go-digest v1.0.0 github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.1.0 github.com/opencontainers/image-spec v1.1.0
@@ -204,7 +205,6 @@ require (
github.com/mitchellh/go-homedir v1.1.0 // indirect github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-ps v1.0.0 // indirect github.com/mitchellh/go-ps v1.0.0 // indirect
github.com/mitchellh/hashstructure/v2 v2.0.2 // indirect github.com/mitchellh/hashstructure/v2 v2.0.2 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/moby/buildkit v0.17.2 // indirect github.com/moby/buildkit v0.17.2 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect github.com/moby/docker-image-spec v1.3.1 // indirect
+35 -35
View File
@@ -6,8 +6,8 @@ import (
"fmt" "fmt"
"net/netip" "net/netip"
"os" "os"
"slices"
"github.com/charmbracelet/huh"
"github.com/docker/cli/cli/streams" "github.com/docker/cli/cli/streams"
"github.com/psviderski/uncloud/internal/cli/config" "github.com/psviderski/uncloud/internal/cli/config"
"github.com/psviderski/uncloud/internal/fs" "github.com/psviderski/uncloud/internal/fs"
@@ -22,7 +22,12 @@ import (
"google.golang.org/protobuf/types/known/emptypb" "google.golang.org/protobuf/types/known/emptypb"
) )
const defaultContextName = "default" const (
// DefaultSSHKeyPath is the fallback location for the SSH private key when provisioning remote machines.
// Used when no key is explicitly provided and SSH agent authentication fails.
DefaultSSHKeyPath = "~/.ssh/id_ed25519"
defaultContextName = "default"
)
type CLI struct { type CLI struct {
Config *config.Config Config *config.Config
@@ -173,7 +178,7 @@ func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions)
return nil, fmt.Errorf("cluster context '%s' already exists", contextName) return nil, fmt.Errorf("cluster context '%s' already exists", contextName)
} }
machineClient, err := cli.provisionRemoteMachine(ctx, *opts.RemoteMachine, opts.Version) machineClient, err := provisionRemoteMachine(ctx, opts.RemoteMachine, opts.Version)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -190,7 +195,7 @@ func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions)
return nil, fmt.Errorf("inspect machine: %w", err) return nil, fmt.Errorf("inspect machine: %w", err)
} }
if minfo.Id != "" { if minfo.Id != "" {
if err = cli.promptResetMachine(); err != nil { if err = promptResetMachine(ctx, machineClient.MachineClient); err != nil {
return nil, err return nil, err
} }
} }
@@ -249,7 +254,7 @@ type AddMachineOptions struct {
Context string Context string
MachineName string MachineName string
PublicIP *netip.Addr PublicIP *netip.Addr
RemoteMachine RemoteMachine RemoteMachine *RemoteMachine
Version string Version string
} }
@@ -272,7 +277,7 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
} }
}() }()
machineClient, err := cli.provisionRemoteMachine(ctx, opts.RemoteMachine, opts.Version) machineClient, err := provisionRemoteMachine(ctx, opts.RemoteMachine, opts.Version)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
@@ -288,7 +293,18 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
return nil, nil, fmt.Errorf("inspect machine: %w", err) return nil, nil, fmt.Errorf("inspect machine: %w", err)
} }
if minfo.Id != "" { if minfo.Id != "" {
if err = cli.promptResetMachine(); err != nil { // Check if the machine is already a member of this cluster.
machines, err := c.ListMachines(ctx, nil)
if err != nil {
return nil, nil, fmt.Errorf("list cluster machines: %w", err)
}
if slices.ContainsFunc(machines, func(m *pb.MachineMember) bool {
return m.Machine.Id == minfo.Id
}) {
return nil, nil, fmt.Errorf("machine is already a member of this cluster (%s)", minfo.Name)
}
if err = promptResetMachine(ctx, machineClient.MachineClient); err != nil {
return nil, nil, err return nil, nil, err
} }
} }
@@ -339,7 +355,7 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
return nil, nil, fmt.Errorf("add machine to cluster (context '%s'): %w", contextName, err) return nil, nil, fmt.Errorf("add machine to cluster (context '%s'): %w", contextName, err)
} }
// List other machines in the cluster to include them in the join request. // Get the most up-to-date list of other machines in the cluster to include them in the join request.
machines, err := c.ListMachines(ctx, nil) machines, err := c.ListMachines(ctx, nil)
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("list cluster machines: %w", err) return nil, nil, fmt.Errorf("list cluster machines: %w", err)
@@ -382,11 +398,20 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
// provisionRemoteMachine installs the Uncloud daemon and dependencies on the remote machine over SSH and returns // provisionRemoteMachine installs the Uncloud daemon and dependencies on the remote machine over SSH and returns
// a machine API client to interact with the machine. The client should be closed after use by the caller. // a machine API client to interact with the machine. The client should be closed after use by the caller.
// The version parameter specifies the version of the Uncloud daemon to install. If empty, the latest version is used. // The version parameter specifies the version of the Uncloud daemon to install. If empty, the latest version is used.
func (cli *CLI) provisionRemoteMachine( // The remoteMachine.SSHKeyPath could be updated to the default SSH key path if it is not set and the SSH agent
ctx context.Context, remoteMachine RemoteMachine, version string, // authentication fails.
func provisionRemoteMachine(
ctx context.Context, remoteMachine *RemoteMachine, version string,
) (*client.Client, error) { ) (*client.Client, error) {
// Provision the remote machine by installing the Uncloud daemon and dependencies over SSH. // Provision the remote machine by installing the Uncloud daemon and dependencies over SSH.
sshClient, err := sshexec.Connect(remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath) sshClient, err := sshexec.Connect(remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath)
// If the SSH connection using SSH agent fails and no key path is provided, try to use the default SSH key.
if err != nil && remoteMachine.KeyPath == "" {
remoteMachine.KeyPath = DefaultSSHKeyPath
sshClient, err = sshexec.Connect(
remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath,
)
}
if err != nil { if err != nil {
return nil, fmt.Errorf( return nil, fmt.Errorf(
"SSH login to remote machine %s: %w", "SSH login to remote machine %s: %w",
@@ -420,31 +445,6 @@ func (cli *CLI) provisionRemoteMachine(
return machineClient, nil return machineClient, nil
} }
func (cli *CLI) promptResetMachine() error {
var confirm bool
form := huh.NewForm(
huh.NewGroup(
huh.NewConfirm().
Title(
"The remote machine is already initialised as a cluster member. Do you want to reset it first?",
).
Affirmative("Yes!").
Negative("No").
Value(&confirm),
),
).WithAccessible(true)
if err := form.Run(); err != nil {
return fmt.Errorf("prompt user to confirm: %w", err)
}
if !confirm {
return fmt.Errorf("remote machine is already initialised as a cluster member")
}
// TODO: implement resetting the remote machine.
return fmt.Errorf("resetting the remote machine is not implemented yet. " +
"Please manually run 'uncloud-uninstall' on the remote machine to fully uninstall Uncloud from it")
}
// ProgressOut returns an output stream for progress writer. // ProgressOut returns an output stream for progress writer.
func (cli *CLI) ProgressOut() *streams.Out { func (cli *CLI) ProgressOut() *streams.Out {
return streams.NewOut(os.Stdout) return streams.NewOut(os.Stdout)
+2 -2
View File
@@ -53,11 +53,11 @@ func (c *Config) Read() error {
func (c *Config) Save() error { func (c *Config) Save() error {
dir, _ := filepath.Split(c.path) dir, _ := filepath.Split(c.path)
if err := os.MkdirAll(dir, 0700); err != nil { if err := os.MkdirAll(dir, 0o700); err != nil {
return fmt.Errorf("create config directory '%s': %w", dir, err) return fmt.Errorf("create config directory '%s': %w", dir, err)
} }
f, err := os.OpenFile(c.path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600) f, err := os.OpenFile(c.path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o600)
if err != nil { if err != nil {
return fmt.Errorf("write config file '%s': %w", c.path, err) return fmt.Errorf("write config file '%s': %w", c.path, err)
} }
+84 -3
View File
@@ -5,12 +5,20 @@ import (
"fmt" "fmt"
"os" "os"
"strings" "strings"
"time"
"github.com/cenkalti/backoff/v4"
"github.com/charmbracelet/huh"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/sshexec" "github.com/psviderski/uncloud/internal/sshexec"
"google.golang.org/protobuf/types/known/emptypb"
) )
// TODO: support pinning the script version to the CLI version. const (
const installScriptURL = "https://raw.githubusercontent.com/psviderski/uncloud/refs/heads/main/scripts/install.sh" // TODO: support pinning the script version to the CLI version.
installScriptURL = "https://raw.githubusercontent.com/psviderski/uncloud/refs/heads/main/scripts/install.sh"
rootUser = "root"
)
type RemoteMachine struct { type RemoteMachine struct {
User string User string
@@ -24,7 +32,7 @@ func installCmd(user string, version string) string {
var env []string var env []string
// Add the SSH user (non-root) to the uncloud group to allow access to the Uncloud daemon unix socket. // Add the SSH user (non-root) to the uncloud group to allow access to the Uncloud daemon unix socket.
if user != "root" { if user != rootUser {
sudoPrefix = "sudo" sudoPrefix = "sudo"
env = append(env, "UNCLOUD_GROUP_ADD_USER="+sshexec.Quote(user)) env = append(env, "UNCLOUD_GROUP_ADD_USER="+sshexec.Quote(user))
} }
@@ -46,6 +54,26 @@ func provisionMachine(ctx context.Context, exec sshexec.Executor, version string
return fmt.Errorf("run whoami: %w", err) return fmt.Errorf("run whoami: %w", err)
} }
if user != rootUser {
// 'sudo -n' is not used because it fails with 'sudo: a password is required' when the user has no password
// in /etc/shadow even though it may have valid sudo access.
out, err := exec.Run(ctx, "sudo true")
if err != nil {
if strings.Contains(out, "password is required") {
return fmt.Errorf(
"user '%[1]s' requires a password for sudo, but Uncloud needs passwordless sudo or root access "+
"to install and configure the uncloudd daemon on the remote machine.\n\n"+
"Possible solutions:\n"+
"1. Use root user or a user with passwordless sudo instead.\n"+
"2. Configure passwordless sudo for the user '%[1]s' by running on the remote machine:\n"+
" echo '%[1]s ALL=(ALL) NOPASSWD:ALL' | sudo tee /etc/sudoers.d/%[1]s",
user)
}
return fmt.Errorf("sudo command failed for user '%s': %w. "+
"Please ensure the user has sudo privileges or use root user instead", user, err)
}
}
cmd := installCmd(user, version) cmd := installCmd(user, version)
fmt.Println("Downloading Uncloud install script:", installScriptURL) fmt.Println("Downloading Uncloud install script:", installScriptURL)
@@ -56,3 +84,56 @@ func provisionMachine(ctx context.Context, exec sshexec.Executor, version string
} }
return nil return nil
} }
func promptResetMachine(ctx context.Context, machineClient pb.MachineClient) error {
var confirm bool
form := huh.NewForm(
huh.NewGroup(
huh.NewConfirm().
Title(
"The remote machine is already initialised as a cluster member. Do you want to reset it first?\n" +
"This will:\n" +
"- Remove all service containers from the machine\n" +
"- Reset the machine to the uninitialised state",
).
Affirmative("Yes!").
Negative("No").
Value(&confirm),
),
).WithAccessible(true)
if err := form.Run(); err != nil {
return fmt.Errorf("prompt user to confirm: %w", err)
}
if !confirm {
return fmt.Errorf("remote machine is already initialised as a cluster member")
}
if _, err := machineClient.Reset(ctx, &pb.ResetRequest{}); err != nil {
return fmt.Errorf("reset remote machine: %w. You can also manually run 'uncloud-uninstall' "+
"on the remote machine to fully uninstall Uncloud from it", err)
}
fmt.Println("Resetting the remote machine...")
if err := waitMachineReady(ctx, machineClient, 1*time.Minute); err != nil {
return fmt.Errorf("wait for machine to be ready after reset: %w", err)
}
return nil
}
// waitMachineReady waits for the machine to be ready to serve requests.
func waitMachineReady(ctx context.Context, machineClient pb.MachineClient, timeout time.Duration) error {
boff := backoff.WithContext(backoff.NewExponentialBackOff(
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(timeout),
), ctx)
inspect := func() error {
_, err := machineClient.Inspect(ctx, &emptypb.Empty{})
if err != nil {
return fmt.Errorf("inspect machine: %w", err)
}
return nil
}
return backoff.Retry(inspect, boff)
}
+3 -2
View File
@@ -5,14 +5,15 @@ import (
"crypto/tls" "crypto/tls"
"errors" "errors"
"fmt" "fmt"
"github.com/cenkalti/backoff/v4"
"golang.org/x/net/http2"
"log/slog" "log/slog"
"net" "net"
"net/http" "net/http"
"net/netip" "net/netip"
"net/url" "net/url"
"time" "time"
"github.com/cenkalti/backoff/v4"
"golang.org/x/net/http2"
) )
const ( const (
+2 -1
View File
@@ -6,11 +6,12 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"github.com/cenkalti/backoff/v4"
"io" "io"
"log/slog" "log/slog"
"net/http" "net/http"
"strconv" "strconv"
"github.com/cenkalti/backoff/v4"
) )
type ChangeType string type ChangeType string
+2 -1
View File
@@ -3,8 +3,9 @@ package daemon
import ( import (
"context" "context"
"fmt" "fmt"
systemd "github.com/coreos/go-systemd/daemon"
"log/slog" "log/slog"
systemd "github.com/coreos/go-systemd/daemon"
"github.com/psviderski/uncloud/internal/machine" "github.com/psviderski/uncloud/internal/machine"
) )
+3 -2
View File
@@ -3,10 +3,11 @@ package daemon
import ( import (
"errors" "errors"
"fmt" "fmt"
"github.com/psviderski/uncloud/internal/machine"
"github.com/psviderski/uncloud/internal/machine/network"
"net/netip" "net/netip"
"os" "os"
"github.com/psviderski/uncloud/internal/machine"
"github.com/psviderski/uncloud/internal/machine/network"
) )
// MachineToken returns the local machine's token that can be used for adding the machine to a cluster. // MachineToken returns the local machine's token that can be used for adding the machine to a cluster.
+3 -2
View File
@@ -4,10 +4,11 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"github.com/cenkalti/backoff/v4"
"github.com/docker/docker/client"
"log/slog" "log/slog"
"time" "time"
"github.com/cenkalti/backoff/v4"
"github.com/docker/docker/client"
) )
// WaitDaemonReady waits for the Docker daemon to start and be ready to serve requests. // WaitDaemonReady waits for the Docker daemon to start and be ready to serve requests.
+350 -107
View File
@@ -124,7 +124,7 @@ func (x DNSRecord_RecordType) Number() protoreflect.EnumNumber {
// Deprecated: Use DNSRecord_RecordType.Descriptor instead. // Deprecated: Use DNSRecord_RecordType.Descriptor instead.
func (DNSRecord_RecordType) EnumDescriptor() ([]byte, []int) { func (DNSRecord_RecordType) EnumDescriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{8, 0} return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{11, 0}
} }
type AddMachineRequest struct { type AddMachineRequest struct {
@@ -339,6 +339,173 @@ func (x *ListMachinesResponse) GetMachines() []*MachineMember {
return nil return nil
} }
type UpdateMachineRequest struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
// Machine to update
MachineId string `protobuf:"bytes,1,opt,name=machine_id,json=machineId,proto3" json:"machine_id,omitempty"`
// Updated machine information
Name *string `protobuf:"bytes,2,opt,name=name,proto3,oneof" json:"name,omitempty"`
PublicIp *IP `protobuf:"bytes,3,opt,name=public_ip,json=publicIp,proto3,oneof" json:"public_ip,omitempty"`
Endpoints []*IPPort `protobuf:"bytes,4,rep,name=endpoints,proto3" json:"endpoints,omitempty"`
}
func (x *UpdateMachineRequest) Reset() {
*x = UpdateMachineRequest{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[4]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *UpdateMachineRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*UpdateMachineRequest) ProtoMessage() {}
func (x *UpdateMachineRequest) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[4]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use UpdateMachineRequest.ProtoReflect.Descriptor instead.
func (*UpdateMachineRequest) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{4}
}
func (x *UpdateMachineRequest) GetMachineId() string {
if x != nil {
return x.MachineId
}
return ""
}
func (x *UpdateMachineRequest) GetName() string {
if x != nil && x.Name != nil {
return *x.Name
}
return ""
}
func (x *UpdateMachineRequest) GetPublicIp() *IP {
if x != nil {
return x.PublicIp
}
return nil
}
func (x *UpdateMachineRequest) GetEndpoints() []*IPPort {
if x != nil {
return x.Endpoints
}
return nil
}
type UpdateMachineResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Machine *MachineInfo `protobuf:"bytes,1,opt,name=machine,proto3" json:"machine,omitempty"`
}
func (x *UpdateMachineResponse) Reset() {
*x = UpdateMachineResponse{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[5]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *UpdateMachineResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*UpdateMachineResponse) ProtoMessage() {}
func (x *UpdateMachineResponse) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[5]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use UpdateMachineResponse.ProtoReflect.Descriptor instead.
func (*UpdateMachineResponse) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{5}
}
func (x *UpdateMachineResponse) GetMachine() *MachineInfo {
if x != nil {
return x.Machine
}
return nil
}
type RemoveMachineRequest struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Id string `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
}
func (x *RemoveMachineRequest) Reset() {
*x = RemoveMachineRequest{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[6]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *RemoveMachineRequest) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*RemoveMachineRequest) ProtoMessage() {}
func (x *RemoveMachineRequest) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[6]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use RemoveMachineRequest.ProtoReflect.Descriptor instead.
func (*RemoveMachineRequest) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{6}
}
func (x *RemoveMachineRequest) GetId() string {
if x != nil {
return x.Id
}
return ""
}
type Domain struct { type Domain struct {
state protoimpl.MessageState state protoimpl.MessageState
sizeCache protoimpl.SizeCache sizeCache protoimpl.SizeCache
@@ -350,7 +517,7 @@ type Domain struct {
func (x *Domain) Reset() { func (x *Domain) Reset() {
*x = Domain{} *x = Domain{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[4] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[7]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -363,7 +530,7 @@ func (x *Domain) String() string {
func (*Domain) ProtoMessage() {} func (*Domain) ProtoMessage() {}
func (x *Domain) ProtoReflect() protoreflect.Message { func (x *Domain) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[4] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[7]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -376,7 +543,7 @@ func (x *Domain) ProtoReflect() protoreflect.Message {
// Deprecated: Use Domain.ProtoReflect.Descriptor instead. // Deprecated: Use Domain.ProtoReflect.Descriptor instead.
func (*Domain) Descriptor() ([]byte, []int) { func (*Domain) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{4} return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{7}
} }
func (x *Domain) GetName() string { func (x *Domain) GetName() string {
@@ -397,7 +564,7 @@ type ReserveDomainRequest struct {
func (x *ReserveDomainRequest) Reset() { func (x *ReserveDomainRequest) Reset() {
*x = ReserveDomainRequest{} *x = ReserveDomainRequest{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[5] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[8]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -410,7 +577,7 @@ func (x *ReserveDomainRequest) String() string {
func (*ReserveDomainRequest) ProtoMessage() {} func (*ReserveDomainRequest) ProtoMessage() {}
func (x *ReserveDomainRequest) ProtoReflect() protoreflect.Message { func (x *ReserveDomainRequest) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[5] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[8]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -423,7 +590,7 @@ func (x *ReserveDomainRequest) ProtoReflect() protoreflect.Message {
// Deprecated: Use ReserveDomainRequest.ProtoReflect.Descriptor instead. // Deprecated: Use ReserveDomainRequest.ProtoReflect.Descriptor instead.
func (*ReserveDomainRequest) Descriptor() ([]byte, []int) { func (*ReserveDomainRequest) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{5} return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{8}
} }
func (x *ReserveDomainRequest) GetEndpoint() string { func (x *ReserveDomainRequest) GetEndpoint() string {
@@ -444,7 +611,7 @@ type CreateDomainRecordsRequest struct {
func (x *CreateDomainRecordsRequest) Reset() { func (x *CreateDomainRecordsRequest) Reset() {
*x = CreateDomainRecordsRequest{} *x = CreateDomainRecordsRequest{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[6] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[9]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -457,7 +624,7 @@ func (x *CreateDomainRecordsRequest) String() string {
func (*CreateDomainRecordsRequest) ProtoMessage() {} func (*CreateDomainRecordsRequest) ProtoMessage() {}
func (x *CreateDomainRecordsRequest) ProtoReflect() protoreflect.Message { func (x *CreateDomainRecordsRequest) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[6] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[9]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -470,7 +637,7 @@ func (x *CreateDomainRecordsRequest) ProtoReflect() protoreflect.Message {
// Deprecated: Use CreateDomainRecordsRequest.ProtoReflect.Descriptor instead. // Deprecated: Use CreateDomainRecordsRequest.ProtoReflect.Descriptor instead.
func (*CreateDomainRecordsRequest) Descriptor() ([]byte, []int) { func (*CreateDomainRecordsRequest) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{6} return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{9}
} }
func (x *CreateDomainRecordsRequest) GetRecords() []*DNSRecord { func (x *CreateDomainRecordsRequest) GetRecords() []*DNSRecord {
@@ -491,7 +658,7 @@ type CreateDomainRecordsResponse struct {
func (x *CreateDomainRecordsResponse) Reset() { func (x *CreateDomainRecordsResponse) Reset() {
*x = CreateDomainRecordsResponse{} *x = CreateDomainRecordsResponse{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[7] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[10]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -504,7 +671,7 @@ func (x *CreateDomainRecordsResponse) String() string {
func (*CreateDomainRecordsResponse) ProtoMessage() {} func (*CreateDomainRecordsResponse) ProtoMessage() {}
func (x *CreateDomainRecordsResponse) ProtoReflect() protoreflect.Message { func (x *CreateDomainRecordsResponse) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[7] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[10]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -517,7 +684,7 @@ func (x *CreateDomainRecordsResponse) ProtoReflect() protoreflect.Message {
// Deprecated: Use CreateDomainRecordsResponse.ProtoReflect.Descriptor instead. // Deprecated: Use CreateDomainRecordsResponse.ProtoReflect.Descriptor instead.
func (*CreateDomainRecordsResponse) Descriptor() ([]byte, []int) { func (*CreateDomainRecordsResponse) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{7} return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{10}
} }
func (x *CreateDomainRecordsResponse) GetRecords() []*DNSRecord { func (x *CreateDomainRecordsResponse) GetRecords() []*DNSRecord {
@@ -540,7 +707,7 @@ type DNSRecord struct {
func (x *DNSRecord) Reset() { func (x *DNSRecord) Reset() {
*x = DNSRecord{} *x = DNSRecord{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[8] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[11]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -553,7 +720,7 @@ func (x *DNSRecord) String() string {
func (*DNSRecord) ProtoMessage() {} func (*DNSRecord) ProtoMessage() {}
func (x *DNSRecord) ProtoReflect() protoreflect.Message { func (x *DNSRecord) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[8] mi := &file_internal_machine_api_pb_cluster_proto_msgTypes[11]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -566,7 +733,7 @@ func (x *DNSRecord) ProtoReflect() protoreflect.Message {
// Deprecated: Use DNSRecord.ProtoReflect.Descriptor instead. // Deprecated: Use DNSRecord.ProtoReflect.Descriptor instead.
func (*DNSRecord) Descriptor() ([]byte, []int) { func (*DNSRecord) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{8} return file_internal_machine_api_pb_cluster_proto_rawDescGZIP(), []int{11}
} }
func (x *DNSRecord) GetName() string { func (x *DNSRecord) GetName() string {
@@ -630,59 +797,87 @@ var file_internal_machine_api_pb_cluster_proto_rawDesc = []byte{
0x6f, 0x6e, 0x73, 0x65, 0x12, 0x2e, 0x0a, 0x08, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x2e, 0x0a, 0x08, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x73,
0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x12, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4d, 0x61, 0x63, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x12, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4d, 0x61, 0x63,
0x68, 0x69, 0x6e, 0x65, 0x4d, 0x65, 0x6d, 0x62, 0x65, 0x72, 0x52, 0x08, 0x6d, 0x61, 0x63, 0x68, 0x68, 0x69, 0x6e, 0x65, 0x4d, 0x65, 0x6d, 0x62, 0x65, 0x72, 0x52, 0x08, 0x6d, 0x61, 0x63, 0x68,
0x69, 0x6e, 0x65, 0x73, 0x22, 0x1c, 0x0a, 0x06, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x12, 0x69, 0x6e, 0x65, 0x73, 0x22, 0xbb, 0x01, 0x0a, 0x14, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x4d,
0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1d, 0x0a,
0x6d, 0x65, 0x22, 0x32, 0x0a, 0x14, 0x52, 0x65, 0x73, 0x65, 0x72, 0x76, 0x65, 0x44, 0x6f, 0x6d, 0x0a, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28,
0x61, 0x69, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1a, 0x0a, 0x08, 0x65, 0x6e, 0x09, 0x52, 0x09, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x64, 0x12, 0x17, 0x0a, 0x04,
0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x65, 0x6e, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x48, 0x00, 0x52, 0x04, 0x6e, 0x61,
0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x22, 0x46, 0x0a, 0x1a, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x6d, 0x65, 0x88, 0x01, 0x01, 0x12, 0x29, 0x0a, 0x09, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f,
0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x52, 0x65, 0x71, 0x69, 0x70, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x07, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49,
0x75, 0x65, 0x73, 0x74, 0x12, 0x28, 0x0a, 0x07, 0x72, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x18, 0x50, 0x48, 0x01, 0x52, 0x08, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x49, 0x70, 0x88, 0x01, 0x01,
0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x4e, 0x53, 0x52, 0x12, 0x29, 0x0a, 0x09, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x73, 0x18, 0x04, 0x20,
0x65, 0x63, 0x6f, 0x72, 0x64, 0x52, 0x07, 0x72, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x22, 0x47, 0x03, 0x28, 0x0b, 0x32, 0x0b, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x50, 0x50, 0x6f, 0x72, 0x74,
0x0a, 0x1b, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x52, 0x09, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x73, 0x42, 0x07, 0x0a, 0x05, 0x5f,
0x63, 0x6f, 0x72, 0x64, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x28, 0x0a, 0x6e, 0x61, 0x6d, 0x65, 0x42, 0x0c, 0x0a, 0x0a, 0x5f, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f,
0x07, 0x72, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x0e, 0x69, 0x70, 0x22, 0x43, 0x0a, 0x15, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x4d, 0x61, 0x63, 0x68,
0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x4e, 0x53, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x52, 0x07, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x2a, 0x0a, 0x07, 0x6d,
0x72, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x22, 0x96, 0x01, 0x0a, 0x09, 0x44, 0x4e, 0x53, 0x52, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x61,
0x65, 0x63, 0x6f, 0x72, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x70, 0x69, 0x2e, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x07,
0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x2d, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x22, 0x26, 0x0a, 0x14, 0x52, 0x65, 0x6d, 0x6f, 0x76,
0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0e, 0x32, 0x19, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x4e, 0x65, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12,
0x53, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x79, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x02, 0x69, 0x64, 0x22,
0x70, 0x65, 0x52, 0x04, 0x74, 0x79, 0x70, 0x65, 0x12, 0x16, 0x0a, 0x06, 0x76, 0x61, 0x6c, 0x75, 0x1c, 0x0a, 0x06, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d,
0x65, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x09, 0x52, 0x06, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x22, 0x32, 0x0a,
0x22, 0x2e, 0x0a, 0x0a, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x79, 0x70, 0x65, 0x12, 0x0f, 0x14, 0x52, 0x65, 0x73, 0x65, 0x72, 0x76, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65,
0x0a, 0x0b, 0x55, 0x4e, 0x53, 0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, 0x12, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x1a, 0x0a, 0x08, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e,
0x05, 0x0a, 0x01, 0x41, 0x10, 0x01, 0x12, 0x08, 0x0a, 0x04, 0x41, 0x41, 0x41, 0x41, 0x10, 0x02, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x08, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e,
0x32, 0x86, 0x03, 0x0a, 0x07, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x12, 0x3d, 0x0a, 0x0a, 0x74, 0x22, 0x46, 0x0a, 0x1a, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69,
0x41, 0x64, 0x64, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x12, 0x16, 0x2e, 0x61, 0x70, 0x69, 0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12,
0x2e, 0x41, 0x64, 0x64, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x28, 0x0a, 0x07, 0x72, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b,
0x73, 0x74, 0x1a, 0x17, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x41, 0x64, 0x64, 0x4d, 0x61, 0x63, 0x68, 0x32, 0x0e, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x4e, 0x53, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64,
0x69, 0x6e, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x41, 0x0a, 0x0c, 0x4c, 0x52, 0x07, 0x72, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x22, 0x47, 0x0a, 0x1b, 0x43, 0x72, 0x65,
0x69, 0x73, 0x74, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x12, 0x16, 0x2e, 0x67, 0x6f,
0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d,
0x70, 0x74, 0x79, 0x1a, 0x19, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x4d, 0x61,
0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x37,
0x0a, 0x0d, 0x52, 0x65, 0x73, 0x65, 0x72, 0x76, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12,
0x19, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x52, 0x65, 0x73, 0x65, 0x72, 0x76, 0x65, 0x44, 0x6f, 0x6d,
0x61, 0x69, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x0b, 0x2e, 0x61, 0x70, 0x69,
0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x30, 0x0a, 0x09, 0x47, 0x65, 0x74, 0x44, 0x6f,
0x6d, 0x61, 0x69, 0x6e, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72,
0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x0b, 0x2e, 0x61,
0x70, 0x69, 0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x34, 0x0a, 0x0d, 0x52, 0x65, 0x6c,
0x65, 0x61, 0x73, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f,
0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70,
0x74, 0x79, 0x1a, 0x0b, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12,
0x58, 0x0a, 0x13, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52,
0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x12, 0x1f, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x43, 0x72, 0x65,
0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73,
0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x20, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x43, 0x72, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x28, 0x0a, 0x07, 0x72, 0x65, 0x63, 0x6f,
0x65, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x72, 0x64, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x0e, 0x2e, 0x61, 0x70, 0x69, 0x2e,
0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x37, 0x5a, 0x35, 0x67, 0x69, 0x74, 0x44, 0x4e, 0x53, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x52, 0x07, 0x72, 0x65, 0x63, 0x6f, 0x72,
0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x70, 0x73, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73, 0x64, 0x73, 0x22, 0x96, 0x01, 0x0a, 0x09, 0x44, 0x4e, 0x53, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64,
0x6b, 0x69, 0x2f, 0x75, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04,
0x6e, 0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x2d, 0x0a, 0x04, 0x74, 0x79, 0x70, 0x65, 0x18, 0x02, 0x20, 0x01,
0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, 0x28, 0x0e, 0x32, 0x19, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x4e, 0x53, 0x52, 0x65, 0x63, 0x6f,
0x72, 0x64, 0x2e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x79, 0x70, 0x65, 0x52, 0x04, 0x74,
0x79, 0x70, 0x65, 0x12, 0x16, 0x0a, 0x06, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x18, 0x03, 0x20,
0x03, 0x28, 0x09, 0x52, 0x06, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x73, 0x22, 0x2e, 0x0a, 0x0a, 0x52,
0x65, 0x63, 0x6f, 0x72, 0x64, 0x54, 0x79, 0x70, 0x65, 0x12, 0x0f, 0x0a, 0x0b, 0x55, 0x4e, 0x53,
0x50, 0x45, 0x43, 0x49, 0x46, 0x49, 0x45, 0x44, 0x10, 0x00, 0x12, 0x05, 0x0a, 0x01, 0x41, 0x10,
0x01, 0x12, 0x08, 0x0a, 0x04, 0x41, 0x41, 0x41, 0x41, 0x10, 0x02, 0x32, 0x92, 0x04, 0x0a, 0x07,
0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x12, 0x3d, 0x0a, 0x0a, 0x41, 0x64, 0x64, 0x4d, 0x61,
0x63, 0x68, 0x69, 0x6e, 0x65, 0x12, 0x16, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x41, 0x64, 0x64, 0x4d,
0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x17, 0x2e,
0x61, 0x70, 0x69, 0x2e, 0x41, 0x64, 0x64, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65,
0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x41, 0x0a, 0x0c, 0x4c, 0x69, 0x73, 0x74, 0x4d, 0x61,
0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x19,
0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4c, 0x69, 0x73, 0x74, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65,
0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x46, 0x0a, 0x0d, 0x55, 0x70, 0x64,
0x61, 0x74, 0x65, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x12, 0x19, 0x2e, 0x61, 0x70, 0x69,
0x2e, 0x55, 0x70, 0x64, 0x61, 0x74, 0x65, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65,
0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1a, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x55, 0x70, 0x64, 0x61,
0x74, 0x65, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73,
0x65, 0x12, 0x42, 0x0a, 0x0d, 0x52, 0x65, 0x6d, 0x6f, 0x76, 0x65, 0x4d, 0x61, 0x63, 0x68, 0x69,
0x6e, 0x65, 0x12, 0x19, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x52, 0x65, 0x6d, 0x6f, 0x76, 0x65, 0x4d,
0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x16, 0x2e,
0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e,
0x45, 0x6d, 0x70, 0x74, 0x79, 0x12, 0x37, 0x0a, 0x0d, 0x52, 0x65, 0x73, 0x65, 0x72, 0x76, 0x65,
0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x19, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x52, 0x65, 0x73,
0x65, 0x72, 0x76, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73,
0x74, 0x1a, 0x0b, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x30,
0x0a, 0x09, 0x47, 0x65, 0x74, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x16, 0x2e, 0x67, 0x6f,
0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d,
0x70, 0x74, 0x79, 0x1a, 0x0b, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e,
0x12, 0x34, 0x0a, 0x0d, 0x52, 0x65, 0x6c, 0x65, 0x61, 0x73, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69,
0x6e, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f,
0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x0b, 0x2e, 0x61, 0x70, 0x69, 0x2e,
0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x12, 0x58, 0x0a, 0x13, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65,
0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x12, 0x1f, 0x2e,
0x61, 0x70, 0x69, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69, 0x6e,
0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x20,
0x2e, 0x61, 0x70, 0x69, 0x2e, 0x43, 0x72, 0x65, 0x61, 0x74, 0x65, 0x44, 0x6f, 0x6d, 0x61, 0x69,
0x6e, 0x52, 0x65, 0x63, 0x6f, 0x72, 0x64, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x42, 0x37, 0x5a, 0x35, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x70,
0x73, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73, 0x6b, 0x69, 0x2f, 0x75, 0x6e, 0x63, 0x6c, 0x6f, 0x75,
0x64, 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69,
0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f,
0x33,
} }
var ( var (
@@ -698,7 +893,7 @@ func file_internal_machine_api_pb_cluster_proto_rawDescGZIP() []byte {
} }
var file_internal_machine_api_pb_cluster_proto_enumTypes = make([]protoimpl.EnumInfo, 2) var file_internal_machine_api_pb_cluster_proto_enumTypes = make([]protoimpl.EnumInfo, 2)
var file_internal_machine_api_pb_cluster_proto_msgTypes = make([]protoimpl.MessageInfo, 9) var file_internal_machine_api_pb_cluster_proto_msgTypes = make([]protoimpl.MessageInfo, 12)
var file_internal_machine_api_pb_cluster_proto_goTypes = []any{ var file_internal_machine_api_pb_cluster_proto_goTypes = []any{
(MachineMember_MembershipState)(0), // 0: api.MachineMember.MembershipState (MachineMember_MembershipState)(0), // 0: api.MachineMember.MembershipState
(DNSRecord_RecordType)(0), // 1: api.DNSRecord.RecordType (DNSRecord_RecordType)(0), // 1: api.DNSRecord.RecordType
@@ -706,43 +901,54 @@ var file_internal_machine_api_pb_cluster_proto_goTypes = []any{
(*AddMachineResponse)(nil), // 3: api.AddMachineResponse (*AddMachineResponse)(nil), // 3: api.AddMachineResponse
(*MachineMember)(nil), // 4: api.MachineMember (*MachineMember)(nil), // 4: api.MachineMember
(*ListMachinesResponse)(nil), // 5: api.ListMachinesResponse (*ListMachinesResponse)(nil), // 5: api.ListMachinesResponse
(*Domain)(nil), // 6: api.Domain (*UpdateMachineRequest)(nil), // 6: api.UpdateMachineRequest
(*ReserveDomainRequest)(nil), // 7: api.ReserveDomainRequest (*UpdateMachineResponse)(nil), // 7: api.UpdateMachineResponse
(*CreateDomainRecordsRequest)(nil), // 8: api.CreateDomainRecordsRequest (*RemoveMachineRequest)(nil), // 8: api.RemoveMachineRequest
(*CreateDomainRecordsResponse)(nil), // 9: api.CreateDomainRecordsResponse (*Domain)(nil), // 9: api.Domain
(*DNSRecord)(nil), // 10: api.DNSRecord (*ReserveDomainRequest)(nil), // 10: api.ReserveDomainRequest
(*NetworkConfig)(nil), // 11: api.NetworkConfig (*CreateDomainRecordsRequest)(nil), // 11: api.CreateDomainRecordsRequest
(*IP)(nil), // 12: api.IP (*CreateDomainRecordsResponse)(nil), // 12: api.CreateDomainRecordsResponse
(*MachineInfo)(nil), // 13: api.MachineInfo (*DNSRecord)(nil), // 13: api.DNSRecord
(*emptypb.Empty)(nil), // 14: google.protobuf.Empty (*NetworkConfig)(nil), // 14: api.NetworkConfig
(*IP)(nil), // 15: api.IP
(*MachineInfo)(nil), // 16: api.MachineInfo
(*IPPort)(nil), // 17: api.IPPort
(*emptypb.Empty)(nil), // 18: google.protobuf.Empty
} }
var file_internal_machine_api_pb_cluster_proto_depIdxs = []int32{ var file_internal_machine_api_pb_cluster_proto_depIdxs = []int32{
11, // 0: api.AddMachineRequest.network:type_name -> api.NetworkConfig 14, // 0: api.AddMachineRequest.network:type_name -> api.NetworkConfig
12, // 1: api.AddMachineRequest.public_ip:type_name -> api.IP 15, // 1: api.AddMachineRequest.public_ip:type_name -> api.IP
13, // 2: api.AddMachineResponse.machine:type_name -> api.MachineInfo 16, // 2: api.AddMachineResponse.machine:type_name -> api.MachineInfo
13, // 3: api.MachineMember.machine:type_name -> api.MachineInfo 16, // 3: api.MachineMember.machine:type_name -> api.MachineInfo
0, // 4: api.MachineMember.state:type_name -> api.MachineMember.MembershipState 0, // 4: api.MachineMember.state:type_name -> api.MachineMember.MembershipState
4, // 5: api.ListMachinesResponse.machines:type_name -> api.MachineMember 4, // 5: api.ListMachinesResponse.machines:type_name -> api.MachineMember
10, // 6: api.CreateDomainRecordsRequest.records:type_name -> api.DNSRecord 15, // 6: api.UpdateMachineRequest.public_ip:type_name -> api.IP
10, // 7: api.CreateDomainRecordsResponse.records:type_name -> api.DNSRecord 17, // 7: api.UpdateMachineRequest.endpoints:type_name -> api.IPPort
1, // 8: api.DNSRecord.type:type_name -> api.DNSRecord.RecordType 16, // 8: api.UpdateMachineResponse.machine:type_name -> api.MachineInfo
2, // 9: api.Cluster.AddMachine:input_type -> api.AddMachineRequest 13, // 9: api.CreateDomainRecordsRequest.records:type_name -> api.DNSRecord
14, // 10: api.Cluster.ListMachines:input_type -> google.protobuf.Empty 13, // 10: api.CreateDomainRecordsResponse.records:type_name -> api.DNSRecord
7, // 11: api.Cluster.ReserveDomain:input_type -> api.ReserveDomainRequest 1, // 11: api.DNSRecord.type:type_name -> api.DNSRecord.RecordType
14, // 12: api.Cluster.GetDomain:input_type -> google.protobuf.Empty 2, // 12: api.Cluster.AddMachine:input_type -> api.AddMachineRequest
14, // 13: api.Cluster.ReleaseDomain:input_type -> google.protobuf.Empty 18, // 13: api.Cluster.ListMachines:input_type -> google.protobuf.Empty
8, // 14: api.Cluster.CreateDomainRecords:input_type -> api.CreateDomainRecordsRequest 6, // 14: api.Cluster.UpdateMachine:input_type -> api.UpdateMachineRequest
3, // 15: api.Cluster.AddMachine:output_type -> api.AddMachineResponse 8, // 15: api.Cluster.RemoveMachine:input_type -> api.RemoveMachineRequest
5, // 16: api.Cluster.ListMachines:output_type -> api.ListMachinesResponse 10, // 16: api.Cluster.ReserveDomain:input_type -> api.ReserveDomainRequest
6, // 17: api.Cluster.ReserveDomain:output_type -> api.Domain 18, // 17: api.Cluster.GetDomain:input_type -> google.protobuf.Empty
6, // 18: api.Cluster.GetDomain:output_type -> api.Domain 18, // 18: api.Cluster.ReleaseDomain:input_type -> google.protobuf.Empty
6, // 19: api.Cluster.ReleaseDomain:output_type -> api.Domain 11, // 19: api.Cluster.CreateDomainRecords:input_type -> api.CreateDomainRecordsRequest
9, // 20: api.Cluster.CreateDomainRecords:output_type -> api.CreateDomainRecordsResponse 3, // 20: api.Cluster.AddMachine:output_type -> api.AddMachineResponse
15, // [15:21] is the sub-list for method output_type 5, // 21: api.Cluster.ListMachines:output_type -> api.ListMachinesResponse
9, // [9:15] is the sub-list for method input_type 7, // 22: api.Cluster.UpdateMachine:output_type -> api.UpdateMachineResponse
9, // [9:9] is the sub-list for extension type_name 18, // 23: api.Cluster.RemoveMachine:output_type -> google.protobuf.Empty
9, // [9:9] is the sub-list for extension extendee 9, // 24: api.Cluster.ReserveDomain:output_type -> api.Domain
0, // [0:9] is the sub-list for field type_name 9, // 25: api.Cluster.GetDomain:output_type -> api.Domain
9, // 26: api.Cluster.ReleaseDomain:output_type -> api.Domain
12, // 27: api.Cluster.CreateDomainRecords:output_type -> api.CreateDomainRecordsResponse
20, // [20:28] is the sub-list for method output_type
12, // [12:20] is the sub-list for method input_type
12, // [12:12] is the sub-list for extension type_name
12, // [12:12] is the sub-list for extension extendee
0, // [0:12] is the sub-list for field type_name
} }
func init() { file_internal_machine_api_pb_cluster_proto_init() } func init() { file_internal_machine_api_pb_cluster_proto_init() }
@@ -802,7 +1008,7 @@ func file_internal_machine_api_pb_cluster_proto_init() {
} }
} }
file_internal_machine_api_pb_cluster_proto_msgTypes[4].Exporter = func(v any, i int) any { file_internal_machine_api_pb_cluster_proto_msgTypes[4].Exporter = func(v any, i int) any {
switch v := v.(*Domain); i { switch v := v.(*UpdateMachineRequest); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -814,7 +1020,7 @@ func file_internal_machine_api_pb_cluster_proto_init() {
} }
} }
file_internal_machine_api_pb_cluster_proto_msgTypes[5].Exporter = func(v any, i int) any { file_internal_machine_api_pb_cluster_proto_msgTypes[5].Exporter = func(v any, i int) any {
switch v := v.(*ReserveDomainRequest); i { switch v := v.(*UpdateMachineResponse); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -826,7 +1032,7 @@ func file_internal_machine_api_pb_cluster_proto_init() {
} }
} }
file_internal_machine_api_pb_cluster_proto_msgTypes[6].Exporter = func(v any, i int) any { file_internal_machine_api_pb_cluster_proto_msgTypes[6].Exporter = func(v any, i int) any {
switch v := v.(*CreateDomainRecordsRequest); i { switch v := v.(*RemoveMachineRequest); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -838,7 +1044,7 @@ func file_internal_machine_api_pb_cluster_proto_init() {
} }
} }
file_internal_machine_api_pb_cluster_proto_msgTypes[7].Exporter = func(v any, i int) any { file_internal_machine_api_pb_cluster_proto_msgTypes[7].Exporter = func(v any, i int) any {
switch v := v.(*CreateDomainRecordsResponse); i { switch v := v.(*Domain); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -850,6 +1056,42 @@ func file_internal_machine_api_pb_cluster_proto_init() {
} }
} }
file_internal_machine_api_pb_cluster_proto_msgTypes[8].Exporter = func(v any, i int) any { file_internal_machine_api_pb_cluster_proto_msgTypes[8].Exporter = func(v any, i int) any {
switch v := v.(*ReserveDomainRequest); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_internal_machine_api_pb_cluster_proto_msgTypes[9].Exporter = func(v any, i int) any {
switch v := v.(*CreateDomainRecordsRequest); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_internal_machine_api_pb_cluster_proto_msgTypes[10].Exporter = func(v any, i int) any {
switch v := v.(*CreateDomainRecordsResponse); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_internal_machine_api_pb_cluster_proto_msgTypes[11].Exporter = func(v any, i int) any {
switch v := v.(*DNSRecord); i { switch v := v.(*DNSRecord); i {
case 0: case 0:
return &v.state return &v.state
@@ -862,13 +1104,14 @@ func file_internal_machine_api_pb_cluster_proto_init() {
} }
} }
} }
file_internal_machine_api_pb_cluster_proto_msgTypes[4].OneofWrappers = []any{}
type x struct{} type x struct{}
out := protoimpl.TypeBuilder{ out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{ File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(), GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_internal_machine_api_pb_cluster_proto_rawDesc, RawDescriptor: file_internal_machine_api_pb_cluster_proto_rawDesc,
NumEnums: 2, NumEnums: 2,
NumMessages: 9, NumMessages: 12,
NumExtensions: 0, NumExtensions: 0,
NumServices: 1, NumServices: 1,
}, },
+20
View File
@@ -11,6 +11,8 @@ import "internal/machine/api/pb/machine.proto";
service Cluster { service Cluster {
rpc AddMachine(AddMachineRequest) returns (AddMachineResponse); rpc AddMachine(AddMachineRequest) returns (AddMachineResponse);
rpc ListMachines(google.protobuf.Empty) returns (ListMachinesResponse); rpc ListMachines(google.protobuf.Empty) returns (ListMachinesResponse);
rpc UpdateMachine(UpdateMachineRequest) returns (UpdateMachineResponse);
rpc RemoveMachine(RemoveMachineRequest) returns (google.protobuf.Empty);
rpc ReserveDomain(ReserveDomainRequest) returns (Domain); rpc ReserveDomain(ReserveDomainRequest) returns (Domain);
rpc GetDomain(google.protobuf.Empty) returns (Domain); rpc GetDomain(google.protobuf.Empty) returns (Domain);
@@ -49,6 +51,24 @@ message ListMachinesResponse {
repeated MachineMember machines = 1; repeated MachineMember machines = 1;
} }
message UpdateMachineRequest {
// Machine to update
string machine_id = 1;
// Updated machine information
optional string name = 2;
optional IP public_ip = 3;
repeated IPPort endpoints = 4;
}
message UpdateMachineResponse {
MachineInfo machine = 1;
}
message RemoveMachineRequest {
string id = 1;
}
message Domain { message Domain {
string name = 1; string name = 1;
} }
@@ -22,6 +22,8 @@ const _ = grpc.SupportPackageIsVersion9
const ( const (
Cluster_AddMachine_FullMethodName = "/api.Cluster/AddMachine" Cluster_AddMachine_FullMethodName = "/api.Cluster/AddMachine"
Cluster_ListMachines_FullMethodName = "/api.Cluster/ListMachines" Cluster_ListMachines_FullMethodName = "/api.Cluster/ListMachines"
Cluster_UpdateMachine_FullMethodName = "/api.Cluster/UpdateMachine"
Cluster_RemoveMachine_FullMethodName = "/api.Cluster/RemoveMachine"
Cluster_ReserveDomain_FullMethodName = "/api.Cluster/ReserveDomain" Cluster_ReserveDomain_FullMethodName = "/api.Cluster/ReserveDomain"
Cluster_GetDomain_FullMethodName = "/api.Cluster/GetDomain" Cluster_GetDomain_FullMethodName = "/api.Cluster/GetDomain"
Cluster_ReleaseDomain_FullMethodName = "/api.Cluster/ReleaseDomain" Cluster_ReleaseDomain_FullMethodName = "/api.Cluster/ReleaseDomain"
@@ -34,6 +36,8 @@ const (
type ClusterClient interface { type ClusterClient interface {
AddMachine(ctx context.Context, in *AddMachineRequest, opts ...grpc.CallOption) (*AddMachineResponse, error) AddMachine(ctx context.Context, in *AddMachineRequest, opts ...grpc.CallOption) (*AddMachineResponse, error)
ListMachines(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*ListMachinesResponse, error) ListMachines(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*ListMachinesResponse, error)
UpdateMachine(ctx context.Context, in *UpdateMachineRequest, opts ...grpc.CallOption) (*UpdateMachineResponse, error)
RemoveMachine(ctx context.Context, in *RemoveMachineRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
ReserveDomain(ctx context.Context, in *ReserveDomainRequest, opts ...grpc.CallOption) (*Domain, error) ReserveDomain(ctx context.Context, in *ReserveDomainRequest, opts ...grpc.CallOption) (*Domain, error)
GetDomain(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*Domain, error) GetDomain(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*Domain, error)
ReleaseDomain(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*Domain, error) ReleaseDomain(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*Domain, error)
@@ -68,6 +72,26 @@ func (c *clusterClient) ListMachines(ctx context.Context, in *emptypb.Empty, opt
return out, nil return out, nil
} }
func (c *clusterClient) UpdateMachine(ctx context.Context, in *UpdateMachineRequest, opts ...grpc.CallOption) (*UpdateMachineResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(UpdateMachineResponse)
err := c.cc.Invoke(ctx, Cluster_UpdateMachine_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *clusterClient) RemoveMachine(ctx context.Context, in *RemoveMachineRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(emptypb.Empty)
err := c.cc.Invoke(ctx, Cluster_RemoveMachine_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *clusterClient) ReserveDomain(ctx context.Context, in *ReserveDomainRequest, opts ...grpc.CallOption) (*Domain, error) { func (c *clusterClient) ReserveDomain(ctx context.Context, in *ReserveDomainRequest, opts ...grpc.CallOption) (*Domain, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...) cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(Domain) out := new(Domain)
@@ -114,6 +138,8 @@ func (c *clusterClient) CreateDomainRecords(ctx context.Context, in *CreateDomai
type ClusterServer interface { type ClusterServer interface {
AddMachine(context.Context, *AddMachineRequest) (*AddMachineResponse, error) AddMachine(context.Context, *AddMachineRequest) (*AddMachineResponse, error)
ListMachines(context.Context, *emptypb.Empty) (*ListMachinesResponse, error) ListMachines(context.Context, *emptypb.Empty) (*ListMachinesResponse, error)
UpdateMachine(context.Context, *UpdateMachineRequest) (*UpdateMachineResponse, error)
RemoveMachine(context.Context, *RemoveMachineRequest) (*emptypb.Empty, error)
ReserveDomain(context.Context, *ReserveDomainRequest) (*Domain, error) ReserveDomain(context.Context, *ReserveDomainRequest) (*Domain, error)
GetDomain(context.Context, *emptypb.Empty) (*Domain, error) GetDomain(context.Context, *emptypb.Empty) (*Domain, error)
ReleaseDomain(context.Context, *emptypb.Empty) (*Domain, error) ReleaseDomain(context.Context, *emptypb.Empty) (*Domain, error)
@@ -134,6 +160,12 @@ func (UnimplementedClusterServer) AddMachine(context.Context, *AddMachineRequest
func (UnimplementedClusterServer) ListMachines(context.Context, *emptypb.Empty) (*ListMachinesResponse, error) { func (UnimplementedClusterServer) ListMachines(context.Context, *emptypb.Empty) (*ListMachinesResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method ListMachines not implemented") return nil, status.Errorf(codes.Unimplemented, "method ListMachines not implemented")
} }
func (UnimplementedClusterServer) UpdateMachine(context.Context, *UpdateMachineRequest) (*UpdateMachineResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method UpdateMachine not implemented")
}
func (UnimplementedClusterServer) RemoveMachine(context.Context, *RemoveMachineRequest) (*emptypb.Empty, error) {
return nil, status.Errorf(codes.Unimplemented, "method RemoveMachine not implemented")
}
func (UnimplementedClusterServer) ReserveDomain(context.Context, *ReserveDomainRequest) (*Domain, error) { func (UnimplementedClusterServer) ReserveDomain(context.Context, *ReserveDomainRequest) (*Domain, error) {
return nil, status.Errorf(codes.Unimplemented, "method ReserveDomain not implemented") return nil, status.Errorf(codes.Unimplemented, "method ReserveDomain not implemented")
} }
@@ -203,6 +235,42 @@ func _Cluster_ListMachines_Handler(srv interface{}, ctx context.Context, dec fun
return interceptor(ctx, in, info, handler) return interceptor(ctx, in, info, handler)
} }
func _Cluster_UpdateMachine_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(UpdateMachineRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ClusterServer).UpdateMachine(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Cluster_UpdateMachine_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ClusterServer).UpdateMachine(ctx, req.(*UpdateMachineRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Cluster_RemoveMachine_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(RemoveMachineRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ClusterServer).RemoveMachine(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Cluster_RemoveMachine_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ClusterServer).RemoveMachine(ctx, req.(*RemoveMachineRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Cluster_ReserveDomain_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { func _Cluster_ReserveDomain_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ReserveDomainRequest) in := new(ReserveDomainRequest)
if err := dec(in); err != nil { if err := dec(in); err != nil {
@@ -290,6 +358,14 @@ var Cluster_ServiceDesc = grpc.ServiceDesc{
MethodName: "ListMachines", MethodName: "ListMachines",
Handler: _Cluster_ListMachines_Handler, Handler: _Cluster_ListMachines_Handler,
}, },
{
MethodName: "UpdateMachine",
Handler: _Cluster_UpdateMachine_Handler,
},
{
MethodName: "RemoveMachine",
Handler: _Cluster_RemoveMachine_Handler,
},
{ {
MethodName: "ReserveDomain", MethodName: "ReserveDomain",
Handler: _Cluster_ReserveDomain_Handler, Handler: _Cluster_ReserveDomain_Handler,
+1 -1
View File
@@ -14,7 +14,7 @@ service Machine {
rpc JoinCluster(JoinClusterRequest) returns (google.protobuf.Empty); rpc JoinCluster(JoinClusterRequest) returns (google.protobuf.Empty);
rpc Token(google.protobuf.Empty) returns (TokenResponse); rpc Token(google.protobuf.Empty) returns (TokenResponse);
rpc Inspect(google.protobuf.Empty) returns (MachineInfo); rpc Inspect(google.protobuf.Empty) returns (MachineInfo);
// Reset restores the machine to a clean state, removing all cluster-related сonfiguration and data. // Reset restores the machine to a clean state, removing all cluster-related configuration and data.
rpc Reset(ResetRequest) returns (google.protobuf.Empty); rpc Reset(ResetRequest) returns (google.protobuf.Empty);
rpc InspectService(InspectServiceRequest) returns (InspectServiceResponse); rpc InspectService(InspectServiceRequest) returns (InspectServiceResponse);
+2 -2
View File
@@ -39,7 +39,7 @@ type MachineClient interface {
JoinCluster(ctx context.Context, in *JoinClusterRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) JoinCluster(ctx context.Context, in *JoinClusterRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
Token(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*TokenResponse, error) Token(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*TokenResponse, error)
Inspect(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*MachineInfo, error) Inspect(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*MachineInfo, error)
// Reset restores the machine to a clean state, removing all cluster-related сonfiguration and data. // Reset restores the machine to a clean state, removing all cluster-related configuration and data.
Reset(ctx context.Context, in *ResetRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) Reset(ctx context.Context, in *ResetRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
InspectService(ctx context.Context, in *InspectServiceRequest, opts ...grpc.CallOption) (*InspectServiceResponse, error) InspectService(ctx context.Context, in *InspectServiceRequest, opts ...grpc.CallOption) (*InspectServiceResponse, error)
} }
@@ -132,7 +132,7 @@ type MachineServer interface {
JoinCluster(context.Context, *JoinClusterRequest) (*emptypb.Empty, error) JoinCluster(context.Context, *JoinClusterRequest) (*emptypb.Empty, error)
Token(context.Context, *emptypb.Empty) (*TokenResponse, error) Token(context.Context, *emptypb.Empty) (*TokenResponse, error)
Inspect(context.Context, *emptypb.Empty) (*MachineInfo, error) Inspect(context.Context, *emptypb.Empty) (*MachineInfo, error)
// Reset restores the machine to a clean state, removing all cluster-related сonfiguration and data. // Reset restores the machine to a clean state, removing all cluster-related configuration and data.
Reset(context.Context, *ResetRequest) (*emptypb.Empty, error) Reset(context.Context, *ResetRequest) (*emptypb.Empty, error)
InspectService(context.Context, *InspectServiceRequest) (*InspectServiceResponse, error) InspectService(context.Context, *InspectServiceRequest) (*InspectServiceResponse, error)
mustEmbedUnimplementedMachineServer() mustEmbedUnimplementedMachineServer()
+2 -1
View File
@@ -2,10 +2,11 @@ package proxy
import ( import (
"fmt" "fmt"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"google.golang.org/grpc/status" "google.golang.org/grpc/status"
"google.golang.org/protobuf/encoding/protowire" "google.golang.org/protobuf/encoding/protowire"
"google.golang.org/protobuf/proto" "google.golang.org/protobuf/proto"
"github.com/psviderski/uncloud/internal/machine/api/pb"
) )
// One2ManyResponder converts upstream responses into messages from upstreams, so that multiple // One2ManyResponder converts upstream responses into messages from upstreams, so that multiple
+2 -1
View File
@@ -2,11 +2,12 @@ package proxy
import ( import (
"context" "context"
"sync"
"github.com/siderolabs/grpc-proxy/proxy" "github.com/siderolabs/grpc-proxy/proxy"
"google.golang.org/grpc/codes" "google.golang.org/grpc/codes"
"google.golang.org/grpc/metadata" "google.golang.org/grpc/metadata"
"google.golang.org/grpc/status" "google.golang.org/grpc/status"
"sync"
) )
// Director manages routing of gRPC requests between local and remote backends. // Director manages routing of gRPC requests between local and remote backends.
+2 -1
View File
@@ -2,11 +2,12 @@ package proxy
import ( import (
"context" "context"
"sync"
"github.com/siderolabs/grpc-proxy/proxy" "github.com/siderolabs/grpc-proxy/proxy"
"google.golang.org/grpc" "google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure" "google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/metadata" "google.golang.org/grpc/metadata"
"sync"
) )
// LocalBackend is a proxy.One2ManyResponder implementation that proxies to a local gRPC server listening on a Unix socket. // LocalBackend is a proxy.One2ManyResponder implementation that proxies to a local gRPC server listening on a Unix socket.
+4 -3
View File
@@ -3,14 +3,15 @@ package proxy
import ( import (
"context" "context"
"fmt" "fmt"
"net/netip"
"sync"
"time"
"github.com/siderolabs/grpc-proxy/proxy" "github.com/siderolabs/grpc-proxy/proxy"
"google.golang.org/grpc" "google.golang.org/grpc"
"google.golang.org/grpc/backoff" "google.golang.org/grpc/backoff"
"google.golang.org/grpc/credentials/insecure" "google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/metadata" "google.golang.org/grpc/metadata"
"net/netip"
"sync"
"time"
) )
// RemoteBackend is a proxy.One2ManyResponder implementation that proxies to a remote gRPC server, injecting machine metadata // RemoteBackend is a proxy.One2ManyResponder implementation that proxies to a remote gRPC server, injecting machine metadata
+17
View File
@@ -0,0 +1,17 @@
package caddyconfig
import (
"fmt"
"github.com/psviderski/uncloud/pkg/api"
)
func GenerateCaddyfile(containers []api.ServiceContainer, verifyResponse string) (string, error) {
return fmt.Sprintf(`http:// {
handle %s {
respond "%s" 200
}
log
}
`, VerifyPath, verifyResponse), nil
}
+51 -27
View File
@@ -23,23 +23,24 @@ const (
// network. // network.
type Controller struct { type Controller struct {
store *store.Store store *store.Store
path string configDir string
verifyResponse string verifyResponse string
log *slog.Logger
} }
func NewController(store *store.Store, path string, verifyResponse string) (*Controller, error) { func NewController(store *store.Store, configDir string, verifyResponse string) (*Controller, error) {
dir := filepath.Dir(path) if err := os.MkdirAll(configDir, 0o750); err != nil {
if err := os.MkdirAll(dir, 0750); err != nil { return nil, fmt.Errorf("create directory for Caddy configuration '%s': %w", configDir, err)
return nil, fmt.Errorf("create parent directory for Caddy configuration '%s': %w", dir, err)
} }
if err := fs.Chown(dir, "", CaddyGroup); err != nil { if err := fs.Chown(configDir, "", CaddyGroup); err != nil {
return nil, fmt.Errorf("change owner of parent directory for Caddy configuration '%s': %w", dir, err) return nil, fmt.Errorf("change owner of directory for Caddy configuration '%s': %w", configDir, err)
} }
return &Controller{ return &Controller{
store: store, store: store,
path: path, configDir: configDir,
verifyResponse: verifyResponse, verifyResponse: verifyResponse,
log: slog.With("component", "caddy-controller"),
}, nil }, nil
} }
@@ -48,14 +49,18 @@ func (c *Controller) Run(ctx context.Context) error {
if err != nil { if err != nil {
return fmt.Errorf("subscribe to container changes: %w", err) return fmt.Errorf("subscribe to container changes: %w", err)
} }
slog.Info("Subscribed to container changes in the cluster to generate Caddy configuration.") c.log.Info("Subscribed to container changes in the cluster to generate Caddy configuration.")
containers, err := c.filterAvailableContainers(containerRecords) containers, err := c.filterAvailableContainers(containerRecords)
if err != nil { if err != nil {
return fmt.Errorf("filter available containers: %w", err) return fmt.Errorf("filter available containers: %w", err)
} }
if err = c.generateConfig(containers); err != nil {
return fmt.Errorf("generate Caddy configuration: %w", err) if err = c.generateCaddyfile(containers); err != nil {
return fmt.Errorf("generate Caddyfile configuration: %w", err)
}
if err = c.generateJSONConfig(containers); err != nil {
return fmt.Errorf("generate Caddy JSON configuration: %w", err)
} }
for { for {
@@ -64,23 +69,27 @@ func (c *Controller) Run(ctx context.Context) error {
if !ok { if !ok {
return fmt.Errorf("containers subscription failed") return fmt.Errorf("containers subscription failed")
} }
slog.Debug("Cluster containers changed, updating Caddy configuration.") c.log.Info("Cluster containers changed, updating Caddy configuration.")
containerRecords, err = c.store.ListContainers(ctx, store.ListOptions{}) containerRecords, err = c.store.ListContainers(ctx, store.ListOptions{})
if err != nil { if err != nil {
slog.Error("Failed to list containers.", "err", err) c.log.Info("Failed to list containers.", "err", err)
continue continue
} }
containers, err = c.filterAvailableContainers(containerRecords) containers, err = c.filterAvailableContainers(containerRecords)
if err != nil { if err != nil {
slog.Error("Failed to filter available containers.", "err", err) c.log.Info("Failed to filter available containers.", "err", err)
continue continue
} }
if err = c.generateConfig(containers); err != nil {
slog.Error("Failed to generate Caddy configuration.", "err", err) if err = c.generateCaddyfile(containers); err != nil {
c.log.Info("Failed to generate Caddyfile configuration.", "err", err)
}
if err = c.generateJSONConfig(containers); err != nil {
c.log.Info("Failed to generate Caddy JSON configuration.", "err", err)
} }
slog.Debug("Updated Caddy configuration.", "path", c.path) c.log.Info("Updated Caddy configuration.", "dir", c.configDir)
case <-ctx.Done(): case <-ctx.Done():
return nil return nil
} }
@@ -95,16 +104,30 @@ func (c *Controller) filterAvailableContainers(
) ([]api.ServiceContainer, error) { ) ([]api.ServiceContainer, error) {
containers := make([]api.ServiceContainer, len(containerRecords)) containers := make([]api.ServiceContainer, len(containerRecords))
for i, cr := range containerRecords { for i, cr := range containerRecords {
containers[i] = api.ServiceContainer{ containers[i] = cr.Container
Container: cr.Container,
// TODO: restore ServiceSpec from the container record once it's saved in the store.
}
} }
return containers, nil return containers, nil
} }
func (c *Controller) generateConfig(containers []api.ServiceContainer) error { func (c *Controller) generateCaddyfile(containers []api.ServiceContainer) error {
config, err := GenerateConfig(containers, c.verifyResponse) caddyfile, err := GenerateCaddyfile(containers, c.verifyResponse)
if err != nil {
return fmt.Errorf("generate Caddyfile: %w", err)
}
caddyfilePath := filepath.Join(c.configDir, "Caddyfile")
if err = os.WriteFile(caddyfilePath, []byte(caddyfile), 0o640); err != nil {
return fmt.Errorf("write Caddyfile to file '%s': %w", caddyfilePath, err)
}
if err = fs.Chown(caddyfilePath, "", CaddyGroup); err != nil {
return fmt.Errorf("change owner of Caddyfile '%s': %w", caddyfilePath, err)
}
return nil
}
func (c *Controller) generateJSONConfig(containers []api.ServiceContainer) error {
config, err := GenerateJSONConfig(containers, c.verifyResponse)
if err != nil { if err != nil {
return err return err
} }
@@ -113,12 +136,13 @@ func (c *Controller) generateConfig(containers []api.ServiceContainer) error {
if err != nil { if err != nil {
return fmt.Errorf("marshal Caddy configuration: %w", err) return fmt.Errorf("marshal Caddy configuration: %w", err)
} }
configPath := filepath.Join(c.configDir, "caddy.json")
if err = os.WriteFile(c.path, configBytes, 0640); err != nil { if err = os.WriteFile(configPath, configBytes, 0o640); err != nil {
return fmt.Errorf("write Caddy configuration to file '%s': %w", c.path, err) return fmt.Errorf("write Caddy configuration to file '%s': %w", configPath, err)
} }
if err = fs.Chown(c.path, "", CaddyGroup); err != nil { if err = fs.Chown(configPath, "", CaddyGroup); err != nil {
return fmt.Errorf("change owner of Caddy configuration file '%s': %w", c.path, err) return fmt.Errorf("change owner of Caddy configuration file '%s': %w", configPath, err)
} }
return nil return nil
@@ -19,7 +19,7 @@ import (
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
) )
func GenerateConfig(containers []api.ServiceContainer, verifyResponse string) (*caddy.Config, error) { func GenerateJSONConfig(containers []api.ServiceContainer, verifyResponse string) (*caddy.Config, error) {
// Maps hostnames to lists of upstreams (container IP:port pairs). // Maps hostnames to lists of upstreams (container IP:port pairs).
httpHostUpstreams := make(map[string][]string) httpHostUpstreams := make(map[string][]string)
httpsHostUpstreams := make(map[string][]string) httpsHostUpstreams := make(map[string][]string)
@@ -378,7 +378,7 @@ func TestGenerateConfig(t *testing.T) {
for _, tt := range tests { for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
config, err := GenerateConfig(tt.containers, "verification-response-body") config, err := GenerateJSONConfig(tt.containers, "verification-response-body")
if tt.wantErr { if tt.wantErr {
assert.Error(t, err) assert.Error(t, err)
@@ -12,9 +12,9 @@ import (
"time" "time"
"github.com/cenkalti/backoff/v4" "github.com/cenkalti/backoff/v4"
"github.com/docker/docker/client"
"github.com/psviderski/uncloud/internal/machine/api/pb" "github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/machine/caddyconfig" "github.com/psviderski/uncloud/internal/machine/caddyconfig"
"github.com/psviderski/uncloud/internal/machine/constants"
"github.com/psviderski/uncloud/internal/machine/corroservice" "github.com/psviderski/uncloud/internal/machine/corroservice"
"github.com/psviderski/uncloud/internal/machine/dns" "github.com/psviderski/uncloud/internal/machine/dns"
"github.com/psviderski/uncloud/internal/machine/docker" "github.com/psviderski/uncloud/internal/machine/docker"
@@ -25,39 +25,42 @@ import (
"google.golang.org/grpc" "google.golang.org/grpc"
) )
const ( // clusterController is the main controller for the machine that is a cluster member. It manages components such as
APIPort = 51000 // the WireGuard network, API server listening the WireGuard network, Corrosion service, Docker network and containers,
) // and others.
type clusterController struct {
type networkController struct {
state *State state *State
store *store.Store store *store.Store
wgnet *network.WireGuardNetwork wgnet *network.WireGuardNetwork
endpointChanges <-chan network.EndpointChangeEvent endpointChanges <-chan network.EndpointChangeEvent
server *grpc.Server server *grpc.Server
corroService corroservice.Service corroService corroservice.Service
dockerCli *client.Client dockerCtrl *docker.Controller
caddyfileCtrl *caddyconfig.Controller // dockerReady is signalled when Docker is configured and ready for containers.
dockerReady chan<- struct{}
caddyconfigCtrl *caddyconfig.Controller
// dnsServer is the embedded internal DNS server for the cluster listening on the machine IP. // dnsServer is the embedded internal DNS server for the cluster listening on the machine IP.
dnsServer *dns.Server dnsServer *dns.Server
dnsResolver *dns.ClusterResolver dnsResolver *dns.ClusterResolver
// stopped is a channel that is closed when the controller is stopped.
stopped chan struct{}
} }
func newNetworkController( func newClusterController(
state *State, state *State,
store *store.Store, store *store.Store,
server *grpc.Server, server *grpc.Server,
corroService corroservice.Service, corroService corroservice.Service,
dockerCli *client.Client, dockerService *docker.Service,
dockerReady chan<- struct{},
caddyfileCtrl *caddyconfig.Controller, caddyfileCtrl *caddyconfig.Controller,
dnsServer *dns.Server, dnsServer *dns.Server,
dnsResolver *dns.ClusterResolver, dnsResolver *dns.ClusterResolver,
) ( ) (*clusterController, error) {
*networkController, error,
) {
slog.Info("Starting WireGuard network.") slog.Info("Starting WireGuard network.")
wgnet, err := network.NewWireGuardNetwork() wgnet, err := network.NewWireGuardNetwork()
if err != nil { if err != nil {
@@ -65,88 +68,97 @@ func newNetworkController(
} }
endpointChanges := wgnet.WatchEndpoints() endpointChanges := wgnet.WatchEndpoints()
return &networkController{ return &clusterController{
state: state, state: state,
store: store, store: store,
wgnet: wgnet, wgnet: wgnet,
endpointChanges: endpointChanges, endpointChanges: endpointChanges,
server: server, server: server,
corroService: corroService, corroService: corroService,
dockerCli: dockerCli, dockerCtrl: docker.NewController(state.ID, dockerService, store),
caddyfileCtrl: caddyfileCtrl, dockerReady: dockerReady,
caddyconfigCtrl: caddyfileCtrl,
dnsServer: dnsServer, dnsServer: dnsServer,
dnsResolver: dnsResolver, dnsResolver: dnsResolver,
stopped: make(chan struct{}),
}, nil }, nil
} }
func (nc *networkController) Run(ctx context.Context) error { func (cc *clusterController) Run(ctx context.Context) error {
defer close(cc.stopped)
if err := firewall.ConfigureIptablesChains(); err != nil { if err := firewall.ConfigureIptablesChains(); err != nil {
return fmt.Errorf("configure iptables chains: %w", err) return fmt.Errorf("configure iptables chains: %w", err)
} }
if err := nc.wgnet.Configure(*nc.state.Network); err != nil { if err := cc.ensureDockerNetwork(ctx); err != nil {
return err
}
slog.Info("Docker network configured.")
if err := cc.wgnet.Configure(*cc.state.Network); err != nil {
return fmt.Errorf("configure WireGuard network: %w", err) return fmt.Errorf("configure WireGuard network: %w", err)
} }
slog.Info("WireGuard network configured.") slog.Info("WireGuard network configured.")
if nc.corroService.Running() { if cc.corroService.Running() {
// Corrosion service was running before the WireGuard network was configured so we need to restart it. // Corrosion service was running before the WireGuard network was configured so we need to restart it.
slog.Info("Restarting corrosion service to apply new configuration with WireGuard network.") slog.Info("Restarting corrosion service to apply new configuration with WireGuard network.")
if err := nc.corroService.Restart(ctx); err != nil { if err := cc.corroService.Restart(ctx); err != nil {
return fmt.Errorf("restart corrosion service: %w", err) return fmt.Errorf("restart corrosion service: %w", err)
} }
slog.Info("Corrosion service restarted.")
} else { } else {
slog.Info("Starting corrosion service.") slog.Info("Starting corrosion service.")
if err := nc.corroService.Start(ctx); err != nil { if err := cc.corroService.Start(ctx); err != nil {
return fmt.Errorf("start corrosion service: %w", err) return fmt.Errorf("start corrosion service: %w", err)
} }
slog.Info("Corrosion service started.")
} }
// TODO: Figure out if we need to manually stop the corrosion service when the context is done or just
// rely on systemd to handle service dependencies on its own.
errGroup, ctx := errgroup.WithContext(ctx) errGroup, ctx := errgroup.WithContext(ctx)
// Start the network API server. Assume the management IP can't be changed when the network is running. // Start the network API server. Assume the management IP can't be changed when the network is running.
apiAddr := net.JoinHostPort(nc.state.Network.ManagementIP.String(), strconv.Itoa(APIPort)) apiAddr := net.JoinHostPort(cc.state.Network.ManagementIP.String(), strconv.Itoa(constants.MachineAPIPort))
listener, err := net.Listen("tcp", apiAddr) listener, err := net.Listen("tcp", apiAddr)
if err != nil { if err != nil {
return fmt.Errorf("listen API port: %w", err) return fmt.Errorf("listen API port: %w", err)
} }
errGroup.Go( errGroup.Go(func() error {
func() error { slog.Info("Starting network API server.", "addr", apiAddr)
slog.Info("Starting network API server.", "addr", apiAddr) if err := cc.server.Serve(listener); err != nil {
if err := nc.server.Serve(listener); err != nil { return fmt.Errorf("network API server failed: %w", err)
return fmt.Errorf("network API server failed: %w", err) }
} return nil
return nil })
},
)
errGroup.Go(func() error { errGroup.Go(func() error {
slog.Info("Starting embedded DNS resolver.") slog.Info("Starting embedded DNS resolver.")
if err := nc.dnsResolver.Run(ctx); err != nil { if err := cc.dnsResolver.Run(ctx); err != nil {
return fmt.Errorf("embedded DNS resolver failed: %w", err) return fmt.Errorf("embedded DNS resolver failed: %w", err)
} }
return nil return nil
}) })
// The Docker network must be created before starting the DNS server because it listens on the machine IP.
errGroup.Go(func() error { errGroup.Go(func() error {
slog.Info("Starting embedded DNS server.") slog.Info("Starting embedded DNS server.")
if err := nc.dnsServer.Run(ctx); err != nil { if err := cc.dnsServer.Run(ctx); err != nil {
return fmt.Errorf("embedded DNS server failed: %w", err) return fmt.Errorf("embedded DNS server failed: %w", err)
} }
return nil return nil
}) })
// Setup Docker network and synchronise containers to the cluster store. // Synchronise Docker containers to the cluster store.
errGroup.Go(func() error { errGroup.Go(func() error {
return nc.prepareAndWatchDocker(ctx) slog.Info("Watching Docker containers and syncing them to cluster store.")
return cc.syncDockerContainers(ctx)
}) })
// Handle machine changes in the cluster. Handling machine and endpoint changes should be done // Handle machine changes in the cluster. Handling machine and endpoint changes should be done
// in separate goroutines to avoid a deadlock when reconfiguring the network. // in separate goroutines to avoid a deadlock when reconfiguring the network.
errGroup.Go(func() error { errGroup.Go(func() error {
if err := nc.handleMachineChanges(ctx); err != nil { if err := cc.handleMachineChanges(ctx); err != nil {
return fmt.Errorf("handle new machines: %w", err) return fmt.Errorf("handle new machines: %w", err)
} }
return nil return nil
@@ -156,24 +168,24 @@ func (nc *networkController) Run(ctx context.Context) error {
errGroup.Go(func() error { errGroup.Go(func() error {
for { for {
select { select {
case e, ok := <-nc.endpointChanges: case e, ok := <-cc.endpointChanges:
if !ok { if !ok {
// The channel was closed, stop watching for changes. // The channel was closed, stop watching for changes.
nc.endpointChanges = nil cc.endpointChanges = nil
return nil return nil
} }
nc.state.mu.Lock() cc.state.mu.Lock()
for i := range nc.state.Network.Peers { for i := range cc.state.Network.Peers {
if nc.state.Network.Peers[i].PublicKey.Equal(e.PublicKey) { if cc.state.Network.Peers[i].PublicKey.Equal(e.PublicKey) {
nc.state.Network.Peers[i].Endpoint = &e.Endpoint cc.state.Network.Peers[i].Endpoint = &e.Endpoint
break break
} }
} }
if err := nc.state.Save(); err != nil { if err := cc.state.Save(); err != nil {
slog.Error("Failed to save machine state.", "err", err) slog.Error("Failed to save machine state.", "err", err)
} }
nc.state.mu.Unlock() cc.state.mu.Unlock()
slog.Debug("Preserved endpoint change in the machine state.", slog.Debug("Preserved endpoint change in the machine state.",
"public_key", e.PublicKey, "endpoint", e.Endpoint) "public_key", e.PublicKey, "endpoint", e.Endpoint)
@@ -184,48 +196,66 @@ func (nc *networkController) Run(ctx context.Context) error {
}) })
errGroup.Go(func() error { errGroup.Go(func() error {
if err := nc.wgnet.Run(ctx); err != nil { if err := cc.wgnet.Run(ctx); err != nil {
return fmt.Errorf("WireGuard network failed: %w", err) return fmt.Errorf("WireGuard network failed: %w", err)
} }
return nil return nil
}) })
errGroup.Go(func() error { errGroup.Go(func() error {
slog.Info("Starting Caddyconfig controller.") slog.Info("Starting caddyconfig controller.")
if err := nc.caddyfileCtrl.Run(ctx); err != nil { if err := cc.caddyconfigCtrl.Run(ctx); err != nil {
//goland:noinspection GoErrorStringFormat return fmt.Errorf("caddyconfig controller failed: %w", err)
return fmt.Errorf("Caddyconfig controller failed: %w", err)
} }
return nil return nil
}) })
// Wait for the context to be done and stop the network API server. // Wait for the context to be done and stop the network API server.
errGroup.Go(func() error { <-ctx.Done()
<-ctx.Done() slog.Info("Stopping network API server.")
slog.Info("Stopping network API server.") // TODO: implement timeout for graceful shutdown.
// TODO: implement timeout for graceful shutdown. cc.server.GracefulStop()
nc.server.GracefulStop() slog.Info("Network API server stopped.")
slog.Info("Network API server stopped.")
return nil
})
return errGroup.Wait() // Wait for all controllers to finish.
err = errGroup.Wait()
// It's safe to stop the Corrosion service after the controllers depending on it and API server are stopped.
// Use a new context with a timeout as the current context is already canceled.
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if corroErr := cc.corroService.Stop(ctx); corroErr != nil {
err = errors.Join(err, fmt.Errorf("stop corrosion service: %w", corroErr))
} else {
slog.Info("Corrosion service stopped.")
}
return err
} }
// prepareAndWatchDocker configures the Docker network and watches local Docker containers to sync them // ensureDockerNetwork ensures that the Docker network is configured and ready for containers.
// to the cluster store. func (cc *clusterController) ensureDockerNetwork(ctx context.Context) error {
func (nc *networkController) prepareAndWatchDocker(ctx context.Context) error { if err := cc.dockerCtrl.WaitDaemonReady(ctx); err != nil {
manager := docker.NewManager(nc.dockerCli, nc.state.ID, nc.store)
if err := manager.WaitDaemonReady(ctx); err != nil {
return fmt.Errorf("wait for Docker daemon: %w", err) return fmt.Errorf("wait for Docker daemon: %w", err)
} }
if err := manager.EnsureUncloudNetwork(ctx, nc.state.Network.Subnet, nc.dnsServer.ListenAddr()); err != nil { if err := cc.dockerCtrl.EnsureUncloudNetwork(
ctx,
cc.state.Network.Subnet,
cc.dnsServer.ListenAddr(),
); err != nil {
return fmt.Errorf("ensure Docker network: %w", err) return fmt.Errorf("ensure Docker network: %w", err)
} }
slog.Info("Docker network configured.")
slog.Info("Watching Docker containers and syncing them to cluster store.") // Signal that Docker is ready for containers.
close(cc.dockerReady)
return nil
}
// syncDockerContainers watches local Docker containers and syncs them to the cluster store.
// TODO: move this to the Docker controller.
func (cc *clusterController) syncDockerContainers(ctx context.Context) error {
// Retry to watch and sync containers until the context is done. // Retry to watch and sync containers until the context is done.
boff := backoff.WithContext(backoff.NewExponentialBackOff( boff := backoff.WithContext(backoff.NewExponentialBackOff(
backoff.WithInitialInterval(100*time.Millisecond), backoff.WithInitialInterval(100*time.Millisecond),
@@ -233,7 +263,7 @@ func (nc *networkController) prepareAndWatchDocker(ctx context.Context) error {
backoff.WithMaxElapsedTime(0), backoff.WithMaxElapsedTime(0),
), ctx) ), ctx)
watchAndSync := func() error { watchAndSync := func() error {
if wErr := manager.WatchAndSyncContainers(ctx); wErr != nil { if wErr := cc.dockerCtrl.WatchAndSyncContainers(ctx); wErr != nil {
slog.Error("Failed to watch and sync containers to cluster store, retrying.", "err", wErr) slog.Error("Failed to watch and sync containers to cluster store, retrying.", "err", wErr)
return wErr return wErr
} }
@@ -251,7 +281,7 @@ func (nc *networkController) prepareAndWatchDocker(ctx context.Context) error {
// handleMachineChanges subscribes to machine changes in the cluster and reconfigures the network peers accordingly // handleMachineChanges subscribes to machine changes in the cluster and reconfigures the network peers accordingly
// when changes occur. // when changes occur.
func (nc *networkController) handleMachineChanges(ctx context.Context) error { func (cc *clusterController) handleMachineChanges(ctx context.Context) error {
for { for {
// Retry to subscribe to machine changes indefinitely until the context is done. // Retry to subscribe to machine changes indefinitely until the context is done.
boff := backoff.WithContext(backoff.NewExponentialBackOff( boff := backoff.WithContext(backoff.NewExponentialBackOff(
@@ -266,7 +296,7 @@ func (nc *networkController) handleMachineChanges(ctx context.Context) error {
err error err error
) )
subscribe := func() error { subscribe := func() error {
if machines, changes, err = nc.store.SubscribeMachines(ctx); err != nil { if machines, changes, err = cc.store.SubscribeMachines(ctx); err != nil {
slog.Info("Failed to subscribe to machine changes, retrying.", "err", err) slog.Info("Failed to subscribe to machine changes, retrying.", "err", err)
} }
return err return err
@@ -284,7 +314,7 @@ func (nc *networkController) handleMachineChanges(ctx context.Context) error {
// completes. Skip configuration now and apply it when the store changes are received. // completes. Skip configuration now and apply it when the store changes are received.
if len(machines) > 0 { if len(machines) > 0 {
slog.Info("Reconfiguring network peers with the current machines.", "machines", len(machines)) slog.Info("Reconfiguring network peers with the current machines.", "machines", len(machines))
if err = nc.configurePeers(machines); err != nil { if err = cc.configurePeers(machines); err != nil {
slog.Error("Failed to configure peers.", "err", err) slog.Error("Failed to configure peers.", "err", err)
} }
} }
@@ -296,11 +326,11 @@ func (nc *networkController) handleMachineChanges(ctx context.Context) error {
// be reworked as well. // be reworked as well.
case <-changes: case <-changes:
slog.Info("Cluster machines changed, reconfiguring network peers.") slog.Info("Cluster machines changed, reconfiguring network peers.")
if machines, err = nc.store.ListMachines(ctx); err != nil { if machines, err = cc.store.ListMachines(ctx); err != nil {
slog.Error("Failed to list machines.", "err", err) slog.Error("Failed to list machines.", "err", err)
continue continue
} }
if err = nc.configurePeers(machines); err != nil { if err = cc.configurePeers(machines); err != nil {
slog.Error("Failed to configure peers.", "err", err) slog.Error("Failed to configure peers.", "err", err)
} }
case <-ctx.Done(): case <-ctx.Done():
@@ -310,23 +340,23 @@ func (nc *networkController) handleMachineChanges(ctx context.Context) error {
} }
} }
func (nc *networkController) configurePeers(machines []*pb.MachineInfo) error { func (cc *clusterController) configurePeers(machines []*pb.MachineInfo) error {
if len(machines) == 0 { if len(machines) == 0 {
return fmt.Errorf("no machines to configure peers") return fmt.Errorf("no machines to configure peers")
} }
nc.state.mu.RLock() cc.state.mu.RLock()
currentPeerEndpoints := make(map[string]*netip.AddrPort, len(nc.state.Network.Peers)) currentPeerEndpoints := make(map[string]*netip.AddrPort, len(cc.state.Network.Peers))
for _, p := range nc.state.Network.Peers { for _, p := range cc.state.Network.Peers {
currentPeerEndpoints[p.PublicKey.String()] = p.Endpoint currentPeerEndpoints[p.PublicKey.String()] = p.Endpoint
} }
nc.state.mu.RUnlock() cc.state.mu.RUnlock()
// Construct the list of peers from the machine configurations ensuring that the current endpoint is preserved. // Construct the list of peers from the machine configurations ensuring that the current endpoint is preserved.
peers := make([]network.PeerConfig, 0, len(machines)-1) peers := make([]network.PeerConfig, 0, len(machines)-1)
for _, m := range machines { for _, m := range machines {
// Skip the current machine. // Skip the current machine.
if m.Id == nc.state.ID { if m.Id == cc.state.ID {
continue continue
} }
if err := m.Network.Validate(); err != nil { if err := m.Network.Validate(); err != nil {
@@ -359,20 +389,37 @@ func (nc *networkController) configurePeers(machines []*pb.MachineInfo) error {
} }
// Preserve the new list of peers in the machine state. // Preserve the new list of peers in the machine state.
nc.state.mu.Lock() cc.state.mu.Lock()
nc.state.Network.Peers = peers cc.state.Network.Peers = peers
err := nc.state.Save() err := cc.state.Save()
nc.state.mu.Unlock() cc.state.mu.Unlock()
if err != nil { if err != nil {
return fmt.Errorf("save machine state: %w", err) return fmt.Errorf("save machine state: %w", err)
} }
nc.state.mu.RLock() cc.state.mu.RLock()
defer nc.state.mu.RUnlock() defer cc.state.mu.RUnlock()
if err = nc.wgnet.Configure(*nc.state.Network); err != nil { if err = cc.wgnet.Configure(*cc.state.Network); err != nil {
return fmt.Errorf("configure network peers: %w", err) return fmt.Errorf("configure network peers: %w", err)
} }
return nil return nil
} }
// TODO: method to shutdown network when leaving a cluster. Regular context cancellation shouldn't bring it down. // Cleanup cleans up the cluster resources such as the WireGuard network, iptables rules, Docker network and containers.
func (cc *clusterController) Cleanup() error {
// Wait for the controller to stop before cleaning up.
<-cc.stopped
var errs []error
if err := cc.dockerCtrl.Cleanup(); err != nil {
errs = append(errs, fmt.Errorf("cleanup Docker resources: %w", err))
}
if err := cc.wgnet.Cleanup(); err != nil {
errs = append(errs, fmt.Errorf("cleanup WireGuard network: %w", err))
}
if err := firewall.CleanupIptablesChains(); err != nil {
errs = append(errs, fmt.Errorf("cleanup iptables chains: %w", err))
}
return errors.Join(errs...)
}
+108 -3
View File
@@ -5,17 +5,18 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb"
"log/slog" "log/slog"
"net/netip" "net/netip"
"time" "time"
"github.com/psviderski/uncloud/internal/corrosion" "github.com/psviderski/uncloud/internal/corrosion"
"github.com/psviderski/uncloud/internal/machine/api/pb" "github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/machine/network" "github.com/psviderski/uncloud/internal/machine/network"
"github.com/psviderski/uncloud/internal/machine/store" "github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/uncloud/internal/secret" "github.com/psviderski/uncloud/internal/secret"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb"
) )
type Cluster struct { type Cluster struct {
@@ -197,6 +198,89 @@ func (c *Cluster) AddMachine(ctx context.Context, req *pb.AddMachineRequest) (*p
return resp, nil return resp, nil
} }
// UpdateMachine updates machine configuration in the cluster.
func (c *Cluster) UpdateMachine(ctx context.Context, req *pb.UpdateMachineRequest) (*pb.UpdateMachineResponse, error) {
if err := c.checkInitialised(ctx); err != nil {
return nil, err
}
if req.MachineId == "" {
return nil, status.Error(codes.InvalidArgument, "machine_id not set")
}
// Get the current machine info
currentMachine, err := c.store.GetMachine(ctx, req.MachineId)
if err != nil {
if errors.Is(err, store.ErrMachineNotFound) {
return nil, status.Errorf(codes.NotFound, "machine not found: %s", req.MachineId)
}
return nil, status.Errorf(codes.Internal, "failed to get machine: %v", err)
}
// Create a copy of the current machine for updating
updatedMachine := &pb.MachineInfo{
Id: currentMachine.Id,
Name: currentMachine.Name,
Network: currentMachine.Network,
PublicIp: currentMachine.PublicIp,
}
// Apply updates from the request
if req.Name != nil {
// Check for empty name
if *req.Name == "" {
return nil, status.Error(codes.InvalidArgument, "machine name cannot be empty")
}
// Check for duplicate names (excluding the current machine)
if *req.Name != currentMachine.Name {
machines, err := c.store.ListMachines(ctx)
if err != nil {
return nil, status.Errorf(codes.Internal, "list machines: %v", err)
}
for _, m := range machines {
if m.Id != req.MachineId && m.Name == *req.Name {
return nil, status.Errorf(codes.AlreadyExists, "machine with name %q already exists", *req.Name)
}
}
}
updatedMachine.Name = *req.Name
}
if req.PublicIp != nil {
// Check if this is an empty IP (used to signal removal)
if len(req.PublicIp.Ip) == 0 {
// User wants to remove public IP
updatedMachine.PublicIp = nil
} else {
// Validate and set the new IP
ip, err := req.PublicIp.ToAddr()
if err != nil {
return nil, status.Errorf(codes.InvalidArgument, "invalid public IP: %v", err)
}
if !ip.IsValid() {
return nil, status.Error(codes.InvalidArgument, "invalid public IP")
}
updatedMachine.PublicIp = req.PublicIp
}
}
if req.Endpoints != nil {
updatedMachine.Network.Endpoints = req.Endpoints
}
// Update the machine in the store
if err = c.store.UpdateMachine(ctx, updatedMachine); err != nil {
if errors.Is(err, store.ErrMachineNotFound) {
return nil, status.Errorf(codes.NotFound, "machine not found: %s", req.MachineId)
}
return nil, status.Errorf(codes.Internal, "update machine: %v", err)
}
slog.Info("Machine configuration updated in the cluster.",
"id", updatedMachine.Id, "name", updatedMachine.Name)
resp := &pb.UpdateMachineResponse{Machine: updatedMachine}
return resp, nil
}
// ListMachines lists all machines in the cluster including their membership states. // ListMachines lists all machines in the cluster including their membership states.
func (c *Cluster) ListMachines(ctx context.Context, _ *emptypb.Empty) (*pb.ListMachinesResponse, error) { func (c *Cluster) ListMachines(ctx context.Context, _ *emptypb.Empty) (*pb.ListMachinesResponse, error) {
if err := c.checkInitialised(ctx); err != nil { if err := c.checkInitialised(ctx); err != nil {
@@ -242,3 +326,24 @@ func (c *Cluster) ListMachines(ctx context.Context, _ *emptypb.Empty) (*pb.ListM
return &pb.ListMachinesResponse{Machines: members}, nil return &pb.ListMachinesResponse{Machines: members}, nil
} }
// RemoveMachine removes a machine from the cluster.
func (c *Cluster) RemoveMachine(ctx context.Context, req *pb.RemoveMachineRequest) (*emptypb.Empty, error) {
if err := c.checkInitialised(ctx); err != nil {
return nil, err
}
if req.Id == "" {
return nil, status.Error(codes.InvalidArgument, "machine ID not set")
}
if err := c.store.DeleteMachine(ctx, req.Id); err != nil {
if errors.Is(err, store.ErrMachineNotFound) {
return nil, status.Errorf(codes.NotFound, "machine not found: %s", req.Id)
}
return nil, status.Errorf(codes.Internal, "delete machine from store: %v", err)
}
slog.Info("Machine removed from the cluster.", "id", req.Id)
return &emptypb.Empty{}, nil
}
+1
View File
@@ -4,6 +4,7 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
"github.com/psviderski/uncloud/internal/dns" "github.com/psviderski/uncloud/internal/dns"
"github.com/psviderski/uncloud/internal/machine/api/pb" "github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/machine/store" "github.com/psviderski/uncloud/internal/machine/store"
+2 -1
View File
@@ -3,8 +3,9 @@ package cluster
import ( import (
"errors" "errors"
"fmt" "fmt"
"go4.org/netipx"
"net/netip" "net/netip"
"go4.org/netipx"
) )
const DefaultSubnetBits = 24 const DefaultSubnetBits = 24
+1
View File
@@ -2,6 +2,7 @@ package cluster
import ( import (
"fmt" "fmt"
"github.com/psviderski/uncloud/internal/secret" "github.com/psviderski/uncloud/internal/secret"
) )
+6
View File
@@ -0,0 +1,6 @@
package constants
const (
// MachineAPIPort is the port for the Machine API service on the management WireGuard network.
MachineAPIPort = 51000
)
+5 -4
View File
@@ -3,10 +3,11 @@ package corroservice
import ( import (
"bytes" "bytes"
"fmt" "fmt"
"github.com/BurntSushi/toml"
"net/netip" "net/netip"
"os" "os"
"path/filepath" "path/filepath"
"github.com/BurntSushi/toml"
"github.com/psviderski/uncloud/internal/fs" "github.com/psviderski/uncloud/internal/fs"
) )
@@ -50,7 +51,7 @@ func (c *Config) Write(path, owner string) error {
if err := encoder.Encode(c); err != nil { if err := encoder.Encode(c); err != nil {
return fmt.Errorf("encode config: %w", err) return fmt.Errorf("encode config: %w", err)
} }
if err := os.WriteFile(path, data.Bytes(), 0600); err != nil { if err := os.WriteFile(path, data.Bytes(), 0o600); err != nil {
return err return err
} }
if err := fs.Chown(path, owner, owner); err != nil { if err := fs.Chown(path, owner, owner); err != nil {
@@ -62,10 +63,10 @@ func (c *Config) Write(path, owner string) error {
func MkDataDir(dir, owner string) error { func MkDataDir(dir, owner string) error {
parent, _ := filepath.Split(dir) parent, _ := filepath.Split(dir)
// Use 0711 for parent directories to allow `owner` to access its nested data directory. // Use 0711 for parent directories to allow `owner` to access its nested data directory.
if err := os.MkdirAll(parent, 0711); err != nil { if err := os.MkdirAll(parent, 0o711); err != nil {
return fmt.Errorf("create directory %q: %w", parent, err) return fmt.Errorf("create directory %q: %w", parent, err)
} }
if err := os.Mkdir(dir, 0700); err != nil { if err := os.Mkdir(dir, 0o700); err != nil {
if !os.IsExist(err) { if !os.IsExist(err) {
return fmt.Errorf("create directory %q: %w", dir, err) return fmt.Errorf("create directory %q: %w", dir, err)
} }
+40 -21
View File
@@ -3,15 +3,16 @@ package corroservice
import ( import (
"context" "context"
"fmt" "fmt"
"io"
"log/slog"
"path/filepath"
"time"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/image" "github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/mount" "github.com/docker/docker/api/types/mount"
"github.com/docker/docker/api/types/network" "github.com/docker/docker/api/types/network"
"github.com/docker/docker/client" "github.com/docker/docker/client"
"io"
"log/slog"
"path/filepath"
"time"
) )
const ( const (
@@ -26,29 +27,47 @@ type DockerService struct {
User string User string
} }
func NewDockerService(cli *client.Client, image, name, dataDir string) *DockerService {
return &DockerService{
Client: cli,
Image: image,
Name: name,
DataDir: dataDir,
}
}
func (s *DockerService) Start(ctx context.Context) error { func (s *DockerService) Start(ctx context.Context) error {
_, err := s.Client.ContainerInspect(ctx, s.Name) _, err := s.Client.ContainerInspect(ctx, s.Name)
if err != nil { if err != nil {
if client.IsErrNotFound(err) { if !client.IsErrNotFound(err) {
return s.startNewContainer(ctx) return fmt.Errorf("inspect container %q: %w", s.Name, err)
}
if err = s.startNewContainer(ctx); err != nil {
return err
}
} else {
// Container already exists.
// TODO: recreate only if the container configuration has to be changed.
if err = s.Client.ContainerRemove(ctx, s.Name, container.RemoveOptions{Force: true}); err != nil {
return fmt.Errorf("remove container %q: %w", s.Name, err)
}
if err = s.startNewContainer(ctx); err != nil {
return err
} }
return fmt.Errorf("inspect container %q: %w", s.Name, err)
}
// TODO: recreate only if the container configuration has to be changed.
if err = s.Client.ContainerRemove(ctx, s.Name, container.RemoveOptions{Force: true}); err != nil {
return fmt.Errorf("remove container %q: %w", s.Name, err)
} }
return s.startNewContainer(ctx) slog.Debug("Waiting for corrosion service to be ready.")
if err = WaitReady(ctx, s.DataDir); err != nil {
return err
}
slog.Debug("Corrosion service is ready.")
return nil
}
func (s *DockerService) Stop(ctx context.Context) error {
if err := s.Client.ContainerStop(ctx, s.Name, container.StopOptions{}); err != nil {
return fmt.Errorf("stop container %q: %w", s.Name, err)
}
slog.Debug("Corrosion Docker container stopped.", "name", s.Name)
if err := s.Client.ContainerRemove(ctx, s.Name, container.RemoveOptions{}); err != nil {
return fmt.Errorf("remove container %q: %w", s.Name, err)
}
slog.Debug("Corrosion Docker container removed.", "name", s.Name)
return nil
} }
func (s *DockerService) Restart(ctx context.Context) error { func (s *DockerService) Restart(ctx context.Context) error {
+54 -1
View File
@@ -1,9 +1,62 @@
package corroservice package corroservice
import "context" import (
"context"
"fmt"
"os"
"path/filepath"
"time"
"github.com/BurntSushi/toml"
"github.com/cenkalti/backoff/v4"
"github.com/psviderski/uncloud/internal/corrosion"
)
type Service interface { type Service interface {
Start(ctx context.Context) error Start(ctx context.Context) error
Stop(ctx context.Context) error
Restart(ctx context.Context) error Restart(ctx context.Context) error
Running() bool Running() bool
} }
// WaitReady waits for the Corrosion service to be ready with the uncloud schema applied.
func WaitReady(ctx context.Context, dataDir string) error {
// Read the config file to get the API address.
configPath := filepath.Join(dataDir, "config.toml")
configData, err := os.ReadFile(configPath)
if err != nil {
return fmt.Errorf("read config file: %w", err)
}
var config Config
if err = toml.Unmarshal(configData, &config); err != nil {
return fmt.Errorf("unmarshal config: %w", err)
}
corro, err := corrosion.NewAPIClient(config.API.Addr)
if err != nil {
return fmt.Errorf("create corrosion API client: %w", err)
}
// Corrosion starts serving the API before applying the schema. Query the cluster table with exponential backoff
// to check if the uncloud schema has been applied.
checkReady := func() error {
rows, err := corro.QueryContext(ctx, "SELECT 1 FROM cluster LIMIT 1")
if err != nil {
return fmt.Errorf("query cluster table: %w", err)
}
defer rows.Close()
return nil
}
b := backoff.NewExponentialBackOff(
backoff.WithInitialInterval(50*time.Millisecond),
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(15*time.Second),
)
if err = backoff.Retry(checkReady, backoff.WithContext(b, ctx)); err != nil {
return fmt.Errorf("corrosion service did not become ready: %w", err)
}
return nil
}
-157
View File
@@ -1,157 +0,0 @@
package corroservice
import (
"bufio"
"context"
"fmt"
"log/slog"
"os/exec"
"path/filepath"
"sync"
"syscall"
"time"
)
const (
DefaultCommand = "corrosion"
DefaultDataDir = "/var/lib/uncloud/corrosion"
)
// SubprocessService implements the Service interface by running the service as a subprocess.
type SubprocessService struct {
Command string
DataDir string
cmd *exec.Cmd
running bool
mu sync.Mutex
cancelWatch context.CancelFunc
}
func DefaultSubprocessService() *SubprocessService {
return &SubprocessService{
Command: DefaultCommand,
DataDir: DefaultDataDir,
}
}
// TODO: maybe stop the process if this ctx is cancelled.
func (s *SubprocessService) Start(ctx context.Context) error {
s.mu.Lock()
defer s.mu.Unlock()
if s.running {
return nil
}
return s.startProcess(ctx)
}
func (s *SubprocessService) Restart(ctx context.Context) error {
s.mu.Lock()
defer s.mu.Unlock()
if s.running {
if err := s.stopProcess(); err != nil {
return fmt.Errorf("stop process: %w", err)
}
}
return s.startProcess(ctx)
}
func (s *SubprocessService) Running() bool {
s.mu.Lock()
defer s.mu.Unlock()
return s.running
}
func (s *SubprocessService) startProcess(ctx context.Context) error {
s.cmd = exec.Command(s.Command, "agent", "-c", filepath.Join(s.DataDir, "config.toml"))
// Redirect stdout and stderr to the logger.
stdout, err := s.cmd.StdoutPipe()
if err != nil {
return fmt.Errorf("create stdout pipe: %w", err)
}
stderr, err := s.cmd.StderrPipe()
if err != nil {
return fmt.Errorf("create stderr pipe: %w", err)
}
go func() {
scanner := bufio.NewScanner(stdout)
for scanner.Scan() {
slog.Info("[corrosion]: " + scanner.Text())
}
// TODO: remove
slog.Info("######## corrosion redirect go routine end ########")
}()
go func() {
scanner := bufio.NewScanner(stderr)
for scanner.Scan() {
slog.Error("[corrosion]: " + scanner.Text())
}
}()
if err = s.cmd.Start(); err != nil {
return fmt.Errorf("start process: %w", err)
}
s.running = true
// Watch for process exit to update running status.
go func() {
if err := s.cmd.Wait(); err != nil {
slog.Error("corrosion process exited with error.", "code", s.cmd.ProcessState.ExitCode(), "err", err)
}
s.mu.Lock()
s.running = false
s.mu.Unlock()
}()
// TODO: figure out the waiting process
// Wait for initialization
//timer := time.NewTimer(2 * time.Second)
//defer timer.Stop()
//select {
////case <-timer.C:
//// s.running = true
//// return nil
//case <-watchCtx.Done():
// return fmt.Errorf("process failed to start")
//case <-ctx.Done():
// s.stopProcess()
// return ctx.Err()
//}
return nil
}
func (s *SubprocessService) stopProcess() error {
if s.cmd == nil || s.cmd.Process == nil {
return nil
}
if err := s.cmd.Process.Signal(syscall.SIGTERM); err != nil {
return fmt.Errorf("send SIGTERM: %w", err)
}
// Wait up to 5 seconds for graceful shutdown before killing the process.
done := make(chan error, 1)
go func() {
done <- s.cmd.Wait()
}()
select {
case <-time.After(5 * time.Second):
if err := s.cmd.Process.Kill(); err != nil {
return fmt.Errorf("kill process: %w", err)
}
case err := <-done:
if err != nil {
return fmt.Errorf("process exited with error: %w", err)
}
}
return nil
}
+15 -12
View File
@@ -5,7 +5,6 @@ import (
"fmt" "fmt"
"log/slog" "log/slog"
"os/exec" "os/exec"
"time"
) )
const DefaultSystemdUnit = "uncloud-corrosion.service" const DefaultSystemdUnit = "uncloud-corrosion.service"
@@ -27,6 +26,15 @@ func (s *SystemdService) Start(ctx context.Context) error {
return s.startOrRestart(ctx, "start") return s.startOrRestart(ctx, "start")
} }
func (s *SystemdService) Stop(_ context.Context) error {
if _, err := exec.Command("systemctl", "stop", s.Unit).Output(); err != nil {
return fmt.Errorf("systemctl stop %s: %w", s.Unit, err)
}
slog.Info("Corrosion systemd service stopped.", "unit", s.Unit)
return nil
}
func (s *SystemdService) Restart(ctx context.Context) error { func (s *SystemdService) Restart(ctx context.Context) error {
return s.startOrRestart(ctx, "restart") return s.startOrRestart(ctx, "restart")
} }
@@ -35,20 +43,15 @@ func (s *SystemdService) startOrRestart(ctx context.Context, cmd string) error {
if _, err := exec.Command("systemctl", cmd, s.Unit).Output(); err != nil { if _, err := exec.Command("systemctl", cmd, s.Unit).Output(); err != nil {
return fmt.Errorf("systemctl %s %s: %w", cmd, s.Unit, err) return fmt.Errorf("systemctl %s %s: %w", cmd, s.Unit, err)
} }
slog.Info(fmt.Sprintf("Corrosion systemd service %sed.", cmd), "unit", s.Unit) slog.Debug(fmt.Sprintf("Corrosion systemd service %sed.", cmd), "unit", s.Unit)
// Optimistically wait for the corrosion service to start and initialise the database schema before proceeding. slog.Debug("Waiting for corrosion service to be ready.")
timer := time.NewTimer(2 * time.Second) if err := WaitReady(ctx, s.DataDir); err != nil {
defer timer.Stop() return err
select {
case <-timer.C:
case <-ctx.Done():
return nil
} }
slog.Debug("Corrosion service is ready.")
// TODO: run a goroutine to check the status of the service and log any errors in the uncloud log.
s.running = true s.running = true
return nil return nil
} }
+2 -2
View File
@@ -14,13 +14,13 @@ const DBFileName = "machine.db"
func NewDB(path string) (*sqlx.DB, error) { func NewDB(path string) (*sqlx.DB, error) {
// Create the database file with 0600 permissions if it doesn't exist, or update permissions if exists. // Create the database file with 0600 permissions if it doesn't exist, or update permissions if exists.
if _, err := os.Stat(path); os.IsNotExist(err) { if _, err := os.Stat(path); os.IsNotExist(err) {
file, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR, 0600) file, err := os.OpenFile(path, os.O_CREATE|os.O_RDWR, 0o600)
if err != nil { if err != nil {
return nil, fmt.Errorf("create SQLite database '%s': %w", path, err) return nil, fmt.Errorf("create SQLite database '%s': %w", path, err)
} }
file.Close() file.Close()
} else { } else {
if err = os.Chmod(path, 0600); err != nil { if err = os.Chmod(path, 0o600); err != nil {
return nil, fmt.Errorf("update SQLite database permissions '%s': %w", path, err) return nil, fmt.Errorf("update SQLite database permissions '%s': %w", path, err)
} }
} }
+2 -8
View File
@@ -5,12 +5,10 @@ import (
"fmt" "fmt"
"log/slog" "log/slog"
"net/netip" "net/netip"
"strings"
"sync" "sync"
"time" "time"
"github.com/psviderski/uncloud/internal/machine/store" "github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/uncloud/pkg/api"
) )
// ClusterResolver implements Resolver by tracking containers in the cluster and resolving service names // ClusterResolver implements Resolver by tracking containers in the cluster and resolving service names
@@ -84,17 +82,13 @@ func (r *ClusterResolver) updateServiceIPs(containers []store.ContainerRecord) {
continue continue
} }
ctr := api.ServiceContainer{Container: record.Container} ctr := record.Container
if ctr.ServiceID() == "" || ctr.ServiceName() == "" { if ctr.ServiceID() == "" || ctr.ServiceName() == "" {
// Container is not part of a service, skip it. // Container is not part of a service, skip it.
continue continue
} }
// TODO: remove normalisation after implementing service name validation: newServiceIPs[ctr.ServiceName()] = append(newServiceIPs[ctr.ServiceName()], ip)
//.https://github.com/psviderski/uncloud/issues/53
serviceName := strings.ToLower(ctr.ServiceName())
newServiceIPs[serviceName] = append(newServiceIPs[serviceName], ip)
// Also add the service ID as a valid lookup. // Also add the service ID as a valid lookup.
newServiceIPs[ctr.ServiceID()] = append(newServiceIPs[ctr.ServiceID()], ip) newServiceIPs[ctr.ServiceID()] = append(newServiceIPs[ctr.ServiceID()], ip)
containersCount++ containersCount++
+16 -2
View File
@@ -196,13 +196,27 @@ func (c *Client) RemoveContainer(ctx context.Context, id string, opts container.
return err return err
} }
// PullOptions defines the options for pulling an image from a remote registry.
// This is a copy of image.PullOptions from the Docker API without the PrivilegeFunc field that is non-serialisable.
type PullOptions struct {
All bool
// RegistryAuth is the base64 encoded credentials for the registry.
RegistryAuth string
Platform string
}
type PullImageMessage struct { type PullImageMessage struct {
Message jsonmessage.JSONMessage Message jsonmessage.JSONMessage
Err error Err error
} }
func (c *Client) PullImage(ctx context.Context, image string) (<-chan PullImageMessage, error) { func (c *Client) PullImage(ctx context.Context, image string, opts PullOptions) (<-chan PullImageMessage, error) {
stream, err := c.grpcClient.PullImage(ctx, &pb.PullImageRequest{Image: image}) optsBytes, err := json.Marshal(opts)
if err != nil {
return nil, fmt.Errorf("marshal options: %w", err)
}
stream, err := c.grpcClient.PullImage(ctx, &pb.PullImageRequest{Image: image, Options: optsBytes})
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -7,12 +7,11 @@ import (
"log/slog" "log/slog"
"time" "time"
dockercontainer "github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/events" "github.com/docker/docker/api/types/events"
"github.com/docker/docker/api/types/filters" "github.com/docker/docker/api/types/filters"
"github.com/docker/docker/client" "github.com/docker/docker/client"
"github.com/psviderski/uncloud/internal/machine/store" "github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/uncloud/pkg/api"
) )
const ( const (
@@ -24,23 +23,26 @@ const (
SyncInterval = 30 * time.Second SyncInterval = 30 * time.Second
) )
type Manager struct { // Controller monitors Docker events and synchronises service containers with the cluster store.
client *client.Client type Controller struct {
// machineID is the ID of the machine where the managed Docker daemon is running. // machineID is the ID of the machine where the managed Docker daemon is running.
machineID string machineID string
client *client.Client
service *Service
store *store.Store store *store.Store
} }
func NewManager(client *client.Client, machineID string, store *store.Store) *Manager { func NewController(machineID string, service *Service, store *store.Store) *Controller {
return &Manager{ return &Controller{
client: client,
machineID: machineID, machineID: machineID,
client: service.Client,
service: service,
store: store, store: store,
} }
} }
// WaitDaemonReady waits for the Docker daemon to start and be ready to serve requests. // WaitDaemonReady waits for the Docker daemon to start and be ready to serve requests.
func (m *Manager) WaitDaemonReady(ctx context.Context) error { func (c *Controller) WaitDaemonReady(ctx context.Context) error {
ticker := time.NewTicker(1 * time.Second) ticker := time.NewTicker(1 * time.Second)
defer ticker.Stop() defer ticker.Stop()
@@ -50,7 +52,7 @@ func (m *Manager) WaitDaemonReady(ctx context.Context) error {
case <-ctx.Done(): case <-ctx.Done():
return ctx.Err() return ctx.Err()
case <-ticker.C: case <-ticker.C:
_, err := m.client.Ping(ctx) _, err := c.client.Ping(ctx)
if err == nil { if err == nil {
ready = true ready = true
break break
@@ -67,7 +69,7 @@ func (m *Manager) WaitDaemonReady(ctx context.Context) error {
return nil return nil
} }
func (m *Manager) WatchAndSyncContainers(ctx context.Context) error { func (c *Controller) WatchAndSyncContainers(ctx context.Context) error {
ctx, cancel := context.WithCancel(ctx) ctx, cancel := context.WithCancel(ctx)
defer cancel() defer cancel()
// Filter only local container events. // Filter only local container events.
@@ -79,9 +81,9 @@ func (m *Manager) WatchAndSyncContainers(ctx context.Context) error {
} }
// Subscribe to Docker events before running the initial sync to avoid missing any events. // Subscribe to Docker events before running the initial sync to avoid missing any events.
eventCh, errCh := m.client.Events(ctx, opts) eventCh, errCh := c.service.Client.Events(ctx, opts)
slog.Debug("Syncing containers to cluster store before processing Docker events.") slog.Debug("Syncing containers to cluster store before processing Docker events.")
if err := m.syncContainersToStore(ctx); err != nil { if err := c.syncContainersToStore(ctx); err != nil {
// The deferred cancel will stop the event subscription. // The deferred cancel will stop the event subscription.
return fmt.Errorf("sync containers to cluster store: %w", err) return fmt.Errorf("sync containers to cluster store: %w", err)
} }
@@ -126,13 +128,13 @@ func (m *Manager) WatchAndSyncContainers(ctx context.Context) error {
"container_name", e.Actor.Attributes["name"], "container_name", e.Actor.Attributes["name"],
"action", e.Action) "action", e.Action)
if err := m.syncContainersToStore(ctx); err != nil { if err := c.syncContainersToStore(ctx); err != nil {
return fmt.Errorf("sync containers to cluster store: %w", err) return fmt.Errorf("sync containers to cluster store: %w", err)
} }
case <-ticker.C: case <-ticker.C:
slog.Debug("Syncing containers to cluster store triggered by a regular interval.", slog.Debug("Syncing containers to cluster store triggered by a regular interval.",
"interval", SyncInterval) "interval", SyncInterval)
if err := m.syncContainersToStore(ctx); err != nil { if err := c.syncContainersToStore(ctx); err != nil {
return fmt.Errorf("sync containers to cluster store: %w", err) return fmt.Errorf("sync containers to cluster store: %w", err)
} }
case err := <-errCh: case err := <-errCh:
@@ -144,32 +146,16 @@ func (m *Manager) WatchAndSyncContainers(ctx context.Context) error {
} }
} }
func (m *Manager) syncContainersToStore(ctx context.Context) error { func (c *Controller) syncContainersToStore(ctx context.Context) error {
storeContainers, err := m.store.ListContainers(ctx, store.ListOptions{MachineIDs: []string{m.machineID}}) storeContainers, err := c.store.ListContainers(ctx, store.ListOptions{MachineIDs: []string{c.machineID}})
if err != nil { if err != nil {
return fmt.Errorf("list containers from store: %w", err) return fmt.Errorf("list containers from store: %w", err)
} }
// List only Uncloud service containers identified by their labels. containers, err := c.service.ListServiceContainers(ctx, "", container.ListOptions{})
containerSummaries, err := m.client.ContainerList(ctx, dockercontainer.ListOptions{
Filters: filters.NewArgs(
filters.Arg("label", api.LabelServiceID),
filters.Arg("label", api.LabelServiceName),
),
})
if err != nil { if err != nil {
// TODO: mark all containers as outdated in the store. // TODO: mark all containers as outdated in the store.
return fmt.Errorf("list Docker containers: %w", err) return fmt.Errorf("list service containers: %w", err)
}
// Inspect each container to get the full container details.
containers := make([]api.Container, len(containerSummaries))
for i, cs := range containerSummaries {
ctr, err := m.client.ContainerInspect(ctx, cs.ID)
if err != nil {
return fmt.Errorf("inspect container '%s': %w", cs.ID, err)
}
containers[i] = api.Container{ContainerJSON: ctr}
} }
// Delete containers from the store that are no longer present in the Docker daemon. // Delete containers from the store that are no longer present in the Docker daemon.
@@ -189,15 +175,15 @@ func (m *Manager) syncContainersToStore(ctx context.Context) error {
var storeErr error var storeErr error
if len(deleteIDs) > 0 { if len(deleteIDs) > 0 {
if err = m.store.DeleteContainers(ctx, store.DeleteOptions{IDs: deleteIDs}); err != nil { if err = c.store.DeleteContainers(ctx, store.DeleteOptions{IDs: deleteIDs}); err != nil {
storeErr = fmt.Errorf("delete containers from store: %w", err) storeErr = fmt.Errorf("delete containers from store: %w", err)
} }
} }
// Create or update the current Docker containers in the store. // Create or update the current Docker containers in the store.
for _, c := range containers { for _, ctr := range containers {
if err = m.store.CreateOrUpdateContainer(ctx, c, m.machineID); err != nil { if err = c.store.CreateOrUpdateContainer(ctx, ctr, c.machineID); err != nil {
storeErr = errors.Join(storeErr, fmt.Errorf("create or update container %q: %w", c.ID, err)) storeErr = errors.Join(storeErr, fmt.Errorf("create or update container '%s': %w", ctr.ID, err))
} }
} }
return storeErr return storeErr
@@ -0,0 +1,19 @@
//go:build darwin
package docker
import (
"context"
"fmt"
"net/netip"
)
// EnsureUncloudNetwork is a stub for Darwin.
func (c *Controller) EnsureUncloudNetwork(ctx context.Context, subnet netip.Prefix, dnsServer netip.Addr) error {
return fmt.Errorf("not supported on Darwin")
}
// Cleanup is a stub for Darwin.
func (c *Controller) Cleanup() error {
return fmt.Errorf("not supported on Darwin")
}
+234
View File
@@ -0,0 +1,234 @@
package docker
import (
"context"
"errors"
"fmt"
"log/slog"
"net/netip"
"strconv"
dockercontainer "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters"
dnetwork "github.com/docker/docker/api/types/network"
"github.com/docker/docker/client"
"github.com/docker/docker/libnetwork/iptables"
"github.com/psviderski/uncloud/internal/machine/dns"
"github.com/psviderski/uncloud/internal/machine/firewall"
"github.com/psviderski/uncloud/internal/machine/network"
"github.com/psviderski/uncloud/pkg/api"
)
// EnsureUncloudNetwork creates the Docker bridge network NetworkName with the provided machine subnet
// if it doesn't exist. If the network exists but has a different subnet, it removes and recreates the network.
// It also configures iptables to allow container access from the WireGuard network.
func (c *Controller) EnsureUncloudNetwork(ctx context.Context, subnet netip.Prefix, dnsServer netip.Addr) error {
// Ensure the Docker network 'uncloud' is created with the correct subnet.
needsCreation := false
nw, err := c.client.NetworkInspect(ctx, NetworkName, dnetwork.InspectOptions{})
if err != nil {
if !client.IsErrNotFound(err) {
return fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err)
}
needsCreation = true
} else if nw.IPAM.Config[0].Subnet != subnet.String() {
// Remove the Docker network if the subnet is different.
// It could be a leftover from a previous incomplete cleanup.
slog.Info(
"Removing Docker network with old subnet.", "name", NetworkName, "subnet", nw.IPAM.Config[0].Subnet,
)
if err = c.client.NetworkRemove(ctx, NetworkName); err != nil {
// It can still fail if the network is in use by a container. Leave it to the user to resolve the issue.
return fmt.Errorf("remove Docker network '%s': %w", NetworkName, err)
}
needsCreation = true
}
if needsCreation {
if _, err = c.client.NetworkCreate(
ctx, NetworkName, dnetwork.CreateOptions{
Driver: "bridge",
Scope: "local",
IPAM: &dnetwork.IPAM{
Config: []dnetwork.IPAMConfig{
{
Subnet: subnet.String(),
},
},
},
Labels: map[string]string{
api.LabelManaged: "",
},
Options: map[string]string{
// Starting with Docker 28.2.0 (https://github.com/moby/moby/pull/49832), we have to explicitly
// allow direct routing from the WireGuard interface to the bridge network.
"com.docker.network.bridge.trusted_host_interfaces": network.WireGuardInterfaceName,
},
},
); err != nil {
return fmt.Errorf("create Docker network '%s': %w", NetworkName, err)
}
slog.Info("Docker network created.", "name", NetworkName, "subnet", subnet.String())
if nw, err = c.client.NetworkInspect(ctx, NetworkName, dnetwork.InspectOptions{}); err != nil {
return fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err)
}
}
// Configure iptables to allow WireGuard network to access containers. The Docker daemon should have already
// created the DOCKER-USER chain at this point.
// TODO: check if this works when firewalld used instead of raw iptables. The Docker daemon has a different
// code path for firewalld.
// Bridge name doesn't seem to be documented but this is the source code where it is generated:
// https://github.com/moby/moby/blob/v27.2.1/libnetwork/drivers/bridge/bridge_linux.go#L664
bridgeName := "br-" + nw.ID[:12]
if err = configureIptables(bridgeName, subnet, dnsServer); err != nil {
return fmt.Errorf("configure iptables for Docker network '%s': %w", NetworkName, err)
}
return nil
}
// configureIptables configures iptables rules for the uncloud Docker network.
func configureIptables(bridgeName string, subnet netip.Prefix, dnsServer netip.Addr) error {
ipt := iptables.GetIptable(iptables.IPv4)
// Allow traffic from other machines and their containers through the WG mesh to the Uncloud containers
// on the machine.
wgRule := []string{
"--in-interface", network.WireGuardInterfaceName,
"--out-interface", bridgeName,
"-j", "ACCEPT",
}
if err := ipt.ProgramRule(iptables.Filter, firewall.DockerUserChain, iptables.Insert, wgRule); err != nil {
return fmt.Errorf("insert iptables rule: %w", err)
}
// Allow DNS queries from Uncloud containers to the embedded DNS server.
for _, proto := range []string{"udp", "tcp"} {
dnsRule := []string{
"--in-interface", bridgeName,
"--dst", dnsServer.String(),
"--protocol", proto,
"--dport", strconv.Itoa(dns.Port),
"-j", "ACCEPT",
}
if err := ipt.ProgramRule(iptables.Filter, firewall.UncloudInputChain, iptables.Insert, dnsRule); err != nil {
return fmt.Errorf("insert iptables rule: %w", err)
}
}
// Skip masquerading for the container traffic going from the uncloud Docker network through the WG mesh.
// https://uncloud.run/blog/connect-docker-containers-across-hosts-wireguard#step-3-configure-ip-routing
skipMasqueradeRule := []string{
"--src", subnet.String(),
"--out-interface", network.WireGuardInterfaceName,
"-j", "RETURN",
}
// Delete and reinsert the rule to ensure it's at the top of the POSTROUTING chain before the MASQUERADE rule
// added by Docker: POSTROUTING -s 10.210.X.0/24 ! -o br-XXX -j MASQUERADE
if err := ipt.ProgramRule(iptables.Nat, "POSTROUTING", iptables.Delete, skipMasqueradeRule); err != nil {
return fmt.Errorf("delete iptables rule: %w", err)
}
if err := ipt.ProgramRule(iptables.Nat, "POSTROUTING", iptables.Insert, skipMasqueradeRule); err != nil {
return fmt.Errorf("insert iptables rule: %w", err)
}
return nil
}
// cleanupIptables deletes the iptables rules for the uncloud Docker network.
func cleanupIptables(bridgeName string, subnet netip.Prefix) error {
ipt := iptables.GetIptable(iptables.IPv4)
// Delete the rule allowing traffic from the WireGuard network to the Docker bridge.
wgRule := []string{
"--in-interface", network.WireGuardInterfaceName,
"--out-interface", bridgeName,
"-j", "ACCEPT",
}
if err := ipt.ProgramRule(iptables.Filter, firewall.DockerUserChain, iptables.Delete, wgRule); err != nil {
return fmt.Errorf("delete iptables rule: %w", err)
}
// Delete the rule that skips masquerading for the container traffic going from the uncloud Docker network
// through the WG mesh.
skipMasqueradeRule := []string{
"--src", subnet.String(),
"--out-interface", network.WireGuardInterfaceName,
"-j", "RETURN",
}
if err := ipt.ProgramRule(iptables.Nat, "POSTROUTING", iptables.Delete, skipMasqueradeRule); err != nil {
return fmt.Errorf("delete iptables rule: %w", err)
}
// Rules in uncloud-owned chains will be automatically cleaned up by the machine cleanup.
return nil
}
// Cleanup removes all uncloud-managed containers and the uncloud Docker network.
func (c *Controller) Cleanup() error {
ctx := context.Background()
var errs []error
// Remove uncloud-managed Docker containers.
containers, err := c.client.ContainerList(ctx, dockercontainer.ListOptions{
All: true, // Include stopped containers.
Filters: filters.NewArgs(
filters.Arg("label", api.LabelManaged),
),
})
if err != nil {
errs = append(errs, fmt.Errorf("list uncloud-managed Docker containers: %w", err))
} else if len(containers) > 0 {
slog.Info("Removing uncloud-managed Docker containers.", "count", len(containers))
removed := 0
for _, ctr := range containers {
err = c.client.ContainerStop(ctx, ctr.ID, dockercontainer.StopOptions{})
if err != nil && !client.IsErrNotFound(err) {
errs = append(errs, fmt.Errorf("stop container '%s': %w", ctr.ID, err))
}
err = c.client.ContainerRemove(ctx, ctr.ID, dockercontainer.RemoveOptions{
// Remove anonymous volumes created by the container.
RemoveVolumes: true,
})
if err == nil {
removed++
} else if !client.IsErrNotFound(err) {
errs = append(errs, fmt.Errorf("remove container '%s': %w", ctr.ID, err))
}
}
slog.Info("Removed uncloud-managed Docker containers.", "count", removed)
}
// Remove the uncloud Docker network and related iptables rules.
nw, err := c.client.NetworkInspect(ctx, NetworkName, dnetwork.InspectOptions{})
if err == nil {
bridgeName := "br-" + nw.ID[:12]
var subnet netip.Prefix
if len(nw.IPAM.Config) > 0 {
subnet, _ = netip.ParsePrefix(nw.IPAM.Config[0].Subnet)
}
if subnet.IsValid() {
if err = cleanupIptables(bridgeName, subnet); err != nil {
errs = append(errs, fmt.Errorf("cleanup iptables for Docker network '%s': %w", NetworkName, err))
} else {
slog.Info("Cleaned up iptables rules for Docker network.", "name", NetworkName, "bridge", bridgeName)
}
}
if err = c.client.NetworkRemove(ctx, NetworkName); err == nil {
slog.Info("Docker network removed.", "name", NetworkName)
} else if !client.IsErrNotFound(err) {
errs = append(errs, fmt.Errorf("remove Docker network '%s': %w", NetworkName, err))
}
} else if !client.IsErrNotFound(err) {
errs = append(errs, fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err))
}
return errors.Join(errs...)
}
-14
View File
@@ -1,14 +0,0 @@
//go:build darwin
package docker
import (
"context"
"fmt"
"net/netip"
)
// EnsureUncloudNetwork is a stub for Darwin.
func (m *Manager) EnsureUncloudNetwork(ctx context.Context, subnet netip.Prefix, dnsServer netip.Addr) error {
return fmt.Errorf("not supported on Darwin")
}
-116
View File
@@ -1,116 +0,0 @@
package docker
import (
"context"
"fmt"
"log/slog"
"net/netip"
"strconv"
dnetwork "github.com/docker/docker/api/types/network"
"github.com/docker/docker/client"
"github.com/docker/docker/libnetwork/iptables"
"github.com/psviderski/uncloud/internal/machine/dns"
"github.com/psviderski/uncloud/internal/machine/firewall"
"github.com/psviderski/uncloud/internal/machine/network"
"github.com/psviderski/uncloud/pkg/api"
)
// EnsureUncloudNetwork creates the Docker bridge network NetworkName with the provided machine subnet
// if it doesn't exist. If the network exists but has a different subnet, it removes and recreates the network.
// It also configures iptables to allow container access from the WireGuard network.
func (m *Manager) EnsureUncloudNetwork(ctx context.Context, subnet netip.Prefix, dnsServer netip.Addr) error {
// Ensure the Docker network 'uncloud' is created with the correct subnet.
needsCreation := false
nw, err := m.client.NetworkInspect(ctx, NetworkName, dnetwork.InspectOptions{})
if err != nil {
if !client.IsErrNotFound(err) {
return fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err)
}
needsCreation = true
} else if nw.IPAM.Config[0].Subnet != subnet.String() {
// Remove the Docker network if the subnet is different.
// It could be a leftover from a previous incomplete cleanup.
slog.Info(
"Removing Docker network with old subnet.", "name", NetworkName, "subnet", nw.IPAM.Config[0].Subnet,
)
if err = m.client.NetworkRemove(ctx, NetworkName); err != nil {
// It can still fail if the network is in use by a container. Leave it to the user to resolve the issue.
return fmt.Errorf("remove Docker network '%s': %w", NetworkName, err)
}
needsCreation = true
}
if needsCreation {
if _, err = m.client.NetworkCreate(
ctx, NetworkName, dnetwork.CreateOptions{
Driver: "bridge",
Scope: "local",
IPAM: &dnetwork.IPAM{
Config: []dnetwork.IPAMConfig{
{
Subnet: subnet.String(),
},
},
},
Labels: map[string]string{
api.LabelManaged: "",
},
Options: map[string]string{
// Starting with Docker 28.2.0 (https://github.com/moby/moby/pull/49832), we have to explicitly
// allow direct routing from the WireGuard interface to the bridge network.
"com.docker.network.bridge.trusted_host_interfaces": network.WireGuardInterfaceName,
},
},
); err != nil {
return fmt.Errorf("create Docker network '%s': %w", NetworkName, err)
}
slog.Info("Docker network created.", "name", NetworkName, "subnet", subnet.String())
if nw, err = m.client.NetworkInspect(ctx, NetworkName, dnetwork.InspectOptions{}); err != nil {
return fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err)
}
}
// Configure iptables to allow WireGuard network to access containers. The Docker daemon should have already
// created the DOCKER-USER chain at this point.
// TODO: check if this works when firewalld used instead of raw iptables. The Docker daemon has a different
// code path for firewalld.
// Bridge name doesn't seem to be documented but this is the source code where it is generated:
// https://github.com/moby/moby/blob/v27.2.1/libnetwork/drivers/bridge/bridge_linux.go#L664
bridgeName := "br-" + nw.ID[:12]
if err = configureIptables(bridgeName, dnsServer); err != nil {
return fmt.Errorf("configure iptables for Docker network '%s': %w", NetworkName, err)
}
return nil
}
// configureIptables configures iptables rules for the uncloud Docker network.
func configureIptables(bridgeName string, dnsServer netip.Addr) error {
ipt := iptables.GetIptable(iptables.IPv4)
// Allow traffic from other machines and their containers through the WG mesh to the Uncloud containers
// on the machine.
wgRule := []string{"--in-interface", network.WireGuardInterfaceName, "--out-interface", bridgeName, "-j", "ACCEPT"}
if err := ipt.ProgramRule(iptables.Filter, firewall.DockerUserChain, iptables.Insert, wgRule); err != nil {
return fmt.Errorf("insert iptables rule: %w", err)
}
// Allow DNS queries from Uncloud containers to the embedded DNS server.
for _, proto := range []string{"udp", "tcp"} {
dnsRule := []string{
"--in-interface", bridgeName,
"--dst", dnsServer.String(),
"--protocol", proto,
"--dport", strconv.Itoa(dns.Port),
"-j", "ACCEPT",
}
if err := ipt.ProgramRule(iptables.Filter, firewall.UncloudInputChain, iptables.Insert, dnsRule); err != nil {
return fmt.Errorf("insert iptables rule: %w", err)
}
}
return nil
}
+72 -55
View File
@@ -2,19 +2,21 @@ package docker
import ( import (
"context" "context"
"database/sql"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"io" "io"
"log/slog" "log/slog"
"net/netip" "net/netip"
"os"
"regexp" "regexp"
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
"github.com/distribution/reference" "github.com/distribution/reference"
dockercommand "github.com/docker/cli/cli/command"
dockerconfig "github.com/docker/cli/cli/config"
"github.com/docker/docker/api/types" "github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters" "github.com/docker/docker/api/types/filters"
@@ -45,20 +47,49 @@ var fullDockerIDRegex = regexp.MustCompile(`^[a-f0-9]{64}$`)
// Server implements the gRPC Docker service that proxies requests to the Docker daemon. // Server implements the gRPC Docker service that proxies requests to the Docker daemon.
type Server struct { type Server struct {
pb.UnimplementedDockerServer pb.UnimplementedDockerServer
client *client.Client client *client.Client
db *sqlx.DB service *Service
db *sqlx.DB
// internalDNSIP is a function that returns the IP address of the internal DNS server. It may return an empty // internalDNSIP is a function that returns the IP address of the internal DNS server. It may return an empty
// address if the address is unknown (e.g. when the machine is not initialised yet). // address if the address is unknown (e.g. when the machine is not initialised yet).
internalDNSIP func() netip.Addr internalDNSIP func() netip.Addr
// networkReady is a function that returns true if the Docker network is ready for containers.
networkReady func() bool
// waitForNetworkReady is a function that waits for the Docker network to be ready for containers.
waitForNetworkReady func(ctx context.Context) error
} }
// NewServer creates a new Docker gRPC server with the provided Docker client. // ServerOption configures the Docker server.
func NewServer(cli *client.Client, db *sqlx.DB, internalDNSIP func() netip.Addr) *Server { type ServerOption func(*Server)
return &Server{
client: cli, // WithNetworkReady sets the network readiness check function.
func WithNetworkReady(networkReady func() bool) ServerOption {
return func(s *Server) {
s.networkReady = networkReady
}
}
// WithWaitForNetworkReady sets the network readiness wait function.
func WithWaitForNetworkReady(waitForNetworkReady func(ctx context.Context) error) ServerOption {
return func(s *Server) {
s.waitForNetworkReady = waitForNetworkReady
}
}
// NewServer creates a new Docker gRPC server with the provided Docker service.
func NewServer(service *Service, db *sqlx.DB, internalDNSIP func() netip.Addr, opts ...ServerOption) *Server {
s := &Server{
client: service.Client,
service: service,
db: db, db: db,
internalDNSIP: internalDNSIP, internalDNSIP: internalDNSIP,
} }
for _, opt := range opts {
opt(s)
}
return s
} }
// CreateContainer creates a new container based on the given configuration. // CreateContainer creates a new container based on the given configuration.
@@ -118,6 +149,15 @@ func (s *Server) InspectContainer(ctx context.Context, req *pb.InspectContainerR
// StartContainer starts a container with the given ID and options. // StartContainer starts a container with the given ID and options.
func (s *Server) StartContainer(ctx context.Context, req *pb.StartContainerRequest) (*emptypb.Empty, error) { func (s *Server) StartContainer(ctx context.Context, req *pb.StartContainerRequest) (*emptypb.Empty, error) {
// Wait for Docker network to be ready before starting the container
if s.waitForNetworkReady != nil {
if err := s.waitForNetworkReady(ctx); err != nil {
return nil, status.Errorf(codes.Unavailable, "Docker network not ready: %v", err)
}
} else if s.networkReady != nil && !s.networkReady() {
return nil, status.Errorf(codes.Unavailable, "Docker network not ready")
}
var opts container.StartOptions var opts container.StartOptions
if len(req.Options) > 0 { if len(req.Options) > 0 {
if err := json.Unmarshal(req.Options, &opts); err != nil { if err := json.Unmarshal(req.Options, &opts); err != nil {
@@ -229,7 +269,6 @@ func (s *Server) RemoveContainer(ctx context.Context, req *pb.RemoveContainerReq
func (s *Server) PullImage(req *pb.PullImageRequest, stream grpc.ServerStreamingServer[pb.JSONMessage]) error { func (s *Server) PullImage(req *pb.PullImageRequest, stream grpc.ServerStreamingServer[pb.JSONMessage]) error {
ctx := stream.Context() ctx := stream.Context()
// TODO: replace with another JSON serializable type (PullOptions.PrivilegeFunc is not serializable).
var opts image.PullOptions var opts image.PullOptions
if len(req.Options) > 0 { if len(req.Options) > 0 {
if err := json.Unmarshal(req.Options, &opts); err != nil { if err := json.Unmarshal(req.Options, &opts); err != nil {
@@ -237,6 +276,14 @@ func (s *Server) PullImage(req *pb.PullImageRequest, stream grpc.ServerStreaming
} }
} }
if opts.RegistryAuth == "" {
// Try to retrieve the authentication token for the image from the default local Docker config file.
dockerConfig := dockerconfig.LoadDefaultConfigFile(os.Stderr)
if encodedAuth, err := dockercommand.RetrieveAuthTokenFromImage(dockerConfig, req.Image); err == nil {
opts.RegistryAuth = encodedAuth
}
}
respBody, err := s.client.ImagePull(ctx, req.Image, opts) respBody, err := s.client.ImagePull(ctx, req.Image, opts)
if err != nil { if err != nil {
return status.Errorf(codes.Internal, err.Error()) return status.Errorf(codes.Internal, err.Error())
@@ -419,6 +466,7 @@ func (s *Server) RemoveVolume(ctx context.Context, req *pb.RemoveVolumeRequest)
} }
// CreateServiceContainer creates a new container for the service with the given specifications. // CreateServiceContainer creates a new container for the service with the given specifications.
// TODO: move the main logic to the Docker service and remove db dependency from the server.
func (s *Server) CreateServiceContainer( func (s *Server) CreateServiceContainer(
ctx context.Context, req *pb.CreateServiceContainerRequest, ctx context.Context, req *pb.CreateServiceContainerRequest,
) (*pb.CreateContainerResponse, error) { ) (*pb.CreateContainerResponse, error) {
@@ -510,10 +558,10 @@ func (s *Server) CreateServiceContainer(
Memory: spec.Container.Resources.Memory, Memory: spec.Container.Resources.Memory,
MemoryReservation: spec.Container.Resources.MemoryReservation, MemoryReservation: spec.Container.Resources.MemoryReservation,
}, },
// Always restart service containers if they exit or a machine restarts. // Restart service containers if they exit or a machine restarts unless they are explicitly stopped.
// For one-off containers and batch jobs we plan to use a different service type/mode. // For one-off containers and batch jobs we plan to use a different service type/mode.
RestartPolicy: container.RestartPolicy{ RestartPolicy: container.RestartPolicy{
Name: container.RestartPolicyAlways, Name: container.RestartPolicyUnlessStopped,
}, },
} }
@@ -673,7 +721,7 @@ func (s *Server) verifyDockerVolumesExist(ctx context.Context, mounts []mount.Mo
func (s *Server) InspectServiceContainer( func (s *Server) InspectServiceContainer(
ctx context.Context, req *pb.InspectContainerRequest, ctx context.Context, req *pb.InspectContainerRequest,
) (*pb.ServiceContainer, error) { ) (*pb.ServiceContainer, error) {
ctr, err := s.client.ContainerInspect(ctx, req.Id) serviceCtr, err := s.service.InspectServiceContainer(ctx, req.Id)
if err != nil { if err != nil {
if client.IsErrNotFound(err) { if client.IsErrNotFound(err) {
return nil, status.Errorf(codes.NotFound, err.Error()) return nil, status.Errorf(codes.NotFound, err.Error())
@@ -681,19 +729,14 @@ func (s *Server) InspectServiceContainer(
return nil, status.Errorf(codes.Internal, err.Error()) return nil, status.Errorf(codes.Internal, err.Error())
} }
ctrBytes, err := json.Marshal(ctr) ctrBytes, err := json.Marshal(serviceCtr.Container)
if err != nil { if err != nil {
return nil, status.Errorf(codes.Internal, "marshal response: %v", err) return nil, status.Errorf(codes.Internal, "marshal container: %v", err)
} }
var specBytes []byte specBytes, err := json.Marshal(serviceCtr.ServiceSpec)
err = s.db.QueryRowContext(ctx, `SELECT service_spec FROM containers WHERE id = $1`, ctr.ID).Scan(&specBytes)
if err != nil { if err != nil {
if errors.Is(err, sql.ErrNoRows) { return nil, status.Errorf(codes.Internal, "marshal service spec: %v", err)
return nil, status.Errorf(codes.NotFound, "service spec not found for container: '%s'", ctr.ID)
}
return nil, status.Errorf(codes.Internal, "get service spec for container '%s' from machine database: %v",
ctr.ID, err)
} }
return &pb.ServiceContainer{ return &pb.ServiceContainer{
@@ -725,54 +768,28 @@ func (s *Server) ListServiceContainers(
return nil, status.Errorf(codes.InvalidArgument, "unmarshal filters: %v", err) return nil, status.Errorf(codes.InvalidArgument, "unmarshal filters: %v", err)
} }
opts.Filters = args opts.Filters = args
} else {
opts.Filters = filters.NewArgs()
} }
} }
// Only uncloud-managed containers that belong to some service.
opts.Filters.Add("label", api.LabelServiceID)
opts.Filters.Add("label", api.LabelManaged)
containerSummaries, err := s.client.ContainerList(ctx, opts) containers, err := s.service.ListServiceContainers(ctx, req.ServiceId, opts)
if err != nil { if err != nil {
return nil, status.Error(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
} }
containers := make([]*pb.ServiceContainer, 0, len(containerSummaries)) // Convert to protobuf format.
for _, cs := range containerSummaries { pbContainers := make([]*pb.ServiceContainer, 0, len(containers))
if req.ServiceId != "" && for _, ctr := range containers {
cs.Labels[api.LabelServiceID] != req.ServiceId && cs.Labels[api.LabelServiceName] != req.ServiceId { ctrBytes, err := json.Marshal(ctr.Container)
continue
}
ctr, err := s.client.ContainerInspect(ctx, cs.ID)
if err != nil {
if client.IsErrNotFound(err) {
// The listed container may have been removed while we were inspecting other containers.
continue
}
return nil, status.Errorf(codes.Internal, "inspect container %s: %v", cs.ID, err)
}
ctrBytes, err := json.Marshal(ctr)
if err != nil { if err != nil {
return nil, status.Errorf(codes.Internal, "marshal container: %v", err) return nil, status.Errorf(codes.Internal, "marshal container: %v", err)
} }
var specBytes []byte specBytes, err := json.Marshal(ctr.ServiceSpec)
err = s.db.QueryRowContext(ctx, `SELECT service_spec FROM containers WHERE id = $1`, ctr.ID).Scan(&specBytes)
if err != nil { if err != nil {
if errors.Is(err, sql.ErrNoRows) { return nil, status.Errorf(codes.Internal, "marshal service spec: %v", err)
// If this happens, there is a bug in the code, or someone manually removed the container from the DB,
// or created a managed container out of band.
slog.Error("Service container not found in machine database.", "id", ctr.ID)
// Just ignore such a container to not fail the list operation as it's not easily recoverable.
continue
}
return nil, status.Errorf(codes.Internal, "get service spec for container '%s' from machine database: %v",
ctr.ID, err)
} }
containers = append(containers, &pb.ServiceContainer{ pbContainers = append(pbContainers, &pb.ServiceContainer{
Container: ctrBytes, Container: ctrBytes,
ServiceSpec: specBytes, ServiceSpec: specBytes,
}) })
@@ -781,7 +798,7 @@ func (s *Server) ListServiceContainers(
return &pb.ListServiceContainersResponse{ return &pb.ListServiceContainersResponse{
Messages: []*pb.MachineServiceContainers{ Messages: []*pb.MachineServiceContainers{
{ {
Containers: containers, Containers: pbContainers,
}, },
}, },
}, nil }, nil
+103
View File
@@ -0,0 +1,103 @@
package docker
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"log/slog"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/client"
"github.com/jmoiron/sqlx"
"github.com/psviderski/uncloud/pkg/api"
)
// Service provides higher-level Docker operations that extends Docker API with Uncloud-specific data
// from the machine database.
type Service struct {
Client *client.Client
db *sqlx.DB
}
// NewService creates a new Docker service instance.
func NewService(client *client.Client, db *sqlx.DB) *Service {
return &Service{
Client: client,
db: db,
}
}
// InspectServiceContainer inspects a Docker container and retrieves its associated ServiceSpec
// from the machine database, returning a complete ServiceContainer.
func (s *Service) InspectServiceContainer(ctx context.Context, nameOrID string) (api.ServiceContainer, error) {
var serviceCtr api.ServiceContainer
ctr, err := s.Client.ContainerInspect(ctx, nameOrID)
if err != nil {
return serviceCtr, err
}
if _, ok := ctr.Config.Labels[api.LabelManaged]; !ok {
return serviceCtr, fmt.Errorf("container '%s' is not managed by Uncloud", nameOrID)
}
serviceCtr.Container = api.Container{ContainerJSON: ctr}
// Retrieve ServiceSpec from the machine database.
var specBytes []byte
err = s.db.QueryRowContext(ctx, `SELECT service_spec FROM containers WHERE id = $1`, ctr.ID).Scan(&specBytes)
if err != nil {
if errors.Is(err, sql.ErrNoRows) {
// If this happens, there is a bug in the code, or someone manually removed the container from the DB,
// or created a managed container out of band or by previous uncloud installation.
return serviceCtr, fmt.Errorf("service spec not found for container '%s' in machine DB", ctr.ID)
}
return serviceCtr, fmt.Errorf("get service spec for container '%s' from machine DB: %w", ctr.ID, err)
}
if err = json.Unmarshal(specBytes, &serviceCtr.ServiceSpec); err != nil {
return serviceCtr, fmt.Errorf("unmarshal service spec for container '%s': %w", ctr.ID, err)
}
return serviceCtr, nil
}
// ListServiceContainers lists Docker containers that belong to the service with the given name or ID.
// If serviceIDOrName is empty, all service containers are returned. The opts parameter allows additional filtering.
func (s *Service) ListServiceContainers(
ctx context.Context, serviceNameOrID string, opts container.ListOptions,
) ([]api.ServiceContainer, error) {
if opts.Filters.Len() == 0 {
opts.Filters = filters.NewArgs()
}
// Add labels to existing filters to list only Uncloud-managed service containers.
opts.Filters.Add("label", api.LabelServiceID)
opts.Filters.Add("label", api.LabelManaged)
containerSummaries, err := s.Client.ContainerList(ctx, opts)
if err != nil {
return nil, err
}
var containers []api.ServiceContainer
for _, cs := range containerSummaries {
// Filter by service name or ID if provided.
if serviceNameOrID != "" &&
cs.Labels[api.LabelServiceID] != serviceNameOrID &&
cs.Labels[api.LabelServiceName] != serviceNameOrID {
continue
}
ctr, err := s.InspectServiceContainer(ctx, cs.ID)
if err != nil {
// Log error but continue with other containers.
slog.Error("Failed to inspect service container.", "service", serviceNameOrID, "id", cs.ID, "err", err)
continue
}
containers = append(containers, ctr)
}
return containers, nil
}
@@ -6,3 +6,8 @@ import "fmt"
func ConfigureIptablesChains() error { func ConfigureIptablesChains() error {
return fmt.Errorf("not supported on Darwin") return fmt.Errorf("not supported on Darwin")
} }
// CleanupIptablesChains is a stub for Darwin.
func CleanupIptablesChains() error {
return fmt.Errorf("not supported on Darwin")
}
+134 -45
View File
@@ -2,10 +2,13 @@ package firewall
import ( import (
"fmt" "fmt"
"log/slog"
"strconv" "strconv"
"strings" "strings"
"github.com/docker/docker/libnetwork/iptables" "github.com/docker/docker/libnetwork/iptables"
"github.com/psviderski/uncloud/internal/machine/constants"
"github.com/psviderski/uncloud/internal/machine/corroservice"
"github.com/psviderski/uncloud/internal/machine/network" "github.com/psviderski/uncloud/internal/machine/network"
) )
@@ -16,59 +19,145 @@ const (
// ConfigureIptablesChains sets up custom iptables chains and initial firewall rules for Uncloud networking. // ConfigureIptablesChains sets up custom iptables chains and initial firewall rules for Uncloud networking.
func ConfigureIptablesChains() error { func ConfigureIptablesChains() error {
// Ensure iptables UNCLOUD-INPUT chain with a RETURN rule exists. All existing rules are flushed. if err := createIptablesChains(); err != nil {
ipt := iptables.GetIptable(iptables.IPv4) return err
if _, err := ipt.NewChain(UncloudInputChain, iptables.Filter); err != nil {
return fmt.Errorf("create iptables chain '%s': %w", UncloudInputChain, err)
}
if err := ipt.RawCombinedOutput("-t", string(iptables.Filter), "-F", UncloudInputChain); err != nil {
return fmt.Errorf("flush iptables chain '%s': %w", UncloudInputChain, err)
}
if err := ipt.AddReturnRule(UncloudInputChain); err != nil {
return fmt.Errorf("add the RETURN rule for iptables chain '%s': %w", UncloudInputChain, err)
} }
// Ensure the main iptables INPUT chain has a jump rule to the UNCLOUD-INPUT chain before any DROP/REJECT rules. ipt4 := iptables.GetIptable(iptables.IPv4)
jumpRule := []string{"-m", "comment", "--comment", "Uncloud-managed", "-j", UncloudInputChain} ipt6 := iptables.GetIptable(iptables.IPv6)
if !ipt.Exists(iptables.Filter, "INPUT", jumpRule...) {
// Look for the first DROP/REJECT rule in the INPUT chain.
out, err := ipt.Raw("-t", string(iptables.Filter), "-L", "INPUT", "--line-numbers")
if err != nil {
return fmt.Errorf("get iptables rules for chain '%s': %w", UncloudInputChain, err)
}
firstRejectRuleNum := 0
for _, line := range strings.Split(string(out), "\n") {
fields := strings.Fields(line)
if len(fields) < 2 {
continue
}
if fields[1] == "DROP" || fields[1] == "REJECT" {
if ruleNum, err := strconv.Atoi(fields[0]); err == nil {
firstRejectRuleNum = ruleNum
break
}
}
}
var addJumpRule []string
if firstRejectRuleNum > 0 {
addJumpRule = append([]string{"-t", string(iptables.Filter), "-I", "INPUT", strconv.Itoa(firstRejectRuleNum)},
jumpRule...)
} else {
addJumpRule = append([]string{"-t", string(iptables.Filter), "-A", "INPUT"}, jumpRule...)
}
if err = ipt.RawCombinedOutput(addJumpRule...); err != nil {
return fmt.Errorf("add iptables rule '%s': %w", strings.Join(addJumpRule, " "), err)
}
}
// Allow WireGuard traffic to the machine. // Allow WireGuard traffic to the machine.
acceptWireGuardRule := []string{"-p", "udp", "--dport", strconv.Itoa(network.WireGuardPort), "-j", "ACCEPT"} acceptWireGuardRule := []string{"-p", "udp", "--dport", strconv.Itoa(network.WireGuardPort), "-j", "ACCEPT"}
err := ipt.ProgramRule(iptables.Filter, UncloudInputChain, iptables.Insert, acceptWireGuardRule) err := ipt4.ProgramRule(iptables.Filter, UncloudInputChain, iptables.Insert, acceptWireGuardRule)
if err != nil { if err != nil {
return fmt.Errorf("insert iptables rule '%s': %w", strings.Join(acceptWireGuardRule, " "), err) return fmt.Errorf("insert iptables rule '%s': %w", strings.Join(acceptWireGuardRule, " "), err)
} }
// Allow cluster machines to access Machine API via the management IPv6 WireGuard network.
acceptMachineAPIRule := []string{
"-i", network.WireGuardInterfaceName,
"-s", "fdcc::/16",
"-p", "tcp",
"--dport", strconv.Itoa(constants.MachineAPIPort),
"-j", "ACCEPT",
}
// Allow Corrosion gossip traffic from cluster machines via the management IPv6 WireGuard network.
acceptCorrosionGossipRule := []string{
"-i", network.WireGuardInterfaceName,
"-s", "fdcc::/16",
"-p", "udp",
"--dport", strconv.Itoa(corroservice.DefaultGossipPort),
"-j", "ACCEPT",
}
for _, rule := range [][]string{acceptMachineAPIRule, acceptCorrosionGossipRule} {
if err = ipt6.ProgramRule(iptables.Filter, UncloudInputChain, iptables.Insert, rule); err != nil {
return fmt.Errorf("insert ip6tables rule '%s': %w", strings.Join(rule, " "), err)
}
}
return nil
}
// createIptablesChains ensures UNCLOUD-INPUT iptables and ip6tables chains exist and
// there are jump rules from the main INPUT chains.
func createIptablesChains() error {
ipt4 := iptables.GetIptable(iptables.IPv4)
ipt6 := iptables.GetIptable(iptables.IPv6)
for i, ipt := range []*iptables.IPTable{ipt4, ipt6} {
iptBin := "iptables"
if i == 1 {
iptBin = "ip6tables"
}
// Ensure UNCLOUD-INPUT chain exists. All existing rules are flushed.
if _, err := ipt.NewChain(UncloudInputChain, iptables.Filter); err != nil {
return fmt.Errorf("create %s chain '%s': %w", iptBin, UncloudInputChain, err)
}
if err := ipt.RawCombinedOutput("-t", string(iptables.Filter), "-F", UncloudInputChain); err != nil {
return fmt.Errorf("flush %s chain '%s': %w", iptBin, UncloudInputChain, err)
}
// Ensure the main INPUT chain has a jump rule to the UNCLOUD-INPUT chain before any DROP/REJECT rules.
jumpRule := []string{"-m", "comment", "--comment", "Uncloud-managed", "-j", UncloudInputChain}
if !ipt.Exists(iptables.Filter, "INPUT", jumpRule...) {
// Look for the first DROP/REJECT rule in the INPUT chain.
out, err := ipt.Raw("-t", string(iptables.Filter), "-L", "INPUT", "--line-numbers")
if err != nil {
return fmt.Errorf("get %s rules for chain '%s': %w", iptBin, UncloudInputChain, err)
}
firstRejectRuleNum := 0
for _, line := range strings.Split(string(out), "\n") {
fields := strings.Fields(line)
if len(fields) < 2 {
continue
}
if fields[1] == "DROP" || fields[1] == "REJECT" {
if ruleNum, err := strconv.Atoi(fields[0]); err == nil {
firstRejectRuleNum = ruleNum
break
}
}
}
var addJumpRule []string
if firstRejectRuleNum > 0 {
addJumpRule = append([]string{
"-t", string(iptables.Filter),
"-I", "INPUT",
strconv.Itoa(firstRejectRuleNum),
}, jumpRule...)
} else {
addJumpRule = append([]string{
"-t", string(iptables.Filter),
"-A", "INPUT",
}, jumpRule...)
}
if err = ipt.RawCombinedOutput(addJumpRule...); err != nil {
return fmt.Errorf("add %s rule '%s': %w", iptBin, strings.Join(addJumpRule, " "), err)
}
}
}
return nil
}
// CleanupIptablesChains removes the custom iptables chains and rules created by ConfigureIptablesChains.
func CleanupIptablesChains() error {
ipt4 := iptables.GetIptable(iptables.IPv4)
ipt6 := iptables.GetIptable(iptables.IPv6)
for i, ipt := range []*iptables.IPTable{ipt4, ipt6} {
iptBin := "iptables"
if i == 1 {
iptBin = "ip6tables"
}
// First, remove the jump rule from INPUT chain to UNCLOUD-INPUT.
jumpRule := []string{"-m", "comment", "--comment", "Uncloud-managed", "-j", UncloudInputChain}
if err := ipt.ProgramRule(iptables.Filter, "INPUT", iptables.Delete, jumpRule); err != nil {
return fmt.Errorf("delete %s jump rule from INPUT: %w", iptBin, err)
}
// Flush all rules from UNCLOUD-INPUT chain as it must be empty before deletion.
if err := ipt.RawCombinedOutput("-t", string(iptables.Filter), "-F", UncloudInputChain); err != nil {
// Chain might not exist which is fine.
if !strings.Contains(err.Error(), "No chain") {
return fmt.Errorf("flush %s chain '%s': %w", iptBin, UncloudInputChain, err)
}
}
// Delete the UNCLOUD-INPUT chain.
if err := ipt.RawCombinedOutput("-t", string(iptables.Filter), "-X", UncloudInputChain); err != nil {
// Chain might not exist which is fine.
if !strings.Contains(err.Error(), "No chain") {
return fmt.Errorf("delete %s chain '%s': %w", iptBin, UncloudInputChain, err)
}
} else {
slog.Info(fmt.Sprintf("Deleted %s chain.", iptBin), "chain", UncloudInputChain)
}
}
return nil return nil
} }
+222 -157
View File
@@ -13,6 +13,7 @@ import (
"path/filepath" "path/filepath"
"slices" "slices"
"strconv" "strconv"
"sync"
"github.com/docker/docker/client" "github.com/docker/docker/client"
"github.com/docker/go-connections/sockets" "github.com/docker/go-connections/sockets"
@@ -23,12 +24,12 @@ import (
apiproxy "github.com/psviderski/uncloud/internal/machine/api/proxy" apiproxy "github.com/psviderski/uncloud/internal/machine/api/proxy"
"github.com/psviderski/uncloud/internal/machine/caddyconfig" "github.com/psviderski/uncloud/internal/machine/caddyconfig"
"github.com/psviderski/uncloud/internal/machine/cluster" "github.com/psviderski/uncloud/internal/machine/cluster"
"github.com/psviderski/uncloud/internal/machine/constants"
"github.com/psviderski/uncloud/internal/machine/corroservice" "github.com/psviderski/uncloud/internal/machine/corroservice"
"github.com/psviderski/uncloud/internal/machine/dns" "github.com/psviderski/uncloud/internal/machine/dns"
machinedocker "github.com/psviderski/uncloud/internal/machine/docker" machinedocker "github.com/psviderski/uncloud/internal/machine/docker"
"github.com/psviderski/uncloud/internal/machine/network" "github.com/psviderski/uncloud/internal/machine/network"
"github.com/psviderski/uncloud/internal/machine/store" "github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/uncloud/pkg/api"
"github.com/siderolabs/grpc-proxy/proxy" "github.com/siderolabs/grpc-proxy/proxy"
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
"google.golang.org/grpc" "google.golang.org/grpc"
@@ -60,9 +61,9 @@ type Config struct {
// DockerClient manages system and user containers using the local Docker daemon. // DockerClient manages system and user containers using the local Docker daemon.
DockerClient *client.Client DockerClient *client.Client
// CaddyConfigPath specifies where the machine generates the Caddy reverse proxy configuration file for routing // CaddyConfigDir specifies the directory where the machine generates the Caddy reverse proxy configuration file
// external traffic to service containers across the internal network. Default is DataDir/caddy/caddy.json. // for routing external traffic to service containers across the internal network. Default is DataDir/caddy.
CaddyConfigPath string CaddyConfigDir string
// DNSUpstreams specifies the upstream DNS servers for the embedded internal DNS server. // DNSUpstreams specifies the upstream DNS servers for the embedded internal DNS server.
DNSUpstreams []netip.AddrPort DNSUpstreams []netip.AddrPort
} }
@@ -127,8 +128,8 @@ func (c *Config) SetDefaults() (*Config, error) {
} }
} }
if cfg.CaddyConfigPath == "" { if cfg.CaddyConfigDir == "" {
cfg.CaddyConfigPath = filepath.Join(cfg.DataDir, "caddy", "caddy.json") cfg.CaddyConfigDir = filepath.Join(cfg.DataDir, "caddy")
} }
return &cfg, nil return &cfg, nil
@@ -149,11 +150,20 @@ type Machine struct {
started chan struct{} started chan struct{}
// initialised is signalled when the machine is configured as a member of a cluster. // initialised is signalled when the machine is configured as a member of a cluster.
initialised chan struct{} initialised chan struct{}
// networkReady is signalled when the Docker network is configured and ready for containers.
networkReady chan struct{}
// resetting is true when the machine is being reset.
resetting bool
// stop cancels the Run method context to stop the machine gracefully.
stop func()
clusterCtrl *clusterController
// store is the cluster store backed by a distributed Corrosion database. // store is the cluster store backed by a distributed Corrosion database.
store *store.Store store *store.Store
cluster *cluster.Cluster cluster *cluster.Cluster
docker *machinedocker.Server // dockerService provides high-level operations for managing Docker containers.
dockerService *machinedocker.Service
dockerServer *machinedocker.Server
// localMachineServer is the gRPC server for the machine API listening on the local Unix socket. // localMachineServer is the gRPC server for the machine API listening on the local Unix socket.
localMachineServer *grpc.Server localMachineServer *grpc.Server
@@ -163,6 +173,9 @@ type Machine struct {
// It proxies requests to the local or remote machine API servers depending on the request targets // It proxies requests to the local or remote machine API servers depending on the request targets
// and aggregates responses. // and aggregates responses.
localProxyServer *grpc.Server localProxyServer *grpc.Server
// mu protects the Machine from concurrent reads and writes.
mu sync.RWMutex
} }
func NewMachine(config *Config) (*Machine, error) { func NewMachine(config *Config) (*Machine, error) {
@@ -210,19 +223,16 @@ func NewMachine(config *Config) (*Machine, error) {
c := cluster.NewCluster(corroStore, corroAdmin) c := cluster.NewCluster(corroStore, corroAdmin)
// Init dependencies for a gRPC Docker server that proxies requests to the local Docker daemon. // Init dependencies for a gRPC Docker server that proxies requests to the local Docker daemon.
dockerCli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
if err != nil {
return nil, fmt.Errorf("create Docker client: %w", err)
}
dbFilePath := filepath.Join(config.DataDir, DBFileName) dbFilePath := filepath.Join(config.DataDir, DBFileName)
db, err := NewDB(dbFilePath) db, err := NewDB(dbFilePath)
if err != nil { if err != nil {
return nil, fmt.Errorf("init machine database: %w", err) return nil, fmt.Errorf("init machine database: %w", err)
} }
dockerService := machinedocker.NewService(config.DockerClient, db)
// Init a local gRPC proxy server that proxies requests to the local or remote machine API servers. // Init a local gRPC proxy server that proxies requests to the local or remote machine API servers.
proxyDirector := apiproxy.NewDirector(config.MachineSockPath, APIPort) proxyDirector := apiproxy.NewDirector(config.MachineSockPath, constants.MachineAPIPort)
localProxyServer := grpc.NewServer( localProxyServer := grpc.NewServer(
grpc.ForceServerCodecV2(proxy.Codec()), grpc.ForceServerCodecV2(proxy.Codec()),
grpc.UnknownServiceHandler( grpc.UnknownServiceHandler(
@@ -235,8 +245,10 @@ func NewMachine(config *Config) (*Machine, error) {
state: state, state: state,
started: make(chan struct{}), started: make(chan struct{}),
initialised: make(chan struct{}, 1), initialised: make(chan struct{}, 1),
networkReady: make(chan struct{}),
store: corroStore, store: corroStore,
cluster: c, cluster: c,
dockerService: dockerService,
localProxyServer: localProxyServer, localProxyServer: localProxyServer,
proxyDirector: proxyDirector, proxyDirector: proxyDirector,
} }
@@ -245,8 +257,10 @@ func NewMachine(config *Config) (*Machine, error) {
internalDNSIP := func() netip.Addr { internalDNSIP := func() netip.Addr {
return m.IP() return m.IP()
} }
m.docker = machinedocker.NewServer(dockerCli, db, internalDNSIP) m.dockerServer = machinedocker.NewServer(dockerService, db, internalDNSIP,
m.localMachineServer = newGRPCServer(m, c, m.docker) machinedocker.WithNetworkReady(m.IsNetworkReady),
machinedocker.WithWaitForNetworkReady(m.WaitForNetworkReady))
m.localMachineServer = newGRPCServer(m, c, m.dockerServer)
if m.Initialised() { if m.Initialised() {
m.initialised <- struct{}{} m.initialised <- struct{}{}
@@ -287,6 +301,9 @@ func (m *Machine) IP() netip.Addr {
} }
func (m *Machine) Run(ctx context.Context) error { func (m *Machine) Run(ctx context.Context) error {
// Create a cancellable context for the Run method to allow stopping the machine gracefully.
ctx, m.stop = context.WithCancel(ctx)
// Docker dependency is essential for the machine to function. Block until it's ready. // Docker dependency is essential for the machine to function. Block until it's ready.
if err := docker.WaitDaemonReady(ctx, m.config.DockerClient); err != nil { if err := docker.WaitDaemonReady(ctx, m.config.DockerClient); err != nil {
return fmt.Errorf("wait for Docker daemon: %w", err) return fmt.Errorf("wait for Docker daemon: %w", err)
@@ -294,7 +311,7 @@ func (m *Machine) Run(ctx context.Context) error {
// Configure and start the corrosion service on the loopback if the machine is not initialised as a cluster // Configure and start the corrosion service on the loopback if the machine is not initialised as a cluster
// member. This provides the store required for the machine to initialise a new cluster on it. Once the machine // member. This provides the store required for the machine to initialise a new cluster on it. Once the machine
// is initialised, the corrosion service is managed by the networkController. // is initialised, the corrosion service is managed by the clusterController.
if !m.Initialised() { if !m.Initialised() {
if err := m.configureCorrosion(); err != nil { if err := m.configureCorrosion(); err != nil {
return fmt.Errorf("configure corrosion service: %w", err) return fmt.Errorf("configure corrosion service: %w", err)
@@ -304,6 +321,7 @@ func (m *Machine) Run(ctx context.Context) error {
if err := m.config.CorrosionService.Start(ctx); err != nil { if err := m.config.CorrosionService.Start(ctx); err != nil {
return fmt.Errorf("start corrosion service: %w", err) return fmt.Errorf("start corrosion service: %w", err)
} }
slog.Info("Corrosion service started.")
} }
// Use an errgroup to coordinate error handling and graceful shutdown of multiple machine components. // Use an errgroup to coordinate error handling and graceful shutdown of multiple machine components.
@@ -314,147 +332,129 @@ func (m *Machine) Run(ctx context.Context) error {
if err != nil { if err != nil {
return fmt.Errorf("listen machine API unix socket %q: %w", m.config.MachineSockPath, err) return fmt.Errorf("listen machine API unix socket %q: %w", m.config.MachineSockPath, err)
} }
errGroup.Go( errGroup.Go(func() error {
func() error { slog.Info("Starting local machine API server.", "path", m.config.MachineSockPath)
slog.Info("Starting local machine API server.", "path", m.config.MachineSockPath) if err := m.localMachineServer.Serve(machineListener); err != nil {
if err := m.localMachineServer.Serve(machineListener); err != nil { return fmt.Errorf("local machine API server failed: %w", err)
return fmt.Errorf("local machine API server failed: %w", err) }
} return nil
return nil })
},
)
// Start the local API proxy server. // Start the local API proxy server.
proxyListener, err := listenUnixSocket(m.config.UncloudSockPath) proxyListener, err := listenUnixSocket(m.config.UncloudSockPath)
if err != nil { if err != nil {
return fmt.Errorf("listen API proxy unix socket %q: %w", m.config.UncloudSockPath, err) return fmt.Errorf("listen API proxy unix socket %q: %w", m.config.UncloudSockPath, err)
} }
errGroup.Go( errGroup.Go(func() error {
func() error { slog.Info("Starting local API proxy server.", "path", m.config.UncloudSockPath)
slog.Info("Starting local API proxy server.", "path", m.config.UncloudSockPath) if err := m.localProxyServer.Serve(proxyListener); err != nil {
if err := m.localProxyServer.Serve(proxyListener); err != nil { return fmt.Errorf("local API proxy server failed: %w", err)
return fmt.Errorf("local API proxy server failed: %w", err) }
} return nil
return nil })
},
)
// Signal that the machine is ready. // Signal that the machine is ready.
close(m.started) close(m.started)
// Control loop for managing components that depend on the machine being initialised as a cluster member. // Wait for the machine to be initialised as a member of a cluster and run the cluster controller.
errGroup.Go( errGroup.Go(func() error {
func() error { if !m.Initialised() {
if !m.Initialised() { slog.Info(
slog.Info( "Waiting for the machine to be initialised as a member of a cluster to start the cluster controller.",
"Waiting for the machine to be initialised as a member of a cluster " + )
"to start the network controller.", }
)
select {
case <-m.initialised:
m.cluster.UpdateMachineID(m.state.ID)
// Ensure the corrosion config is up to date, including a new gossip address if the machine
// has just joined a cluster.
if err := m.configureCorrosion(); err != nil {
return fmt.Errorf("configure corrosion service: %w", err)
}
slog.Info("Configured corrosion service.", "dir", m.config.CorrosionDir)
slog.Info("Starting cluster controller.")
// Update the proxy director's local address to the machine's management IP address, allowing
// the proxy to identify which requests should be proxied to the local machine API server.
m.proxyDirector.UpdateLocalAddress(m.state.Network.ManagementIP.String())
proxyServer := grpc.NewServer(
grpc.ForceServerCodecV2(proxy.Codec()),
grpc.UnknownServiceHandler(
proxy.TransparentHandler(m.proxyDirector.Director),
),
)
// Create a new caddyconfig controller for managing the Caddy reverse proxy configuration.
// It will also serve the current machine ID at /.uncloud-verify to verify Caddy reachability.
caddyconfigCtrl, err := caddyconfig.NewController(m.store, m.config.CaddyConfigDir, m.state.ID)
if err != nil {
return fmt.Errorf("create caddyconfig controller: %w", err)
} }
var ctrl *networkController dnsResolver := dns.NewClusterResolver(m.store)
// Error channel for communicating the termination of the network controller. dnsServer, err := dns.NewServer(m.IP(), dnsResolver, m.config.DNSUpstreams)
errCh := make(chan error) if err != nil {
return fmt.Errorf("create embedded DNS server: %w", err)
for {
select {
// Wait for the machine to be initialised as a member of a cluster to start the network controller.
// It can be reset when leaving the cluster and then re-initialised again with a new configuration.
case <-m.initialised:
var err error
m.cluster.UpdateMachineID(m.state.ID)
// Ensure the corrosion config is up to date, including a new gossip address if the machine
// has just joined a cluster.
if err = m.configureCorrosion(); err != nil {
return fmt.Errorf("configure corrosion service: %w", err)
}
slog.Info("Configured corrosion service.", "dir", m.config.CorrosionDir)
slog.Info("Starting network controller.")
// Update the proxy director's local address to the machine's management IP address, allowing
// the proxy to identify which requests should be proxied to the local machine API server.
m.proxyDirector.UpdateLocalAddress(m.state.Network.ManagementIP.String())
proxyServer := grpc.NewServer(
grpc.ForceServerCodecV2(proxy.Codec()),
grpc.UnknownServiceHandler(
proxy.TransparentHandler(m.proxyDirector.Director),
),
)
// Create a new Caddyfile controller for managing the Caddy reverse proxy configuration.
// It will also serve the current machine ID at /.uncloud-verify to verify Caddy reachability.
caddyfileCtrl, err := caddyconfig.NewController(m.store, m.config.CaddyConfigPath, m.state.ID)
if err != nil {
return fmt.Errorf("create Caddyfile controller: %w", err)
}
dnsResolver := dns.NewClusterResolver(m.store)
dnsServer, err := dns.NewServer(m.IP(), dnsResolver, m.config.DNSUpstreams)
if err != nil {
return fmt.Errorf("create embedded DNS server: %w", err)
}
ctrl, err = newNetworkController(
m.state,
m.store,
proxyServer,
m.config.CorrosionService,
m.config.DockerClient,
caddyfileCtrl,
dnsServer,
dnsResolver,
)
if err != nil {
return fmt.Errorf("initialise network controller: %w", err)
}
go func() {
if err = ctrl.Run(ctx); err != nil {
errCh <- fmt.Errorf("run network controller: %w", err)
} else {
slog.Info("Network controller stopped.")
errCh <- nil
}
}()
case err := <-errCh:
if err != nil {
return err
}
ctrl = nil
case <-ctx.Done():
// Wait for the network controller to stop before returning.
if ctrl != nil {
if err := <-errCh; err != nil {
return err
}
}
return nil
}
} }
},
) m.mu.Lock()
m.clusterCtrl, err = newClusterController(
m.state,
m.store,
proxyServer,
m.config.CorrosionService,
m.dockerService,
m.networkReady,
caddyconfigCtrl,
dnsServer,
dnsResolver,
)
m.mu.Unlock()
if err != nil {
return fmt.Errorf("initialise cluster controller: %w", err)
}
if err = m.clusterCtrl.Run(ctx); err != nil {
return fmt.Errorf("run cluster controller: %w", err)
}
slog.Info("Cluster controller stopped.")
case <-ctx.Done():
// The context was cancelled before the machine was initialised.
}
return nil
})
// Shutdown goroutine. // Shutdown goroutine.
errGroup.Go( errGroup.Go(func() error {
func() error { var err error
<-ctx.Done()
slog.Info("Stopping local machine API server.")
// TODO: implement timeout for graceful shutdown.
m.localMachineServer.GracefulStop()
slog.Info("Local machine API server stopped.")
slog.Info("Stopping local API proxy server.") <-ctx.Done()
// TODO: implement timeout for graceful shutdown. slog.Info("Stopping local machine API server.")
m.localProxyServer.GracefulStop() // TODO: implement timeout for graceful shutdown.
// Close the proxy director to close all backend connections. m.localMachineServer.GracefulStop()
m.proxyDirector.Close() slog.Info("Local machine API server stopped.")
slog.Info("Local API proxy server stopped.")
m.config.DockerClient.Close() slog.Info("Stopping local API proxy server.")
return nil // TODO: implement timeout for graceful shutdown.
}, m.localProxyServer.GracefulStop()
) // Close the proxy director to close all backend connections.
m.proxyDirector.Close()
slog.Info("Local API proxy server stopped.")
// Clean up the machine data and resources if the machine shutdown was initiated by a reset.
if m.resetting {
slog.Info("Cleaning up machine data and resources.")
if err = m.cleanup(); err != nil {
slog.Error("Failed to clean up machine data and resources.", "err", err)
}
}
m.config.DockerClient.Close()
return err
})
return errGroup.Wait() return errGroup.Wait()
} }
@@ -483,7 +483,7 @@ func listenUnixSocket(path string) (net.Listener, error) {
// Ensure the parent directory exists and has the correct group permissions. // Ensure the parent directory exists and has the correct group permissions.
parent, _ := filepath.Split(path) parent, _ := filepath.Split(path)
if err = os.MkdirAll(parent, 0750); err != nil { if err = os.MkdirAll(parent, 0o750); err != nil {
return nil, fmt.Errorf("create directory %q: %w", parent, err) return nil, fmt.Errorf("create directory %q: %w", parent, err)
} }
if err = os.Chown(parent, -1, gid); err != nil { if err = os.Chown(parent, -1, gid); err != nil {
@@ -537,15 +537,38 @@ func (m *Machine) configureCorrosion() error {
return fmt.Errorf("write corrosion config: %w", err) return fmt.Errorf("write corrosion config: %w", err)
} }
if err := os.WriteFile(schemaPath, []byte(store.Schema), 0644); err != nil { if err := os.WriteFile(schemaPath, []byte(store.Schema), 0o644); err != nil {
return fmt.Errorf("write corrosion schema: %w", err) return fmt.Errorf("write corrosion schema: %w", err)
} }
return nil return nil
} }
// cleanup removes the machine resources and persistent state.
func (m *Machine) cleanup() error {
var errs []error
m.mu.RLock()
clusterCtrl := m.clusterCtrl
m.mu.RUnlock()
if clusterCtrl != nil {
if err := clusterCtrl.Cleanup(); err != nil {
errs = append(errs, fmt.Errorf("cleanup cluster resources: %w", err))
}
}
if err := os.RemoveAll(m.config.DataDir); err != nil {
errs = append(errs,
fmt.Errorf("remove data directory with persistent machine state '%s': %w", m.config.DataDir, err))
} else {
slog.Info("Removed data directory storing persistent machine state.", "path", m.config.DataDir)
}
return errors.Join(errs...)
}
// CheckPrerequisites verifies if the machine meets all necessary system requirements to participate in the cluster. // CheckPrerequisites verifies if the machine meets all necessary system requirements to participate in the cluster.
func (m *Machine) CheckPrerequisites(ctx context.Context, _ *emptypb.Empty) (*pb.CheckPrerequisitesResponse, error) { func (m *Machine) CheckPrerequisites(_ context.Context, _ *emptypb.Empty) (*pb.CheckPrerequisitesResponse, error) {
// Check DNS port (UDP) availability. // Check DNS port (UDP) availability.
if err := checkDNSPortAvailable(); err != nil { if err := checkDNSPortAvailable(); err != nil {
return &pb.CheckPrerequisitesResponse{ return &pb.CheckPrerequisitesResponse{
@@ -776,20 +799,62 @@ func (m *Machine) Inspect(_ context.Context, _ *emptypb.Empty) (*pb.MachineInfo,
}, nil }, nil
} }
// Reset restores the machine to a clean state, removing all cluster-related сonfiguration and data and scheduling // IsNetworkReady returns true if the Docker network is ready for containers.
// a graceful shutdown. The uncloud daemon will restart the machine if managed by systemd. func (m *Machine) IsNetworkReady() bool {
func (m *Machine) Reset(ctx context.Context, _ *pb.ResetRequest) (*emptypb.Empty, error) { if !m.Initialised() {
// If machine is not initialized, there's no network to check
return false
}
// Check if network is ready by checking if the networkReady channel has been closed
select {
case <-m.networkReady:
return true
default:
return false
}
}
// WaitForNetworkReady waits for the Docker network to be ready for containers.
// It returns nil when the network is ready or an error if the context is cancelled.
func (m *Machine) WaitForNetworkReady(ctx context.Context) error {
if !m.Initialised() {
// If machine is not initialized, there's no network to wait for
return nil
}
// Wait for network to be ready or context to be cancelled
select {
case <-m.networkReady:
return nil
case <-ctx.Done():
return ctx.Err()
}
}
// Reset restores the machine to a clean state, scheduling a graceful shutdown and removing all cluster-related
// configuration and resource. The uncloud daemon will restart the machine if managed by systemd.
func (m *Machine) Reset(_ context.Context, _ *pb.ResetRequest) (*emptypb.Empty, error) {
if !m.Initialised() {
return nil, nil
}
// Check if the machine is already being reset to avoid concurrent resets.
m.mu.Lock()
if m.resetting {
m.mu.Unlock()
return nil, status.Error(codes.FailedPrecondition, "machine is already being reset")
}
m.resetting = true
m.mu.Unlock()
slog.Info("Resetting machine to a clean state.") slog.Info("Resetting machine to a clean state.")
// Trigger the machine shutdown. The resetting boolean informs the machine to clean up its resources on shutdown.
// We can't clean up the resources synchronously here because this is an RPC call that depends on the running
// gRPC server and network.
m.stop()
// TODO: stop and remove all managed service containers. return &emptypb.Empty{}, nil
// TODO: check if the request is coming from the unix or network socket. For the network socket, the reset should
// be called in a separate goroutine to avoid blocking the RPC response.
// TODO: stop the network controller
// TODO: implement and call Cleanup on the network controller to remove Docker network, WG interface, iptables
// rules, corrosion state, ?stop corrosion service.
// TODO: stop the machine and remove the machine.json state. The daemon should restart it to a clean state.
return &emptypb.Empty{}, status.Error(codes.Unimplemented, "reset machine is not implemented yet")
} }
// InspectService returns detailed information about a service and its containers stored in the cluster store. // InspectService returns detailed information about a service and its containers stored in the cluster store.
@@ -824,7 +889,7 @@ func (m *Machine) InspectService(
} }
} }
ctr := api.ServiceContainer{Container: records[0].Container} ctr := records[0].Container
svc := &pb.Service{ svc := &pb.Service{
Id: ctr.ServiceID(), Id: ctr.ServiceID(),
Name: ctr.ServiceName(), Name: ctr.ServiceName(),
+1 -1
View File
@@ -14,7 +14,7 @@ func MachineIP(subnet netip.Prefix) netip.Addr {
} }
// ManagementIP returns the IPv6 address of a peer derived from the first 14 bytes of its public key. // ManagementIP returns the IPv6 address of a peer derived from the first 14 bytes of its public key.
// This address is intended for cluster management traffic. // This address always starts with fdcc: and is intended for cluster management traffic.
func ManagementIP(publicKey secret.Secret) netip.Addr { func ManagementIP(publicKey secret.Secret) netip.Addr {
bytes := [16]byte{0xfd, 0xcc} bytes := [16]byte{0xfd, 0xcc}
copy(bytes[2:], publicKey[:14]) copy(bytes[2:], publicKey[:14])
+2 -1
View File
@@ -1,11 +1,12 @@
package network package network
import ( import (
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
"log/slog" "log/slog"
"net/netip" "net/netip"
"slices" "slices"
"time" "time"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
) )
const ( const (
+4 -3
View File
@@ -3,13 +3,14 @@ package tunnel
import ( import (
"context" "context"
"fmt" "fmt"
"golang.zx2c4.com/wireguard/conn"
"golang.zx2c4.com/wireguard/device"
"golang.zx2c4.com/wireguard/tun/netstack"
"net" "net"
"net/netip" "net/netip"
"time" "time"
"github.com/psviderski/uncloud/internal/secret" "github.com/psviderski/uncloud/internal/secret"
"golang.zx2c4.com/wireguard/conn"
"golang.zx2c4.com/wireguard/device"
"golang.zx2c4.com/wireguard/tun/netstack"
) )
const ( const (
+2 -1
View File
@@ -2,10 +2,11 @@ package network
import ( import (
"fmt" "fmt"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
"net/netip" "net/netip"
"time" "time"
"github.com/psviderski/uncloud/internal/secret" "github.com/psviderski/uncloud/internal/secret"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
) )
const ( const (
@@ -24,3 +24,7 @@ func (n *WireGuardNetwork) Run(ctx context.Context) error {
func (n *WireGuardNetwork) WatchEndpoints() <-chan EndpointChangeEvent { func (n *WireGuardNetwork) WatchEndpoints() <-chan EndpointChangeEvent {
return nil return nil
} }
func (n *WireGuardNetwork) Cleanup() error {
return errors.New("not implemented on darwin")
}
+53 -14
View File
@@ -6,18 +6,19 @@ import (
"context" "context"
"errors" "errors"
"fmt" "fmt"
"github.com/vishvananda/netlink"
"go4.org/netipx"
"golang.org/x/sys/unix"
"golang.zx2c4.com/wireguard/wgctrl"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
"log/slog" "log/slog"
"net" "net"
"net/netip" "net/netip"
"slices" "slices"
"sync" "sync"
"time" "time"
"github.com/psviderski/uncloud/internal/secret" "github.com/psviderski/uncloud/internal/secret"
"github.com/vishvananda/netlink"
"go4.org/netipx"
"golang.org/x/sys/unix"
"golang.zx2c4.com/wireguard/wgctrl"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
) )
type WireGuardNetwork struct { type WireGuardNetwork struct {
@@ -26,6 +27,8 @@ type WireGuardNetwork struct {
peers map[string]*peer peers map[string]*peer
// watchers is a list of channels that are notified when the endpoints of the peers change. // watchers is a list of channels that are notified when the endpoints of the peers change.
watchers []chan EndpointChangeEvent watchers []chan EndpointChangeEvent
// running indicates whether the network control loop (Run) is currently running.
running bool
// mu synchronises concurrent network configuration changes. // mu synchronises concurrent network configuration changes.
mu sync.Mutex mu sync.Mutex
} }
@@ -78,19 +81,15 @@ func (n *WireGuardNetwork) Configure(config Config) error {
} }
slog.Info("Configured WireGuard interface.", "name", n.link.Attrs().Name) slog.Info("Configured WireGuard interface.", "name", n.link.Attrs().Name)
machinePrefix := netip.PrefixFrom(MachineIP(config.Subnet), config.Subnet.Bits())
managementPrefix, err := addrToSingleIPPrefix(config.ManagementIP) managementPrefix, err := addrToSingleIPPrefix(config.ManagementIP)
if err != nil { if err != nil {
return fmt.Errorf("parse management IP: %w", err) return fmt.Errorf("parse management IP: %w", err)
} }
addrs := []netip.Prefix{managementPrefix, machinePrefix} addrs := []netip.Prefix{managementPrefix}
if err = n.updateAddresses(addrs); err != nil { if err = n.updateAddresses(addrs); err != nil {
return err return err
} }
slog.Info( slog.Info("Updated addresses of the WireGuard interface.", "name", n.link.Attrs().Name, "addrs", addrs)
"Updated addresses of the WireGuard interface.",
"name", n.link.Attrs().Name, "addrs", addrs,
)
// Bring the WireGuard interface up if it's not already up. // Bring the WireGuard interface up if it's not already up.
if n.link.Attrs().Flags&unix.IFF_UP != unix.IFF_UP { if n.link.Attrs().Flags&unix.IFF_UP != unix.IFF_UP {
@@ -99,7 +98,7 @@ func (n *WireGuardNetwork) Configure(config Config) error {
} }
slog.Info("Brought WireGuard interface up.", "name", n.link.Attrs().Name) slog.Info("Brought WireGuard interface up.", "name", n.link.Attrs().Name)
} }
if err = n.updatePeerRoutes(); err != nil { if err = n.updatePeerRoutes(MachineIP(config.Subnet)); err != nil {
return err return err
} }
slog.Info( slog.Info(
@@ -129,7 +128,7 @@ func (n *WireGuardNetwork) configureDevice(config Config) error {
// rotations and connection disruptions. // rotations and connection disruptions.
n.peers = make(map[string]*peer, len(config.Peers)) n.peers = make(map[string]*peer, len(config.Peers))
wgPeers := make(map[string]*wgtypes.Peer, len(dev.Peers)) wgPeers := make(map[string]*wgtypes.Peer, len(dev.Peers))
for i, _ := range dev.Peers { for i := range dev.Peers {
wgPeers[secret.Secret(dev.Peers[i].PublicKey[:]).String()] = &dev.Peers[i] wgPeers[secret.Secret(dev.Peers[i].PublicKey[:]).String()] = &dev.Peers[i]
} }
for _, pc := range config.Peers { for _, pc := range config.Peers {
@@ -202,7 +201,7 @@ func (n *WireGuardNetwork) updateAddresses(addrs []netip.Prefix) error {
// updatePeerRoutes adds routes to the peers via the WireGuard interface and removes old routes to peers // updatePeerRoutes adds routes to the peers via the WireGuard interface and removes old routes to peers
// that are no longer in the configuration. // that are no longer in the configuration.
func (n *WireGuardNetwork) updatePeerRoutes() error { func (n *WireGuardNetwork) updatePeerRoutes(machineIP netip.Addr) error {
// Build a set of compacted IP ranges for all peers. // Build a set of compacted IP ranges for all peers.
var ipsetBuilder netipx.IPSetBuilder var ipsetBuilder netipx.IPSetBuilder
for _, p := range n.peers { for _, p := range n.peers {
@@ -222,11 +221,18 @@ func (n *WireGuardNetwork) updatePeerRoutes() error {
// Add routes to the computed IP ranges via the WireGuard link. // Add routes to the computed IP ranges via the WireGuard link.
for _, prefix := range ipset.Prefixes() { for _, prefix := range ipset.Prefixes() {
dst := prefixToIPNet(prefix) dst := prefixToIPNet(prefix)
var src net.IP
// Use the machine IP as the source address for IPv4 routes to other peers.
if prefix.Addr().Is4() {
src = machineIP.AsSlice()
}
if err = netlink.RouteAdd( if err = netlink.RouteAdd(
&netlink.Route{ &netlink.Route{
LinkIndex: n.link.Attrs().Index, LinkIndex: n.link.Attrs().Index,
Scope: netlink.SCOPE_LINK, Scope: netlink.SCOPE_LINK,
Dst: &dst, Dst: &dst,
Src: src,
}, },
); err != nil { ); err != nil {
if !errors.Is(err, unix.EEXIST) { if !errors.Is(err, unix.EEXIST) {
@@ -272,6 +278,14 @@ func (n *WireGuardNetwork) Run(ctx context.Context) error {
} }
defer wg.Close() defer wg.Close()
n.mu.Lock()
if n.running {
n.mu.Unlock()
return errors.New("network is already running")
}
n.running = true
n.mu.Unlock()
ticker := time.NewTicker(1 * time.Second) ticker := time.NewTicker(1 * time.Second)
for { for {
select { select {
@@ -290,6 +304,11 @@ func (n *WireGuardNetwork) Run(ctx context.Context) error {
for _, ch := range n.watchers { for _, ch := range n.watchers {
close(ch) close(ch)
} }
n.mu.Lock()
n.running = false
n.mu.Unlock()
return nil return nil
} }
} }
@@ -427,3 +446,23 @@ func (n *WireGuardNetwork) notifyWatchers(ctx context.Context, events []Endpoint
} }
return nil return nil
} }
// Cleanup deletes the WireGuard link. The network must not be running when this method is called.
func (n *WireGuardNetwork) Cleanup() error {
n.mu.Lock()
defer n.mu.Unlock()
if n.running {
return errors.New("network is still running, stop it before cleanup")
}
// Delete the WireGuard link.
name := n.link.Attrs().Name
if err := netlink.LinkDel(n.link); err != nil {
return fmt.Errorf("delete WireGuard link %q: %w", name, err)
}
n.link = nil
slog.Info("Deleted WireGuard interface.", "name", name)
return nil
}
+3 -2
View File
@@ -6,6 +6,7 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"sync" "sync"
"github.com/psviderski/uncloud/internal/machine/network" "github.com/psviderski/uncloud/internal/machine/network"
) )
@@ -71,7 +72,7 @@ func (c *State) Save() error {
return fmt.Errorf("state path not set") return fmt.Errorf("state path not set")
} }
dir, _ := filepath.Split(c.path) dir, _ := filepath.Split(c.path)
if err := os.MkdirAll(dir, 0711); err != nil { if err := os.MkdirAll(dir, 0o711); err != nil {
return fmt.Errorf("create state directory %q: %w", dir, err) return fmt.Errorf("create state directory %q: %w", dir, err)
} }
@@ -79,5 +80,5 @@ func (c *State) Save() error {
if err != nil { if err != nil {
return err return err
} }
return os.WriteFile(c.path, data, 0600) return os.WriteFile(c.path, data, 0o600)
} }
+8 -5
View File
@@ -4,11 +4,12 @@ import (
"context" "context"
"encoding/json" "encoding/json"
"fmt" "fmt"
sq "github.com/Masterminds/squirrel"
"github.com/psviderski/uncloud/pkg/api"
"log/slog" "log/slog"
"strings" "strings"
"time" "time"
sq "github.com/Masterminds/squirrel"
"github.com/psviderski/uncloud/pkg/api"
) )
const ( const (
@@ -22,7 +23,7 @@ const (
) )
type ContainerRecord struct { type ContainerRecord struct {
Container api.Container Container api.ServiceContainer
MachineID string MachineID string
SyncStatus string SyncStatus string
UpdatedAt time.Time UpdatedAt time.Time
@@ -47,10 +48,12 @@ type DeleteOptions struct {
// CreateOrUpdateContainer creates a new container record or updates an existing one in the store database. // CreateOrUpdateContainer creates a new container record or updates an existing one in the store database.
// The container is associated with the given machine ID that indicates which machine the container is running on. // The container is associated with the given machine ID that indicates which machine the container is running on.
func (s *Store) CreateOrUpdateContainer(ctx context.Context, ctr api.Container, machineID string) error { func (s *Store) CreateOrUpdateContainer(ctx context.Context, ctr api.ServiceContainer, machineID string) error {
// Remove the environment variables from the container record before storing it in the database // Remove the environment variables from the container record before storing it in the database
// to avoid leaking secrets. // to avoid leaking secrets.
ctr.Config.Env = nil ctr.Config.Env = nil
ctr.ServiceSpec.Container.Env = nil
cJSON, err := json.Marshal(ctr) cJSON, err := json.Marshal(ctr)
if err != nil { if err != nil {
return fmt.Errorf("marshal container: %w", err) return fmt.Errorf("marshal container: %w", err)
@@ -117,7 +120,7 @@ func (s *Store) ListContainers(ctx context.Context, opts ListOptions) ([]Contain
return nil, fmt.Errorf("scan container record: %w", err) return nil, fmt.Errorf("scan container record: %w", err)
} }
var c api.Container var c api.ServiceContainer
if err = json.Unmarshal([]byte(cJSON), &c); err != nil { if err = json.Unmarshal([]byte(cJSON), &c); err != nil {
return nil, fmt.Errorf("unmarshal container: %w", err) return nil, fmt.Errorf("unmarshal container: %w", err)
} }
+90 -2
View File
@@ -5,17 +5,19 @@ import (
_ "embed" _ "embed"
"errors" "errors"
"fmt" "fmt"
"google.golang.org/protobuf/encoding/protojson"
"log/slog" "log/slog"
"github.com/psviderski/uncloud/internal/corrosion" "github.com/psviderski/uncloud/internal/corrosion"
"github.com/psviderski/uncloud/internal/machine/api/pb" "github.com/psviderski/uncloud/internal/machine/api/pb"
"google.golang.org/protobuf/encoding/protojson"
) )
var ( var (
//go:embed schema.sql //go:embed schema.sql
Schema string Schema string
ErrKeyNotFound = errors.New("key not found") ErrKeyNotFound = errors.New("key not found")
ErrMachineNotFound = errors.New("machine not found")
) )
// Store is a cluster store backed by a distributed Corrosion database. // Store is a cluster store backed by a distributed Corrosion database.
@@ -66,6 +68,53 @@ func (s *Store) CreateMachine(ctx context.Context, m *pb.MachineInfo) error {
return nil return nil
} }
func (s *Store) GetMachine(ctx context.Context, machineID string) (*pb.MachineInfo, error) {
if machineID == "" {
return nil, fmt.Errorf("machine ID cannot be empty")
}
rows, err := s.corro.QueryContext(ctx, "SELECT info FROM machines WHERE id = ?", machineID)
if err != nil {
return nil, fmt.Errorf("query machine: %w", err)
}
defer rows.Close()
if !rows.Next() {
if err := rows.Err(); err != nil {
return nil, fmt.Errorf("query error: %w", err)
}
return nil, fmt.Errorf("%w: %s", ErrMachineNotFound, machineID)
}
var mJSON string
if err = rows.Scan(&mJSON); err != nil {
return nil, fmt.Errorf("scan machine info: %w", err)
}
if mJSON == "" {
return nil, fmt.Errorf("machine info is empty for id %s", machineID)
}
protojsonParser := protojson.UnmarshalOptions{DiscardUnknown: true}
var m pb.MachineInfo
if err = protojsonParser.Unmarshal([]byte(mJSON), &m); err != nil {
return nil, fmt.Errorf("unmarshal machine info for id %s: %w", machineID, err)
}
// Validate the unmarshaled data. just in case
if m.Id != machineID {
return nil, fmt.Errorf("machine ID mismatch: expected %s, got %s", machineID, m.Id)
}
if m.Network != nil {
if err = m.Network.Validate(); err != nil {
return nil, fmt.Errorf("invalid network configuration for machine %s: %w", m.Id, err)
}
}
return &m, nil
}
func (s *Store) ListMachines(ctx context.Context) ([]*pb.MachineInfo, error) { func (s *Store) ListMachines(ctx context.Context) ([]*pb.MachineInfo, error) {
rows, err := s.corro.QueryContext(ctx, "SELECT info FROM machines ORDER BY name") rows, err := s.corro.QueryContext(ctx, "SELECT info FROM machines ORDER BY name")
if err != nil { if err != nil {
@@ -95,6 +144,45 @@ func (s *Store) ListMachines(ctx context.Context) ([]*pb.MachineInfo, error) {
return machines, nil return machines, nil
} }
func (s *Store) UpdateMachine(ctx context.Context, m *pb.MachineInfo) error {
if m == nil {
return fmt.Errorf("machine info cannot be nil")
}
if m.Id == "" {
return fmt.Errorf("machine ID cannot be empty")
}
mJSON, err := protojson.Marshal(m)
if err != nil {
return fmt.Errorf("marshal machine info: %w", err)
}
result, err := s.corro.ExecContext(ctx, "UPDATE machines SET info = ? WHERE id = ?", string(mJSON), m.Id)
if err != nil {
return fmt.Errorf("update machine: %w", err)
}
// Check if machine exists
if result.RowsAffected == 0 {
return fmt.Errorf("%w: %s", ErrMachineNotFound, m.Id)
}
return nil
}
func (s *Store) DeleteMachine(ctx context.Context, id string) error {
result, err := s.corro.ExecContext(ctx, "DELETE FROM machines WHERE id = ?", id)
if err != nil {
return fmt.Errorf("delete machine: %w", err)
}
// Check if machine was deleted.
if result.RowsAffected == 0 {
return fmt.Errorf("%w: %s", ErrMachineNotFound, id)
}
return nil
}
// SubscribeMachines returns a list of machines and a channel that signals changes to the list. The channel doesn't // SubscribeMachines returns a list of machines and a channel that signals changes to the list. The channel doesn't
// receive any values, it just signals when a machine has been added, updated, or deleted in the database. // receive any values, it just signals when a machine has been added, updated, or deleted in the database.
func (s *Store) SubscribeMachines(ctx context.Context) ([]*pb.MachineInfo, <-chan struct{}, error) { func (s *Store) SubscribeMachines(ctx context.Context) ([]*pb.MachineInfo, <-chan struct{}, error) {
+1
View File
@@ -6,6 +6,7 @@ import (
"fmt" "fmt"
"net/netip" "net/netip"
"strings" "strings"
"github.com/psviderski/uncloud/internal/secret" "github.com/psviderski/uncloud/internal/secret"
) )
+2 -1
View File
@@ -3,9 +3,10 @@ package sshexec
import ( import (
"context" "context"
"fmt" "fmt"
"golang.org/x/crypto/ssh"
"io" "io"
"strings" "strings"
"golang.org/x/crypto/ssh"
) )
type Remote struct { type Remote struct {
+1
View File
@@ -2,6 +2,7 @@ package ucind
import ( import (
"errors" "errors"
"github.com/docker/docker/client" "github.com/docker/docker/client"
) )
View File
+2 -1
View File
@@ -20,7 +20,7 @@ Uncloud stores its configuration in `~/.config/uncloud/config.yaml`. If you wish
### Initialisation ### Initialisation
Begin by initialising the first node in your cluster with `uc machine init [USER@HOST:PORT]`. If you do not have a need for Caddy reverse proxy, you may disable this feature with `--no-caddy`. Begin by initialising the first node in your cluster with `uc machine init [USER@HOST:PORT]`. If you do not have a need for Caddy reverse proxy, you may disable this feature with `--no-caddy`. If you want to avoid using uncloud's managed DNS service, add the `--no-dns` flag.
This command will idempotently install Docker, uncloudd, uncloud-corrosion. If Caddy is enabled, it will set up a reverse proxy. If Uncloud DNS is enabled, it will create a DNS A record for the machine's public IP address under `*.[CLUSTER ID].cluster.uncloud.run`. This command will idempotently install Docker, uncloudd, uncloud-corrosion. If Caddy is enabled, it will set up a reverse proxy. If Uncloud DNS is enabled, it will create a DNS A record for the machine's public IP address under `*.[CLUSTER ID].cluster.uncloud.run`.
@@ -37,6 +37,7 @@ Uncloud (uncloud.run) DNS can be managed with the `uc dns` subcommand.
* To reserve a domain name, run `uc dns reserve` * To reserve a domain name, run `uc dns reserve`
* To release a domain name, run `uc dns release`. * To release a domain name, run `uc dns release`.
* To see the domain name, run `uc dns show` * To see the domain name, run `uc dns show`
* To avoid using the Uncloud managed DNS service, use the `--no-dns` flag on your `uc machine init` command.
### Running a service ### Running a service
+21
View File
@@ -0,0 +1,21 @@
package api
import "strings"
// CaddySpec is the Caddy reverse proxy configuration for a service.
type CaddySpec struct {
// Config contains the Caddy config (Caddyfile) content. It must not conflict with the Caddy configs
// of other services.
Config string
}
func (c *CaddySpec) Equals(other *CaddySpec) bool {
if c == nil {
return other == nil || strings.TrimSpace(other.Config) == ""
}
if other == nil {
return strings.TrimSpace(c.Config) == ""
}
return strings.TrimSpace(c.Config) == strings.TrimSpace(other.Config)
}
+2
View File
@@ -42,6 +42,8 @@ type ImageClient interface {
type MachineClient interface { type MachineClient interface {
InspectMachine(ctx context.Context, id string) (*pb.MachineMember, error) InspectMachine(ctx context.Context, id string) (*pb.MachineMember, error)
ListMachines(ctx context.Context, filter *MachineFilter) (MachineMembersList, error) ListMachines(ctx context.Context, filter *MachineFilter) (MachineMembersList, error)
UpdateMachine(ctx context.Context, req *pb.UpdateMachineRequest) (*pb.MachineInfo, error)
RenameMachine(ctx context.Context, nameOrID, newName string) (*pb.MachineInfo, error)
} }
type ServiceClient interface { type ServiceClient interface {
+3 -2
View File
@@ -1,10 +1,11 @@
package api package api
import ( import (
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"net/netip" "net/netip"
"testing" "testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
) )
func TestPortSpec_Validate(t *testing.T) { func TestPortSpec_Validate(t *testing.T) {
+20 -2
View File
@@ -6,6 +6,7 @@ import (
"maps" "maps"
"regexp" "regexp"
"slices" "slices"
"strings"
"github.com/distribution/reference" "github.com/distribution/reference"
"github.com/google/go-cmp/cmp" "github.com/google/go-cmp/cmp"
@@ -42,6 +43,10 @@ func ValidateServiceID(id string) bool {
// ServiceSpec defines the desired state of a service. // ServiceSpec defines the desired state of a service.
// ATTENTION: after changing this struct, verify if deploy.EvalContainerSpecChange needs to be updated. // ATTENTION: after changing this struct, verify if deploy.EvalContainerSpecChange needs to be updated.
type ServiceSpec struct { type ServiceSpec struct {
// Caddy is the optional Caddy reverse proxy configuration for the service.
// Caddy and Ports cannot be specified simultaneously.
Caddy *CaddySpec `json:",omitempty"`
// Container defines the desired state of each container in the service.
Container ContainerSpec Container ContainerSpec
// Mode is the replication mode of the service. Default is ServiceModeReplicated if empty. // Mode is the replication mode of the service. Default is ServiceModeReplicated if empty.
Mode string Mode string
@@ -49,6 +54,7 @@ type ServiceSpec struct {
// Placement defines the placement constraints for the service. // Placement defines the placement constraints for the service.
Placement Placement Placement Placement
// Ports defines what service ports to publish to make the service accessible outside the cluster. // Ports defines what service ports to publish to make the service accessible outside the cluster.
// Caddy and Ports cannot be specified simultaneously.
Ports []PortSpec Ports []PortSpec
// Replicas is the number of containers to run for the service. Only valid for a replicated service. // Replicas is the number of containers to run for the service. Only valid for a replicated service.
Replicas uint `json:",omitempty"` Replicas uint `json:",omitempty"`
@@ -112,10 +118,17 @@ func (s *ServiceSpec) Validate() error {
return fmt.Errorf("service name too long (max 63 characters): %q", s.Name) return fmt.Errorf("service name too long (max 63 characters): %q", s.Name)
} }
if !dnsLabelRegexp.MatchString(s.Name) { if !dnsLabelRegexp.MatchString(s.Name) {
return fmt.Errorf("invalid service name: %q. must be 1-63 characters, lowercase letters, numbers, and dashes only; must start and end with a letter or number", s.Name) return fmt.Errorf("invalid service name: %q. must be 1-63 characters, lowercase letters, numbers, "+
"and dashes only; must start and end with a letter or number", s.Name)
} }
} }
// Validate that Caddy and Ports are not used together.
if s.Caddy != nil && strings.TrimSpace(s.Caddy.Config) != "" && len(s.Ports) > 0 {
return fmt.Errorf("ports and Caddy configuration cannot be specified simultaneously: " +
"Caddy config is auto-generated from ports, use only one of them")
}
for _, p := range s.Ports { for _, p := range s.Ports {
if (p.Mode == "" || p.Mode == PortModeIngress) && if (p.Mode == "" || p.Mode == PortModeIngress) &&
p.Protocol != ProtocolHTTP && p.Protocol != ProtocolHTTPS { p.Protocol != ProtocolHTTP && p.Protocol != ProtocolHTTPS {
@@ -151,11 +164,16 @@ func (s *ServiceSpec) Validate() error {
func (s *ServiceSpec) Clone() ServiceSpec { func (s *ServiceSpec) Clone() ServiceSpec {
spec := *s spec := *s
if s.Caddy != nil {
caddyCopy := *s.Caddy
spec.Caddy = &caddyCopy
}
spec.Container = s.Container.Clone()
if s.Ports != nil { if s.Ports != nil {
spec.Ports = make([]PortSpec, len(s.Ports)) spec.Ports = make([]PortSpec, len(s.Ports))
copy(spec.Ports, s.Ports) copy(spec.Ports, s.Ports)
} }
spec.Container = s.Container.Clone()
if s.Volumes != nil { if s.Volumes != nil {
spec.Volumes = make([]VolumeSpec, len(s.Volumes)) spec.Volumes = make([]VolumeSpec, len(s.Volumes))

Some files were not shown because too many files have changed in this diff Show More