Compare commits

...
77 Commits
Author SHA1 Message Date
Pasha Sviderski aef4b2213f docs: rewrite Rolling deployments doc from Deployment strategies 2026-03-01 19:51:47 +10:00
Pasha Sviderski 44013da332 chore: format AGENTS, instruct to not use em dashes 2026-03-01 19:38:23 +10:00
Pasha Sviderski b48ebe2acc chore: UNCLOUD_HEALTH_MONITOR_PERIOD accepts duration (10s, 500ms, 0) 2026-03-01 08:08:57 +10:00
Pasha Sviderski eaa4455320 docs: generate CLI reference docs 2026-02-28 16:29:05 +10:00
Pasha Sviderski ce3cb8a34f feat: add --skip-health flag to bypass health monitoring during container deployment 2026-02-28 16:26:48 +10:00
Pasha Sviderski e6705599b3 chore: rename env var UNCLOUD_DEFAULT_HEALTH_MONITOR_PERIOD_MS -> UNCLOUD_HEALTH_MONITOR_PERIOD_MS 2026-02-28 16:03:36 +10:00
Pasha Sviderski 7b88040c73 fix: rolling container replacement: if new container fails, rollback and run old container only if it was running 2026-02-28 14:29:48 +10:00
Pasha Sviderski 7a9eac593f feat: implement container health monitoring and rollback during rolling deployment (closes #24) 2026-02-27 21:36:40 +10:00
Pasha Sviderski 17c1bb5c21 feat: add WaitContainerHealthy client method and health check utilities 2026-02-27 21:28:11 +10:00
Pasha Sviderski a5a9ef5444 chore: init container healthcheck const 2026-02-26 19:39:49 +10:00
Pasha Sviderski 2940965e31 fix: recreate container on ulimit changes, add test 2026-02-25 16:38:44 +10:00
Pasha Sviderski 185cf19a0c chore: minor full-spec test change 2026-02-25 16:03:28 +10:00
Pasha Sviderski 4443fa9e0e chore: update full-spec test to include latest implemented Compose attributes 2026-02-25 16:01:31 +10:00
Pasha Sviderski 3056e2af11 feat: add support for healthcheck in Compose 2026-02-25 14:54:24 +10:00
Pasha Sviderski 7bfaf31138 fix: image tag template formatting in docs 2026-02-24 16:42:55 +10:00
Pasha Sviderski f8d1945544 chore: add TODO for ReplaceContainerOperation 2026-02-24 15:30:23 +10:00
Pasha Sviderski 93d7f29d31 chore: remove TODO about encapsulating Client in deploy operations 2026-02-24 13:17:59 +10:00
Pasha Sviderski 18ebd29032 chore: split deploy operations into separate files in operation package 2026-02-24 11:42:47 +10:00
Pasha Sviderski fe56d6d708 chore: always run proxy to virtualised Docker on macOS for 'image push' except OrbStack 2026-02-23 20:06:23 +10:00
14056cc584 fix: format IPv6 addr correclty for HTTP URL (#254)
* fix: format IPv6 addr correclty for HTTP URL

* test: cover changes

* test: refactor

---------

Co-authored-by: gm0stache <gm0stache@noreply.codeberg.org>
2026-02-16 21:19:52 +01:00
Dimitar DimitrovandGitHub d3ee7399eb fix: support image push when using Rancher Desktop locally (#251)
Rancher Desktop is a free alternative to Docker Desktop (which prohibits
commercial use) and has gained significant popularity. When running
Rancher Desktop on non-native Docker platforms such as macOS and
Windows, the secondary tunnel from the host OS to the Docker's Linux
machine is necessary. At the moment, the check for this seems to be
based on the vhost name of the docker engine's VM.

For Rancher Desktop it seemms to be `lima-rancher-desktop`:

  $ docker info --format 'json' | jq .Name

This might not be the best approach but is simple and should save a lot
of headaches for a lot of people until a potentially better solution is
devised.

See also: https://github.com/psviderski/unregistry/issues/62
2026-02-15 23:04:57 +01:00
Pasha Sviderski 5c54e9d059 chore: refactor device mapping to be compliant with Compose (CDI requests) 2026-02-11 17:44:38 +00:00
Justin BradfordandGitHub ab6f856987 feat: Add support for compose devices mappings (#250)
* feat: Add support for compose `devices` mappings

https://docs.docker.com/reference/compose-file/services/#devices
```
services:
  foo:
    devices:
      - "/dev/ttyUSB0:/dev/ttyUSB0"
      - "/dev/sda:/dev/xvda:rwm"
```

* Lint fix
2026-02-11 17:00:18 +00:00
Pasha Sviderski 6e59657fe2 chore: global reconciliation: stop all running conflicting containers, not only replace the first one 2026-02-09 22:29:25 +00:00
Pasha Sviderski 215f21439d chore: lint and fix e2e deploy tests with ReplaceContainerOperation 2026-02-09 09:53:51 +00:00
NickandGitHub 53bf446502 deploy.update_config.order support to start-first or stop-first when replacing containers (#248) 2026-02-09 09:31:19 +00:00
Pasha Sviderski 2588279e52 chore: rename AI.md to AGENTS.md 2026-02-03 18:32:12 +01:00
Pasha Sviderski f48dc250a9 chore: fix landing navbar github/discord buttons wrap on mobile 2026-02-03 17:23:38 +01:00
Zasda Yusuf MikailandGitHub 15d9ceb4d1 feat: auto-provision volumes on all machines for global services (#243)
* feat: auto-provision volumes on all machines for global services

* fix: schedule global service volumes only on machines that need them

* fix: use union of eligible machines for volumes shared by global services

* test: restore global with missing volume fails test

* refactor: simplify volume scheduler by removing redundant check

Move isVolumeSharedBetweenGlobalAndReplicated check earlier to fail
fast, then use isVolumeForGlobalService instead of the now-redundant
isVolumeOnlyForGlobalServices function.
2026-02-02 12:30:57 +01:00
Pasha Sviderski a4a8e70f9c fix: sshcli tests 2026-01-28 19:00:53 +10:00
Pasha Sviderski 1d4dbd96ce chore: minor sshcli refactgor 2026-01-28 18:17:55 +10:00
Pasha Sviderski 476a2d57ca fix: reset already initialised machine on 'uc machine init' when using ssh+cli connection 2026-01-28 18:08:57 +10:00
Pasha Sviderski ce46a67d6d fix: ssh args for machine init/add when using ssh+cli 2026-01-28 17:44:54 +10:00
Pasha Sviderski 87e49dfb62 chore: label temporary socat proxy containers created by 'uc image push' with uncloud.managed 2026-01-28 17:40:15 +10:00
Pasha Sviderski 808bda67d4 chore: remove unused constants 2026-01-28 17:28:35 +10:00
Pasha Sviderski c91a964513 chore: make ssh+cli connections reuse one SSH connection via control socket. Fix image push 2026-01-28 16:49:08 +10:00
Pasha Sviderski e0a63a3f49 chore: typo 2026-01-27 11:32:18 +10:00
Pasha Sviderski 0b2e1bb7a2 chore: update newsletter form to substack 2026-01-26 21:10:40 +10:00
Pasha Sviderski 7c4f739d07 chore: specify machines to deploy website to 2026-01-26 18:03:46 +10:00
NickandGitHub 3bd9400051 feat: add support for ulimits in compose and container creation (#221) (#244)
* feat: add support for ulimits in compose and container creation ([#221](https://github.com/ipaddicting/uncloud/issues/221))
* refactor: move Ulimits to ContainerResources and use map, also removed GEMINI.md.
2026-01-20 16:17:54 +10:00
Pasha Sviderski 1caa5ccf0e docs: generate CLI reference docs for wg commands 2026-01-10 18:37:40 +11:00
Pasha Sviderski 76affeb6b2 chore: minor formatting for 'wg show' command, enrich Unimplemented error with >=0.16 requirement 2026-01-10 18:36:56 +11:00
Justin BradfordandGitHub a945291371 feat: Add uc wg show command to inspect a machine's uncloud wireguard network (#161)
* feat: Add `uc wg show` command to inspect a machine's uncloud wireguard network (draft / work in progress)

* Compile proto files with `make proto-mise`

* Fix lint errors

* Output peers in tabwriter table

* Lookup machine name for wireguard peer public key

* Add --machine flag to `uc wg show` to proxy call to specific machine

* Small refactor to move wg show logic out into a separate function
2026-01-10 16:49:35 +11:00
06c3fba96e feat: add support for sysctls compose key (#239)
* feat: add support for `sysctls` compose key

* lint

---------

Co-authored-by: Pasha Sviderski <me@psviderski.name>
2026-01-10 15:53:56 +11:00
Pasha Sviderski 7e739dd2f0 docs: update compose matrix 'build' supported, link capabilities man page 2026-01-10 15:27:42 +11:00
Andrey ViktorovandGitHub 6c2f85d38a feat: add support for cap_add and cap_drop compose keys (#238) 2026-01-10 15:13:03 +11:00
Pasha Sviderski 1ea1d15845 docs: update README subtitle 2026-01-04 16:22:46 +10:00
Anton OvchinnikovandGitHub 5e0d4d0f05 build: Generate CLI docs in GHA (#236)
* build: Generate CLI docs in GHA
* fix: Move no-tty logic outside cobra definitions
2026-01-02 10:23:42 +10:00
Anton OvchinnikovandGitHub 9a1d2e0d46 fix: Equals and Clone for configs (#235) 2025-12-30 20:32:19 +10:00
Anton Ovchinnikov 878f96a4b7 Revert "build: Generate CLI docs in GHA"
This reverts commit 9efaab2beb.
2025-12-30 01:51:31 +01:00
Anton Ovchinnikov 9efaab2beb build: Generate CLI docs in GHA 2025-12-30 01:22:56 +01:00
Anton OvchinnikovandGitHub 2c34ba7eff fix(configs): Create non-existent parent directories automatically (#233) 2025-12-29 11:38:06 +01:00
Anton Ovchinnikov 57e2e22266 docs: Add missing docs for uc start/stop 2025-12-28 17:01:01 +01:00
Anton OvchinnikovandGitHub 4ec5c15254 docs: Link to iximiuz Labs tutorials (#230) 2025-12-27 15:33:12 +10:00
Anton Ovchinnikov 33a3683196 chore: Regenerate CLI docs 2025-12-26 12:23:59 +01:00
Pasha Sviderski 4be1ea3506 fix: machine name for failed machines in 'uc ps' 2025-12-23 19:49:21 +10:00
Pasha Sviderski f86e63c851 chore: reduce the default gRPC connection timeout to proxied machines 20->10s 2025-12-23 19:48:57 +10:00
Pasha Sviderski f6cf73190f chore: wait for FailedPrecondition in addition to Unavailable for backward compatibility 2025-12-23 19:48:18 +10:00
Pasha Sviderski 5308c87651 chore: update 'uc machine init/add' to wait for cluster readiness, confirm caddy deployment on added machine 2025-12-23 19:47:26 +10:00
Pasha Sviderski 9bd5ffcd2d chore: update ucind cluster to wait for initial store sync and cluster readiness 2025-12-23 19:02:04 +10:00
Pasha Sviderski 8d8acd5410 chore: wait for the initial store sync and cluster components before serving cluster gRPC requests 2025-12-23 19:00:09 +10:00
Pasha Sviderski 4d97c30a9c chore: enable gRPC auto retries for transient Unavailable failures up to ~8s 2025-12-23 18:54:08 +10:00
Pasha Sviderski aa71ab0220 fix: e2e caddy test after changing the config header 2025-12-22 21:22:56 +10:00
Pasha Sviderski 2345ee9856 docs: improve Caddyfile header and note about misconfigured caddy service, minor docs update 2025-12-22 16:31:15 +10:00
Justin BradfordandGitHub 65e9cdd444 refactor: use WaitGroup.Go to replace wg.Add(1)/go/wg.Done() boilerplate (#222)
https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/modernize#hdr-Analyzer_waitgroup
2025-12-19 21:07:29 +10:00
Pasha Sviderski 85a2615db1 chore: add 'ctx conn' alias for 'ctx connection' command 2025-12-19 20:53:30 +10:00
Pasha Sviderski 777615bc56 chore: add TODO to remove extra checks after upgrading clusters to perform store sync 2025-12-19 20:52:46 +10:00
Pasha Sviderski bac34ebd0c chore: save machine state file atomically 2025-12-19 20:12:46 +10:00
Pasha Sviderski b2322e1c49 fix: WireGuard peers misconfiguration on joined machine by waiting for the initial store sync 2025-12-19 19:52:13 +10:00
Pasha Sviderski 436671457d chore: pass current store DB version when adding new machine to cluster 2025-12-19 18:51:49 +10:00
Pasha Sviderski fc847093d2 chore: new InspectMachine gRPC method that to return store DB version (lamport time) 2025-12-19 16:29:14 +10:00
Pasha Sviderski d2a7af744d fix: uncloud daemon crash when listing partially replicated container records 2025-12-18 16:32:26 +10:00
Pasha Sviderski 5acf557675 chore: add --yes flag to 'uc machine init/add' to auto-confirm machine reset 2025-12-17 17:36:54 +10:00
Pasha Sviderski 0b32cd2095 chore: upgrade unregistry 0.3.1 -> 0.4.1 with a fix for pushing big images 2025-12-16 20:08:40 +10:00
Pasha Sviderski 425b974331 fix: panic when initialising new cluster with --connect 2025-12-16 11:28:11 +10:00
Pasha Sviderski db09dec855 chore: add more tests for MatchesDockerVolume and update error message when volume spec doesn't match 2025-12-15 18:12:12 +10:00
Justin BradfordandGitHub 20cdcdf917 fix: Set volume driver name to "local" if not specified (#219) 2025-12-15 17:50:36 +10:00
113 changed files with 5997 additions and 1236 deletions
+6
View File
@@ -36,6 +36,12 @@ jobs:
git diff --exit-code || git diff --exit-code ||
(echo "go.mod or go.sum has changed. Please run 'go mod tidy' and commit the changes." && exit 1) (echo "go.mod or go.sum has changed. Please run 'go mod tidy' and commit the changes." && exit 1)
- name: Generate docs
run: |
make cli-docs
git diff --exit-code ||
(echo "Documentation is out of date. Please run 'make cli-docs' and commit the changes." && exit 1)
- name: Run tests - name: Run tests
run: | run: |
make ucind-image make ucind-image
+3
View File
@@ -7,6 +7,8 @@
*.dll *.dll
*.so *.so
*.dylib *.dylib
/uncloud
/uncloudd
# OS X # OS X
.DS_Store .DS_Store
@@ -37,3 +39,4 @@ node_modules/
# VS Code # VS Code
.vscode .vscode
.devcontainer/
+38 -31
View File
@@ -1,10 +1,14 @@
# AI.md - Uncloud Project Guide # AGENTS.md - Uncloud Project Guide
This document provides comprehensive information about the Uncloud project for AI assistants to understand the codebase, architecture, and development practices. This document provides comprehensive information about the Uncloud project for AI assistants to understand the codebase,
architecture, and development practices.
## Project Overview ## Project Overview
**Uncloud** is a lightweight clustering and container orchestration tool that enables deployment and management of web applications across cloud VMs and bare metal servers. It creates a secure WireGuard mesh network between Docker hosts and provides automatic service discovery, load balancing, HTTPS ingress, and simple CLI commands for application management. **Uncloud** is a lightweight clustering and container orchestration tool that enables deployment and management of web
applications across cloud VMs and bare metal servers. It creates a secure WireGuard mesh network between Docker hosts
and provides automatic service discovery, load balancing, HTTPS ingress, and simple CLI commands for application
management.
### Key Characteristics ### Key Characteristics
@@ -68,29 +72,29 @@ This document provides comprehensive information about the Uncloud project for A
- **`cmd/`**: Contains main applications - **`cmd/`**: Contains main applications
- `uncloud/`: CLI tool with subcommands for machine, service, volume management - `uncloud/`: CLI tool with subcommands for machine, service, volume management
- `uncloudd/`: Daemon that runs on each machine - `uncloudd/`: Daemon that runs on each machine
- `ucind/`: Development cluster management for testing - `ucind/`: Development cluster management for testing
- **`internal/`**: Internal implementation packages - **`internal/`**: Internal implementation packages
- `cli/`: Command-line interface logic - `cli/`: Command-line interface logic
- `machine/`: Machine lifecycle and state management - `machine/`: Machine lifecycle and state management
- `daemon/`: Daemon implementation and gRPC services - `daemon/`: Daemon implementation and gRPC services
- `dns/`: Internal DNS server for service discovery - `dns/`: Internal DNS server for service discovery
- **`pkg/`**: Public API packages for external use - **`pkg/`**: Public API packages for external use
- `api/`: Core API types and definitions - `api/`: Core API types and definitions
- `client/`: Client libraries for interacting with Uncloud - `client/`: Client libraries for interacting with Uncloud
- **`experiment/`**: Experimental features and prototypes - **`experiment/`**: Experimental features and prototypes
- **`scripts/`**: Installation and utility scripts - **`scripts/`**: Installation and utility scripts
- **`test/`**: Test suites and test infrastructure - **`test/`**: Test suites and test infrastructure
- **`website/`**: Documentation website (Docusaurus) - **`website/`**: Documentation website (Docusaurus)
- `landing/`: Landing page - `landing/`: Landing page
- `docs/`: User documentation - `docs/`: User documentation
- **`misc/`**: Design documents and guides - **`misc/`**: Design documents and guides
@@ -100,25 +104,25 @@ This document provides comprehensive information about the Uncloud project for A
```go ```go
// Networking and orchestration // Networking and orchestration
github.com/docker/docker // Docker API client github.com/docker/docker // Docker API client
github.com/docker/compose/v2 // Docker Compose integration github.com/docker/compose/v2 // Docker Compose integration
golang.zx2c4.com/wireguard // WireGuard implementation golang.zx2c4.com/wireguard // WireGuard implementation
github.com/hashicorp/serf // Gossip protocol github.com/hashicorp/serf // Gossip protocol
// State management // State management
github.com/ipfs/go-ds-crdt // CRDT distributed storage github.com/ipfs/go-ds-crdt // CRDT distributed storage
github.com/dgraph-io/badger/v3 // Embedded database github.com/dgraph-io/badger/v3 // Embedded database
// Web proxy // Web proxy
github.com/caddyserver/caddy/v2 // HTTP server and reverse proxy github.com/caddyserver/caddy/v2 // HTTP server and reverse proxy
// CLI and UX // CLI and UX
github.com/spf13/cobra // CLI framework github.com/spf13/cobra // CLI framework
github.com/charmbracelet/huh // Interactive forms github.com/charmbracelet/huh // Interactive forms
// gRPC and networking // gRPC and networking
google.golang.org/grpc // gRPC framework google.golang.org/grpc // gRPC framework
github.com/siderolabs/grpc-proxy // gRPC proxy for forwarding github.com/siderolabs/grpc-proxy // gRPC proxy for forwarding
``` ```
## Development Workflow ## Development Workflow
@@ -189,9 +193,9 @@ uc context use <name> # Switch context
### Testing ### Testing
- Test files and locations - Test files and locations
- Unit tests alongside source files (`*_test.go`) - Unit tests alongside source files (`*_test.go`)
- Integration tests in `test/e2e/` - Integration tests in `test/e2e/`
- Test fixtures in `test/fixtures/` - Test fixtures in `test/fixtures/`
- Use table driven tests whenever possible - Use table driven tests whenever possible
### Dependencies ### Dependencies
@@ -225,8 +229,8 @@ uc context use <name> # Switch context
### Logs and Monitoring ### Logs and Monitoring
- Systemd services (getting logs via `journalctl -u SERVICE_NAME`) - Systemd services (getting logs via `journalctl -u SERVICE_NAME`)
- `uncloud` -- Uncloud daemon - `uncloud` -- Uncloud daemon
- `uncloud-corrosion` -- Corrosion process - `uncloud-corrosion` -- Corrosion process
- Machine daemon logs - Machine daemon logs
- Container logs via Docker - Container logs via Docker
@@ -247,7 +251,8 @@ uc context use <name> # Switch context
- `Makefile`: Build and development tasks - `Makefile`: Build and development tasks
- `Dockerfile`: Container build instructions forUncloud-in-Docker (used for testing) - `Dockerfile`: Container build instructions forUncloud-in-Docker (used for testing)
This document should help AI assistants understand the project structure, make informed suggestions, and contribute effectively to the Uncloud codebase. This document should help AI assistants understand the project structure, make informed suggestions, and contribute
effectively to the Uncloud codebase.
## Documentation ## Documentation
@@ -255,4 +260,6 @@ Instructions when generating documentation:
- Use conversational language write as if you were speaking to a friend. - Use conversational language write as if you were speaking to a friend.
- Keep sentences simple, optimize for clarity and understanding. - Keep sentences simple, optimize for clarity and understanding.
- Place the subject before the action whenever possible. Example: prefer “The function loads data” over “Data is loaded by the function.” - Do not use em dashes (—) or semicolons (;) in sentences. Instead, break complex sentences into simpler ones.
- Place the subject before the action whenever possible. Example: prefer "The function loads data" over "Data is loaded
by the function."
+1 -1
View File
@@ -1 +1 @@
./AI.md ./AGENTS.md
+11 -1
View File
@@ -1,7 +1,7 @@
<div align="center"> <div align="center">
<img src="./website/landing/images/logo-title.svg#gh-light-mode-only" alt="Uncloud logo"/> <img src="./website/landing/images/logo-title.svg#gh-light-mode-only" alt="Uncloud logo"/>
<img src="./website/landing/images/logo-title-dark.svg#gh-dark-mode-only" alt="Uncloud logo"/> <img src="./website/landing/images/logo-title-dark.svg#gh-dark-mode-only" alt="Uncloud logo"/>
<p><strong>▸ Docker simplicity. Multi-machine power ◂</strong></p> <p><strong>▸ Deploy and scale containerised apps across servers without Swarm or Kubernetes overhead ◂</strong></p>
<p> <p>
<a href="https://uncloud.run/docs"><img src="https://img.shields.io/badge/Docs-blue.svg?style=for-the-badge&logo=gitbook&logoColor=white" alt="Documentation"></a> <a href="https://uncloud.run/docs"><img src="https://img.shields.io/badge/Docs-blue.svg?style=for-the-badge&logo=gitbook&logoColor=white" alt="Documentation"></a>
@@ -281,6 +281,16 @@ using the CLI or directly over SSH. They all have the complete cluster state and
machine is a full backup of your control plane. machine is a full backup of your control plane.
</details> </details>
## 🧪 Interactive tutorials
To give you a chance to play with Uncloud without even leaving your browser, we're providing interactive tutorials and playgrounds on the [iximiuz Labs](https://labs.iximiuz.com/) platform.
Available tutorials:
1. [Setting up a new Uncloud cluster](https://labs.iximiuz.com/tutorials/uncloud-create-cluster-ebebf72b) - the tutorial walks you through creating a new cluster with two machines and then deploying a simple web service to it.
You can also launch the [Uncloud playground](https://labs.iximiuz.com/playgrounds/uncloud-cluster-64523f7c) where you can play with an already initialized Uncloud cluster.
## 🏗 Project status ## 🏗 Project status
Uncloud is currently in active development and is **not ready for production use**. Features may change significantly Uncloud is currently in active development and is **not ready for production use**. Features may change significantly
+3 -2
View File
@@ -11,8 +11,9 @@ import (
func NewConnectionCommand() *cobra.Command { func NewConnectionCommand() *cobra.Command {
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "connection", Use: "connection",
Short: "Choose a new default connection for the current context.", Aliases: []string{"conn"},
Short: "Choose a new default connection for the current context.",
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
return selectConnection(uncli) return selectConnection(uncli)
+16 -10
View File
@@ -14,18 +14,20 @@ import (
"github.com/psviderski/uncloud/pkg/client" "github.com/psviderski/uncloud/pkg/client"
"github.com/psviderski/uncloud/pkg/client/compose" "github.com/psviderski/uncloud/pkg/client/compose"
"github.com/psviderski/uncloud/pkg/client/deploy" "github.com/psviderski/uncloud/pkg/client/deploy"
"github.com/psviderski/uncloud/pkg/client/deploy/operation"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
type deployOptions struct { type deployOptions struct {
cli.BuildServicesOptions cli.BuildServicesOptions
files []string files []string
profiles []string profiles []string
services []string services []string
noBuild bool noBuild bool
recreate bool recreate bool
yes bool skipHealth bool
yes bool
} }
// NewDeployCommand creates a new command to deploy services from a Compose file. // NewDeployCommand creates a new command to deploy services from a Compose file.
@@ -60,6 +62,10 @@ func NewDeployCommand() *cobra.Command {
"One or more Compose profiles to enable.") "One or more Compose profiles to enable.")
cmd.Flags().BoolVar(&opts.recreate, "recreate", false, cmd.Flags().BoolVar(&opts.recreate, "recreate", false,
"Recreate containers even if their configuration and image haven't changed.") "Recreate containers even if their configuration and image haven't changed.")
cmd.Flags().BoolVar(&opts.skipHealth, "skip-health", false,
"Skip the monitoring period and health checks after starting new containers. Useful for faster emergency "+
"deployments.\n"+
"Warning: This may cause downtime if new containers fail to start properly.")
cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false, cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
"Auto-confirm deployment plan. Should be explicitly set when running non-interactively,\n"+ "Auto-confirm deployment plan. Should be explicitly set when running non-interactively,\n"+
"e.g., in CI/CD pipelines. [$UNCLOUD_AUTO_CONFIRM]") "e.g., in CI/CD pipelines. [$UNCLOUD_AUTO_CONFIRM]")
@@ -148,9 +154,9 @@ func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
fmt.Println() fmt.Println()
} }
var strategy deploy.Strategy strategy := &deploy.RollingStrategy{
if opts.recreate { ForceRecreate: opts.recreate,
strategy = &deploy.RollingStrategy{ForceRecreate: true} SkipHealthMonitor: opts.skipHealth,
} }
composeDeploy, err := compose.NewDeploymentWithStrategy(ctx, clusterClient, project, strategy) composeDeploy, err := compose.NewDeploymentWithStrategy(ctx, clusterClient, project, strategy)
if err != nil { if err != nil {
@@ -198,7 +204,7 @@ func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
}, uncli.ProgressOut(), "Deploying services") }, uncli.ProgressOut(), "Deploying services")
} }
func printPlan(ctx context.Context, cli *client.Client, plan deploy.SequenceOperation) error { func printPlan(ctx context.Context, cli *client.Client, plan operation.SequenceOperation) error {
for _, op := range plan.Operations { for _, op := range plan.Operations {
svcPlan, ok := op.(*deploy.Plan) svcPlan, ok := op.(*deploy.Plan)
if !ok { if !ok {
+79 -55
View File
@@ -8,7 +8,8 @@ import (
"strings" "strings"
"time" "time"
"github.com/cenkalti/backoff/v4" "github.com/charmbracelet/huh/spinner"
"github.com/charmbracelet/lipgloss"
"github.com/docker/compose/v2/pkg/progress" "github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/cmd/uncloud/caddy" "github.com/psviderski/uncloud/cmd/uncloud/caddy"
"github.com/psviderski/uncloud/internal/cli" "github.com/psviderski/uncloud/internal/cli"
@@ -16,8 +17,6 @@ import (
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client" "github.com/psviderski/uncloud/pkg/client"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
) )
type addOptions struct { type addOptions struct {
@@ -27,6 +26,7 @@ type addOptions struct {
publicIP string publicIP string
sshKey string sshKey string
version string version string
yes bool
} }
func NewAddCommand() *cobra.Command { func NewAddCommand() *cobra.Command {
@@ -41,6 +41,8 @@ Connection methods:
ssh+cli://user@host - Use system SSH command (supports ProxyJump, SSH config)`, ssh+cli://user@host - Use system SSH command (supports ProxyJump, SSH config)`,
Args: cobra.ExactArgs(1), Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
cli.BindEnvToFlag(cmd, "yes", "UNCLOUD_AUTO_CONFIRM")
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
// Determine if SSH CLI needs to be used and strip scheme // Determine if SSH CLI needs to be used and strip scheme
@@ -88,6 +90,9 @@ Connection methods:
&opts.version, "version", "latest", &opts.version, "version", "latest",
"Version of the Uncloud daemon to install on the machine.", "Version of the Uncloud daemon to install on the machine.",
) )
cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
"Auto-confirm prompts (e.g., resetting an already initialised machine).\n"+
"Should be explicitly set when running non-interactively, e.g., in CI/CD pipelines. [$UNCLOUD_AUTO_CONFIRM]")
return cmd return cmd
} }
@@ -113,6 +118,7 @@ func add(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine,
RemoteMachine: remoteMachine, RemoteMachine: remoteMachine,
SkipInstall: opts.noInstall, SkipInstall: opts.noInstall,
Version: opts.version, Version: opts.version,
AutoConfirm: opts.yes,
}) })
if err != nil { if err != nil {
return err return err
@@ -124,79 +130,97 @@ func add(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine,
return nil return nil
} }
// Wait for the cluster to be initialised to be able to deploy the Caddy service. // Wait for the cluster to be initialised on the machine to be able to deploy the Caddy service.
fmt.Println("Waiting for the machine to be ready...") err = spinner.New().
fmt.Println() Title(" Waiting for the machine to join the cluster...").
if err = waitClusterInitialised(ctx, machineClient); err != nil { Type(spinner.MiniDot).
return fmt.Errorf("wait for cluster to be initialised on machine: %w", err) Style(lipgloss.NewStyle().Foreground(lipgloss.Color("3"))).
TitleStyle(lipgloss.NewStyle()).
ActionWithErr(func(ctx context.Context) error {
return machineClient.WaitClusterReady(ctx, 5*time.Minute)
}).
Run()
if err != nil {
return fmt.Errorf("wait for machine to join the cluster: %w", err)
} }
fmt.Println("Machine joined the cluster.")
// TODO: scale the existing Caddy service to the new machine instead of running a new deployment
// that may cause a small downtime.
// Deploy a Caddy service container to the added machine. If caddy service is already deployed on other machines, // Deploy a Caddy service container to the added machine. If caddy service is already deployed on other machines,
// use the deployed image version. Otherwise, use the latest version. // use the deployed image version.
// NOTE: We use the cluster client to inspect and scale the Caddy service because the newly added machine may have // NOTE: We use the cluster client to inspect and scale the Caddy service because the newly added machine may have
// issues accessing the Machine API of existing machines in the cluster. // issues accessing the Machine API of existing machines in the cluster.
// See the issue for more details: https://github.com/psviderski/uncloud/issues/65. // See the issue for more details: https://github.com/psviderski/uncloud/issues/65.
caddyImage := "" caddyImage := ""
caddySvc, err := clusterClient.InspectService(ctx, client.CaddyServiceName) caddySvc, err := clusterClient.InspectService(ctx, client.CaddyServiceName)
if err != nil { if err != nil {
if !errors.Is(err, api.ErrNotFound) { if errors.Is(err, api.ErrNotFound) {
return fmt.Errorf("inspect caddy service: %w", err) // Caddy service is not deployed.
return nil
} }
} else { return fmt.Errorf("inspect caddy service: %w", err)
caddyImage = caddySvc.Containers[0].Container.Config.Image }
// Find the latest created container and use its image. caddyImage = caddySvc.Containers[0].Container.Config.Image
var latestCreated time.Time // Find the latest created container and use its image.
for _, c := range caddySvc.Containers[1:] { var latestCreated time.Time
created, err := time.Parse(time.RFC3339Nano, c.Container.Created) for _, c := range caddySvc.Containers[1:] {
if err != nil { created, err := time.Parse(time.RFC3339Nano, c.Container.Created)
continue if err != nil {
} continue
if created.After(latestCreated) { }
latestCreated = created if created.After(latestCreated) {
caddyImage = c.Container.Config.Image latestCreated = created
} caddyImage = c.Container.Config.Image
} }
} }
// TODO: scale the existing Caddy service to the new machine instead of running a new deployment
// that may cause a small downtime.
d, err := clusterClient.NewCaddyDeployment(caddyImage, "", api.Placement{}) d, err := clusterClient.NewCaddyDeployment(caddyImage, "", api.Placement{})
if err != nil { if err != nil {
return fmt.Errorf("create caddy deployment: %w", err) return fmt.Errorf("create caddy deployment: %w", err)
} }
err = progress.RunWithTitle(ctx, func(ctx context.Context) error { plan, err := d.Plan(ctx)
if _, err = d.Run(ctx); err != nil {
return fmt.Errorf("deploy caddy: %w", err)
}
return nil
}, uncli.ProgressOut(), fmt.Sprintf("Deploying service %s", d.Spec.Name))
if err != nil { if err != nil {
return err return fmt.Errorf("plan caddy deployment: %w", err)
}
fmt.Println()
if len(plan.Operations) == 0 {
fmt.Printf("%s service is up to date.\n", client.CaddyServiceName)
} else {
// Initialise a machine and container name resolver to properly format the plan output.
resolver, err := clusterClient.ServiceOperationNameResolver(ctx, caddySvc)
if err != nil {
return fmt.Errorf("create machine and container name resolver for service operations: %w", err)
}
fmt.Println("caddy deployment plan:")
fmt.Println(plan.Format(resolver))
fmt.Println()
if !opts.yes {
confirmed, err := cli.Confirm()
if err != nil {
return fmt.Errorf("confirm deployment: %w", err)
}
if !confirmed {
fmt.Println("Cancelled. No changes were made.")
return nil
}
}
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
if _, err = d.Run(ctx); err != nil {
return fmt.Errorf("deploy caddy: %w", err)
}
return nil
}, uncli.ProgressOut(), fmt.Sprintf("Deploying service %s (%s mode)", d.Spec.Name, d.Spec.Mode))
if err != nil {
return err
}
} }
fmt.Println() fmt.Println()
return caddy.UpdateDomainRecords(ctx, machineClient, uncli.ProgressOut()) return caddy.UpdateDomainRecords(ctx, machineClient, uncli.ProgressOut())
} }
func waitClusterInitialised(ctx context.Context, client *client.Client) error {
boff := backoff.WithContext(backoff.NewExponentialBackOff(
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(5*time.Minute),
), ctx)
check := func() error {
_, err := client.ListMachines(ctx, nil)
if err == nil {
return nil
}
statusErr := status.Convert(err)
if statusErr.Code() == codes.FailedPrecondition {
return err
}
return backoff.Permanent(err)
}
return backoff.Retry(check, boff)
}
+38 -9
View File
@@ -5,7 +5,10 @@ import (
"fmt" "fmt"
"net/netip" "net/netip"
"strings" "strings"
"time"
"github.com/charmbracelet/huh/spinner"
"github.com/charmbracelet/lipgloss"
"github.com/docker/compose/v2/pkg/progress" "github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/cmd/uncloud/caddy" "github.com/psviderski/uncloud/cmd/uncloud/caddy"
"github.com/psviderski/uncloud/cmd/uncloud/dns" "github.com/psviderski/uncloud/cmd/uncloud/dns"
@@ -28,12 +31,13 @@ type initOptions struct {
sshKey string sshKey string
version string version string
context string context string
yes bool
} }
func NewInitCommand() *cobra.Command { func NewInitCommand() *cobra.Command {
opts := initOptions{} opts := initOptions{}
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "init [USER@HOST:PORT]", Use: "init [schema://]USER@HOST[:PORT]",
Short: "Initialise a new cluster with a remote machine as the first member.", Short: "Initialise a new cluster with a remote machine as the first member.",
Long: `Initialise a new cluster by setting up a remote machine as the first member. Long: `Initialise a new cluster by setting up a remote machine as the first member.
This command creates a new context in your Uncloud config to manage the cluster. This command creates a new context in your Uncloud config to manage the cluster.
@@ -56,6 +60,8 @@ Connection methods:
// TODO: support initialising a cluster on the local machine. // TODO: support initialising a cluster on the local machine.
Args: cobra.MaximumNArgs(1), Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
cli.BindEnvToFlag(cmd, "yes", "UNCLOUD_AUTO_CONFIRM")
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
var remoteMachine *cli.RemoteMachine var remoteMachine *cli.RemoteMachine
@@ -82,6 +88,11 @@ Connection methods:
return initCluster(cmd.Context(), uncli, remoteMachine, opts) return initCluster(cmd.Context(), uncli, remoteMachine, opts)
}, },
} }
cmd.Flags().StringVarP(
&opts.context, "context", "c", cli.DefaultContextName,
"Name of the new context to be created in the Uncloud config to manage the cluster.",
)
cmd.Flags().StringVar(&opts.dnsEndpoint, "dns-endpoint", dns.DefaultUncloudDNSAPIEndpoint, cmd.Flags().StringVar(&opts.dnsEndpoint, "dns-endpoint", dns.DefaultUncloudDNSAPIEndpoint,
"API endpoint for the Uncloud DNS service.") "API endpoint for the Uncloud DNS service.")
cmd.Flags().StringVarP( cmd.Flags().StringVarP(
@@ -119,15 +130,19 @@ Connection methods:
&opts.version, "version", "latest", &opts.version, "version", "latest",
"Version of the Uncloud daemon to install on the machine.", "Version of the Uncloud daemon to install on the machine.",
) )
cmd.Flags().StringVarP( cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
&opts.context, "context", "c", cli.DefaultContextName, "Auto-confirm prompts (e.g., resetting an already initialised machine).\n"+
"Name of the new context to be created in the Uncloud config to manage the cluster.", "Should be explicitly set when running non-interactively, e.g., in CI/CD pipelines. [$UNCLOUD_AUTO_CONFIRM]")
)
return cmd return cmd
} }
func initCluster(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine, opts initOptions) error { func initCluster(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine, opts initOptions) error {
if uncli.Config == nil {
// Config is nil when connecting directly to a remote machine (--connect) without using Uncloud config.
return fmt.Errorf("do not specify --connect when initialising a new cluster")
}
netPrefix, err := netip.ParsePrefix(opts.network) netPrefix, err := netip.ParsePrefix(opts.network)
if err != nil { if err != nil {
return fmt.Errorf("parse network CIDR: %w", err) return fmt.Errorf("parse network CIDR: %w", err)
@@ -154,20 +169,34 @@ func initCluster(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteM
RemoteMachine: remoteMachine, RemoteMachine: remoteMachine,
SkipInstall: opts.noInstall, SkipInstall: opts.noInstall,
Version: opts.version, Version: opts.version,
AutoConfirm: opts.yes,
}) })
if err != nil { if err != nil {
return err return err
} }
defer client.Close() defer client.Close()
// Since the cluster API needs a few moments to become ready after cluster initialisation,
// we keep the user informed during this wait. We wait here even if no Caddy or DNS is requested
// as the cluster needs to be ready so that commands such as 'uc machine ls' work immediately after init.
err = spinner.New().
Title(" Waiting for the cluster to be ready...").
Type(spinner.MiniDot).
Style(lipgloss.NewStyle().Foreground(lipgloss.Color("3"))).
TitleStyle(lipgloss.NewStyle()).
ActionWithErr(func(ctx context.Context) error {
return client.WaitClusterReady(ctx, 1*time.Minute)
}).
Run()
if err != nil {
return fmt.Errorf("wait for cluster to be ready: %w", err)
}
fmt.Println("Cluster is ready.")
if opts.noCaddy && opts.noDNS { if opts.noCaddy && opts.noDNS {
return nil return nil
} }
// Deploy the Caddy service to the initialised machine.
// The creation of a deployment plan talks to cluster API. Since the API needs a few moments to become available
// after cluster initialisation, we keep the user informed during this wait.
fmt.Println("Waiting for the machine to be ready...")
fmt.Println() fmt.Println()
if !opts.noDNS { if !opts.noDNS {
+3 -1
View File
@@ -14,6 +14,7 @@ import (
"github.com/psviderski/uncloud/cmd/uncloud/machine" "github.com/psviderski/uncloud/cmd/uncloud/machine"
"github.com/psviderski/uncloud/cmd/uncloud/service" "github.com/psviderski/uncloud/cmd/uncloud/service"
"github.com/psviderski/uncloud/cmd/uncloud/volume" "github.com/psviderski/uncloud/cmd/uncloud/volume"
"github.com/psviderski/uncloud/cmd/uncloud/wg"
"github.com/psviderski/uncloud/internal/cli" "github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/cli/config" "github.com/psviderski/uncloud/internal/cli/config"
"github.com/psviderski/uncloud/internal/fs" "github.com/psviderski/uncloud/internal/fs"
@@ -40,8 +41,8 @@ func main() {
SilenceErrors: true, SilenceErrors: true,
PersistentPreRunE: func(cmd *cobra.Command, args []string) error { PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
cli.BindEnvToFlag(cmd, "connect", "UNCLOUD_CONNECT") cli.BindEnvToFlag(cmd, "connect", "UNCLOUD_CONNECT")
cli.BindEnvToFlag(cmd, "uncloud-config", "UNCLOUD_CONFIG")
cli.BindEnvToFlag(cmd, "context", "UNCLOUD_CONTEXT") cli.BindEnvToFlag(cmd, "context", "UNCLOUD_CONTEXT")
cli.BindEnvToFlag(cmd, "uncloud-config", "UNCLOUD_CONFIG")
var conn *config.MachineConnection var conn *config.MachineConnection
if opts.connect != "" { if opts.connect != "" {
@@ -134,6 +135,7 @@ func main() {
service.NewStartCommand("service"), service.NewStartCommand("service"),
service.NewStopCommand("service"), service.NewStopCommand("service"),
volume.NewRootCommand(), volume.NewRootCommand(),
wg.NewRootCommand(),
) )
cobra.CheckErr(cmd.Execute()) cobra.CheckErr(cmd.Execute())
} }
+7 -7
View File
@@ -205,18 +205,12 @@ func collectContainers(ctx context.Context, cli *client.Client) ([]containerInfo
var containers []containerInfo var containers []containerInfo
for _, msc := range machineContainers { for _, msc := range machineContainers {
machineName := "unknown"
// Metadata can be nil if the request was broadcasted to only one machine. // Metadata can be nil if the request was broadcasted to only one machine.
if msc.Metadata == nil && len(machineContainers) > 1 { if msc.Metadata == nil && len(machineContainers) > 1 {
return nil, fmt.Errorf("something went wrong with gRPC proxy: metadata is missing for a machine response") return nil, fmt.Errorf("something went wrong with gRPC proxy: metadata is missing for a machine response")
} }
if msc.Metadata != nil && msc.Metadata.Error != "" {
client.PrintWarning(fmt.Sprintf("failed to list containers on machine %s: %s", machineName,
msc.Metadata.Error))
continue
}
machineName := "unknown"
if msc.Metadata != nil { if msc.Metadata != nil {
var ok bool var ok bool
machineName, ok = machinesNamesByIP[msc.Metadata.Machine] machineName, ok = machinesNamesByIP[msc.Metadata.Machine]
@@ -231,6 +225,12 @@ func collectContainers(ctx context.Context, cli *client.Client) ([]containerInfo
} }
} }
if msc.Metadata != nil && msc.Metadata.Error != "" {
client.PrintWarning(fmt.Sprintf("failed to list containers on machine %s: %s", machineName,
msc.Metadata.Error))
continue
}
for _, ctr := range msc.Containers { for _, ctr := range msc.Containers {
if ctr.Container.State == nil || ctr.Container.Config == nil { if ctr.Container.State == nil || ctr.Container.Config == nil {
continue continue
+6 -1
View File
@@ -59,7 +59,7 @@ If the service has multiple replicas and no container ID is specified, the comma
execCmd.Flags().BoolVarP(&opts.detach, "detach", "d", false, "Detached mode: run command in the background") execCmd.Flags().BoolVarP(&opts.detach, "detach", "d", false, "Detached mode: run command in the background")
execCmd.Flags().BoolVarP(&opts.noTty, "no-tty", "T", !cli.IsStdoutTerminal(), execCmd.Flags().BoolVarP(&opts.noTty, "no-tty", "T", false,
"Disable pseudo-TTY allocation. By default 'uc exec' allocates a TTY when connected to a terminal.") "Disable pseudo-TTY allocation. By default 'uc exec' allocates a TTY when connected to a terminal.")
// Keep "-i" and "-t" flags hidden for compatibility with docker exec // Keep "-i" and "-t" flags hidden for compatibility with docker exec
@@ -82,6 +82,11 @@ If the service has multiple replicas and no container ID is specified, the comma
} }
func runExec(ctx context.Context, uncli *cli.CLI, serviceName string, command []string, opts execCliOptions) error { func runExec(ctx context.Context, uncli *cli.CLI, serviceName string, command []string, opts execCliOptions) error {
// Disable TTY allocation if not connected to a terminal
if !cli.IsStdoutTerminal() {
opts.noTty = true
}
if !opts.detach { if !opts.detach {
// Check if we're trying to attach to a TTY from a non-TTY client, e.g. // Check if we're trying to attach to a TTY from a non-TTY client, e.g.
// when doing an 'cmd | uc exec ...' // when doing an 'cmd | uc exec ...'
+132
View File
@@ -0,0 +1,132 @@
package wg
import (
"context"
"fmt"
"os"
"strings"
"text/tabwriter"
"time"
"github.com/docker/go-units"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
"golang.zx2c4.com/wireguard/wgctrl/wgtypes"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "wg",
Short: "Inspect WireGuard network",
}
cmd.AddCommand(newShowCommand())
return cmd
}
type showOptions struct {
machine string
}
func newShowCommand() *cobra.Command {
opts := showOptions{}
cmd := &cobra.Command{
Use: "show",
Short: "Show WireGuard network configuration for a machine.",
Long: "Show the WireGuard network configuration for the machine currently connected to " +
"(or specified by the global --connect flag).",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return runShow(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVarP(&opts.machine, "machine", "m", "",
"Name or ID of the machine to show the configuration for. (default is connected machine)")
return cmd
}
func runShow(ctx context.Context, uncli *cli.CLI, opts showOptions) error {
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connection failed: %w", err)
}
defer client.Close()
if opts.machine != "" {
// Proxy requests to the specified machine.
ctx, _, err = client.ProxyMachinesContext(ctx, []string{opts.machine})
if err != nil {
return err
}
}
resp, err := client.MachineClient.InspectWireGuardNetwork(ctx, nil)
if err != nil {
if status.Code(err) == codes.Unimplemented {
return fmt.Errorf("inspect WireGuard network: "+
"make sure the target machine is running uncloudd daemon version >= 0.16.0: %w", err)
}
return err
}
machines, err := client.ListMachines(ctx, nil)
if err != nil {
return fmt.Errorf("list machines: %w", err)
}
machinesNamesByPublicKey := make(map[string]string)
for _, m := range machines {
publicKey := wgtypes.Key(m.Machine.Network.PublicKey).String()
machinesNamesByPublicKey[publicKey] = m.Machine.Name
}
// Fetch the machine's name for more descriptive output
inspectResp, err := client.MachineClient.InspectMachine(ctx, nil)
if err == nil {
fmt.Printf("Machine name: %s\n", inspectResp.Machines[0].Machine.Name)
}
fmt.Printf("WireGuard interface: %s\n", resp.InterfaceName)
fmt.Printf("WireGuard public key: %s\n", wgtypes.Key(resp.PublicKey).String())
fmt.Printf("WireGuard port: %d\n", resp.ListenPort)
fmt.Println()
if len(resp.Peers) == 0 {
fmt.Println("No WireGuard peers configured.")
return nil
}
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
if _, err = fmt.Fprintln(tw, "PEER\tPUBLIC KEY\tENDPOINT\tHANDSHAKE\tRECEIVED\tSENT\tALLOWED IPS"); err != nil {
return fmt.Errorf("write header: %w", err)
}
for _, peer := range resp.Peers {
machineName, ok := machinesNamesByPublicKey[wgtypes.Key(peer.PublicKey).String()]
if !ok {
machineName = "(unknown)"
}
lastHandshake := ""
if peer.LastHandshakeTime != nil {
lastHandshake = time.Since(peer.LastHandshakeTime.AsTime()).Round(time.Second).String() + " ago"
}
_, err = fmt.Fprintf(
tw,
"%s\t%s\t%s\t%s\t%s\t%s\t%s\n",
machineName,
wgtypes.Key(peer.PublicKey).String(),
peer.Endpoint,
lastHandshake,
units.HumanSize(float64(peer.ReceiveBytes)),
units.HumanSize(float64(peer.TransmitBytes)),
strings.Join(peer.AllowedIps, ", "),
)
if err != nil {
return fmt.Errorf("write row: %w", err)
}
}
return tw.Flush()
}
+2 -2
View File
@@ -46,7 +46,7 @@ require (
github.com/muesli/termenv v0.16.0 github.com/muesli/termenv v0.16.0
github.com/opencontainers/go-digest v1.0.0 github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.1.1 github.com/opencontainers/image-spec v1.1.1
github.com/psviderski/unregistry v0.3.1 github.com/psviderski/unregistry v0.4.1
github.com/siderolabs/discovery-api v0.1.4 github.com/siderolabs/discovery-api v0.1.4
github.com/siderolabs/discovery-client v0.1.9 github.com/siderolabs/discovery-client v0.1.9
github.com/siderolabs/grpc-proxy v0.5.1 github.com/siderolabs/grpc-proxy v0.5.1
@@ -66,6 +66,7 @@ require (
google.golang.org/grpc v1.74.2 google.golang.org/grpc v1.74.2
google.golang.org/protobuf v1.36.9 google.golang.org/protobuf v1.36.9
modernc.org/sqlite v1.36.3 modernc.org/sqlite v1.36.3
tags.cncf.io/container-device-interface v1.0.1
) )
require ( require (
@@ -415,5 +416,4 @@ require (
sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 // indirect sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.4.2 // indirect sigs.k8s.io/structured-merge-diff/v4 v4.4.2 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect sigs.k8s.io/yaml v1.4.0 // indirect
tags.cncf.io/container-device-interface v1.0.1 // indirect
) )
+2 -2
View File
@@ -1040,8 +1040,8 @@ github.com/prometheus/procfs v0.0.3/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDa
github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A= github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A=
github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc= github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk= github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
github.com/psviderski/unregistry v0.3.1 h1:2xQ3uMNMyXAASqq9+kGUgNMIFOvPSMIFGdz47uDfjhQ= github.com/psviderski/unregistry v0.4.1 h1:KedQTjyj8hHbzygK9uw5YS10amMZ5cjSYHqSjUUBhoE=
github.com/psviderski/unregistry v0.3.1/go.mod h1:DXLnzrskO5NuRlzZhpbyo1E6KpirnctUvq6WtCGkpUQ= github.com/psviderski/unregistry v0.4.1/go.mod h1:DXLnzrskO5NuRlzZhpbyo1E6KpirnctUvq6WtCGkpUQ=
github.com/quic-go/qpack v0.5.1 h1:giqksBPnT/HDtZ6VhtFKgoLOWmlyo9Ei6u9PqzIMbhI= github.com/quic-go/qpack v0.5.1 h1:giqksBPnT/HDtZ6VhtFKgoLOWmlyo9Ei6u9PqzIMbhI=
github.com/quic-go/qpack v0.5.1/go.mod h1:+PC4XFrEskIVkcLzpEkbLqq1uCoxPhQuvK5rH1ZgaEg= github.com/quic-go/qpack v0.5.1/go.mod h1:+PC4XFrEskIVkcLzpEkbLqq1uCoxPhQuvK5rH1ZgaEg=
github.com/quic-go/quic-go v0.48.2 h1:wsKXZPeGWpMpCGSWqOcqpW2wZYic/8T3aqiOID0/KWE= github.com/quic-go/quic-go v0.48.2 h1:wsKXZPeGWpMpCGSWqOcqpW2wZYic/8T3aqiOID0/KWE=
+51 -9
View File
@@ -162,6 +162,7 @@ type InitClusterOptions struct {
RemoteMachine *RemoteMachine RemoteMachine *RemoteMachine
SkipInstall bool SkipInstall bool
Version string Version string
AutoConfirm bool
} }
// InitCluster initialises a new cluster on a remote machine and returns a client to interact with the cluster. // InitCluster initialises a new cluster on a remote machine and returns a client to interact with the cluster.
@@ -171,7 +172,7 @@ func (cli *CLI) InitCluster(ctx context.Context, opts InitClusterOptions) (*clie
return cli.initRemoteMachine(ctx, opts) return cli.initRemoteMachine(ctx, opts)
} }
// TODO: implement local machine initialisation // TODO: implement local machine initialisation
return nil, fmt.Errorf("local machine initialisation is not implemented yet") return nil, fmt.Errorf("local machine initialisation is not implemented yet. Please specify a remote machine")
} }
func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions) (*client.Client, error) { func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions) (*client.Client, error) {
@@ -197,7 +198,12 @@ func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions)
return nil, fmt.Errorf("inspect machine: %w", err) return nil, fmt.Errorf("inspect machine: %w", err)
} }
if minfo.Id != "" { if minfo.Id != "" {
if err = promptResetMachine(ctx, machineClient.MachineClient); err != nil { if !opts.AutoConfirm {
if err = promptResetMachine(); err != nil {
return nil, err
}
}
if err = resetAndWaitMachine(ctx, machineClient.MachineClient); err != nil {
return nil, err return nil, err
} }
} }
@@ -246,9 +252,17 @@ func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions)
MachineID: resp.Machine.Id, MachineID: resp.Machine.Id,
} }
if opts.RemoteMachine.UseSSHCLI { if opts.RemoteMachine.UseSSHCLI {
connCfg.SSHCLI = config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port) connCfg.SSHCLI = config.NewSSHDestination(
opts.RemoteMachine.User,
opts.RemoteMachine.Host,
opts.RemoteMachine.Port,
)
} else { } else {
connCfg.SSH = config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port) connCfg.SSH = config.NewSSHDestination(
opts.RemoteMachine.User,
opts.RemoteMachine.Host,
opts.RemoteMachine.Port,
)
} }
cli.Config.Contexts[contextName].Connections = append(cli.Config.Contexts[contextName].Connections, connCfg) cli.Config.Contexts[contextName].Connections = append(cli.Config.Contexts[contextName].Connections, connCfg)
if err = cli.Config.Save(); err != nil { if err = cli.Config.Save(); err != nil {
@@ -289,6 +303,7 @@ type AddMachineOptions struct {
RemoteMachine *RemoteMachine RemoteMachine *RemoteMachine
SkipInstall bool SkipInstall bool
Version string Version string
AutoConfirm bool
} }
// AddMachine provisions a remote machine and adds it to the cluster. It returns a cluster client and a machine client. // AddMachine provisions a remote machine and adds it to the cluster. It returns a cluster client and a machine client.
@@ -318,6 +333,7 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
}() }()
// Check if the machine is already initialised as a cluster member and prompt the user to reset it first. // Check if the machine is already initialised as a cluster member and prompt the user to reset it first.
// TODO: refactor to use client.InspectMachine.
minfo, err := machineClient.Inspect(ctx, &emptypb.Empty{}) minfo, err := machineClient.Inspect(ctx, &emptypb.Empty{})
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("inspect machine: %w", err) return nil, nil, fmt.Errorf("inspect machine: %w", err)
@@ -334,7 +350,12 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
return nil, nil, fmt.Errorf("machine is already a member of this cluster (%s)", minfo.Name) return nil, nil, fmt.Errorf("machine is already a member of this cluster (%s)", minfo.Name)
} }
if err = promptResetMachine(ctx, machineClient.MachineClient); err != nil { if !opts.AutoConfirm {
if err = promptResetMachine(); err != nil {
return nil, nil, err
}
}
if err = resetAndWaitMachine(ctx, machineClient.MachineClient); err != nil {
return nil, nil, err return nil, nil, err
} }
} }
@@ -385,6 +406,18 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
return nil, nil, fmt.Errorf("add machine to cluster (context '%s'): %w", contextName, err) return nil, nil, fmt.Errorf("add machine to cluster (context '%s'): %w", contextName, err)
} }
// Get the current store DB version from the cluster to pass to the join request.
var storeDBVersion int64
inspectResp, err := c.MachineClient.InspectMachine(ctx, &emptypb.Empty{})
if err != nil {
// TODO(lhf): remove Unimplemented check when v0.17.0 is released.
if status.Convert(err).Code() != codes.Unimplemented {
return nil, nil, fmt.Errorf("inspect current cluster machine: %w", err)
}
} else {
storeDBVersion = inspectResp.Machines[0].StoreDbVersion
}
// Get the most up-to-date list of other machines in the cluster to include them in the join request. // Get the most up-to-date list of other machines in the cluster to include them in the join request.
machines, err := c.ListMachines(ctx, nil) machines, err := c.ListMachines(ctx, nil)
if err != nil { if err != nil {
@@ -399,8 +432,9 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
// Configure the remote machine to join the cluster. // Configure the remote machine to join the cluster.
joinReq := &pb.JoinClusterRequest{ joinReq := &pb.JoinClusterRequest{
Machine: addResp.Machine, Machine: addResp.Machine,
OtherMachines: otherMachines, OtherMachines: otherMachines,
MinStoreDbVersion: storeDBVersion,
} }
if _, err = machineClient.JoinCluster(ctx, joinReq); err != nil { if _, err = machineClient.JoinCluster(ctx, joinReq); err != nil {
return nil, nil, fmt.Errorf("join cluster: %w", err) return nil, nil, fmt.Errorf("join cluster: %w", err)
@@ -415,9 +449,17 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
MachineID: addResp.Machine.Id, MachineID: addResp.Machine.Id,
} }
if opts.RemoteMachine.UseSSHCLI { if opts.RemoteMachine.UseSSHCLI {
connCfg.SSHCLI = config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port) connCfg.SSHCLI = config.NewSSHDestination(
opts.RemoteMachine.User,
opts.RemoteMachine.Host,
opts.RemoteMachine.Port,
)
} else { } else {
connCfg.SSH = config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port) connCfg.SSH = config.NewSSHDestination(
opts.RemoteMachine.User,
opts.RemoteMachine.Host,
opts.RemoteMachine.Port,
)
} }
cli.Config.Contexts[contextName].Connections = append(cli.Config.Contexts[contextName].Connections, connCfg) cli.Config.Contexts[contextName].Connections = append(cli.Config.Contexts[contextName].Connections, connCfg)
if err = cli.Config.Save(); err != nil { if err = cli.Config.Save(); err != nil {
+13 -17
View File
@@ -11,11 +11,6 @@ import (
"github.com/psviderski/uncloud/internal/secret" "github.com/psviderski/uncloud/internal/secret"
) )
const (
DefaultSSHUser = "root"
DefaultSSHPort = 22
)
type MachineConnection struct { type MachineConnection struct {
SSH SSHDestination `yaml:"ssh,omitempty"` SSH SSHDestination `yaml:"ssh,omitempty"`
SSHCLI SSHDestination `yaml:"ssh_cli,omitempty"` SSHCLI SSHDestination `yaml:"ssh_cli,omitempty"`
@@ -30,7 +25,7 @@ type MachineConnection struct {
MachineID string `yaml:"machine_id,omitempty"` MachineID string `yaml:"machine_id,omitempty"`
} }
func (c MachineConnection) String() string { func (c *MachineConnection) String() string {
if c.SSH != "" { if c.SSH != "" {
return "ssh://" + string(c.SSH) return "ssh://" + string(c.SSH)
} else if c.SSHCLI != "" { } else if c.SSHCLI != "" {
@@ -69,36 +64,37 @@ func (c *MachineConnection) Validate() error {
} }
// SSHDestination represents an SSH destination string in the canonical form of "user@host:port". // SSHDestination represents an SSH destination string in the canonical form of "user@host:port".
// The default user "root" and port 22 can be omitted. // Empty user or port components are omitted.
type SSHDestination string type SSHDestination string
// NewSSHDestination constructs an SSHDestination from user, host, and port components.
// If user is empty, it is omitted.
// If port is 0, it is omitted.
func NewSSHDestination(user, host string, port int) SSHDestination { func NewSSHDestination(user, host string, port int) SSHDestination {
dst := host dst := host
if port != 0 && port != DefaultSSHPort { if port != 0 {
dst = net.JoinHostPort(host, strconv.Itoa(port)) dst = net.JoinHostPort(host, strconv.Itoa(port))
} }
if user == "" { if user != "" {
user = DefaultSSHUser dst = fmt.Sprintf("%s@%s", user, dst)
} }
dst = user + "@" + dst
return SSHDestination(dst) return SSHDestination(dst)
} }
// Parse parses the SSH destination string into user, host, and port components.
// If user is not specified, it returns an empty string.
// If port is not specified, it returns 0.
func (d SSHDestination) Parse() (user string, host string, port int, err error) { func (d SSHDestination) Parse() (user string, host string, port int, err error) {
host = string(d) host = string(d)
if strings.Contains(host, "@") { if strings.Contains(host, "@") {
user, host, _ = strings.Cut(host, "@") user, host, _ = strings.Cut(host, "@")
} }
if user == "" {
user = DefaultSSHUser
}
h, p, sErr := net.SplitHostPort(host) h, p, sErr := net.SplitHostPort(host)
if sErr == nil { if sErr == nil {
host = h host = h
port, err = strconv.Atoi(p) port, err = strconv.Atoi(p)
} }
if port == 0 {
port = DefaultSSHPort
}
return return
} }
+13 -1
View File
@@ -2,6 +2,7 @@ package cli
import ( import (
"context" "context"
"errors"
"fmt" "fmt"
"os" "os"
"strings" "strings"
@@ -86,7 +87,13 @@ func provisionMachine(ctx context.Context, exec sshexec.Executor, version string
return nil return nil
} }
func promptResetMachine(ctx context.Context, machineClient pb.MachineClient) error { func promptResetMachine() error {
if !IsStdinTerminal() {
return errors.New("the remote machine is already initialised as a cluster member; " +
"cannot ask to confirm reset in non-interactive mode, " +
"use --yes flag or set UNCLOUD_AUTO_CONFIRM=true to auto-confirm")
}
var confirm bool var confirm bool
form := huh.NewForm( form := huh.NewForm(
huh.NewGroup( huh.NewGroup(
@@ -110,10 +117,15 @@ func promptResetMachine(ctx context.Context, machineClient pb.MachineClient) err
return fmt.Errorf("remote machine is already initialised as a cluster member") return fmt.Errorf("remote machine is already initialised as a cluster member")
} }
return nil
}
func resetAndWaitMachine(ctx context.Context, machineClient pb.MachineClient) error {
if _, err := machineClient.Reset(ctx, &pb.ResetRequest{}); err != nil { if _, err := machineClient.Reset(ctx, &pb.ResetRequest{}); err != nil {
return fmt.Errorf("reset remote machine: %w. You can also manually run 'uncloud-uninstall' "+ return fmt.Errorf("reset remote machine: %w. You can also manually run 'uncloud-uninstall' "+
"on the remote machine to fully uninstall Uncloud from it", err) "on the remote machine to fully uninstall Uncloud from it", err)
} }
fmt.Println("Resetting the remote machine...") fmt.Println("Resetting the remote machine...")
if err := waitMachineReady(ctx, machineClient, 1*time.Minute); err != nil { if err := waitMachineReady(ctx, machineClient, 1*time.Minute); err != nil {
return fmt.Errorf("wait for machine to be ready after reset: %w", err) return fmt.Errorf("wait for machine to be ready after reset: %w", err)
+573 -173
View File
@@ -10,6 +10,7 @@ import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect" protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl" protoimpl "google.golang.org/protobuf/runtime/protoimpl"
emptypb "google.golang.org/protobuf/types/known/emptypb" emptypb "google.golang.org/protobuf/types/known/emptypb"
timestamppb "google.golang.org/protobuf/types/known/timestamppb"
reflect "reflect" reflect "reflect"
sync "sync" sync "sync"
) )
@@ -371,6 +372,8 @@ type JoinClusterRequest struct {
Machine *MachineInfo `protobuf:"bytes,1,opt,name=machine,proto3" json:"machine,omitempty"` Machine *MachineInfo `protobuf:"bytes,1,opt,name=machine,proto3" json:"machine,omitempty"`
OtherMachines []*MachineInfo `protobuf:"bytes,3,rep,name=other_machines,json=otherMachines,proto3" json:"other_machines,omitempty"` OtherMachines []*MachineInfo `protobuf:"bytes,3,rep,name=other_machines,json=otherMachines,proto3" json:"other_machines,omitempty"`
// Minimum store database version the new machine should sync to before starting cluster operations.
MinStoreDbVersion int64 `protobuf:"varint,4,opt,name=min_store_db_version,json=minStoreDbVersion,proto3" json:"min_store_db_version,omitempty"`
} }
func (x *JoinClusterRequest) Reset() { func (x *JoinClusterRequest) Reset() {
@@ -419,6 +422,125 @@ func (x *JoinClusterRequest) GetOtherMachines() []*MachineInfo {
return nil return nil
} }
func (x *JoinClusterRequest) GetMinStoreDbVersion() int64 {
if x != nil {
return x.MinStoreDbVersion
}
return 0
}
type InspectMachineResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
// Must contain only one repeated messages field to allow broadcasting InspectMachine requests to multiple machines.
Machines []*MachineDetails `protobuf:"bytes,1,rep,name=machines,proto3" json:"machines,omitempty"`
}
func (x *InspectMachineResponse) Reset() {
*x = InspectMachineResponse{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[6]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *InspectMachineResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*InspectMachineResponse) ProtoMessage() {}
func (x *InspectMachineResponse) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[6]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use InspectMachineResponse.ProtoReflect.Descriptor instead.
func (*InspectMachineResponse) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{6}
}
func (x *InspectMachineResponse) GetMachines() []*MachineDetails {
if x != nil {
return x.Machines
}
return nil
}
type MachineDetails struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
Metadata *Metadata `protobuf:"bytes,1,opt,name=metadata,proto3" json:"metadata,omitempty"`
Machine *MachineInfo `protobuf:"bytes,2,opt,name=machine,proto3" json:"machine,omitempty"`
// Current Corrosion cr-sqlite database version (Lamport timestamp) of the cluster store.
StoreDbVersion int64 `protobuf:"varint,3,opt,name=store_db_version,json=storeDbVersion,proto3" json:"store_db_version,omitempty"`
}
func (x *MachineDetails) Reset() {
*x = MachineDetails{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[7]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *MachineDetails) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*MachineDetails) ProtoMessage() {}
func (x *MachineDetails) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[7]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use MachineDetails.ProtoReflect.Descriptor instead.
func (*MachineDetails) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{7}
}
func (x *MachineDetails) GetMetadata() *Metadata {
if x != nil {
return x.Metadata
}
return nil
}
func (x *MachineDetails) GetMachine() *MachineInfo {
if x != nil {
return x.Machine
}
return nil
}
func (x *MachineDetails) GetStoreDbVersion() int64 {
if x != nil {
return x.StoreDbVersion
}
return 0
}
type TokenResponse struct { type TokenResponse struct {
state protoimpl.MessageState state protoimpl.MessageState
sizeCache protoimpl.SizeCache sizeCache protoimpl.SizeCache
@@ -430,7 +552,7 @@ type TokenResponse struct {
func (x *TokenResponse) Reset() { func (x *TokenResponse) Reset() {
*x = TokenResponse{} *x = TokenResponse{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[6] mi := &file_internal_machine_api_pb_machine_proto_msgTypes[8]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -443,7 +565,7 @@ func (x *TokenResponse) String() string {
func (*TokenResponse) ProtoMessage() {} func (*TokenResponse) ProtoMessage() {}
func (x *TokenResponse) ProtoReflect() protoreflect.Message { func (x *TokenResponse) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[6] mi := &file_internal_machine_api_pb_machine_proto_msgTypes[8]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -456,7 +578,7 @@ func (x *TokenResponse) ProtoReflect() protoreflect.Message {
// Deprecated: Use TokenResponse.ProtoReflect.Descriptor instead. // Deprecated: Use TokenResponse.ProtoReflect.Descriptor instead.
func (*TokenResponse) Descriptor() ([]byte, []int) { func (*TokenResponse) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{6} return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{8}
} }
func (x *TokenResponse) GetToken() string { func (x *TokenResponse) GetToken() string {
@@ -475,7 +597,7 @@ type ResetRequest struct {
func (x *ResetRequest) Reset() { func (x *ResetRequest) Reset() {
*x = ResetRequest{} *x = ResetRequest{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[7] mi := &file_internal_machine_api_pb_machine_proto_msgTypes[9]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -488,7 +610,7 @@ func (x *ResetRequest) String() string {
func (*ResetRequest) ProtoMessage() {} func (*ResetRequest) ProtoMessage() {}
func (x *ResetRequest) ProtoReflect() protoreflect.Message { func (x *ResetRequest) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[7] mi := &file_internal_machine_api_pb_machine_proto_msgTypes[9]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -501,7 +623,7 @@ func (x *ResetRequest) ProtoReflect() protoreflect.Message {
// Deprecated: Use ResetRequest.ProtoReflect.Descriptor instead. // Deprecated: Use ResetRequest.ProtoReflect.Descriptor instead.
func (*ResetRequest) Descriptor() ([]byte, []int) { func (*ResetRequest) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{7} return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{9}
} }
type Service struct { type Service struct {
@@ -518,7 +640,7 @@ type Service struct {
func (x *Service) Reset() { func (x *Service) Reset() {
*x = Service{} *x = Service{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[8] mi := &file_internal_machine_api_pb_machine_proto_msgTypes[10]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -531,7 +653,7 @@ func (x *Service) String() string {
func (*Service) ProtoMessage() {} func (*Service) ProtoMessage() {}
func (x *Service) ProtoReflect() protoreflect.Message { func (x *Service) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[8] mi := &file_internal_machine_api_pb_machine_proto_msgTypes[10]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -544,7 +666,7 @@ func (x *Service) ProtoReflect() protoreflect.Message {
// Deprecated: Use Service.ProtoReflect.Descriptor instead. // Deprecated: Use Service.ProtoReflect.Descriptor instead.
func (*Service) Descriptor() ([]byte, []int) { func (*Service) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{8} return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{10}
} }
func (x *Service) GetId() string { func (x *Service) GetId() string {
@@ -586,7 +708,7 @@ type InspectServiceRequest struct {
func (x *InspectServiceRequest) Reset() { func (x *InspectServiceRequest) Reset() {
*x = InspectServiceRequest{} *x = InspectServiceRequest{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[9] mi := &file_internal_machine_api_pb_machine_proto_msgTypes[11]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -599,7 +721,7 @@ func (x *InspectServiceRequest) String() string {
func (*InspectServiceRequest) ProtoMessage() {} func (*InspectServiceRequest) ProtoMessage() {}
func (x *InspectServiceRequest) ProtoReflect() protoreflect.Message { func (x *InspectServiceRequest) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[9] mi := &file_internal_machine_api_pb_machine_proto_msgTypes[11]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -612,7 +734,7 @@ func (x *InspectServiceRequest) ProtoReflect() protoreflect.Message {
// Deprecated: Use InspectServiceRequest.ProtoReflect.Descriptor instead. // Deprecated: Use InspectServiceRequest.ProtoReflect.Descriptor instead.
func (*InspectServiceRequest) Descriptor() ([]byte, []int) { func (*InspectServiceRequest) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{9} return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{11}
} }
func (x *InspectServiceRequest) GetId() string { func (x *InspectServiceRequest) GetId() string {
@@ -633,7 +755,7 @@ type InspectServiceResponse struct {
func (x *InspectServiceResponse) Reset() { func (x *InspectServiceResponse) Reset() {
*x = InspectServiceResponse{} *x = InspectServiceResponse{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[10] mi := &file_internal_machine_api_pb_machine_proto_msgTypes[12]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -646,7 +768,7 @@ func (x *InspectServiceResponse) String() string {
func (*InspectServiceResponse) ProtoMessage() {} func (*InspectServiceResponse) ProtoMessage() {}
func (x *InspectServiceResponse) ProtoReflect() protoreflect.Message { func (x *InspectServiceResponse) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[10] mi := &file_internal_machine_api_pb_machine_proto_msgTypes[12]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -659,7 +781,7 @@ func (x *InspectServiceResponse) ProtoReflect() protoreflect.Message {
// Deprecated: Use InspectServiceResponse.ProtoReflect.Descriptor instead. // Deprecated: Use InspectServiceResponse.ProtoReflect.Descriptor instead.
func (*InspectServiceResponse) Descriptor() ([]byte, []int) { func (*InspectServiceResponse) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{10} return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{12}
} }
func (x *InspectServiceResponse) GetService() *Service { func (x *InspectServiceResponse) GetService() *Service {
@@ -669,6 +791,164 @@ func (x *InspectServiceResponse) GetService() *Service {
return nil return nil
} }
type InspectWireGuardNetworkResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
InterfaceName string `protobuf:"bytes,1,opt,name=interface_name,json=interfaceName,proto3" json:"interface_name,omitempty"`
PublicKey []byte `protobuf:"bytes,2,opt,name=public_key,json=publicKey,proto3" json:"public_key,omitempty"`
ListenPort int32 `protobuf:"varint,3,opt,name=listen_port,json=listenPort,proto3" json:"listen_port,omitempty"`
Peers []*WireGuardPeer `protobuf:"bytes,4,rep,name=peers,proto3" json:"peers,omitempty"`
}
func (x *InspectWireGuardNetworkResponse) Reset() {
*x = InspectWireGuardNetworkResponse{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[13]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *InspectWireGuardNetworkResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*InspectWireGuardNetworkResponse) ProtoMessage() {}
func (x *InspectWireGuardNetworkResponse) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[13]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use InspectWireGuardNetworkResponse.ProtoReflect.Descriptor instead.
func (*InspectWireGuardNetworkResponse) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{13}
}
func (x *InspectWireGuardNetworkResponse) GetInterfaceName() string {
if x != nil {
return x.InterfaceName
}
return ""
}
func (x *InspectWireGuardNetworkResponse) GetPublicKey() []byte {
if x != nil {
return x.PublicKey
}
return nil
}
func (x *InspectWireGuardNetworkResponse) GetListenPort() int32 {
if x != nil {
return x.ListenPort
}
return 0
}
func (x *InspectWireGuardNetworkResponse) GetPeers() []*WireGuardPeer {
if x != nil {
return x.Peers
}
return nil
}
type WireGuardPeer struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
PublicKey []byte `protobuf:"bytes,1,opt,name=public_key,json=publicKey,proto3" json:"public_key,omitempty"`
Endpoint string `protobuf:"bytes,2,opt,name=endpoint,proto3" json:"endpoint,omitempty"`
LastHandshakeTime *timestamppb.Timestamp `protobuf:"bytes,3,opt,name=last_handshake_time,json=lastHandshakeTime,proto3" json:"last_handshake_time,omitempty"`
ReceiveBytes int64 `protobuf:"varint,4,opt,name=receive_bytes,json=receiveBytes,proto3" json:"receive_bytes,omitempty"`
TransmitBytes int64 `protobuf:"varint,5,opt,name=transmit_bytes,json=transmitBytes,proto3" json:"transmit_bytes,omitempty"`
AllowedIps []string `protobuf:"bytes,6,rep,name=allowed_ips,json=allowedIps,proto3" json:"allowed_ips,omitempty"`
}
func (x *WireGuardPeer) Reset() {
*x = WireGuardPeer{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[14]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *WireGuardPeer) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*WireGuardPeer) ProtoMessage() {}
func (x *WireGuardPeer) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[14]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use WireGuardPeer.ProtoReflect.Descriptor instead.
func (*WireGuardPeer) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{14}
}
func (x *WireGuardPeer) GetPublicKey() []byte {
if x != nil {
return x.PublicKey
}
return nil
}
func (x *WireGuardPeer) GetEndpoint() string {
if x != nil {
return x.Endpoint
}
return ""
}
func (x *WireGuardPeer) GetLastHandshakeTime() *timestamppb.Timestamp {
if x != nil {
return x.LastHandshakeTime
}
return nil
}
func (x *WireGuardPeer) GetReceiveBytes() int64 {
if x != nil {
return x.ReceiveBytes
}
return 0
}
func (x *WireGuardPeer) GetTransmitBytes() int64 {
if x != nil {
return x.TransmitBytes
}
return 0
}
func (x *WireGuardPeer) GetAllowedIps() []string {
if x != nil {
return x.AllowedIps
}
return nil
}
type Service_Container struct { type Service_Container struct {
state protoimpl.MessageState state protoimpl.MessageState
sizeCache protoimpl.SizeCache sizeCache protoimpl.SizeCache
@@ -682,7 +962,7 @@ type Service_Container struct {
func (x *Service_Container) Reset() { func (x *Service_Container) Reset() {
*x = Service_Container{} *x = Service_Container{}
if protoimpl.UnsafeEnabled { if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[11] mi := &file_internal_machine_api_pb_machine_proto_msgTypes[15]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi) ms.StoreMessageInfo(mi)
} }
@@ -695,7 +975,7 @@ func (x *Service_Container) String() string {
func (*Service_Container) ProtoMessage() {} func (*Service_Container) ProtoMessage() {}
func (x *Service_Container) ProtoReflect() protoreflect.Message { func (x *Service_Container) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_machine_proto_msgTypes[11] mi := &file_internal_machine_api_pb_machine_proto_msgTypes[15]
if protoimpl.UnsafeEnabled && x != nil { if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil { if ms.LoadMessageInfo() == nil {
@@ -708,7 +988,7 @@ func (x *Service_Container) ProtoReflect() protoreflect.Message {
// Deprecated: Use Service_Container.ProtoReflect.Descriptor instead. // Deprecated: Use Service_Container.ProtoReflect.Descriptor instead.
func (*Service_Container) Descriptor() ([]byte, []int) { func (*Service_Container) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{8, 0} return file_internal_machine_api_pb_machine_proto_rawDescGZIP(), []int{10, 0}
} }
func (x *Service_Container) GetMachineId() string { func (x *Service_Container) GetMachineId() string {
@@ -732,113 +1012,170 @@ var file_internal_machine_api_pb_machine_proto_rawDesc = []byte{
0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x70, 0x62, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x70, 0x62, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e,
0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x03, 0x61, 0x70, 0x69, 0x1a, 0x1b, 0x67, 0x6f, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x03, 0x61, 0x70, 0x69, 0x1a, 0x1b, 0x67, 0x6f,
0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x65, 0x6d, 0x6f, 0x67, 0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x65, 0x6d,
0x70, 0x74, 0x79, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x24, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x70, 0x74, 0x79, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x1f, 0x67, 0x6f, 0x6f, 0x67, 0x6c,
0x6e, 0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x74, 0x69, 0x6d, 0x65, 0x73,
0x70, 0x62, 0x2f, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x74, 0x61, 0x6d, 0x70, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x24, 0x69, 0x6e, 0x74, 0x65,
0x85, 0x01, 0x0a, 0x0b, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x12, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69,
0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x02, 0x69, 0x64, 0x12, 0x2f, 0x70, 0x62, 0x2f, 0x63, 0x6f, 0x6d, 0x6d, 0x6f, 0x6e, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f,
0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6e, 0x22, 0x85, 0x01, 0x0a, 0x0b, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66, 0x6f,
0x61, 0x6d, 0x65, 0x12, 0x2c, 0x0a, 0x07, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x18, 0x03, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x02, 0x69, 0x64,
0x20, 0x01, 0x28, 0x0b, 0x32, 0x12, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4e, 0x65, 0x74, 0x77, 0x6f, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04,
0x72, 0x6b, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x52, 0x07, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x2c, 0x0a, 0x07, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x18,
0x6b, 0x12, 0x24, 0x0a, 0x09, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f, 0x69, 0x70, 0x18, 0x04, 0x03, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x12, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4e, 0x65, 0x74, 0x77,
0x20, 0x01, 0x28, 0x0b, 0x32, 0x07, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x50, 0x52, 0x08, 0x70, 0x6f, 0x72, 0x6b, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x52, 0x07, 0x6e, 0x65, 0x74, 0x77, 0x6f,
0x75, 0x62, 0x6c, 0x69, 0x63, 0x49, 0x70, 0x22, 0xae, 0x01, 0x0a, 0x0d, 0x4e, 0x65, 0x74, 0x77, 0x72, 0x6b, 0x12, 0x24, 0x0a, 0x09, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f, 0x69, 0x70, 0x18,
0x6f, 0x72, 0x6b, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x12, 0x25, 0x0a, 0x06, 0x73, 0x75, 0x62, 0x04, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x07, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x50, 0x52, 0x08,
0x6e, 0x65, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0d, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x49, 0x70, 0x22, 0xae, 0x01, 0x0a, 0x0d, 0x4e, 0x65, 0x74,
0x49, 0x50, 0x50, 0x72, 0x65, 0x66, 0x69, 0x78, 0x52, 0x06, 0x73, 0x75, 0x62, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x12, 0x25, 0x0a, 0x06, 0x73, 0x75,
0x12, 0x2c, 0x0a, 0x0d, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x5f, 0x69, 0x62, 0x6e, 0x65, 0x74, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0d, 0x2e, 0x61, 0x70, 0x69,
0x70, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x07, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x50, 0x2e, 0x49, 0x50, 0x50, 0x72, 0x65, 0x66, 0x69, 0x78, 0x52, 0x06, 0x73, 0x75, 0x62, 0x6e, 0x65,
0x52, 0x0c, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x49, 0x70, 0x12, 0x29, 0x74, 0x12, 0x2c, 0x0a, 0x0d, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x5f,
0x0a, 0x09, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x69, 0x70, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x07, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49,
0x0b, 0x32, 0x0b, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x50, 0x50, 0x6f, 0x72, 0x74, 0x52, 0x09, 0x50, 0x52, 0x0c, 0x6d, 0x61, 0x6e, 0x61, 0x67, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x49, 0x70, 0x12,
0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x73, 0x12, 0x1d, 0x0a, 0x0a, 0x70, 0x75, 0x62, 0x29, 0x0a, 0x09, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x73, 0x18, 0x03, 0x20, 0x03,
0x6c, 0x69, 0x63, 0x5f, 0x6b, 0x65, 0x79, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x09, 0x70, 0x28, 0x0b, 0x32, 0x0b, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x50, 0x50, 0x6f, 0x72, 0x74, 0x52,
0x75, 0x62, 0x6c, 0x69, 0x63, 0x4b, 0x65, 0x79, 0x22, 0x50, 0x0a, 0x1a, 0x43, 0x68, 0x65, 0x63, 0x09, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x73, 0x12, 0x1d, 0x0a, 0x0a, 0x70, 0x75,
0x6b, 0x50, 0x72, 0x65, 0x72, 0x65, 0x71, 0x75, 0x69, 0x73, 0x69, 0x74, 0x65, 0x73, 0x52, 0x65, 0x62, 0x6c, 0x69, 0x63, 0x5f, 0x6b, 0x65, 0x79, 0x18, 0x04, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x09,
0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x1c, 0x0a, 0x09, 0x73, 0x61, 0x74, 0x69, 0x73, 0x66, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x4b, 0x65, 0x79, 0x22, 0x50, 0x0a, 0x1a, 0x43, 0x68, 0x65,
0x69, 0x65, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x08, 0x52, 0x09, 0x73, 0x61, 0x74, 0x69, 0x73, 0x63, 0x6b, 0x50, 0x72, 0x65, 0x72, 0x65, 0x71, 0x75, 0x69, 0x73, 0x69, 0x74, 0x65, 0x73, 0x52,
0x66, 0x69, 0x65, 0x64, 0x12, 0x14, 0x0a, 0x05, 0x65, 0x72, 0x72, 0x6f, 0x72, 0x18, 0x02, 0x20, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x1c, 0x0a, 0x09, 0x73, 0x61, 0x74, 0x69, 0x73,
0x01, 0x28, 0x09, 0x52, 0x05, 0x65, 0x72, 0x72, 0x6f, 0x72, 0x22, 0xc3, 0x01, 0x0a, 0x12, 0x49, 0x66, 0x69, 0x65, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x08, 0x52, 0x09, 0x73, 0x61, 0x74, 0x69,
0x6e, 0x69, 0x74, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x73, 0x66, 0x69, 0x65, 0x64, 0x12, 0x14, 0x0a, 0x05, 0x65, 0x72, 0x72, 0x6f, 0x72, 0x18, 0x02,
0x74, 0x12, 0x20, 0x0a, 0x0b, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x4e, 0x61, 0x6d, 0x65, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x65, 0x72, 0x72, 0x6f, 0x72, 0x22, 0xc3, 0x01, 0x0a, 0x12,
0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x4e, 0x49, 0x6e, 0x69, 0x74, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x52, 0x65, 0x71, 0x75, 0x65,
0x61, 0x6d, 0x65, 0x12, 0x27, 0x0a, 0x07, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x18, 0x02, 0x73, 0x74, 0x12, 0x20, 0x0a, 0x0b, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x4e, 0x61, 0x6d,
0x20, 0x01, 0x28, 0x0b, 0x32, 0x0d, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x50, 0x50, 0x72, 0x65, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0b, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65,
0x66, 0x69, 0x78, 0x52, 0x07, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x12, 0x26, 0x0a, 0x09, 0x4e, 0x61, 0x6d, 0x65, 0x12, 0x27, 0x0a, 0x07, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x18,
0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f, 0x69, 0x70, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x0d, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x50, 0x50, 0x72,
0x07, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x50, 0x48, 0x00, 0x52, 0x08, 0x70, 0x75, 0x62, 0x6c, 0x65, 0x66, 0x69, 0x78, 0x52, 0x07, 0x6e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x12, 0x26, 0x0a,
0x69, 0x63, 0x49, 0x70, 0x12, 0x26, 0x0a, 0x0e, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f, 0x69, 0x09, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f, 0x69, 0x70, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b,
0x70, 0x5f, 0x61, 0x75, 0x74, 0x6f, 0x18, 0x04, 0x20, 0x01, 0x28, 0x08, 0x48, 0x00, 0x52, 0x0c, 0x32, 0x07, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x50, 0x48, 0x00, 0x52, 0x08, 0x70, 0x75, 0x62,
0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x49, 0x70, 0x41, 0x75, 0x74, 0x6f, 0x42, 0x12, 0x0a, 0x10, 0x6c, 0x69, 0x63, 0x49, 0x70, 0x12, 0x26, 0x0a, 0x0e, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f,
0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f, 0x69, 0x70, 0x5f, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x69, 0x70, 0x5f, 0x61, 0x75, 0x74, 0x6f, 0x18, 0x04, 0x20, 0x01, 0x28, 0x08, 0x48, 0x00, 0x52,
0x22, 0x41, 0x0a, 0x13, 0x49, 0x6e, 0x69, 0x74, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x52, 0x0c, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x49, 0x70, 0x41, 0x75, 0x74, 0x6f, 0x42, 0x12, 0x0a,
0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x2a, 0x0a, 0x07, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x10, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f, 0x69, 0x70, 0x5f, 0x63, 0x6f, 0x6e, 0x66, 0x69,
0x6e, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4d, 0x67, 0x22, 0x41, 0x0a, 0x13, 0x49, 0x6e, 0x69, 0x74, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72,
0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x07, 0x6d, 0x61, 0x63, 0x68, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x2a, 0x0a, 0x07, 0x6d, 0x61, 0x63, 0x68,
0x69, 0x6e, 0x65, 0x22, 0x79, 0x0a, 0x12, 0x4a, 0x6f, 0x69, 0x6e, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x69, 0x6e, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x61, 0x70, 0x69, 0x2e,
0x65, 0x72, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x2a, 0x0a, 0x07, 0x6d, 0x61, 0x63, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x07, 0x6d, 0x61, 0x63,
0x68, 0x69, 0x6e, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x61, 0x70, 0x69, 0x68, 0x69, 0x6e, 0x65, 0x22, 0xaa, 0x01, 0x0a, 0x12, 0x4a, 0x6f, 0x69, 0x6e, 0x43, 0x6c, 0x75,
0x2e, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x07, 0x6d, 0x61, 0x73, 0x74, 0x65, 0x72, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x2a, 0x0a, 0x07, 0x6d,
0x63, 0x68, 0x69, 0x6e, 0x65, 0x12, 0x37, 0x0a, 0x0e, 0x6f, 0x74, 0x68, 0x65, 0x72, 0x5f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x61,
0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x70, 0x69, 0x2e, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x07,
0x61, 0x70, 0x69, 0x2e, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x12, 0x37, 0x0a, 0x0e, 0x6f, 0x74, 0x68, 0x65, 0x72,
0x0d, 0x6f, 0x74, 0x68, 0x65, 0x72, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x22, 0x25, 0x5f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x18, 0x03, 0x20, 0x03, 0x28, 0x0b, 0x32,
0x0a, 0x0d, 0x54, 0x6f, 0x6b, 0x65, 0x6e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x10, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66,
0x14, 0x0a, 0x05, 0x74, 0x6f, 0x6b, 0x65, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x05, 0x6f, 0x52, 0x0d, 0x6f, 0x74, 0x68, 0x65, 0x72, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x73,
0x74, 0x6f, 0x6b, 0x65, 0x6e, 0x22, 0x0e, 0x0a, 0x0c, 0x52, 0x65, 0x73, 0x65, 0x74, 0x52, 0x65, 0x12, 0x2f, 0x0a, 0x14, 0x6d, 0x69, 0x6e, 0x5f, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x5f, 0x64, 0x62,
0x71, 0x75, 0x65, 0x73, 0x74, 0x22, 0xc3, 0x01, 0x0a, 0x07, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x5f, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x18, 0x04, 0x20, 0x01, 0x28, 0x03, 0x52, 0x11,
0x65, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x02, 0x69, 0x6d, 0x69, 0x6e, 0x53, 0x74, 0x6f, 0x72, 0x65, 0x44, 0x62, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f,
0x64, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x6e, 0x22, 0x49, 0x0a, 0x16, 0x49, 0x6e, 0x73, 0x70, 0x65, 0x63, 0x74, 0x4d, 0x61, 0x63, 0x68,
0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x6d, 0x6f, 0x64, 0x65, 0x18, 0x03, 0x20, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x2f, 0x0a, 0x08, 0x6d,
0x01, 0x28, 0x09, 0x52, 0x04, 0x6d, 0x6f, 0x64, 0x65, 0x12, 0x36, 0x0a, 0x0a, 0x63, 0x6f, 0x6e, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x18, 0x01, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x13, 0x2e,
0x74, 0x61, 0x69, 0x6e, 0x65, 0x72, 0x73, 0x18, 0x04, 0x20, 0x03, 0x28, 0x0b, 0x32, 0x16, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x44, 0x65, 0x74, 0x61, 0x69,
0x61, 0x70, 0x69, 0x2e, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x2e, 0x43, 0x6f, 0x6e, 0x74, 0x6c, 0x73, 0x52, 0x08, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x73, 0x22, 0x91, 0x01, 0x0a,
0x61, 0x69, 0x6e, 0x65, 0x72, 0x52, 0x0a, 0x63, 0x6f, 0x6e, 0x74, 0x61, 0x69, 0x6e, 0x65, 0x72, 0x0e, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x44, 0x65, 0x74, 0x61, 0x69, 0x6c, 0x73, 0x12,
0x73, 0x1a, 0x48, 0x0a, 0x09, 0x43, 0x6f, 0x6e, 0x74, 0x61, 0x69, 0x6e, 0x65, 0x72, 0x12, 0x1d, 0x29, 0x0a, 0x08, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x18, 0x01, 0x20, 0x01, 0x28,
0x0a, 0x0a, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x5f, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x0b, 0x32, 0x0d, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61,
0x28, 0x09, 0x52, 0x09, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x64, 0x12, 0x1c, 0x0a, 0x52, 0x08, 0x6d, 0x65, 0x74, 0x61, 0x64, 0x61, 0x74, 0x61, 0x12, 0x2a, 0x0a, 0x07, 0x6d, 0x61,
0x09, 0x63, 0x6f, 0x6e, 0x74, 0x61, 0x69, 0x6e, 0x65, 0x72, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x10, 0x2e, 0x61, 0x70,
0x52, 0x09, 0x63, 0x6f, 0x6e, 0x74, 0x61, 0x69, 0x6e, 0x65, 0x72, 0x22, 0x27, 0x0a, 0x15, 0x49, 0x69, 0x2e, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x52, 0x07, 0x6d,
0x6e, 0x73, 0x70, 0x65, 0x63, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x52, 0x65, 0x71, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x12, 0x28, 0x0a, 0x10, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x5f,
0x75, 0x65, 0x73, 0x74, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x64, 0x62, 0x5f, 0x76, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e, 0x18, 0x03, 0x20, 0x01, 0x28, 0x03,
0x52, 0x02, 0x69, 0x64, 0x22, 0x40, 0x0a, 0x16, 0x49, 0x6e, 0x73, 0x70, 0x65, 0x63, 0x74, 0x53, 0x52, 0x0e, 0x73, 0x74, 0x6f, 0x72, 0x65, 0x44, 0x62, 0x56, 0x65, 0x72, 0x73, 0x69, 0x6f, 0x6e,
0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x26, 0x22, 0x25, 0x0a, 0x0d, 0x54, 0x6f, 0x6b, 0x65, 0x6e, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73,
0x0a, 0x07, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x65, 0x12, 0x14, 0x0a, 0x05, 0x74, 0x6f, 0x6b, 0x65, 0x6e, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09,
0x0c, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x52, 0x07, 0x73, 0x52, 0x05, 0x74, 0x6f, 0x6b, 0x65, 0x6e, 0x22, 0x0e, 0x0a, 0x0c, 0x52, 0x65, 0x73, 0x65, 0x74,
0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x32, 0xc3, 0x03, 0x0a, 0x07, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x22, 0xc3, 0x01, 0x0a, 0x07, 0x53, 0x65, 0x72, 0x76,
0x6e, 0x65, 0x12, 0x4d, 0x0a, 0x12, 0x43, 0x68, 0x65, 0x63, 0x6b, 0x50, 0x72, 0x65, 0x72, 0x65, 0x69, 0x63, 0x65, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52,
0x71, 0x75, 0x69, 0x73, 0x69, 0x74, 0x65, 0x73, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x02, 0x69, 0x64, 0x12, 0x12, 0x0a, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x02, 0x20, 0x01, 0x28,
0x09, 0x52, 0x04, 0x6e, 0x61, 0x6d, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x6d, 0x6f, 0x64, 0x65, 0x18,
0x03, 0x20, 0x01, 0x28, 0x09, 0x52, 0x04, 0x6d, 0x6f, 0x64, 0x65, 0x12, 0x36, 0x0a, 0x0a, 0x63,
0x6f, 0x6e, 0x74, 0x61, 0x69, 0x6e, 0x65, 0x72, 0x73, 0x18, 0x04, 0x20, 0x03, 0x28, 0x0b, 0x32,
0x16, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x2e, 0x43, 0x6f,
0x6e, 0x74, 0x61, 0x69, 0x6e, 0x65, 0x72, 0x52, 0x0a, 0x63, 0x6f, 0x6e, 0x74, 0x61, 0x69, 0x6e,
0x65, 0x72, 0x73, 0x1a, 0x48, 0x0a, 0x09, 0x43, 0x6f, 0x6e, 0x74, 0x61, 0x69, 0x6e, 0x65, 0x72,
0x12, 0x1d, 0x0a, 0x0a, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x5f, 0x69, 0x64, 0x18, 0x01,
0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x64, 0x12,
0x1c, 0x0a, 0x09, 0x63, 0x6f, 0x6e, 0x74, 0x61, 0x69, 0x6e, 0x65, 0x72, 0x18, 0x02, 0x20, 0x01,
0x28, 0x0c, 0x52, 0x09, 0x63, 0x6f, 0x6e, 0x74, 0x61, 0x69, 0x6e, 0x65, 0x72, 0x22, 0x27, 0x0a,
0x15, 0x49, 0x6e, 0x73, 0x70, 0x65, 0x63, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x52,
0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x0e, 0x0a, 0x02, 0x69, 0x64, 0x18, 0x01, 0x20, 0x01,
0x28, 0x09, 0x52, 0x02, 0x69, 0x64, 0x22, 0x40, 0x0a, 0x16, 0x49, 0x6e, 0x73, 0x70, 0x65, 0x63,
0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x12, 0x26, 0x0a, 0x07, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28,
0x0b, 0x32, 0x0c, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x52,
0x07, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x22, 0xb2, 0x01, 0x0a, 0x1f, 0x49, 0x6e, 0x73,
0x70, 0x65, 0x63, 0x74, 0x57, 0x69, 0x72, 0x65, 0x47, 0x75, 0x61, 0x72, 0x64, 0x4e, 0x65, 0x74,
0x77, 0x6f, 0x72, 0x6b, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x25, 0x0a, 0x0e,
0x69, 0x6e, 0x74, 0x65, 0x72, 0x66, 0x61, 0x63, 0x65, 0x5f, 0x6e, 0x61, 0x6d, 0x65, 0x18, 0x01,
0x20, 0x01, 0x28, 0x09, 0x52, 0x0d, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x66, 0x61, 0x63, 0x65, 0x4e,
0x61, 0x6d, 0x65, 0x12, 0x1d, 0x0a, 0x0a, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f, 0x6b, 0x65,
0x79, 0x18, 0x02, 0x20, 0x01, 0x28, 0x0c, 0x52, 0x09, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x4b,
0x65, 0x79, 0x12, 0x1f, 0x0a, 0x0b, 0x6c, 0x69, 0x73, 0x74, 0x65, 0x6e, 0x5f, 0x70, 0x6f, 0x72,
0x74, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05, 0x52, 0x0a, 0x6c, 0x69, 0x73, 0x74, 0x65, 0x6e, 0x50,
0x6f, 0x72, 0x74, 0x12, 0x28, 0x0a, 0x05, 0x70, 0x65, 0x65, 0x72, 0x73, 0x18, 0x04, 0x20, 0x03,
0x28, 0x0b, 0x32, 0x12, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x57, 0x69, 0x72, 0x65, 0x47, 0x75, 0x61,
0x72, 0x64, 0x50, 0x65, 0x65, 0x72, 0x52, 0x05, 0x70, 0x65, 0x65, 0x72, 0x73, 0x22, 0x83, 0x02,
0x0a, 0x0d, 0x57, 0x69, 0x72, 0x65, 0x47, 0x75, 0x61, 0x72, 0x64, 0x50, 0x65, 0x65, 0x72, 0x12,
0x1d, 0x0a, 0x0a, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x5f, 0x6b, 0x65, 0x79, 0x18, 0x01, 0x20,
0x01, 0x28, 0x0c, 0x52, 0x09, 0x70, 0x75, 0x62, 0x6c, 0x69, 0x63, 0x4b, 0x65, 0x79, 0x12, 0x1a,
0x0a, 0x08, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09,
0x52, 0x08, 0x65, 0x6e, 0x64, 0x70, 0x6f, 0x69, 0x6e, 0x74, 0x12, 0x4a, 0x0a, 0x13, 0x6c, 0x61,
0x73, 0x74, 0x5f, 0x68, 0x61, 0x6e, 0x64, 0x73, 0x68, 0x61, 0x6b, 0x65, 0x5f, 0x74, 0x69, 0x6d,
0x65, 0x18, 0x03, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65,
0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74,
0x61, 0x6d, 0x70, 0x52, 0x11, 0x6c, 0x61, 0x73, 0x74, 0x48, 0x61, 0x6e, 0x64, 0x73, 0x68, 0x61,
0x6b, 0x65, 0x54, 0x69, 0x6d, 0x65, 0x12, 0x23, 0x0a, 0x0d, 0x72, 0x65, 0x63, 0x65, 0x69, 0x76,
0x65, 0x5f, 0x62, 0x79, 0x74, 0x65, 0x73, 0x18, 0x04, 0x20, 0x01, 0x28, 0x03, 0x52, 0x0c, 0x72,
0x65, 0x63, 0x65, 0x69, 0x76, 0x65, 0x42, 0x79, 0x74, 0x65, 0x73, 0x12, 0x25, 0x0a, 0x0e, 0x74,
0x72, 0x61, 0x6e, 0x73, 0x6d, 0x69, 0x74, 0x5f, 0x62, 0x79, 0x74, 0x65, 0x73, 0x18, 0x05, 0x20,
0x01, 0x28, 0x03, 0x52, 0x0d, 0x74, 0x72, 0x61, 0x6e, 0x73, 0x6d, 0x69, 0x74, 0x42, 0x79, 0x74,
0x65, 0x73, 0x12, 0x1f, 0x0a, 0x0b, 0x61, 0x6c, 0x6c, 0x6f, 0x77, 0x65, 0x64, 0x5f, 0x69, 0x70,
0x73, 0x18, 0x06, 0x20, 0x03, 0x28, 0x09, 0x52, 0x0a, 0x61, 0x6c, 0x6c, 0x6f, 0x77, 0x65, 0x64,
0x49, 0x70, 0x73, 0x32, 0xe3, 0x04, 0x0a, 0x07, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x12,
0x4d, 0x0a, 0x12, 0x43, 0x68, 0x65, 0x63, 0x6b, 0x50, 0x72, 0x65, 0x72, 0x65, 0x71, 0x75, 0x69,
0x73, 0x69, 0x74, 0x65, 0x73, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x1f, 0x2e,
0x61, 0x70, 0x69, 0x2e, 0x43, 0x68, 0x65, 0x63, 0x6b, 0x50, 0x72, 0x65, 0x72, 0x65, 0x71, 0x75,
0x69, 0x73, 0x69, 0x74, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x40,
0x0a, 0x0b, 0x49, 0x6e, 0x69, 0x74, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x12, 0x17, 0x2e,
0x61, 0x70, 0x69, 0x2e, 0x49, 0x6e, 0x69, 0x74, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x52,
0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x18, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x6e, 0x69,
0x74, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65,
0x12, 0x3e, 0x0a, 0x0b, 0x4a, 0x6f, 0x69, 0x6e, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x12,
0x17, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4a, 0x6f, 0x69, 0x6e, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65,
0x72, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c,
0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79,
0x1a, 0x1f, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x43, 0x68, 0x65, 0x63, 0x6b, 0x50, 0x72, 0x65, 0x72, 0x12, 0x33, 0x0a, 0x05, 0x54, 0x6f, 0x6b, 0x65, 0x6e, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67,
0x65, 0x71, 0x75, 0x69, 0x73, 0x69, 0x74, 0x65, 0x73, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73,
0x65, 0x12, 0x40, 0x0a, 0x0b, 0x49, 0x6e, 0x69, 0x74, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72,
0x12, 0x17, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x6e, 0x69, 0x74, 0x43, 0x6c, 0x75, 0x73, 0x74,
0x65, 0x72, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x18, 0x2e, 0x61, 0x70, 0x69, 0x2e,
0x49, 0x6e, 0x69, 0x74, 0x43, 0x6c, 0x75, 0x73, 0x74, 0x65, 0x72, 0x52, 0x65, 0x73, 0x70, 0x6f,
0x6e, 0x73, 0x65, 0x12, 0x3e, 0x0a, 0x0b, 0x4a, 0x6f, 0x69, 0x6e, 0x43, 0x6c, 0x75, 0x73, 0x74,
0x65, 0x72, 0x12, 0x17, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4a, 0x6f, 0x69, 0x6e, 0x43, 0x6c, 0x75,
0x73, 0x74, 0x65, 0x72, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x16, 0x2e, 0x67, 0x6f,
0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d,
0x70, 0x74, 0x79, 0x12, 0x33, 0x0a, 0x05, 0x54, 0x6f, 0x6b, 0x65, 0x6e, 0x12, 0x16, 0x2e, 0x67,
0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45,
0x6d, 0x70, 0x74, 0x79, 0x1a, 0x12, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x54, 0x6f, 0x6b, 0x65, 0x6e,
0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x33, 0x0a, 0x07, 0x49, 0x6e, 0x73, 0x70,
0x65, 0x63, 0x74, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f,
0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x10, 0x2e, 0x61, 0x70,
0x69, 0x2e, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x12, 0x32, 0x0a,
0x05, 0x52, 0x65, 0x73, 0x65, 0x74, 0x12, 0x11, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x52, 0x65, 0x73,
0x65, 0x74, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67,
0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74,
0x79, 0x12, 0x49, 0x0a, 0x0e, 0x49, 0x6e, 0x73, 0x70, 0x65, 0x63, 0x74, 0x53, 0x65, 0x72, 0x76, 0x79, 0x1a, 0x12, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x54, 0x6f, 0x6b, 0x65, 0x6e, 0x52, 0x65, 0x73,
0x69, 0x63, 0x65, 0x12, 0x1a, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x6e, 0x73, 0x70, 0x65, 0x63, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x33, 0x0a, 0x07, 0x49, 0x6e, 0x73, 0x70, 0x65, 0x63, 0x74,
0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62,
0x1b, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x6e, 0x73, 0x70, 0x65, 0x63, 0x74, 0x53, 0x65, 0x72, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x10, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x4d,
0x76, 0x69, 0x63, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x37, 0x5a, 0x35, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x49, 0x6e, 0x66, 0x6f, 0x12, 0x45, 0x0a, 0x0e, 0x49, 0x6e,
0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x70, 0x73, 0x76, 0x69, 0x64, 0x73, 0x70, 0x65, 0x63, 0x74, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x12, 0x16, 0x2e, 0x67,
0x65, 0x72, 0x73, 0x6b, 0x69, 0x2f, 0x75, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x69, 0x6e, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45,
0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x2f, 0x61, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x1b, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x6e, 0x73, 0x70, 0x65,
0x70, 0x69, 0x2f, 0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, 0x63, 0x74, 0x4d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73,
0x65, 0x12, 0x57, 0x0a, 0x17, 0x49, 0x6e, 0x73, 0x70, 0x65, 0x63, 0x74, 0x57, 0x69, 0x72, 0x65,
0x47, 0x75, 0x61, 0x72, 0x64, 0x4e, 0x65, 0x74, 0x77, 0x6f, 0x72, 0x6b, 0x12, 0x16, 0x2e, 0x67,
0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45,
0x6d, 0x70, 0x74, 0x79, 0x1a, 0x24, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x6e, 0x73, 0x70, 0x65,
0x63, 0x74, 0x57, 0x69, 0x72, 0x65, 0x47, 0x75, 0x61, 0x72, 0x64, 0x4e, 0x65, 0x74, 0x77, 0x6f,
0x72, 0x6b, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x32, 0x0a, 0x05, 0x52, 0x65,
0x73, 0x65, 0x74, 0x12, 0x11, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x52, 0x65, 0x73, 0x65, 0x74, 0x52,
0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x12, 0x49,
0x0a, 0x0e, 0x49, 0x6e, 0x73, 0x70, 0x65, 0x63, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65,
0x12, 0x1a, 0x2e, 0x61, 0x70, 0x69, 0x2e, 0x49, 0x6e, 0x73, 0x70, 0x65, 0x63, 0x74, 0x53, 0x65,
0x72, 0x76, 0x69, 0x63, 0x65, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x1b, 0x2e, 0x61,
0x70, 0x69, 0x2e, 0x49, 0x6e, 0x73, 0x70, 0x65, 0x63, 0x74, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63,
0x65, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x37, 0x5a, 0x35, 0x67, 0x69, 0x74,
0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x70, 0x73, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73,
0x6b, 0x69, 0x2f, 0x75, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72,
0x6e, 0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f,
0x70, 0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
} }
var ( var (
@@ -853,57 +1190,72 @@ func file_internal_machine_api_pb_machine_proto_rawDescGZIP() []byte {
return file_internal_machine_api_pb_machine_proto_rawDescData return file_internal_machine_api_pb_machine_proto_rawDescData
} }
var file_internal_machine_api_pb_machine_proto_msgTypes = make([]protoimpl.MessageInfo, 12) var file_internal_machine_api_pb_machine_proto_msgTypes = make([]protoimpl.MessageInfo, 16)
var file_internal_machine_api_pb_machine_proto_goTypes = []any{ var file_internal_machine_api_pb_machine_proto_goTypes = []any{
(*MachineInfo)(nil), // 0: api.MachineInfo (*MachineInfo)(nil), // 0: api.MachineInfo
(*NetworkConfig)(nil), // 1: api.NetworkConfig (*NetworkConfig)(nil), // 1: api.NetworkConfig
(*CheckPrerequisitesResponse)(nil), // 2: api.CheckPrerequisitesResponse (*CheckPrerequisitesResponse)(nil), // 2: api.CheckPrerequisitesResponse
(*InitClusterRequest)(nil), // 3: api.InitClusterRequest (*InitClusterRequest)(nil), // 3: api.InitClusterRequest
(*InitClusterResponse)(nil), // 4: api.InitClusterResponse (*InitClusterResponse)(nil), // 4: api.InitClusterResponse
(*JoinClusterRequest)(nil), // 5: api.JoinClusterRequest (*JoinClusterRequest)(nil), // 5: api.JoinClusterRequest
(*TokenResponse)(nil), // 6: api.TokenResponse (*InspectMachineResponse)(nil), // 6: api.InspectMachineResponse
(*ResetRequest)(nil), // 7: api.ResetRequest (*MachineDetails)(nil), // 7: api.MachineDetails
(*Service)(nil), // 8: api.Service (*TokenResponse)(nil), // 8: api.TokenResponse
(*InspectServiceRequest)(nil), // 9: api.InspectServiceRequest (*ResetRequest)(nil), // 9: api.ResetRequest
(*InspectServiceResponse)(nil), // 10: api.InspectServiceResponse (*Service)(nil), // 10: api.Service
(*Service_Container)(nil), // 11: api.Service.Container (*InspectServiceRequest)(nil), // 11: api.InspectServiceRequest
(*IP)(nil), // 12: api.IP (*InspectServiceResponse)(nil), // 12: api.InspectServiceResponse
(*IPPrefix)(nil), // 13: api.IPPrefix (*InspectWireGuardNetworkResponse)(nil), // 13: api.InspectWireGuardNetworkResponse
(*IPPort)(nil), // 14: api.IPPort (*WireGuardPeer)(nil), // 14: api.WireGuardPeer
(*emptypb.Empty)(nil), // 15: google.protobuf.Empty (*Service_Container)(nil), // 15: api.Service.Container
(*IP)(nil), // 16: api.IP
(*IPPrefix)(nil), // 17: api.IPPrefix
(*IPPort)(nil), // 18: api.IPPort
(*Metadata)(nil), // 19: api.Metadata
(*timestamppb.Timestamp)(nil), // 20: google.protobuf.Timestamp
(*emptypb.Empty)(nil), // 21: google.protobuf.Empty
} }
var file_internal_machine_api_pb_machine_proto_depIdxs = []int32{ var file_internal_machine_api_pb_machine_proto_depIdxs = []int32{
1, // 0: api.MachineInfo.network:type_name -> api.NetworkConfig 1, // 0: api.MachineInfo.network:type_name -> api.NetworkConfig
12, // 1: api.MachineInfo.public_ip:type_name -> api.IP 16, // 1: api.MachineInfo.public_ip:type_name -> api.IP
13, // 2: api.NetworkConfig.subnet:type_name -> api.IPPrefix 17, // 2: api.NetworkConfig.subnet:type_name -> api.IPPrefix
12, // 3: api.NetworkConfig.management_ip:type_name -> api.IP 16, // 3: api.NetworkConfig.management_ip:type_name -> api.IP
14, // 4: api.NetworkConfig.endpoints:type_name -> api.IPPort 18, // 4: api.NetworkConfig.endpoints:type_name -> api.IPPort
13, // 5: api.InitClusterRequest.network:type_name -> api.IPPrefix 17, // 5: api.InitClusterRequest.network:type_name -> api.IPPrefix
12, // 6: api.InitClusterRequest.public_ip:type_name -> api.IP 16, // 6: api.InitClusterRequest.public_ip:type_name -> api.IP
0, // 7: api.InitClusterResponse.machine:type_name -> api.MachineInfo 0, // 7: api.InitClusterResponse.machine:type_name -> api.MachineInfo
0, // 8: api.JoinClusterRequest.machine:type_name -> api.MachineInfo 0, // 8: api.JoinClusterRequest.machine:type_name -> api.MachineInfo
0, // 9: api.JoinClusterRequest.other_machines:type_name -> api.MachineInfo 0, // 9: api.JoinClusterRequest.other_machines:type_name -> api.MachineInfo
11, // 10: api.Service.containers:type_name -> api.Service.Container 7, // 10: api.InspectMachineResponse.machines:type_name -> api.MachineDetails
8, // 11: api.InspectServiceResponse.service:type_name -> api.Service 19, // 11: api.MachineDetails.metadata:type_name -> api.Metadata
15, // 12: api.Machine.CheckPrerequisites:input_type -> google.protobuf.Empty 0, // 12: api.MachineDetails.machine:type_name -> api.MachineInfo
3, // 13: api.Machine.InitCluster:input_type -> api.InitClusterRequest 15, // 13: api.Service.containers:type_name -> api.Service.Container
5, // 14: api.Machine.JoinCluster:input_type -> api.JoinClusterRequest 10, // 14: api.InspectServiceResponse.service:type_name -> api.Service
15, // 15: api.Machine.Token:input_type -> google.protobuf.Empty 14, // 15: api.InspectWireGuardNetworkResponse.peers:type_name -> api.WireGuardPeer
15, // 16: api.Machine.Inspect:input_type -> google.protobuf.Empty 20, // 16: api.WireGuardPeer.last_handshake_time:type_name -> google.protobuf.Timestamp
7, // 17: api.Machine.Reset:input_type -> api.ResetRequest 21, // 17: api.Machine.CheckPrerequisites:input_type -> google.protobuf.Empty
9, // 18: api.Machine.InspectService:input_type -> api.InspectServiceRequest 3, // 18: api.Machine.InitCluster:input_type -> api.InitClusterRequest
2, // 19: api.Machine.CheckPrerequisites:output_type -> api.CheckPrerequisitesResponse 5, // 19: api.Machine.JoinCluster:input_type -> api.JoinClusterRequest
4, // 20: api.Machine.InitCluster:output_type -> api.InitClusterResponse 21, // 20: api.Machine.Token:input_type -> google.protobuf.Empty
15, // 21: api.Machine.JoinCluster:output_type -> google.protobuf.Empty 21, // 21: api.Machine.Inspect:input_type -> google.protobuf.Empty
6, // 22: api.Machine.Token:output_type -> api.TokenResponse 21, // 22: api.Machine.InspectMachine:input_type -> google.protobuf.Empty
0, // 23: api.Machine.Inspect:output_type -> api.MachineInfo 21, // 23: api.Machine.InspectWireGuardNetwork:input_type -> google.protobuf.Empty
15, // 24: api.Machine.Reset:output_type -> google.protobuf.Empty 9, // 24: api.Machine.Reset:input_type -> api.ResetRequest
10, // 25: api.Machine.InspectService:output_type -> api.InspectServiceResponse 11, // 25: api.Machine.InspectService:input_type -> api.InspectServiceRequest
19, // [19:26] is the sub-list for method output_type 2, // 26: api.Machine.CheckPrerequisites:output_type -> api.CheckPrerequisitesResponse
12, // [12:19] is the sub-list for method input_type 4, // 27: api.Machine.InitCluster:output_type -> api.InitClusterResponse
12, // [12:12] is the sub-list for extension type_name 21, // 28: api.Machine.JoinCluster:output_type -> google.protobuf.Empty
12, // [12:12] is the sub-list for extension extendee 8, // 29: api.Machine.Token:output_type -> api.TokenResponse
0, // [0:12] is the sub-list for field type_name 0, // 30: api.Machine.Inspect:output_type -> api.MachineInfo
6, // 31: api.Machine.InspectMachine:output_type -> api.InspectMachineResponse
13, // 32: api.Machine.InspectWireGuardNetwork:output_type -> api.InspectWireGuardNetworkResponse
21, // 33: api.Machine.Reset:output_type -> google.protobuf.Empty
12, // 34: api.Machine.InspectService:output_type -> api.InspectServiceResponse
26, // [26:35] is the sub-list for method output_type
17, // [17:26] is the sub-list for method input_type
17, // [17:17] is the sub-list for extension type_name
17, // [17:17] is the sub-list for extension extendee
0, // [0:17] is the sub-list for field type_name
} }
func init() { file_internal_machine_api_pb_machine_proto_init() } func init() { file_internal_machine_api_pb_machine_proto_init() }
@@ -986,7 +1338,7 @@ func file_internal_machine_api_pb_machine_proto_init() {
} }
} }
file_internal_machine_api_pb_machine_proto_msgTypes[6].Exporter = func(v any, i int) any { file_internal_machine_api_pb_machine_proto_msgTypes[6].Exporter = func(v any, i int) any {
switch v := v.(*TokenResponse); i { switch v := v.(*InspectMachineResponse); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -998,7 +1350,7 @@ func file_internal_machine_api_pb_machine_proto_init() {
} }
} }
file_internal_machine_api_pb_machine_proto_msgTypes[7].Exporter = func(v any, i int) any { file_internal_machine_api_pb_machine_proto_msgTypes[7].Exporter = func(v any, i int) any {
switch v := v.(*ResetRequest); i { switch v := v.(*MachineDetails); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -1010,7 +1362,7 @@ func file_internal_machine_api_pb_machine_proto_init() {
} }
} }
file_internal_machine_api_pb_machine_proto_msgTypes[8].Exporter = func(v any, i int) any { file_internal_machine_api_pb_machine_proto_msgTypes[8].Exporter = func(v any, i int) any {
switch v := v.(*Service); i { switch v := v.(*TokenResponse); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -1022,7 +1374,7 @@ func file_internal_machine_api_pb_machine_proto_init() {
} }
} }
file_internal_machine_api_pb_machine_proto_msgTypes[9].Exporter = func(v any, i int) any { file_internal_machine_api_pb_machine_proto_msgTypes[9].Exporter = func(v any, i int) any {
switch v := v.(*InspectServiceRequest); i { switch v := v.(*ResetRequest); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -1034,7 +1386,7 @@ func file_internal_machine_api_pb_machine_proto_init() {
} }
} }
file_internal_machine_api_pb_machine_proto_msgTypes[10].Exporter = func(v any, i int) any { file_internal_machine_api_pb_machine_proto_msgTypes[10].Exporter = func(v any, i int) any {
switch v := v.(*InspectServiceResponse); i { switch v := v.(*Service); i {
case 0: case 0:
return &v.state return &v.state
case 1: case 1:
@@ -1046,6 +1398,54 @@ func file_internal_machine_api_pb_machine_proto_init() {
} }
} }
file_internal_machine_api_pb_machine_proto_msgTypes[11].Exporter = func(v any, i int) any { file_internal_machine_api_pb_machine_proto_msgTypes[11].Exporter = func(v any, i int) any {
switch v := v.(*InspectServiceRequest); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_internal_machine_api_pb_machine_proto_msgTypes[12].Exporter = func(v any, i int) any {
switch v := v.(*InspectServiceResponse); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_internal_machine_api_pb_machine_proto_msgTypes[13].Exporter = func(v any, i int) any {
switch v := v.(*InspectWireGuardNetworkResponse); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_internal_machine_api_pb_machine_proto_msgTypes[14].Exporter = func(v any, i int) any {
switch v := v.(*WireGuardPeer); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
file_internal_machine_api_pb_machine_proto_msgTypes[15].Exporter = func(v any, i int) any {
switch v := v.(*Service_Container); i { switch v := v.(*Service_Container); i {
case 0: case 0:
return &v.state return &v.state
@@ -1068,7 +1468,7 @@ func file_internal_machine_api_pb_machine_proto_init() {
GoPackagePath: reflect.TypeOf(x{}).PkgPath(), GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_internal_machine_api_pb_machine_proto_rawDesc, RawDescriptor: file_internal_machine_api_pb_machine_proto_rawDesc,
NumEnums: 0, NumEnums: 0,
NumMessages: 12, NumMessages: 16,
NumExtensions: 0, NumExtensions: 0,
NumServices: 1, NumServices: 1,
}, },
+36
View File
@@ -5,6 +5,7 @@ package api;
option go_package = "github.com/psviderski/uncloud/internal/machine/api/pb"; option go_package = "github.com/psviderski/uncloud/internal/machine/api/pb";
import "google/protobuf/empty.proto"; import "google/protobuf/empty.proto";
import "google/protobuf/timestamp.proto";
import "internal/machine/api/pb/common.proto"; import "internal/machine/api/pb/common.proto";
service Machine { service Machine {
@@ -13,7 +14,12 @@ service Machine {
rpc InitCluster(InitClusterRequest) returns (InitClusterResponse); rpc InitCluster(InitClusterRequest) returns (InitClusterResponse);
rpc JoinCluster(JoinClusterRequest) returns (google.protobuf.Empty); rpc JoinCluster(JoinClusterRequest) returns (google.protobuf.Empty);
rpc Token(google.protobuf.Empty) returns (TokenResponse); rpc Token(google.protobuf.Empty) returns (TokenResponse);
// Deprecated: use InspectMachine instead.
rpc Inspect(google.protobuf.Empty) returns (MachineInfo); rpc Inspect(google.protobuf.Empty) returns (MachineInfo);
// InspectMachine retrieves detailed information about the machine. Supports broadcasting to multiple machines.
rpc InspectMachine(google.protobuf.Empty) returns (InspectMachineResponse);
// InspectWireGuardNetwork retrieves the current WireGuard network configuration and peer status.
rpc InspectWireGuardNetwork(google.protobuf.Empty) returns (InspectWireGuardNetworkResponse);
// Reset restores the machine to a clean state, removing all cluster-related configuration and data. // Reset restores the machine to a clean state, removing all cluster-related configuration and data.
rpc Reset(ResetRequest) returns (google.protobuf.Empty); rpc Reset(ResetRequest) returns (google.protobuf.Empty);
@@ -58,6 +64,20 @@ message InitClusterResponse {
message JoinClusterRequest { message JoinClusterRequest {
MachineInfo machine = 1; MachineInfo machine = 1;
repeated MachineInfo other_machines = 3; repeated MachineInfo other_machines = 3;
// Minimum store database version the new machine should sync to before starting cluster operations.
int64 min_store_db_version = 4;
}
message InspectMachineResponse {
// Must contain only one repeated messages field to allow broadcasting InspectMachine requests to multiple machines.
repeated MachineDetails machines = 1;
}
message MachineDetails {
Metadata metadata = 1;
MachineInfo machine = 2;
// Current Corrosion cr-sqlite database version (Lamport timestamp) of the cluster store.
int64 store_db_version = 3;
} }
message TokenResponse { message TokenResponse {
@@ -88,3 +108,19 @@ message InspectServiceRequest {
message InspectServiceResponse { message InspectServiceResponse {
Service service = 1; Service service = 1;
} }
message InspectWireGuardNetworkResponse {
string interface_name = 1;
bytes public_key = 2;
int32 listen_port = 3;
repeated WireGuardPeer peers = 4;
}
message WireGuardPeer {
bytes public_key = 1;
string endpoint = 2;
google.protobuf.Timestamp last_handshake_time = 3;
int64 receive_bytes = 4;
int64 transmit_bytes = 5;
repeated string allowed_ips = 6;
}
+89 -7
View File
@@ -20,13 +20,15 @@ import (
const _ = grpc.SupportPackageIsVersion9 const _ = grpc.SupportPackageIsVersion9
const ( const (
Machine_CheckPrerequisites_FullMethodName = "/api.Machine/CheckPrerequisites" Machine_CheckPrerequisites_FullMethodName = "/api.Machine/CheckPrerequisites"
Machine_InitCluster_FullMethodName = "/api.Machine/InitCluster" Machine_InitCluster_FullMethodName = "/api.Machine/InitCluster"
Machine_JoinCluster_FullMethodName = "/api.Machine/JoinCluster" Machine_JoinCluster_FullMethodName = "/api.Machine/JoinCluster"
Machine_Token_FullMethodName = "/api.Machine/Token" Machine_Token_FullMethodName = "/api.Machine/Token"
Machine_Inspect_FullMethodName = "/api.Machine/Inspect" Machine_Inspect_FullMethodName = "/api.Machine/Inspect"
Machine_Reset_FullMethodName = "/api.Machine/Reset" Machine_InspectMachine_FullMethodName = "/api.Machine/InspectMachine"
Machine_InspectService_FullMethodName = "/api.Machine/InspectService" Machine_InspectWireGuardNetwork_FullMethodName = "/api.Machine/InspectWireGuardNetwork"
Machine_Reset_FullMethodName = "/api.Machine/Reset"
Machine_InspectService_FullMethodName = "/api.Machine/InspectService"
) )
// MachineClient is the client API for Machine service. // MachineClient is the client API for Machine service.
@@ -38,7 +40,12 @@ type MachineClient interface {
InitCluster(ctx context.Context, in *InitClusterRequest, opts ...grpc.CallOption) (*InitClusterResponse, error) InitCluster(ctx context.Context, in *InitClusterRequest, opts ...grpc.CallOption) (*InitClusterResponse, error)
JoinCluster(ctx context.Context, in *JoinClusterRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) JoinCluster(ctx context.Context, in *JoinClusterRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
Token(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*TokenResponse, error) Token(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*TokenResponse, error)
// Deprecated: use InspectMachine instead.
Inspect(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*MachineInfo, error) Inspect(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*MachineInfo, error)
// InspectMachine retrieves detailed information about the machine. Supports broadcasting to multiple machines.
InspectMachine(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*InspectMachineResponse, error)
// InspectWireGuardNetwork retrieves the current WireGuard network configuration and peer status.
InspectWireGuardNetwork(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*InspectWireGuardNetworkResponse, error)
// Reset restores the machine to a clean state, removing all cluster-related configuration and data. // Reset restores the machine to a clean state, removing all cluster-related configuration and data.
Reset(ctx context.Context, in *ResetRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) Reset(ctx context.Context, in *ResetRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
InspectService(ctx context.Context, in *InspectServiceRequest, opts ...grpc.CallOption) (*InspectServiceResponse, error) InspectService(ctx context.Context, in *InspectServiceRequest, opts ...grpc.CallOption) (*InspectServiceResponse, error)
@@ -102,6 +109,26 @@ func (c *machineClient) Inspect(ctx context.Context, in *emptypb.Empty, opts ...
return out, nil return out, nil
} }
func (c *machineClient) InspectMachine(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*InspectMachineResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(InspectMachineResponse)
err := c.cc.Invoke(ctx, Machine_InspectMachine_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *machineClient) InspectWireGuardNetwork(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*InspectWireGuardNetworkResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(InspectWireGuardNetworkResponse)
err := c.cc.Invoke(ctx, Machine_InspectWireGuardNetwork_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *machineClient) Reset(ctx context.Context, in *ResetRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) { func (c *machineClient) Reset(ctx context.Context, in *ResetRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...) cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(emptypb.Empty) out := new(emptypb.Empty)
@@ -131,7 +158,12 @@ type MachineServer interface {
InitCluster(context.Context, *InitClusterRequest) (*InitClusterResponse, error) InitCluster(context.Context, *InitClusterRequest) (*InitClusterResponse, error)
JoinCluster(context.Context, *JoinClusterRequest) (*emptypb.Empty, error) JoinCluster(context.Context, *JoinClusterRequest) (*emptypb.Empty, error)
Token(context.Context, *emptypb.Empty) (*TokenResponse, error) Token(context.Context, *emptypb.Empty) (*TokenResponse, error)
// Deprecated: use InspectMachine instead.
Inspect(context.Context, *emptypb.Empty) (*MachineInfo, error) Inspect(context.Context, *emptypb.Empty) (*MachineInfo, error)
// InspectMachine retrieves detailed information about the machine. Supports broadcasting to multiple machines.
InspectMachine(context.Context, *emptypb.Empty) (*InspectMachineResponse, error)
// InspectWireGuardNetwork retrieves the current WireGuard network configuration and peer status.
InspectWireGuardNetwork(context.Context, *emptypb.Empty) (*InspectWireGuardNetworkResponse, error)
// Reset restores the machine to a clean state, removing all cluster-related configuration and data. // Reset restores the machine to a clean state, removing all cluster-related configuration and data.
Reset(context.Context, *ResetRequest) (*emptypb.Empty, error) Reset(context.Context, *ResetRequest) (*emptypb.Empty, error)
InspectService(context.Context, *InspectServiceRequest) (*InspectServiceResponse, error) InspectService(context.Context, *InspectServiceRequest) (*InspectServiceResponse, error)
@@ -160,6 +192,12 @@ func (UnimplementedMachineServer) Token(context.Context, *emptypb.Empty) (*Token
func (UnimplementedMachineServer) Inspect(context.Context, *emptypb.Empty) (*MachineInfo, error) { func (UnimplementedMachineServer) Inspect(context.Context, *emptypb.Empty) (*MachineInfo, error) {
return nil, status.Errorf(codes.Unimplemented, "method Inspect not implemented") return nil, status.Errorf(codes.Unimplemented, "method Inspect not implemented")
} }
func (UnimplementedMachineServer) InspectMachine(context.Context, *emptypb.Empty) (*InspectMachineResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method InspectMachine not implemented")
}
func (UnimplementedMachineServer) InspectWireGuardNetwork(context.Context, *emptypb.Empty) (*InspectWireGuardNetworkResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method InspectWireGuardNetwork not implemented")
}
func (UnimplementedMachineServer) Reset(context.Context, *ResetRequest) (*emptypb.Empty, error) { func (UnimplementedMachineServer) Reset(context.Context, *ResetRequest) (*emptypb.Empty, error) {
return nil, status.Errorf(codes.Unimplemented, "method Reset not implemented") return nil, status.Errorf(codes.Unimplemented, "method Reset not implemented")
} }
@@ -277,6 +315,42 @@ func _Machine_Inspect_Handler(srv interface{}, ctx context.Context, dec func(int
return interceptor(ctx, in, info, handler) return interceptor(ctx, in, info, handler)
} }
func _Machine_InspectMachine_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(emptypb.Empty)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(MachineServer).InspectMachine(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Machine_InspectMachine_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(MachineServer).InspectMachine(ctx, req.(*emptypb.Empty))
}
return interceptor(ctx, in, info, handler)
}
func _Machine_InspectWireGuardNetwork_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(emptypb.Empty)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(MachineServer).InspectWireGuardNetwork(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Machine_InspectWireGuardNetwork_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(MachineServer).InspectWireGuardNetwork(ctx, req.(*emptypb.Empty))
}
return interceptor(ctx, in, info, handler)
}
func _Machine_Reset_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { func _Machine_Reset_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ResetRequest) in := new(ResetRequest)
if err := dec(in); err != nil { if err := dec(in); err != nil {
@@ -340,6 +414,14 @@ var Machine_ServiceDesc = grpc.ServiceDesc{
MethodName: "Inspect", MethodName: "Inspect",
Handler: _Machine_Inspect_Handler, Handler: _Machine_Inspect_Handler,
}, },
{
MethodName: "InspectMachine",
Handler: _Machine_InspectMachine_Handler,
},
{
MethodName: "InspectWireGuardNetwork",
Handler: _Machine_InspectWireGuardNetwork_Handler,
},
{ {
MethodName: "Reset", MethodName: "Reset",
Handler: _Machine_Reset_Handler, Handler: _Machine_Reset_Handler,
+3 -1
View File
@@ -80,6 +80,8 @@ func (b *RemoteBackend) GetConnection(ctx context.Context, _ string) (context.Co
backoffConfig.MaxDelay = 15 * time.Second backoffConfig.MaxDelay = 15 * time.Second
var err error var err error
// This client keeps retrying connection in the background indefinitely even after the first connection fails
// and the Unavailable error is returned to the caller.
b.conn, err = grpc.NewClient( b.conn, err = grpc.NewClient(
b.target, b.target,
grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithTransportCredentials(insecure.NewCredentials()),
@@ -88,7 +90,7 @@ func (b *RemoteBackend) GetConnection(ctx context.Context, _ string) (context.Co
// Not published as a constant in gRPC library. // Not published as a constant in gRPC library.
// See: https://github.com/grpc/grpc-go/blob/d5dee5fdbdeb52f6ea10b37b2cc7ce37814642d7/clientconn.go#L55-L56 // See: https://github.com/grpc/grpc-go/blob/d5dee5fdbdeb52f6ea10b37b2cc7ce37814642d7/clientconn.go#L55-L56
// Each connection attempt can take up to MinConnectTimeout. // Each connection attempt can take up to MinConnectTimeout.
MinConnectTimeout: 20 * time.Second, MinConnectTimeout: 10 * time.Second,
}), }),
grpc.WithDefaultCallOptions( grpc.WithDefaultCallOptions(
grpc.ForceCodecV2(proxy.Codec()), grpc.ForceCodecV2(proxy.Codec()),
+8 -3
View File
@@ -12,14 +12,16 @@ import (
"strconv" "strconv"
"strings" "strings"
"text/template" "text/template"
"time"
"github.com/psviderski/uncloud/internal/machine/store" "github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
) )
const ( const (
caddyfileHeader = `# This file is autogenerated by Uncloud based on the configuration of running services. caddyfileHeaderFmt = `# Caddyfile autogenerated by Uncloud (DO NOT EDIT): %s
# Do not edit manually. Any manual changes will be overwritten on the next update. # Automatically updated on service or health status changes.
# Docs: https://uncloud.run/docs/concepts/ingress/overview
` `
caddyfileTemplate = `# Health check endpoint to verify Caddy reachability on this machine. caddyfileTemplate = `# Health check endpoint to verify Caddy reachability on this machine.
http:// { http:// {
@@ -56,7 +58,9 @@ https://{{$hostname}} {
}{{end}} }{{end}}
` `
caddyfileUnavailabeFooter = `# NOTE: User-defined configs for services were skipped because Caddy is not running on this machine caddyfileUnavailabeFooter = `# NOTE: User-defined configs for services were skipped because Caddy is not running on this machine
# or the latest generated config is invalid. Please check the Caddy logs if it's running. # (not accessible via the shared admin socket /run/uncloud/caddy/admin.sock) or the latest
# generated config is invalid. Please check the service 'caddy' is running (uc inspect caddy)
# and its logs for more details (uc logs caddy).
` `
) )
@@ -120,6 +124,7 @@ func (g *CaddyfileGenerator) Generate(
return "", fmt.Errorf("generate base Caddyfile from service ports: %w", err) return "", fmt.Errorf("generate base Caddyfile from service ports: %w", err)
} }
caddyfileHeader := fmt.Sprintf(caddyfileHeaderFmt, time.Now().UTC().Format(time.RFC3339))
if !includeCustom { if !includeCustom {
return fmt.Sprintf("%s\n%s\n%s", caddyfileHeader, caddyfile, caddyfileUnavailabeFooter), nil return fmt.Sprintf("%s\n%s\n%s", caddyfileHeader, caddyfile, caddyfileUnavailabeFooter), nil
} }
+30 -13
View File
@@ -3,6 +3,7 @@ package caddyconfig
import ( import (
"context" "context"
"errors" "errors"
"regexp"
"strings" "strings"
"testing" "testing"
"time" "time"
@@ -17,8 +18,17 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
const testCaddyfileHeader = `# This file is autogenerated by Uncloud based on the configuration of running services. // generatedTimestampRegex matches the "(DO NOT EDIT): <RFC3339 timestamp>" part of the header.
# Do not edit manually. Any manual changes will be overwritten on the next update. var generatedTimestampRegex = regexp.MustCompile(`\(DO NOT EDIT\): \d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z`)
// normaliseGeneratedTimestamp replaces dynamic timestamps with a placeholder for test comparison.
func normaliseGeneratedTimestamp(caddyfile string) string {
return generatedTimestampRegex.ReplaceAllString(caddyfile, "(DO NOT EDIT): TIMESTAMP_PLACEHOLDER")
}
const testCaddyfileHeader = `# Caddyfile autogenerated by Uncloud (DO NOT EDIT): TIMESTAMP_PLACEHOLDER
# Automatically updated on service or health status changes.
# Docs: https://uncloud.run/docs/concepts/ingress/overview
# Health check endpoint to verify Caddy reachability on this machine. # Health check endpoint to verify Caddy reachability on this machine.
http:// { http:// {
@@ -190,7 +200,7 @@ https://secure.example.com {
} }
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, tt.want, config, "Generated Caddyfile doesn't match") assert.Equal(t, tt.want, normaliseGeneratedTimestamp(config), "Generated Caddyfile doesn't match")
}) })
} }
} }
@@ -216,8 +226,9 @@ func TestCaddyfileGeneratorWithCustomConfigs(t *testing.T) {
time.Now(), time.Now(),
), ),
}, },
want: `# This file is autogenerated by Uncloud based on the configuration of running services. want: `# Caddyfile autogenerated by Uncloud (DO NOT EDIT): TIMESTAMP_PLACEHOLDER
# Do not edit manually. Any manual changes will be overwritten on the next update. # Automatically updated on service or health status changes.
# Docs: https://uncloud.run/docs/concepts/ingress/overview
# User-defined global config from service 'caddy'. # User-defined global config from service 'caddy'.
# Global Caddy configuration # Global Caddy configuration
@@ -412,8 +423,9 @@ web.example.com {
time.Now(), time.Now(),
), ),
}, },
want: `# This file is autogenerated by Uncloud based on the configuration of running services. want: `# Caddyfile autogenerated by Uncloud (DO NOT EDIT): TIMESTAMP_PLACEHOLDER
# Do not edit manually. Any manual changes will be overwritten on the next update. # Automatically updated on service or health status changes.
# Docs: https://uncloud.run/docs/concepts/ingress/overview
# User-defined global config from service 'caddy'. # User-defined global config from service 'caddy'.
# Global config # Global config
@@ -649,8 +661,9 @@ badconfig.com {
time.Now(), time.Now(),
), ),
}, },
want: `# This file is autogenerated by Uncloud based on the configuration of running services. want: `# Caddyfile autogenerated by Uncloud (DO NOT EDIT): TIMESTAMP_PLACEHOLDER
# Do not edit manually. Any manual changes will be overwritten on the next update. # Automatically updated on service or health status changes.
# Docs: https://uncloud.run/docs/concepts/ingress/overview
# User-defined global config from service 'caddy'. # User-defined global config from service 'caddy'.
# Global config from test machine # Global config from test machine
@@ -831,7 +844,7 @@ valid.example.com {
} }
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, tt.want, config, "Generated Caddyfile doesn't match") assert.Equal(t, tt.want, normaliseGeneratedTimestamp(config), "Generated Caddyfile doesn't match")
}) })
} }
} }
@@ -926,7 +939,9 @@ http://api.example.com {
} }
# NOTE: User-defined configs for services were skipped because Caddy is not running on this machine # NOTE: User-defined configs for services were skipped because Caddy is not running on this machine
# or the latest generated config is invalid. Please check the Caddy logs if it's running. # (not accessible via the shared admin socket /run/uncloud/caddy/admin.sock) or the latest
# generated config is invalid. Please check the service 'caddy' is running (uc inspect caddy)
# and its logs for more details (uc logs caddy).
`, `,
}, },
{ {
@@ -950,7 +965,9 @@ http://api.example.com {
} }
# NOTE: User-defined configs for services were skipped because Caddy is not running on this machine # NOTE: User-defined configs for services were skipped because Caddy is not running on this machine
# or the latest generated config is invalid. Please check the Caddy logs if it's running. # (not accessible via the shared admin socket /run/uncloud/caddy/admin.sock) or the latest
# generated config is invalid. Please check the service 'caddy' is running (uc inspect caddy)
# and its logs for more details (uc logs caddy).
`, `,
}, },
} }
@@ -964,7 +981,7 @@ http://api.example.com {
config, err := generator.Generate(ctx, tt.containers, false) config, err := generator.Generate(ctx, tt.containers, false)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, tt.want, config, "Generated Caddyfile doesn't match") assert.Equal(t, tt.want, normaliseGeneratedTimestamp(config), "Generated Caddyfile doesn't match")
}) })
} }
} }
+171 -52
View File
@@ -40,7 +40,9 @@ type clusterController struct {
corroService corroservice.Service corroService corroservice.Service
dockerCtrl *docker.Controller dockerCtrl *docker.Controller
// dockerReady is signalled when Docker is configured and ready for containers. // dockerReady is signalled when Docker is configured and ready for containers.
dockerReady chan<- struct{} dockerReady chan<- struct{}
// clusterReady is signalled when the cluster controller has finished initializing all components.
clusterReady chan<- struct{}
caddyconfigCtrl *caddyconfig.Controller caddyconfigCtrl *caddyconfig.Controller
// dnsServer is the embedded internal DNS server for the cluster listening on the machine IP. // dnsServer is the embedded internal DNS server for the cluster listening on the machine IP.
@@ -60,6 +62,7 @@ func newClusterController(
corroService corroservice.Service, corroService corroservice.Service,
dockerService *docker.Service, dockerService *docker.Service,
dockerReady chan<- struct{}, dockerReady chan<- struct{},
clusterReady chan<- struct{},
caddyfileCtrl *caddyconfig.Controller, caddyfileCtrl *caddyconfig.Controller,
dnsServer *dns.Server, dnsServer *dns.Server,
dnsResolver *dns.ClusterResolver, dnsResolver *dns.ClusterResolver,
@@ -81,6 +84,7 @@ func newClusterController(
corroService: corroService, corroService: corroService,
dockerCtrl: docker.NewController(state.ID, dockerService, store), dockerCtrl: docker.NewController(state.ID, dockerService, store),
dockerReady: dockerReady, dockerReady: dockerReady,
clusterReady: clusterReady,
caddyconfigCtrl: caddyfileCtrl, caddyconfigCtrl: caddyfileCtrl,
dnsServer: dnsServer, dnsServer: dnsServer,
dnsResolver: dnsResolver, dnsResolver: dnsResolver,
@@ -123,6 +127,21 @@ func (cc *clusterController) Run(ctx context.Context) error {
errGroup, ctx := errgroup.WithContext(ctx) errGroup, ctx := errgroup.WithContext(ctx)
// Start the WireGuard control loop before waiting for store sync. This ensures endpoint rotation happens
// while waiting, allowing Corrosion to connect to peers.
errGroup.Go(func() error {
if err := cc.wgnet.Run(ctx); err != nil {
return fmt.Errorf("WireGuard network failed: %w", err)
}
return nil
})
// Watch for WireGuard peer endpoint changes and update the machine state accordingly.
errGroup.Go(func() error {
cc.handleEndpointChanges(ctx)
return nil
})
// Start the network API server. Assume the management IP can't be changed when the network is running. // Start the network API server. Assume the management IP can't be changed when the network is running.
apiAddr := net.JoinHostPort(cc.state.Network.ManagementIP.String(), strconv.Itoa(constants.MachineAPIPort)) apiAddr := net.JoinHostPort(cc.state.Network.ManagementIP.String(), strconv.Itoa(constants.MachineAPIPort))
listener, err := net.Listen("tcp", apiAddr) listener, err := net.Listen("tcp", apiAddr)
@@ -137,6 +156,22 @@ func (cc *clusterController) Run(ctx context.Context) error {
return nil return nil
}) })
// Wait for the store database to sync to the minimum version before starting store-dependent components.
// This prevents issues with using partially replicated data when the machine just joined the cluster,
// e.g., an empty machine list causing WireGuard peer misconfiguration.
cc.waitStoreSync(ctx)
// Check if waitStoreSync exited because the context was cancelled. Return early in that case.
if ctx.Err() != nil {
cc.stopAPIServer()
err := errGroup.Wait()
if corroErr := cc.stopCorrosion(); corroErr != nil {
err = errors.Join(err, corroErr)
}
return err
}
errGroup.Go(func() error { errGroup.Go(func() error {
slog.Info("Starting embedded DNS resolver.") slog.Info("Starting embedded DNS resolver.")
if err := cc.dnsResolver.Run(ctx); err != nil { if err := cc.dnsResolver.Run(ctx); err != nil {
@@ -169,44 +204,6 @@ func (cc *clusterController) Run(ctx context.Context) error {
return nil return nil
}) })
// Watch for WireGuard peer endpoint changes and update the machine state accordingly.
errGroup.Go(func() error {
for {
select {
case e, ok := <-cc.endpointChanges:
if !ok {
// The channel was closed, stop watching for changes.
cc.endpointChanges = nil
return nil
}
cc.state.mu.Lock()
for i := range cc.state.Network.Peers {
if cc.state.Network.Peers[i].PublicKey.Equal(e.PublicKey) {
cc.state.Network.Peers[i].Endpoint = &e.Endpoint
break
}
}
if err := cc.state.Save(); err != nil {
slog.Error("Failed to save machine state.", "err", err)
}
cc.state.mu.Unlock()
slog.Debug("Preserved endpoint change in the machine state.",
"public_key", e.PublicKey, "endpoint", e.Endpoint)
case <-ctx.Done():
return nil
}
}
})
errGroup.Go(func() error {
if err := cc.wgnet.Run(ctx); err != nil {
return fmt.Errorf("WireGuard network failed: %w", err)
}
return nil
})
errGroup.Go(func() error { errGroup.Go(func() error {
slog.Info("Starting caddyconfig controller.") slog.Info("Starting caddyconfig controller.")
if err := cc.caddyconfigCtrl.Run(ctx); err != nil { if err := cc.caddyconfigCtrl.Run(ctx); err != nil {
@@ -225,12 +222,13 @@ func (cc *clusterController) Run(ctx context.Context) error {
}) })
} }
// Wait for the context to be done and stop the network API server. // Signal that the cluster controller has finished starting all components.
close(cc.clusterReady)
slog.Info("Cluster controller finished starting all components.")
// Wait for the context to be done and stop all servers and controllers.
<-ctx.Done() <-ctx.Done()
slog.Info("Stopping network API server.")
// TODO: implement timeout for graceful shutdown. cc.stopAPIServer()
cc.server.GracefulStop()
slog.Info("Network API server stopped.")
// Stop the unregistry server with a timeout if it was started. // Stop the unregistry server with a timeout if it was started.
if cc.unregistry != nil { if cc.unregistry != nil {
@@ -248,19 +246,50 @@ func (cc *clusterController) Run(ctx context.Context) error {
// Wait for all controllers to finish. // Wait for all controllers to finish.
err = errGroup.Wait() err = errGroup.Wait()
// It's safe to stop the Corrosion service after the controllers depending on it and API server are stopped. // Stop Corrosion after all controllers depending on it and API server are stopped.
// Use a new context with a timeout as the current context is already canceled. if corroErr := cc.stopCorrosion(); corroErr != nil {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) err = errors.Join(err, corroErr)
defer cancel()
if corroErr := cc.corroService.Stop(ctx); corroErr != nil {
err = errors.Join(err, fmt.Errorf("stop corrosion service: %w", corroErr))
} else {
slog.Info("Corrosion service stopped.")
} }
return err return err
} }
// stopAPIServer gracefully stops the network API server with a timeout.
func (cc *clusterController) stopAPIServer() {
timeout := 10 * time.Second
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
stopped := make(chan struct{})
go func() {
slog.Info("Stopping network API server.")
cc.server.GracefulStop()
close(stopped)
}()
select {
case <-ctx.Done():
slog.Warn("Network API server graceful stop timed out, forcing stop.", "timeout", timeout)
cc.server.Stop()
case <-stopped:
}
slog.Info("Network API server stopped.")
}
// stopCorrosion stops the Corrosion service with a timeout.
func (cc *clusterController) stopCorrosion() error {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := cc.corroService.Stop(ctx); err != nil {
return fmt.Errorf("stop corrosion service: %w", err)
}
slog.Info("Corrosion service stopped.")
return nil
}
// ensureDockerNetwork ensures that the Docker network is configured and ready for containers. // ensureDockerNetwork ensures that the Docker network is configured and ready for containers.
func (cc *clusterController) ensureDockerNetwork(ctx context.Context) error { func (cc *clusterController) ensureDockerNetwork(ctx context.Context) error {
if err := cc.dockerCtrl.WaitDaemonReady(ctx); err != nil { if err := cc.dockerCtrl.WaitDaemonReady(ctx); err != nil {
@@ -281,6 +310,95 @@ func (cc *clusterController) ensureDockerNetwork(ctx context.Context) error {
return nil return nil
} }
// handleEndpointChanges watches for WireGuard peer endpoint changes and persists them to the machine state.
func (cc *clusterController) handleEndpointChanges(ctx context.Context) {
for {
select {
case e, ok := <-cc.endpointChanges:
if !ok {
// The channel was closed, stop watching for changes.
cc.endpointChanges = nil
return
}
cc.state.mu.Lock()
for i := range cc.state.Network.Peers {
if cc.state.Network.Peers[i].PublicKey.Equal(e.PublicKey) {
cc.state.Network.Peers[i].Endpoint = &e.Endpoint
break
}
}
if err := cc.state.Save(); err != nil {
slog.Error("Failed to save machine state.", "err", err)
}
cc.state.mu.Unlock()
slog.Debug("Preserved endpoint change in the machine state.",
"public_key", e.PublicKey, "endpoint", e.Endpoint)
case <-ctx.Done():
return
}
}
}
// waitStoreSync waits for the store database to sync to the minimum required DB version if set in the machine state.
// Blocks until synced or context is cancelled.
func (cc *clusterController) waitStoreSync(ctx context.Context) {
minVersion := cc.state.MinStoreDBVersion
if minVersion == 0 {
return
}
slog.Info("Waiting for the initial cluster store sync.", "min_version", minVersion)
ticker := time.NewTicker(500 * time.Millisecond)
defer ticker.Stop()
var (
lastVersion int64
lastLogTime time.Time
lastErrLogTime time.Time
)
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
version, err := cc.store.DBVersion(ctx)
if err != nil {
// Log errors at most once every 5 seconds.
if time.Since(lastErrLogTime) >= 5*time.Second {
slog.Error("Failed to get the cluster store DB version, retrying.", "err", err)
lastErrLogTime = time.Now()
}
continue
}
if version >= minVersion {
slog.Info("Cluster store completed the initial sync.", "version", version, "min_version", minVersion)
// Clear MinStoreDBVersion so next restart doesn't wait for sync.
cc.state.mu.Lock()
cc.state.MinStoreDBVersion = 0
if err := cc.state.Save(); err != nil {
slog.Error("Failed to save machine state after the initial cluster store sync.", "err", err)
}
cc.state.mu.Unlock()
return
}
// Log progress only once a second.
if version != lastVersion && time.Since(lastLogTime) >= 1*time.Second {
slog.Info("Syncing cluster store.", "version", version, "min_version", minVersion)
lastLogTime = time.Now()
lastVersion = version
}
}
}
}
// syncDockerContainers watches local Docker containers and syncs them to the cluster store. // syncDockerContainers watches local Docker containers and syncs them to the cluster store.
// TODO: move this to the Docker controller. // TODO: move this to the Docker controller.
func (cc *clusterController) syncDockerContainers(ctx context.Context) error { func (cc *clusterController) syncDockerContainers(ctx context.Context) error {
@@ -340,6 +458,7 @@ func (cc *clusterController) handleMachineChanges(ctx context.Context) error {
// The machine store may be empty when a machine first joins the cluster, before store synchronization // The machine store may be empty when a machine first joins the cluster, before store synchronization
// completes. Skip configuration now and apply it when the store changes are received. // completes. Skip configuration now and apply it when the store changes are received.
// TODO: remove this check after releasing 0.17.0 and assuming cluster machines wait for store sync on join.
if len(machines) > 0 { if len(machines) > 0 {
slog.Info("Reconfiguring network peers with the current machines.", "machines", len(machines)) slog.Info("Reconfiguring network peers with the current machines.", "machines", len(machines))
if err = cc.configurePeers(machines); err != nil { if err = cc.configurePeers(machines); err != nil {
+48 -51
View File
@@ -26,12 +26,19 @@ type Cluster struct {
corroAdmin *corrosion.AdminClient corroAdmin *corrosion.AdminClient
// machineID is the ID of the current machine that is running the cluster service. // machineID is the ID of the current machine that is running the cluster service.
machineID string machineID string
// initialised is closed when the machine is configured as a member of a cluster.
initialised <-chan struct{}
// ready is closed when the cluster controller has finished starting all components
// and the machine is ready to serve cluster requests.
ready <-chan struct{}
} }
func NewCluster(store *store.Store, corroAdmin *corrosion.AdminClient) *Cluster { func NewCluster(store *store.Store, corroAdmin *corrosion.AdminClient, initialised, ready <-chan struct{}) *Cluster {
return &Cluster{ return &Cluster{
store: store, store: store,
corroAdmin: corroAdmin, corroAdmin: corroAdmin,
initialised: initialised,
ready: ready,
} }
} }
@@ -41,67 +48,45 @@ func (c *Cluster) UpdateMachineID(mid string) {
} }
func (c *Cluster) Init(ctx context.Context, network netip.Prefix) error { func (c *Cluster) Init(ctx context.Context, network netip.Prefix) error {
initialised, err := c.Initialised(ctx) select {
if err != nil { case <-c.initialised:
return err return fmt.Errorf("cluster is already initialised on this machine")
} default:
if initialised {
return fmt.Errorf("cluster is already initialised")
} }
if err = c.store.Put(ctx, "network", network.String()); err != nil { if err := c.store.Put(ctx, "network", network.String()); err != nil {
return fmt.Errorf("put network to store: %w", err) return fmt.Errorf("put network to store: %w", err)
} }
if err = c.store.Put(ctx, "created_at", time.Now().UTC().Format(time.RFC3339)); err != nil { if err := c.store.Put(ctx, "created_at", time.Now().UTC().Format(time.RFC3339)); err != nil {
return fmt.Errorf("put created_at to store: %w", err) return fmt.Errorf("put created_at to store: %w", err)
} }
return nil return nil
} }
func (c *Cluster) Initialised(ctx context.Context) (bool, error) { // checkReady checks if the machine is ready to serve cluster requests (store synced, cluster components started).
var createdAt string func (c *Cluster) checkReady() error {
if err := c.store.Get(ctx, "created_at", &createdAt); err != nil { select {
if errors.Is(err, store.ErrKeyNotFound) { case <-c.ready:
return false, nil return nil
} default:
return false, status.Errorf(codes.Internal, "get created_at from store: %v", err) return status.Error(codes.Unavailable, "machine is not ready to serve cluster requests")
} }
return true, nil
}
func (c *Cluster) checkInitialised(ctx context.Context) error {
initialised, err := c.Initialised(ctx)
if err != nil {
return err
}
if !initialised {
return status.Error(codes.FailedPrecondition, "cluster is not initialised")
}
return nil
}
func (c *Cluster) Network(ctx context.Context) (netip.Prefix, error) {
if err := c.checkInitialised(ctx); err != nil {
return netip.Prefix{}, err
}
var net string
if err := c.store.Get(ctx, "network", &net); err != nil {
return netip.Prefix{}, status.Errorf(codes.Internal, "get network from store: %v", err)
}
prefix, err := netip.ParsePrefix(net)
if err != nil {
return netip.Prefix{}, status.Errorf(codes.Internal, "parse network prefix: %v", err)
}
return prefix, nil
} }
// AddMachine adds a machine to the cluster. // AddMachine adds a machine to the cluster.
func (c *Cluster) AddMachine(ctx context.Context, req *pb.AddMachineRequest) (*pb.AddMachineResponse, error) { func (c *Cluster) AddMachine(ctx context.Context, req *pb.AddMachineRequest) (*pb.AddMachineResponse, error) {
if err := c.checkInitialised(ctx); err != nil { if err := c.checkReady(); err != nil {
return nil, err return nil, err
} }
return c.AddMachineWithoutReadyCheck(ctx, req)
}
// AddMachineWithoutReadyCheck adds a machine to the cluster without checking if the cluster is ready.
// This is used internally during cluster initialisation to add the first machine.
func (c *Cluster) AddMachineWithoutReadyCheck(
ctx context.Context, req *pb.AddMachineRequest,
) (*pb.AddMachineResponse, error) {
if req.Network == nil { if req.Network == nil {
return nil, status.Error(codes.InvalidArgument, "network not set") return nil, status.Error(codes.InvalidArgument, "network not set")
} }
@@ -162,7 +147,7 @@ func (c *Cluster) AddMachine(ctx context.Context, req *pb.AddMachineRequest) (*p
manageIP = pb.NewIP(network.ManagementIP(req.Network.PublicKey)) manageIP = pb.NewIP(network.ManagementIP(req.Network.PublicKey))
} }
// Allocate a subnet for the machine from the cluster network. // Allocate a subnet for the machine from the cluster network.
clusterNetwork, err := c.Network(ctx) clusterNetwork, err := c.network(ctx)
if err != nil { if err != nil {
return nil, status.Errorf(codes.Internal, "get cluster network: %v", err) return nil, status.Errorf(codes.Internal, "get cluster network: %v", err)
} }
@@ -198,9 +183,21 @@ func (c *Cluster) AddMachine(ctx context.Context, req *pb.AddMachineRequest) (*p
return resp, nil return resp, nil
} }
func (c *Cluster) network(ctx context.Context) (netip.Prefix, error) {
var net string
if err := c.store.Get(ctx, "network", &net); err != nil {
return netip.Prefix{}, status.Errorf(codes.Internal, "get network from store: %v", err)
}
prefix, err := netip.ParsePrefix(net)
if err != nil {
return netip.Prefix{}, status.Errorf(codes.Internal, "parse network prefix: %v", err)
}
return prefix, nil
}
// UpdateMachine updates machine configuration in the cluster. // UpdateMachine updates machine configuration in the cluster.
func (c *Cluster) UpdateMachine(ctx context.Context, req *pb.UpdateMachineRequest) (*pb.UpdateMachineResponse, error) { func (c *Cluster) UpdateMachine(ctx context.Context, req *pb.UpdateMachineRequest) (*pb.UpdateMachineResponse, error) {
if err := c.checkInitialised(ctx); err != nil { if err := c.checkReady(); err != nil {
return nil, err return nil, err
} }
@@ -283,7 +280,7 @@ func (c *Cluster) UpdateMachine(ctx context.Context, req *pb.UpdateMachineReques
// ListMachines lists all machines in the cluster including their membership states. // ListMachines lists all machines in the cluster including their membership states.
func (c *Cluster) ListMachines(ctx context.Context, _ *emptypb.Empty) (*pb.ListMachinesResponse, error) { func (c *Cluster) ListMachines(ctx context.Context, _ *emptypb.Empty) (*pb.ListMachinesResponse, error) {
if err := c.checkInitialised(ctx); err != nil { if err := c.checkReady(); err != nil {
return nil, err return nil, err
} }
@@ -329,7 +326,7 @@ func (c *Cluster) ListMachines(ctx context.Context, _ *emptypb.Empty) (*pb.ListM
// RemoveMachine removes a machine from the cluster. // RemoveMachine removes a machine from the cluster.
func (c *Cluster) RemoveMachine(ctx context.Context, req *pb.RemoveMachineRequest) (*emptypb.Empty, error) { func (c *Cluster) RemoveMachine(ctx context.Context, req *pb.RemoveMachineRequest) (*emptypb.Empty, error) {
if err := c.checkInitialised(ctx); err != nil { if err := c.checkReady(); err != nil {
return nil, err return nil, err
} }
+4 -4
View File
@@ -25,7 +25,7 @@ type uncloudDNSDomain struct {
} }
func (c *Cluster) ReserveDomain(ctx context.Context, req *pb.ReserveDomainRequest) (*pb.Domain, error) { func (c *Cluster) ReserveDomain(ctx context.Context, req *pb.ReserveDomainRequest) (*pb.Domain, error) {
if err := c.checkInitialised(ctx); err != nil { if err := c.checkReady(); err != nil {
return nil, err return nil, err
} }
@@ -64,7 +64,7 @@ func (c *Cluster) ReserveDomain(ctx context.Context, req *pb.ReserveDomainReques
} }
func (c *Cluster) GetDomain(ctx context.Context, _ *emptypb.Empty) (*pb.Domain, error) { func (c *Cluster) GetDomain(ctx context.Context, _ *emptypb.Empty) (*pb.Domain, error) {
if err := c.checkInitialised(ctx); err != nil { if err := c.checkReady(); err != nil {
return nil, err return nil, err
} }
@@ -95,7 +95,7 @@ func (c *Cluster) storedDomain(ctx context.Context) (uncloudDNSDomain, error) {
} }
func (c *Cluster) ReleaseDomain(ctx context.Context, _ *emptypb.Empty) (*pb.Domain, error) { func (c *Cluster) ReleaseDomain(ctx context.Context, _ *emptypb.Empty) (*pb.Domain, error) {
if err := c.checkInitialised(ctx); err != nil { if err := c.checkReady(); err != nil {
return nil, err return nil, err
} }
@@ -115,7 +115,7 @@ func (c *Cluster) ReleaseDomain(ctx context.Context, _ *emptypb.Empty) (*pb.Doma
func (c *Cluster) CreateDomainRecords( func (c *Cluster) CreateDomainRecords(
ctx context.Context, req *pb.CreateDomainRecordsRequest, ctx context.Context, req *pb.CreateDomainRecordsRequest,
) (*pb.CreateDomainRecordsResponse, error) { ) (*pb.CreateDomainRecordsResponse, error) {
if err := c.checkInitialised(ctx); err != nil { if err := c.checkReady(); err != nil {
return nil, err return nil, err
} }
+65 -10
View File
@@ -33,6 +33,7 @@ import (
"github.com/docker/docker/client" "github.com/docker/docker/client"
"github.com/docker/docker/pkg/stdcopy" "github.com/docker/docker/pkg/stdcopy"
"github.com/docker/go-connections/nat" "github.com/docker/go-connections/nat"
"github.com/docker/go-units"
"github.com/google/go-containerregistry/pkg/authn" "github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name" "github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote" "github.com/google/go-containerregistry/pkg/v1/remote"
@@ -566,6 +567,22 @@ func (s *Server) CreateServiceContainer(
if spec.Mode == "" { if spec.Mode == "" {
config.Labels[api.LabelServiceMode] = api.ServiceModeReplicated config.Labels[api.LabelServiceMode] = api.ServiceModeReplicated
} }
if hc := spec.Container.Healthcheck; hc != nil {
if hc.Disable {
config.Healthcheck = &container.HealthConfig{
Test: []string{"NONE"},
}
} else {
config.Healthcheck = &container.HealthConfig{
Test: hc.Test,
Interval: hc.Interval,
Timeout: hc.Timeout,
StartPeriod: hc.StartPeriod,
StartInterval: hc.StartInterval,
Retries: int(hc.Retries),
}
}
}
// TODO: do not set the ports as container labels once migrated to retrieve them from the spec in DB. // TODO: do not set the ports as container labels once migrated to retrieve them from the spec in DB.
var err error var err error
@@ -605,6 +622,8 @@ func (s *Server) CreateServiceContainer(
} }
} }
hostConfig := &container.HostConfig{ hostConfig := &container.HostConfig{
CapAdd: spec.Container.CapAdd,
CapDrop: spec.Container.CapDrop,
Binds: spec.Container.Volumes, Binds: spec.Container.Volumes,
Init: spec.Container.Init, Init: spec.Container.Init,
Mounts: mounts, Mounts: mounts,
@@ -614,13 +633,16 @@ func (s *Server) CreateServiceContainer(
NanoCPUs: spec.Container.Resources.CPU, NanoCPUs: spec.Container.Resources.CPU,
Memory: spec.Container.Resources.Memory, Memory: spec.Container.Resources.Memory,
MemoryReservation: spec.Container.Resources.MemoryReservation, MemoryReservation: spec.Container.Resources.MemoryReservation,
Devices: toDockerDevices(spec.Container.Resources.Devices),
DeviceRequests: spec.Container.Resources.DeviceReservations, DeviceRequests: spec.Container.Resources.DeviceReservations,
Ulimits: toDockerUlimits(spec.Container.Resources.Ulimits),
}, },
// Restart service containers if they exit or a machine restarts unless they are explicitly stopped. // Restart service containers if they exit or a machine restarts unless they are explicitly stopped.
// For one-off containers and batch jobs we plan to use a different service type/mode. // For one-off containers and batch jobs we plan to use a different service type/mode.
RestartPolicy: container.RestartPolicy{ RestartPolicy: container.RestartPolicy{
Name: container.RestartPolicyUnlessStopped, Name: container.RestartPolicyUnlessStopped,
}, },
Sysctls: spec.Container.Sysctls,
} }
// Configure the container to use the internal DNS server if it's available. // Configure the container to use the internal DNS server if it's available.
@@ -801,14 +823,17 @@ func (s *Server) injectConfigs(ctx context.Context, containerID string, configs
} }
// copyContentToContainer copies content directly to a file in the container using Docker's CopyToContainer API. // copyContentToContainer copies content directly to a file in the container using Docker's CopyToContainer API.
// It will create any intermediate directories in the target path that don't exist.
func (s *Server) copyContentToContainer(ctx context.Context, containerID string, content []byte, targetPath string, uid *uint64, gid *uint64, fileMode os.FileMode) error { func (s *Server) copyContentToContainer(ctx context.Context, containerID string, content []byte, targetPath string, uid *uint64, gid *uint64, fileMode os.FileMode) error {
// Create a tar archive containing the file
var buf bytes.Buffer var buf bytes.Buffer
tw := tar.NewWriter(&buf) tw := tar.NewWriter(&buf)
// Create tar header // Trim leading slash(es) to avoid double slashes in the tar path
tarPath := strings.TrimPrefix(targetPath, "/")
// Create tar header with full path
header := &tar.Header{ header := &tar.Header{
Name: filepath.Base(targetPath), Name: tarPath,
Size: int64(len(content)), Size: int64(len(content)),
Mode: int64(fileMode), Mode: int64(fileMode),
ModTime: time.Now(), ModTime: time.Now(),
@@ -834,16 +859,12 @@ func (s *Server) copyContentToContainer(ctx context.Context, containerID string,
return fmt.Errorf("close tar writer: %w", err) return fmt.Errorf("close tar writer: %w", err)
} }
// Copy the tar archive to the container // Always extract to root. The tar archive contains the full path, so tar will
targetDir := filepath.Dir(targetPath) // automatically create any intermediate directories that don't exist in the container.
if targetDir == "." {
targetDir = "/"
}
if err := s.client.CopyToContainer( if err := s.client.CopyToContainer(
ctx, ctx,
containerID, containerID,
targetDir, "/",
&buf, &buf,
container.CopyToContainerOptions{CopyUIDGID: true}, container.CopyToContainerOptions{CopyUIDGID: true},
); err != nil { ); err != nil {
@@ -875,6 +896,40 @@ func toDockerBindOptions(opts *api.BindOptions) *mount.BindOptions {
return dockerOpts return dockerOpts
} }
func toDockerUlimits(ulimits map[string]api.Ulimit) []*units.Ulimit {
if len(ulimits) == 0 {
return nil
}
dockerUlimits := make([]*units.Ulimit, 0, len(ulimits))
for name, u := range ulimits {
dockerUlimits = append(dockerUlimits, &units.Ulimit{
Name: name,
Soft: u.Soft,
Hard: u.Hard,
})
}
return dockerUlimits
}
func toDockerDevices(devices []api.DeviceMapping) []container.DeviceMapping {
if len(devices) == 0 {
return nil
}
dockerDevices := make([]container.DeviceMapping, 0, len(devices))
for _, d := range devices {
dockerDevices = append(dockerDevices, container.DeviceMapping{
PathOnHost: d.HostPath,
PathInContainer: d.ContainerPath,
CgroupPermissions: d.CgroupPermissions,
})
}
return dockerDevices
}
// verifyDockerVolumesExist checks if the Docker named volumes referenced in the mounts exist on the machine. // verifyDockerVolumesExist checks if the Docker named volumes referenced in the mounts exist on the machine.
func (s *Server) verifyDockerVolumesExist(ctx context.Context, mounts []mount.Mount) error { func (s *Server) verifyDockerVolumesExist(ctx context.Context, mounts []mount.Mount) error {
for _, m := range mounts { for _, m := range mounts {
+97 -8
View File
@@ -33,10 +33,12 @@ import (
"github.com/psviderski/unregistry" "github.com/psviderski/unregistry"
"github.com/siderolabs/grpc-proxy/proxy" "github.com/siderolabs/grpc-proxy/proxy"
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
"golang.zx2c4.com/wireguard/wgctrl"
"google.golang.org/grpc" "google.golang.org/grpc"
"google.golang.org/grpc/codes" "google.golang.org/grpc/codes"
"google.golang.org/grpc/status" "google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb" "google.golang.org/protobuf/types/known/emptypb"
"google.golang.org/protobuf/types/known/timestamppb"
) )
const ( const (
@@ -174,10 +176,13 @@ type Machine struct {
state *State state *State
// started is closed when the machine is ready to serve requests on the local API server. // started is closed when the machine is ready to serve requests on the local API server.
started chan struct{} started chan struct{}
// initialised is signalled when the machine is configured as a member of a cluster. // initialised is closed when the machine is configured as a member of a cluster.
initialised chan struct{} initialised chan struct{}
// networkReady is signalled when the Docker network is configured and ready for containers. // networkReady is closed when the Docker network is configured and ready for containers.
networkReady chan struct{} networkReady chan struct{}
// clusterReady is closed when the cluster controller has finished starting all components
// and the machine is ready to serve cluster requests.
clusterReady chan struct{}
// resetting is true when the machine is being reset. // resetting is true when the machine is being reset.
resetting bool resetting bool
// stop cancels the Run method context to stop the machine gracefully. // stop cancels the Run method context to stop the machine gracefully.
@@ -246,7 +251,10 @@ func NewMachine(config *Config) (*Machine, error) {
if err != nil { if err != nil {
return nil, fmt.Errorf("create corrosion admin client: %w", err) return nil, fmt.Errorf("create corrosion admin client: %w", err)
} }
c := cluster.NewCluster(corroStore, corroAdmin)
initialised := make(chan struct{})
clusterReady := make(chan struct{})
c := cluster.NewCluster(corroStore, corroAdmin, initialised, clusterReady)
// Init dependencies for a gRPC Docker server that proxies requests to the local Docker daemon. // Init dependencies for a gRPC Docker server that proxies requests to the local Docker daemon.
dbFilePath := filepath.Join(config.DataDir, DBFileName) dbFilePath := filepath.Join(config.DataDir, DBFileName)
@@ -269,8 +277,9 @@ func NewMachine(config *Config) (*Machine, error) {
config: *config, config: *config,
state: state, state: state,
started: make(chan struct{}), started: make(chan struct{}),
initialised: make(chan struct{}, 1), initialised: initialised,
networkReady: make(chan struct{}), networkReady: make(chan struct{}),
clusterReady: clusterReady,
store: corroStore, store: corroStore,
cluster: c, cluster: c,
dockerService: dockerService, dockerService: dockerService,
@@ -294,7 +303,7 @@ func NewMachine(config *Config) (*Machine, error) {
m.localMachineServer = newGRPCServer(m, c, m.dockerServer, caddyServer) m.localMachineServer = newGRPCServer(m, c, m.dockerServer, caddyServer)
if m.Initialised() { if m.Initialised() {
m.initialised <- struct{}{} close(m.initialised)
} }
return m, nil return m, nil
@@ -475,6 +484,7 @@ func (m *Machine) Run(ctx context.Context) error {
m.config.CorrosionService, m.config.CorrosionService,
m.dockerService, m.dockerService,
m.networkReady, m.networkReady,
m.clusterReady,
caddyconfigCtrl, caddyconfigCtrl,
dnsServer, dnsServer,
dnsResolver, dnsResolver,
@@ -722,7 +732,7 @@ func (m *Machine) InitCluster(ctx context.Context, req *pb.InitClusterRequest) (
addReq.PublicIp = pb.NewIP(publicIP) addReq.PublicIp = pb.NewIP(publicIP)
} }
addResp, err := m.cluster.AddMachine(ctx, addReq) addResp, err := m.cluster.AddMachineWithoutReadyCheck(ctx, addReq)
if err != nil { if err != nil {
return nil, status.Errorf(codes.Internal, "add machine to cluster: %v", err) return nil, status.Errorf(codes.Internal, "add machine to cluster: %v", err)
} }
@@ -749,7 +759,7 @@ func (m *Machine) InitCluster(ctx context.Context, req *pb.InitClusterRequest) (
} }
slog.Info("Cluster initialised with machine.", "id", m.state.ID, "machine", m.state.Name) slog.Info("Cluster initialised with machine.", "id", m.state.ID, "machine", m.state.Name)
// Signal that the machine is initialised as a member of a cluster. // Signal that the machine is initialised as a member of a cluster.
m.initialised <- struct{}{} close(m.initialised)
resp := &pb.InitClusterResponse{ resp := &pb.InitClusterResponse{
Machine: addResp.Machine, Machine: addResp.Machine,
@@ -792,6 +802,7 @@ func (m *Machine) JoinCluster(_ context.Context, req *pb.JoinClusterRequest) (*e
PrivateKey: m.state.Network.PrivateKey, PrivateKey: m.state.Network.PrivateKey,
PublicKey: m.state.Network.PublicKey, PublicKey: m.state.Network.PublicKey,
} }
m.state.MinStoreDBVersion = req.MinStoreDbVersion
// Build a peers config from other cluster machines. // Build a peers config from other cluster machines.
m.state.Network.Peers = make([]network.PeerConfig, 0, len(req.OtherMachines)) m.state.Network.Peers = make([]network.PeerConfig, 0, len(req.OtherMachines))
@@ -830,7 +841,7 @@ func (m *Machine) JoinCluster(_ context.Context, req *pb.JoinClusterRequest) (*e
"peers", len(m.state.Network.Peers), "peers", len(m.state.Network.Peers),
) )
// Signal that the machine is initialised as a member of a cluster. // Signal that the machine is initialised as a member of a cluster.
m.initialised <- struct{}{} close(m.initialised)
return &emptypb.Empty{}, nil return &emptypb.Empty{}, nil
} }
@@ -864,6 +875,7 @@ func (m *Machine) Token(_ context.Context, _ *emptypb.Empty) (*pb.TokenResponse,
return &pb.TokenResponse{Token: tokenStr}, nil return &pb.TokenResponse{Token: tokenStr}, nil
} }
// Deprecated: use InspectMachine instead.
func (m *Machine) Inspect(_ context.Context, _ *emptypb.Empty) (*pb.MachineInfo, error) { func (m *Machine) Inspect(_ context.Context, _ *emptypb.Empty) (*pb.MachineInfo, error) {
return &pb.MachineInfo{ return &pb.MachineInfo{
Id: m.state.ID, Id: m.state.ID,
@@ -876,6 +888,31 @@ func (m *Machine) Inspect(_ context.Context, _ *emptypb.Empty) (*pb.MachineInfo,
}, nil }, nil
} }
func (m *Machine) InspectMachine(ctx context.Context, _ *emptypb.Empty) (*pb.InspectMachineResponse, error) {
dbVersion, err := m.store.DBVersion(ctx)
if err != nil {
return nil, status.Errorf(codes.Internal, "get database version of the cluster store: %v", err)
}
return &pb.InspectMachineResponse{
Machines: []*pb.MachineDetails{
{
// Metadata is injected by the gRPC proxy.
Machine: &pb.MachineInfo{
Id: m.state.ID,
Name: m.state.Name,
Network: &pb.NetworkConfig{
Subnet: pb.NewIPPrefix(m.state.Network.Subnet),
ManagementIp: pb.NewIP(m.state.Network.ManagementIP),
PublicKey: m.state.Network.PublicKey,
},
},
StoreDbVersion: dbVersion,
},
},
}, nil
}
// IsNetworkReady returns true if the Docker network is ready for containers. // IsNetworkReady returns true if the Docker network is ready for containers.
func (m *Machine) IsNetworkReady() bool { func (m *Machine) IsNetworkReady() bool {
if !m.Initialised() { if !m.Initialised() {
@@ -909,6 +946,58 @@ func (m *Machine) WaitForNetworkReady(ctx context.Context) error {
} }
} }
// InspectWireGuardNetwork retrieves the current WireGuard network configuration and peer status.
func (m *Machine) InspectWireGuardNetwork(
_ context.Context, _ *emptypb.Empty,
) (*pb.InspectWireGuardNetworkResponse, error) {
deviceName := network.WireGuardInterfaceName
wg, err := wgctrl.New()
if err != nil {
return nil, fmt.Errorf("create WireGuard client: %w", err)
}
defer wg.Close()
dev, err := wg.Device(deviceName)
if err != nil {
return nil, fmt.Errorf("get WireGuard device '%s': %w", deviceName, err)
}
peers := make([]*pb.WireGuardPeer, len(dev.Peers))
for i, p := range dev.Peers {
var lastHandshake *timestamppb.Timestamp
if !p.LastHandshakeTime.IsZero() {
lastHandshake = timestamppb.New(p.LastHandshakeTime)
}
allowedIPs := make([]string, len(p.AllowedIPs))
for j, ip := range p.AllowedIPs {
allowedIPs[j] = ip.String()
}
var endpoint string
if p.Endpoint != nil {
endpoint = p.Endpoint.String()
}
peers[i] = &pb.WireGuardPeer{
PublicKey: p.PublicKey[:],
Endpoint: endpoint,
LastHandshakeTime: lastHandshake,
ReceiveBytes: p.ReceiveBytes,
TransmitBytes: p.TransmitBytes,
AllowedIps: allowedIPs,
}
}
return &pb.InspectWireGuardNetworkResponse{
InterfaceName: dev.Name,
PublicKey: dev.PublicKey[:],
ListenPort: int32(dev.ListenPort),
Peers: peers,
}, nil
}
// Reset restores the machine to a clean state, scheduling a graceful shutdown and removing all cluster-related // Reset restores the machine to a clean state, scheduling a graceful shutdown and removing all cluster-related
// configuration and resource. The uncloud daemon will restart the machine if managed by systemd. // configuration and resource. The uncloud daemon will restart the machine if managed by systemd.
func (m *Machine) Reset(_ context.Context, _ *pb.ResetRequest) (*emptypb.Empty, error) { func (m *Machine) Reset(_ context.Context, _ *pb.ResetRequest) (*emptypb.Empty, error) {
+37 -2
View File
@@ -24,6 +24,9 @@ type State struct {
Name string Name string
// Network specifies the network configuration for this machine. // Network specifies the network configuration for this machine.
Network *network.Config Network *network.Config
// MinStoreDBVersion is the latest database version of one of the existing cluster machines at the time this machine
// joined the cluster. The machine should sync to at least this version before starting any cluster operations.
MinStoreDBVersion int64 `json:",omitempty"`
// path is the file path config is read from and saved to. // path is the file path config is read from and saved to.
path string path string
@@ -66,7 +69,7 @@ func (c *State) Encode() ([]byte, error) {
return data, nil return data, nil
} }
// Save writes the state data to the file at the given path. // Save atomically writes the state data to the file at the configured path.
func (c *State) Save() error { func (c *State) Save() error {
if c.path == "" { if c.path == "" {
return fmt.Errorf("state path not set") return fmt.Errorf("state path not set")
@@ -80,5 +83,37 @@ func (c *State) Save() error {
if err != nil { if err != nil {
return err return err
} }
return os.WriteFile(c.path, data, 0o600)
// Write to a temporary file and rename for atomic save. CreateTemp creates a file with mode 0o600.
tmpFile, err := os.CreateTemp(dir, ".machine.json.*")
if err != nil {
return fmt.Errorf("create temp file: %w", err)
}
tmpPath := tmpFile.Name()
// Clean up temp file on error.
defer func() {
if err != nil {
os.Remove(tmpPath)
}
}()
if _, err = tmpFile.Write(data); err != nil {
tmpFile.Close()
return fmt.Errorf("write temp file: %w", err)
}
// Without fsync, the data may only be in the OS buffer cache. If the system crashes before the OS flushes
// it to disk, the file could be empty or corrupted even after rename.
if err = tmpFile.Sync(); err != nil {
tmpFile.Close()
return fmt.Errorf("sync temp file: %w", err)
}
if err = tmpFile.Close(); err != nil {
return fmt.Errorf("close temp file: %w", err)
}
if err = os.Rename(tmpPath, c.path); err != nil {
return fmt.Errorf("rename temp file: %w", err)
}
return nil
} }
+35 -6
View File
@@ -82,7 +82,7 @@ func (s *Store) CreateOrUpdateContainer(ctx context.Context, ctr api.ServiceCont
// ListContainers returns a list of container records from the store database that match the given options. // ListContainers returns a list of container records from the store database that match the given options.
func (s *Store) ListContainers(ctx context.Context, opts ListOptions) ([]ContainerRecord, error) { func (s *Store) ListContainers(ctx context.Context, opts ListOptions) ([]ContainerRecord, error) {
q := sq.Select("container", "machine_id", "sync_status", "updated_at").From("containers"). q := sq.Select("id", "container", "machine_id", "sync_status", "updated_at").From("containers").
Where(sq.Eq{"sync_status": SyncStatusSynced}) Where(sq.Eq{"sync_status": SyncStatusSynced})
if len(opts.MachineIDs) > 0 { if len(opts.MachineIDs) > 0 {
@@ -112,14 +112,23 @@ func (s *Store) ListContainers(ctx context.Context, opts ListOptions) ([]Contain
defer rows.Close() defer rows.Close()
var containers []ContainerRecord var containers []ContainerRecord
var cJSON, machineID, syncStatus, updatedAtStr string var id, cJSON, machineID, syncStatus, updatedAtStr string
var updatedAt time.Time var updatedAt time.Time
skipped := 0
for rows.Next() { for rows.Next() {
if err = rows.Scan(&cJSON, &machineID, &syncStatus, &updatedAtStr); err != nil { if err = rows.Scan(&id, &cJSON, &machineID, &syncStatus, &updatedAtStr); err != nil {
return nil, fmt.Errorf("scan container record: %w", err) return nil, fmt.Errorf("scan container record: %w", err)
} }
// Skip containers with empty JSON data. This can happen during partial replication
// when cr-sqlite has created the row but the container column hasn't been synced yet.
if cJSON == "" || cJSON == "{}" {
slog.Debug("Skipping container with empty data in the store (partial replication?).", "id", id)
skipped++
continue
}
var c api.ServiceContainer var c api.ServiceContainer
if err = json.Unmarshal([]byte(cJSON), &c); err != nil { if err = json.Unmarshal([]byte(cJSON), &c); err != nil {
return nil, fmt.Errorf("unmarshal container: %w", err) return nil, fmt.Errorf("unmarshal container: %w", err)
@@ -135,6 +144,11 @@ func (s *Store) ListContainers(ctx context.Context, opts ListOptions) ([]Contain
}) })
} }
if skipped > 0 {
slog.Warn("Listing containers from the store skipped empty records (possibly due to partial replication).",
"skipped", skipped, "valid", len(containers))
}
return containers, nil return containers, nil
} }
@@ -166,7 +180,7 @@ func (s *Store) DeleteContainers(ctx context.Context, opts DeleteOptions) error
// receive any values, it just signals when a container(s) has been added, updated, or deleted in the database. // receive any values, it just signals when a container(s) has been added, updated, or deleted in the database.
func (s *Store) SubscribeContainers(ctx context.Context) ([]ContainerRecord, <-chan struct{}, error) { func (s *Store) SubscribeContainers(ctx context.Context) ([]ContainerRecord, <-chan struct{}, error) {
// TODO: figure out whether we need sync_status at all (not used at the moment). // TODO: figure out whether we need sync_status at all (not used at the moment).
q := sq.Select("container", "machine_id", "sync_status", "updated_at").From("containers"). q := sq.Select("id", "container", "machine_id", "sync_status", "updated_at").From("containers").
Where(sq.Eq{"sync_status": SyncStatusSynced}) Where(sq.Eq{"sync_status": SyncStatusSynced})
query, args, err := q.ToSql() query, args, err := q.ToSql()
if err != nil { if err != nil {
@@ -179,15 +193,24 @@ func (s *Store) SubscribeContainers(ctx context.Context) ([]ContainerRecord, <-c
} }
var containers []ContainerRecord var containers []ContainerRecord
var cJSON, updatedAtStr string var id, cJSON, updatedAtStr string
skipped := 0
rows := sub.Rows() rows := sub.Rows()
for rows.Next() { for rows.Next() {
var cr ContainerRecord var cr ContainerRecord
if err = rows.Scan(&cJSON, &cr.MachineID, &cr.SyncStatus, &updatedAtStr); err != nil { if err = rows.Scan(&id, &cJSON, &cr.MachineID, &cr.SyncStatus, &updatedAtStr); err != nil {
return nil, nil, err return nil, nil, err
} }
// Skip containers with empty JSON data. This can happen during partial replication
// when cr-sqlite has created the row but the container column hasn't been synced yet.
if cJSON == "" || cJSON == "{}" {
slog.Debug("Skipping container with empty data in the store (partial replication?).", "id", id)
skipped++
continue
}
if err = json.Unmarshal([]byte(cJSON), &cr.Container); err != nil { if err = json.Unmarshal([]byte(cJSON), &cr.Container); err != nil {
return nil, nil, fmt.Errorf("unmarshal container: %w", err) return nil, nil, fmt.Errorf("unmarshal container: %w", err)
} }
@@ -196,6 +219,12 @@ func (s *Store) SubscribeContainers(ctx context.Context) ([]ContainerRecord, <-c
} }
containers = append(containers, cr) containers = append(containers, cr)
} }
if skipped > 0 {
slog.Warn("Container subscription skipped empty records in the store (possibly due to partial replication).",
"skipped", skipped, "valid", len(containers))
}
events, err := sub.Changes() events, err := sub.Changes()
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("get subscription changes: %w", err) return nil, nil, fmt.Errorf("get subscription changes: %w", err)
+58 -6
View File
@@ -56,6 +56,25 @@ func (s *Store) Delete(ctx context.Context, key string) error {
return err return err
} }
// DBVersion returns the current cr-sqlite database version (Lamport timestamp).
func (s *Store) DBVersion(ctx context.Context) (int64, error) {
rows, err := s.corro.QueryContext(ctx, "SELECT crsql_db_version()")
if err != nil {
return 0, fmt.Errorf("query crsql_db_version(): %w", err)
}
defer rows.Close()
if !rows.Next() {
return 0, fmt.Errorf("no result from crsql_db_version()")
}
var version int64
if err = rows.Scan(&version); err != nil {
return 0, fmt.Errorf("scan db version: %w", err)
}
return version, nil
}
func (s *Store) CreateMachine(ctx context.Context, m *pb.MachineInfo) error { func (s *Store) CreateMachine(ctx context.Context, m *pb.MachineInfo) error {
mJSON, err := protojson.Marshal(m) mJSON, err := protojson.Marshal(m)
if err != nil { if err != nil {
@@ -116,19 +135,29 @@ func (s *Store) GetMachine(ctx context.Context, machineID string) (*pb.MachineIn
} }
func (s *Store) ListMachines(ctx context.Context) ([]*pb.MachineInfo, error) { func (s *Store) ListMachines(ctx context.Context) ([]*pb.MachineInfo, error) {
rows, err := s.corro.QueryContext(ctx, "SELECT info FROM machines ORDER BY name") rows, err := s.corro.QueryContext(ctx, "SELECT id, info FROM machines ORDER BY name")
if err != nil { if err != nil {
return nil, err return nil, err
} }
defer rows.Close() defer rows.Close()
var machines []*pb.MachineInfo var machines []*pb.MachineInfo
skipped := 0
for rows.Next() { for rows.Next() {
var mJSON string var id, mJSON string
if err = rows.Scan(&mJSON); err != nil { if err = rows.Scan(&id, &mJSON); err != nil {
return nil, err return nil, err
} }
// Skip machines with empty JSON data. This can happen during partial replication
// when cr-sqlite has created the row but the info column hasn't been synced yet.
if mJSON == "" || mJSON == "{}" {
slog.Debug("Skipping machine with empty data in the store (partial replication?).", "id", id)
skipped++
continue
}
protojsonParser := protojson.UnmarshalOptions{DiscardUnknown: true} protojsonParser := protojson.UnmarshalOptions{DiscardUnknown: true}
var m pb.MachineInfo var m pb.MachineInfo
if err = protojsonParser.Unmarshal([]byte(mJSON), &m); err != nil { if err = protojsonParser.Unmarshal([]byte(mJSON), &m); err != nil {
@@ -141,6 +170,12 @@ func (s *Store) ListMachines(ctx context.Context) ([]*pb.MachineInfo, error) {
} }
machines = append(machines, &m) machines = append(machines, &m)
} }
if skipped > 0 {
slog.Warn("Listing machines from the store skipped empty records (possibly due to partial replication).",
"skipped", skipped, "valid", len(machines))
}
return machines, nil return machines, nil
} }
@@ -186,24 +221,41 @@ func (s *Store) DeleteMachine(ctx context.Context, id string) error {
// SubscribeMachines returns a list of machines and a channel that signals changes to the list. The channel doesn't // SubscribeMachines returns a list of machines and a channel that signals changes to the list. The channel doesn't
// receive any values, it just signals when a machine has been added, updated, or deleted in the database. // receive any values, it just signals when a machine has been added, updated, or deleted in the database.
func (s *Store) SubscribeMachines(ctx context.Context) ([]*pb.MachineInfo, <-chan struct{}, error) { func (s *Store) SubscribeMachines(ctx context.Context) ([]*pb.MachineInfo, <-chan struct{}, error) {
sub, err := s.corro.SubscribeContext(ctx, "SELECT info FROM machines ORDER BY name", nil, false) sub, err := s.corro.SubscribeContext(ctx, "SELECT id, info FROM machines ORDER BY name", nil, false)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
rows := sub.Rows() rows := sub.Rows()
var machines []*pb.MachineInfo var machines []*pb.MachineInfo
skipped := 0
for rows.Next() { for rows.Next() {
var mJSON string var id, mJSON string
if err = rows.Scan(&mJSON); err != nil { if err = rows.Scan(&id, &mJSON); err != nil {
return nil, nil, err return nil, nil, err
} }
// Skip machines with empty JSON data. This can happen during partial replication
// when cr-sqlite has created the row but the info column hasn't been synced yet.
if mJSON == "" || mJSON == "{}" {
slog.Debug("Skipping machine with empty data in the store (partial replication?).", "id", id)
skipped++
continue
}
var m pb.MachineInfo var m pb.MachineInfo
if err = protojson.Unmarshal([]byte(mJSON), &m); err != nil { if err = protojson.Unmarshal([]byte(mJSON), &m); err != nil {
return nil, nil, fmt.Errorf("unmarshal machine info: %w", err) return nil, nil, fmt.Errorf("unmarshal machine info: %w", err)
} }
machines = append(machines, &m) machines = append(machines, &m)
} }
if skipped > 0 {
slog.Warn("Machine subscription skipped empty records in the store (possibly due to partial replication).",
"skipped", skipped, "valid", len(machines))
}
events, err := sub.Changes() events, err := sub.Changes()
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("get subscription changes: %w", err) return nil, nil, fmt.Errorf("get subscription changes: %w", err)
+11
View File
@@ -4,6 +4,7 @@ import (
"fmt" "fmt"
"net" "net"
"os" "os"
osuser "os/user"
"strconv" "strconv"
"time" "time"
@@ -13,7 +14,17 @@ import (
) )
func Connect(user, host string, port int, sshKeyPath string) (*ssh.Client, error) { func Connect(user, host string, port int, sshKeyPath string) (*ssh.Client, error) {
// Use the current OS user if no user is specified to be make it consistent with ssh CLI behavior.
if user == "" {
if u, err := osuser.Current(); err == nil {
user = u.Username
}
}
if port == 0 {
port = 22
}
addr := net.JoinHostPort(host, strconv.Itoa(port)) addr := net.JoinHostPort(host, strconv.Itoa(port))
// Try to connect using SSH agent only. // Try to connect using SSH agent only.
agentAuth, agentClose, agentErr := sshAgentAuth() agentAuth, agentClose, agentErr := sshAgentAuth()
if agentErr == nil { if agentErr == nil {
+7 -2
View File
@@ -30,7 +30,7 @@ func NewSSHCLIRemote(user, host string, port int, keyPath string) *SSHCLIRemote
func (r *SSHCLIRemote) buildSSHArgs() []string { func (r *SSHCLIRemote) buildSSHArgs() []string {
args := []string{"-o", "ConnectTimeout=5"} args := []string{"-o", "ConnectTimeout=5"}
if r.port != 0 && r.port != 22 { if r.port != 0 {
args = append(args, "-p", strconv.Itoa(r.port)) args = append(args, "-p", strconv.Itoa(r.port))
} }
@@ -38,7 +38,12 @@ func (r *SSHCLIRemote) buildSSHArgs() []string {
args = append(args, "-i", r.keyPath) args = append(args, "-i", r.keyPath)
} }
args = append(args, r.user+"@"+r.host) dst := r.host
if r.user != "" {
dst = fmt.Sprintf("%s@%s", r.user, dst)
}
args = append(args, dst)
return args return args
} }
+27 -15
View File
@@ -18,29 +18,49 @@ func TestSSHCLIRemote_buildSSHArgs(t *testing.T) {
expected []string expected []string
}{ }{
{ {
name: "default port, no key", name: "host only",
host: "example.com",
expected: []string{"-o", "ConnectTimeout=5", "example.com"},
},
{
name: "with user",
user: "root", user: "root",
host: "example.com", host: "example.com",
port: 22,
keyPath: "",
expected: []string{"-o", "ConnectTimeout=5", "root@example.com"}, expected: []string{"-o", "ConnectTimeout=5", "root@example.com"},
}, },
{ {
name: "custom port", name: "with port",
host: "example.com",
port: 2222,
expected: []string{"-o", "ConnectTimeout=5", "-p", "2222", "example.com"},
},
{
name: "with key",
host: "example.com",
keyPath: "/path/to/key",
expected: []string{"-o", "ConnectTimeout=5", "-i", "/path/to/key", "example.com"},
},
{
name: "user and port",
user: "ubuntu", user: "ubuntu",
host: "192.168.1.10", host: "192.168.1.10",
port: 2222, port: 2222,
keyPath: "",
expected: []string{"-o", "ConnectTimeout=5", "-p", "2222", "ubuntu@192.168.1.10"}, expected: []string{"-o", "ConnectTimeout=5", "-p", "2222", "ubuntu@192.168.1.10"},
}, },
{ {
name: "with key path", name: "user and key",
user: "root", user: "root",
host: "example.com", host: "example.com",
port: 22,
keyPath: "/path/to/key", keyPath: "/path/to/key",
expected: []string{"-o", "ConnectTimeout=5", "-i", "/path/to/key", "root@example.com"}, expected: []string{"-o", "ConnectTimeout=5", "-i", "/path/to/key", "root@example.com"},
}, },
{
name: "port and key",
host: "example.com",
port: 22,
keyPath: "~/.ssh/id_rsa",
expected: []string{"-o", "ConnectTimeout=5", "-p", "22", "-i", "~/.ssh/id_rsa", "example.com"},
},
{ {
name: "all options", name: "all options",
user: "admin", user: "admin",
@@ -49,14 +69,6 @@ func TestSSHCLIRemote_buildSSHArgs(t *testing.T) {
keyPath: "~/.ssh/id_rsa", keyPath: "~/.ssh/id_rsa",
expected: []string{"-o", "ConnectTimeout=5", "-p", "2222", "-i", "~/.ssh/id_rsa", "admin@server.local"}, expected: []string{"-o", "ConnectTimeout=5", "-p", "2222", "-i", "~/.ssh/id_rsa", "admin@server.local"},
}, },
{
name: "port zero (default)",
user: "root",
host: "example.com",
port: 0,
keyPath: "",
expected: []string{"-o", "ConnectTimeout=5", "root@example.com"},
},
} }
for _, tt := range tests { for _, tt := range tests {
+46 -16
View File
@@ -119,17 +119,16 @@ func (p *Provisioner) CreateCluster(ctx context.Context, name string, opts Creat
func (p *Provisioner) initCluster(ctx context.Context, machines []Machine) error { func (p *Provisioner) initCluster(ctx context.Context, machines []Machine) error {
// Init a new cluster on the first machine. // Init a new cluster on the first machine.
initMachine := machines[0] initMachine := machines[0]
if err := WaitMachineReady(ctx, initMachine, 30*time.Second); err != nil {
return fmt.Errorf("wait for machine %q to be ready: %w", initMachine.Name, err)
}
initClient, err := initMachine.Connect(ctx) initClient, err := initMachine.Connect(ctx)
if err != nil { if err != nil {
return fmt.Errorf("create machine client over TCP '%s': %w", initMachine.APIAddress, err) return fmt.Errorf("create machine client over TCP '%s': %w", initMachine.APIAddress, err)
} }
defer initClient.Close() defer initClient.Close()
if err := initClient.WaitMachineReady(ctx, 30*time.Second); err != nil {
return fmt.Errorf("wait for machine %q to be ready: %w", initMachine.Name, err)
}
req := &pb.InitClusterRequest{ req := &pb.InitClusterRequest{
MachineName: initMachine.Name, MachineName: initMachine.Name,
Network: pb.NewIPPrefix(cluster.DefaultNetwork), Network: pb.NewIPPrefix(cluster.DefaultNetwork),
@@ -138,14 +137,22 @@ func (p *Provisioner) initCluster(ctx context.Context, machines []Machine) error
if err != nil { if err != nil {
return fmt.Errorf("init cluster: %w", err) return fmt.Errorf("init cluster: %w", err)
} }
fmt.Printf("Cluster %q initialised with machine %q\n", initMachine.ClusterName, initResp.Machine.Name) fmt.Printf("Cluster %q initialised with machine %q\n", initMachine.ClusterName, initResp.Machine.Name)
fmt.Printf("Waiting for cluster to be ready...")
if err = initClient.WaitClusterReady(ctx, 30*time.Second); err != nil {
return fmt.Errorf("wait for cluster to be ready: %w", err)
}
fmt.Println(" done.")
// Get the current store DB version from the init machine to pass to the join requests.
inspectResp, err := initClient.MachineClient.InspectMachine(ctx, &emptypb.Empty{})
if err != nil {
return fmt.Errorf("inspect init machine: %w", err)
}
// Join the rest of the machines to the cluster. // Join the rest of the machines to the cluster.
for _, m := range machines[1:] { for _, m := range machines[1:] {
if err = WaitMachineReady(ctx, m, 30*time.Second); err != nil {
return fmt.Errorf("wait for machine %q to be ready: %w", m.Name, err)
}
cli, err := m.Connect(ctx) cli, err := m.Connect(ctx)
if err != nil { if err != nil {
return fmt.Errorf("create machine client over TCP '%s': %w", m.APIAddress, err) return fmt.Errorf("create machine client over TCP '%s': %w", m.APIAddress, err)
@@ -153,6 +160,10 @@ func (p *Provisioner) initCluster(ctx context.Context, machines []Machine) error
//goland:noinspection GoDeferInLoop //goland:noinspection GoDeferInLoop
defer cli.Close() defer cli.Close()
if err := cli.WaitMachineReady(ctx, 30*time.Second); err != nil {
return fmt.Errorf("wait for machine %q to be ready: %w", m.Name, err)
}
tokenResp, err := cli.Token(ctx, &emptypb.Empty{}) tokenResp, err := cli.Token(ctx, &emptypb.Empty{})
if err != nil { if err != nil {
return fmt.Errorf("get machine token: %w", err) return fmt.Errorf("get machine token: %w", err)
@@ -181,8 +192,9 @@ func (p *Provisioner) initCluster(ctx context.Context, machines []Machine) error
// Configure the machine to join the cluster. // Configure the machine to join the cluster.
joinReq := &pb.JoinClusterRequest{ joinReq := &pb.JoinClusterRequest{
Machine: addResp.Machine, Machine: addResp.Machine,
OtherMachines: []*pb.MachineInfo{initResp.Machine}, OtherMachines: []*pb.MachineInfo{initResp.Machine},
MinStoreDbVersion: inspectResp.Machines[0].StoreDbVersion,
} }
if _, err = cli.JoinCluster(ctx, joinReq); err != nil { if _, err = cli.JoinCluster(ctx, joinReq); err != nil {
return fmt.Errorf("join cluster: %w", err) return fmt.Errorf("join cluster: %w", err)
@@ -256,16 +268,16 @@ func (p *Provisioner) InspectCluster(ctx context.Context, name string) (Cluster,
// WaitClusterReady waits for all machines in the cluster to be ready and UP. // WaitClusterReady waits for all machines in the cluster to be ready and UP.
func (p *Provisioner) WaitClusterReady(ctx context.Context, c Cluster, timeout time.Duration) error { func (p *Provisioner) WaitClusterReady(ctx context.Context, c Cluster, timeout time.Duration) error {
firstMachine := c.Machines[0] firstMachine := c.Machines[0]
if err := WaitMachineReady(ctx, firstMachine, timeout); err != nil {
return fmt.Errorf("wait for machine '%s' to be ready: %w", firstMachine.Name, err)
}
cli, err := firstMachine.Connect(ctx) cli, err := firstMachine.Connect(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to machine over TCP '%s': %w", firstMachine.APIAddress, err) return fmt.Errorf("connect to machine over TCP '%s': %w", firstMachine.APIAddress, err)
} }
defer cli.Close() defer cli.Close()
if err = cli.WaitClusterReady(ctx, timeout); err != nil {
return fmt.Errorf("wait for cluster to be ready: %w", err)
}
boff := backoff.WithContext(backoff.NewExponentialBackOff( boff := backoff.WithContext(backoff.NewExponentialBackOff(
backoff.WithInitialInterval(100*time.Millisecond), backoff.WithInitialInterval(100*time.Millisecond),
backoff.WithMaxInterval(1*time.Second), backoff.WithMaxInterval(1*time.Second),
@@ -289,7 +301,25 @@ func (p *Provisioner) WaitClusterReady(ctx context.Context, c Cluster, timeout t
} }
return nil return nil
} }
return backoff.Retry(checkMachinesUp, boff) if err = backoff.Retry(checkMachinesUp, boff); err != nil {
return err
}
// Wait for each machine to sync the cluster store and be ready to serve cluster requests.
for _, m := range c.Machines[1:] {
mcli, err := m.Connect(ctx)
if err != nil {
return fmt.Errorf("connect to machine over TCP '%s': %w", m.APIAddress, err)
}
//goland:noinspection GoDeferInLoop
defer mcli.Close()
if err = mcli.WaitClusterReady(ctx, timeout); err != nil {
return fmt.Errorf("wait for cluster to be ready on machine '%s': %w", m.Name, err)
}
}
return nil
} }
func (p *Provisioner) RemoveCluster(ctx context.Context, name string) error { func (p *Provisioner) RemoveCluster(ctx context.Context, name string) error {
-25
View File
@@ -9,7 +9,6 @@ import (
"net/netip" "net/netip"
"time" "time"
"github.com/cenkalti/backoff/v4"
"github.com/containerd/errdefs" "github.com/containerd/errdefs"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/image" "github.com/docker/docker/api/types/image"
@@ -17,7 +16,6 @@ import (
"github.com/psviderski/uncloud/internal/secret" "github.com/psviderski/uncloud/internal/secret"
"github.com/psviderski/uncloud/pkg/client" "github.com/psviderski/uncloud/pkg/client"
"github.com/psviderski/uncloud/pkg/client/connector" "github.com/psviderski/uncloud/pkg/client/connector"
"google.golang.org/protobuf/types/known/emptypb"
) )
const ( const (
@@ -194,26 +192,3 @@ func randomMachineName() (string, error) {
} }
return "machine-" + suffix, nil return "machine-" + suffix, nil
} }
// WaitMachineReady waits for the machine API to respond.
func WaitMachineReady(ctx context.Context, m Machine, timeout time.Duration) error {
cli, err := m.Connect(ctx)
if err != nil {
return fmt.Errorf("connect to machine over TCP '%s': %w", m.APIAddress, err)
}
defer cli.Close()
boff := backoff.WithContext(backoff.NewExponentialBackOff(
backoff.WithInitialInterval(100*time.Millisecond),
backoff.WithMaxInterval(10*time.Second),
backoff.WithMaxElapsedTime(timeout),
), ctx)
inspect := func() error {
if _, err := cli.Inspect(ctx, &emptypb.Empty{}); err != nil {
return fmt.Errorf("inspect machine: %w", err)
}
return nil
}
return backoff.Retry(inspect, boff)
}
+10 -2
View File
@@ -21,11 +21,14 @@ type ContainerClient interface {
CreateContainer( CreateContainer(
ctx context.Context, serviceID string, spec ServiceSpec, machineID string, ctx context.Context, serviceID string, spec ServiceSpec, machineID string,
) (container.CreateResponse, error) ) (container.CreateResponse, error)
ExecContainer(ctx context.Context, serviceNameOrID, containerNameOrID string, config ExecOptions) (int, error)
InspectContainer(ctx context.Context, serviceNameOrID, containerNameOrID string) (MachineServiceContainer, error) InspectContainer(ctx context.Context, serviceNameOrID, containerNameOrID string) (MachineServiceContainer, error)
RemoveContainer(ctx context.Context, serviceNameOrID, containerNameOrID string, opts container.RemoveOptions) error
StartContainer(ctx context.Context, serviceNameOrID, containerNameOrID string) error StartContainer(ctx context.Context, serviceNameOrID, containerNameOrID string) error
StopContainer(ctx context.Context, serviceNameOrID, containerNameOrID string, opts container.StopOptions) error StopContainer(ctx context.Context, serviceNameOrID, containerNameOrID string, opts container.StopOptions) error
ExecContainer(ctx context.Context, serviceNameOrID, containerNameOrID string, config ExecOptions) (int, error) RemoveContainer(ctx context.Context, serviceNameOrID, containerNameOrID string, opts container.RemoveOptions) error
WaitContainerHealthy(
ctx context.Context, serviceNameOrID, containerNameOrID string, opts WaitContainerHealthyOptions,
) error
} }
type DNSClient interface { type DNSClient interface {
@@ -57,3 +60,8 @@ type VolumeClient interface {
ListVolumes(ctx context.Context, filter *VolumeFilter) ([]MachineVolume, error) ListVolumes(ctx context.Context, filter *VolumeFilter) ([]MachineVolume, error)
RemoveVolume(ctx context.Context, machineNameOrID, volumeName string, force bool) error RemoveVolume(ctx context.Context, machineNameOrID, volumeName string, force bool) error
} }
// AsPtr returns a pointer to the given value. Useful for optional fields in API structs.
func AsPtr[T any](v T) *T {
return &v
}
+71
View File
@@ -6,6 +6,7 @@ import (
"fmt" "fmt"
"os" "os"
"path/filepath" "path/filepath"
"sort"
"strconv" "strconv"
) )
@@ -98,6 +99,76 @@ func (c *ConfigMount) Validate() error {
return nil return nil
} }
// Compare compares this ConfigMount with another.
// Returns:
//
// -1 if c < other
// 0 if c == other
// +1 if c > other
func (c *ConfigMount) Compare(other *ConfigMount) int {
if c.ConfigName != other.ConfigName {
if c.ConfigName < other.ConfigName {
return -1
}
return 1
}
if c.ContainerPath != other.ContainerPath {
if c.ContainerPath < other.ContainerPath {
return -1
}
return 1
}
if c.Uid != other.Uid {
if c.Uid < other.Uid {
return -1
}
return 1
}
if c.Gid != other.Gid {
if c.Gid < other.Gid {
return -1
}
return 1
}
// Compare Mode (handle nil cases)
if c.Mode == nil && other.Mode != nil {
return -1
}
if c.Mode != nil && other.Mode == nil {
return 1
}
if c.Mode != nil && other.Mode != nil {
if *c.Mode < *other.Mode {
return -1
}
if *c.Mode > *other.Mode {
return 1
}
}
return 0
}
// Equals compares two ConfigMount instances for equality.
func (c *ConfigMount) Equals(other *ConfigMount) bool {
return c.Compare(other) == 0
}
func (c *ConfigMount) Clone() ConfigMount {
clone := *c
if c.Mode != nil {
mode := *c.Mode
clone.Mode = &mode
}
return clone
}
// sortConfigMounts sorts a slice of ConfigMount instances.
func sortConfigMounts(mounts []ConfigMount) {
sort.Slice(mounts, func(i, j int) bool {
return mounts[i].Compare(&mounts[j]) < 0
})
}
// ValidateConfigsAndMounts takes config specs and config mounts and validates that all mounts refer to existing specs // ValidateConfigsAndMounts takes config specs and config mounts and validates that all mounts refer to existing specs
func ValidateConfigsAndMounts(configs []ConfigSpec, mounts []ConfigMount) error { func ValidateConfigsAndMounts(configs []ConfigSpec, mounts []ConfigMount) error {
configMap := make(map[string]struct{}) configMap := make(map[string]struct{})
+77
View File
@@ -3,7 +3,9 @@ package api
import ( import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"io"
"net/netip" "net/netip"
"os"
"strings" "strings"
"time" "time"
@@ -41,6 +43,19 @@ func (c *Container) CreatedTime() time.Time {
return c.created return c.created
} }
// HasHealthcheck returns true if the container has a health check configured.
func (c *Container) HasHealthcheck() bool {
hc := c.Config.Healthcheck
if hc == nil {
return false
}
if len(hc.Test) > 0 && hc.Test[0] == "NONE" {
return false
}
return true
}
// Healthy determines if the container is running and healthy. // Healthy determines if the container is running and healthy.
// A running container with no health check configured is considered healthy. // A running container with no health check configured is considered healthy.
func (c *Container) Healthy() bool { func (c *Container) Healthy() bool {
@@ -135,6 +150,10 @@ func (c *Container) UnmarshalJSON(data []byte) error {
} }
*c = Container(temp) *c = Container(temp)
if c.ContainerJSONBase == nil {
return fmt.Errorf("container data is missing mandatory base fields: %s", data)
}
c.Name = strings.TrimPrefix(c.Name, "/") c.Name = strings.TrimPrefix(c.Name, "/")
return nil return nil
@@ -240,3 +259,61 @@ func (c *ServiceContainer) UnmarshalJSON(data []byte) error {
return nil return nil
} }
// DefaultHealthMonitorPeriod is the default duration (5 seconds) to wait before checking that the container is still
// running and not restarting. Can be overridden with the UNCLOUD_HEALTH_MONITOR_PERIOD environment variable
// (e.g. "10s" or "0").
var DefaultHealthMonitorPeriod = defaultHealthMonitorPeriod()
func defaultHealthMonitorPeriod() time.Duration {
if v, ok := os.LookupEnv("UNCLOUD_HEALTH_MONITOR_PERIOD"); ok {
if d, err := time.ParseDuration(v); err == nil {
return d
}
}
return 5 * time.Second
}
// WaitContainerHealthyOptions configures the behaviour of WaitContainerHealthy.
type WaitContainerHealthyOptions struct {
// MonitorPeriod is how long to wait before checking that the container is still running and not restarting.
// Containers with a health check that become healthy before the period ends succeed early.
// nil means use the default DefaultHealthMonitorPeriod.
// Zero skips the monitoring and checks the container's health immediately after starting.
MonitorPeriod *time.Duration
}
// ExecOptions contains configuration for executing a command in a container.
type ExecOptions struct {
// Command is the command to run in the container.
Command []string
// AttachStdin attaches the stdin stream to the exec session.
AttachStdin bool
// AttachStdout attaches the stdout stream to the exec session.
AttachStdout bool
// AttachStderr attaches the stderr stream to the exec session.
AttachStderr bool
// Tty allocates a pseudo-TTY for the exec session.
Tty bool
// Detach runs the command in the background without attaching to streams.
Detach bool
//// Not yet implemented fields
// User specifies the user to run the command as.
User string
// Privileged runs the command in privileged mode.
Privileged bool
// WorkingDir sets the working directory for the command.
WorkingDir string
// Env sets environment variables for the command.
Env []string
// Client-side only fields (not serialized, not sent to server)
// Stdin is the input stream. Defaults to os.Stdin if nil.
Stdin io.Reader `json:"-"`
// Stdout is the output stream. Defaults to os.Stdout if nil.
Stdout io.Writer `json:"-"`
// Stderr is the error stream. Defaults to os.Stderr if nil.
Stderr io.Writer `json:"-"`
}
-37
View File
@@ -1,37 +0,0 @@
package api
import "io"
// ExecOptions contains configuration for executing a command in a container.
type ExecOptions struct {
// Command is the command to run in the container.
Command []string
// AttachStdin attaches the stdin stream to the exec session.
AttachStdin bool
// AttachStdout attaches the stdout stream to the exec session.
AttachStdout bool
// AttachStderr attaches the stderr stream to the exec session.
AttachStderr bool
// Tty allocates a pseudo-TTY for the exec session.
Tty bool
// Detach runs the command in the background without attaching to streams.
Detach bool
//// Not yet implemented fields
// User specifies the user to run the command as.
User string
// Privileged runs the command in privileged mode.
Privileged bool
// WorkingDir sets the working directory for the command.
WorkingDir string
// Env sets environment variables for the command.
Env []string
// Client-side only fields (not serialized, not sent to server)
// Stdin is the input stream. Defaults to os.Stdin if nil.
Stdin io.Reader `json:"-"`
// Stdout is the output stream. Defaults to os.Stdout if nil.
Stdout io.Writer `json:"-"`
// Stderr is the error stream. Defaults to os.Stderr if nil.
Stderr io.Writer `json:"-"`
}
+20 -1
View File
@@ -17,6 +17,25 @@ type ContainerResources struct {
// MemoryReservation is the minimum amount of memory (in bytes) the container needs to run efficiently. // MemoryReservation is the minimum amount of memory (in bytes) the container needs to run efficiently.
// TODO: implement a placement constraint that checks available memory on machines. // TODO: implement a placement constraint that checks available memory on machines.
MemoryReservation int64 MemoryReservation int64
// Device reservations/requests for access to things like GPUs // Devices provides direct access to host devices.
Devices []DeviceMapping
// DeviceReservations requests for access to things like GPUs.
DeviceReservations []container.DeviceRequest DeviceReservations []container.DeviceRequest
// Ulimits defines the resource limits for the container.
Ulimits map[string]Ulimit
}
// DeviceMapping represents a device mapping between host and container.
type DeviceMapping struct {
// HostPath is the path to the device on the host.
HostPath string
// ContainerPath is the path to the device in the container.
ContainerPath string
// CgroupPermissions is the cgroup permissions for the device (e.g., "rwm").
CgroupPermissions string
}
type Ulimit struct {
Soft int64
Hard int64
} }
+100 -2
View File
@@ -7,6 +7,7 @@ import (
"regexp" "regexp"
"slices" "slices"
"strings" "strings"
"time"
mapset "github.com/deckarep/golang-set/v2" mapset "github.com/deckarep/golang-set/v2"
"github.com/distribution/reference" "github.com/distribution/reference"
@@ -19,6 +20,13 @@ const (
ServiceModeReplicated = "replicated" ServiceModeReplicated = "replicated"
ServiceModeGlobal = "global" ServiceModeGlobal = "global"
// UpdateOrderStartFirst starts the new container before stopping the old one.
// This minimizes downtime but briefly runs both containers.
UpdateOrderStartFirst = "start-first"
// UpdateOrderStopFirst stops the old container before starting the new one.
// This prevents data corruption for stateful services but causes brief downtime.
UpdateOrderStopFirst = "stop-first"
// PullPolicyAlways means the image is always pulled from the registry. // PullPolicyAlways means the image is always pulled from the registry.
PullPolicyAlways = "always" PullPolicyAlways = "always"
// PullPolicyMissing means the image is pulled from the registry only if it's not available on the machine where // PullPolicyMissing means the image is pulled from the registry only if it's not available on the machine where
@@ -59,6 +67,8 @@ type ServiceSpec struct {
Ports []PortSpec Ports []PortSpec
// Replicas is the number of containers to run for the service. Only valid for a replicated service. // Replicas is the number of containers to run for the service. Only valid for a replicated service.
Replicas uint `json:",omitempty"` Replicas uint `json:",omitempty"`
// UpdateConfig configures how the service is updated during a deployment.
UpdateConfig UpdateConfig `json:",omitempty"`
// Volumes is list of data volumes that can be mounted into the container. // Volumes is list of data volumes that can be mounted into the container.
Volumes []VolumeSpec Volumes []VolumeSpec
// Configs is list of configuration objects that can be mounted into the container. // Configs is list of configuration objects that can be mounted into the container.
@@ -225,13 +235,20 @@ func (s *ServiceSpec) Clone() ServiceSpec {
// ContainerSpec defines the desired state of a container in a service. // ContainerSpec defines the desired state of a container in a service.
// ATTENTION: after changing this struct, verify if deploy.EvalContainerSpecChange needs to be updated. // ATTENTION: after changing this struct, verify if deploy.EvalContainerSpecChange needs to be updated.
type ContainerSpec struct { type ContainerSpec struct {
// Specifies which additional capabilities should be added for the container.
CapAdd []string
// Specifies which capabilities should be dropped from the container.
CapDrop []string
// Command overrides the default CMD of the image to be executed when running a container. // Command overrides the default CMD of the image to be executed when running a container.
Command []string Command []string
// Entrypoint overrides the default ENTRYPOINT of the image. // Entrypoint overrides the default ENTRYPOINT of the image.
Entrypoint []string Entrypoint []string
// Env defines the environment variables to set inside the container. // Env defines the environment variables to set inside the container.
Env EnvVars Env EnvVars
Image string // Healthcheck defines the health check configuration for the container or overrides the health check options
// defined in the image. If nil, the image's default health check is used.
Healthcheck *HealthcheckSpec `json:",omitempty"`
Image string
// Run a custom init inside the container. If nil, use the daemon's configured settings. // Run a custom init inside the container. If nil, use the daemon's configured settings.
Init *bool Init *bool
// LogDriver overrides the default logging driver for the container. Each Docker daemon can have its own default. // LogDriver overrides the default logging driver for the container. Each Docker daemon can have its own default.
@@ -243,6 +260,8 @@ type ContainerSpec struct {
PullPolicy string PullPolicy string
// Resource allocation for the container. // Resource allocation for the container.
Resources ContainerResources Resources ContainerResources
// Namespaced kernel parameters to be set in container
Sysctls map[string]string
// User overrides the default user of the image used to run the container. Format: user|UID[:group|GID]. // User overrides the default user of the image used to run the container. Format: user|UID[:group|GID].
User string User string
// VolumeMounts specifies how volumes are mounted into the container filesystem. // VolumeMounts specifies how volumes are mounted into the container filesystem.
@@ -290,18 +309,32 @@ func (s *ContainerSpec) Equals(spec ContainerSpec) bool {
orig := s.SetDefaults() orig := s.SetDefaults()
spec = spec.SetDefaults() spec = spec.SetDefaults()
// Volumes
slices.Sort(orig.Volumes) slices.Sort(orig.Volumes)
slices.Sort(spec.Volumes) slices.Sort(spec.Volumes)
// Volume mounts
sortVolumeMounts(orig.VolumeMounts) sortVolumeMounts(orig.VolumeMounts)
sortVolumeMounts(spec.VolumeMounts) sortVolumeMounts(spec.VolumeMounts)
// Config mounts
sortConfigMounts(orig.ConfigMounts)
sortConfigMounts(spec.ConfigMounts)
return cmp.Equal(orig, spec, cmpopts.EquateEmpty()) return cmp.Equal(orig, spec, cmpopts.EquateEmpty())
} }
func (s *ContainerSpec) Clone() ContainerSpec { func (s *ContainerSpec) Clone() ContainerSpec {
spec := *s spec := *s
if s.CapAdd != nil {
spec.CapAdd = make([]string, len(s.CapAdd))
copy(spec.CapAdd, s.CapAdd)
}
if s.CapDrop != nil {
spec.CapDrop = make([]string, len(s.CapDrop))
copy(spec.CapDrop, s.CapDrop)
}
if s.Command != nil { if s.Command != nil {
spec.Command = make([]string, len(s.Command)) spec.Command = make([]string, len(s.Command))
copy(spec.Command, s.Command) copy(spec.Command, s.Command)
@@ -310,6 +343,17 @@ func (s *ContainerSpec) Clone() ContainerSpec {
spec.Entrypoint = make([]string, len(s.Entrypoint)) spec.Entrypoint = make([]string, len(s.Entrypoint))
copy(spec.Entrypoint, s.Entrypoint) copy(spec.Entrypoint, s.Entrypoint)
} }
if s.Env != nil {
spec.Env = make(EnvVars, len(s.Env))
for k, v := range s.Env {
spec.Env[k] = v
}
}
if s.Healthcheck != nil {
hc := *s.Healthcheck
hc.Test = slices.Clone(s.Healthcheck.Test)
spec.Healthcheck = &hc
}
if s.LogDriver != nil { if s.LogDriver != nil {
logDriver := *s.LogDriver logDriver := *s.LogDriver
if s.LogDriver.Options != nil { if s.LogDriver.Options != nil {
@@ -325,6 +369,27 @@ func (s *ContainerSpec) Clone() ContainerSpec {
spec.VolumeMounts = make([]VolumeMount, len(s.VolumeMounts)) spec.VolumeMounts = make([]VolumeMount, len(s.VolumeMounts))
copy(spec.VolumeMounts, s.VolumeMounts) copy(spec.VolumeMounts, s.VolumeMounts)
} }
if s.ConfigMounts != nil {
spec.ConfigMounts = make([]ConfigMount, len(s.ConfigMounts))
for i, cm := range s.ConfigMounts {
spec.ConfigMounts[i] = cm.Clone()
}
}
if s.Sysctls != nil {
spec.Sysctls = make(map[string]string, len(s.Sysctls))
for k, v := range s.Sysctls {
spec.Sysctls[k] = v
}
}
if s.Resources.Ulimits != nil {
spec.Resources.Ulimits = maps.Clone(s.Resources.Ulimits)
}
if s.Resources.Devices != nil {
spec.Resources.Devices = slices.Clone(s.Resources.Devices)
}
if s.Resources.DeviceReservations != nil {
spec.Resources.DeviceReservations = slices.Clone(s.Resources.DeviceReservations)
}
return spec return spec
} }
@@ -343,6 +408,26 @@ func (e EnvVars) ToSlice() []string {
return env return env
} }
// HealthcheckSpec defines the health check configuration for a container.
type HealthcheckSpec struct {
// Test is the command used to check health.
// Formats: ["CMD", args...], ["CMD-SHELL", "command"], or ["NONE"] to disable.
Test []string `json:",omitempty"`
// Interval is the time between health checks.
// Zero means to inherit the value from the image or use the Docker default (30s) if not defined in the image.
Interval time.Duration `json:",omitempty"`
// Timeout is how long to wait before considering the checck to have hung.
Timeout time.Duration `json:",omitempty"`
// StartPeriod is the initialisation time for a container before the retries start to count down.
StartPeriod time.Duration `json:",omitempty"`
// StartInterval is the time between health checks during the start period.
StartInterval time.Duration `json:",omitempty"`
// Retries is the number of consecutive failures needed to consider a container unhealthy.
Retries uint `json:",omitempty"`
// Disable disables the health check defined in the image. true is equivalent to setting Test to ["NONE"].
Disable bool `json:",omitempty"`
}
type LogDriver struct { type LogDriver struct {
// Name of the logging driver to use. // Name of the logging driver to use.
Name string Name string
@@ -350,6 +435,19 @@ type LogDriver struct {
Options map[string]string Options map[string]string
} }
// UpdateConfig configures how a service is updated during a deployment.
type UpdateConfig struct {
// Order specifies the order of operations during an update.
// Valid values are "start-first" (default for stateless services) and "stop-first" (default for services with
// volumes). Empty value means the strategy will determine the order based on service characteristics.
Order string `json:",omitempty"`
// MonitorPeriod is how long to wait after starting a container before checking that it's still running
// and not restarting. Containers with a health check that become healthy before the period ends succeed early.
// nil means use the default api.DefaultHealthMonitorPeriod.
// Zero skips the monitoring and checks the container's health immediately after starting.
MonitorPeriod *time.Duration `json:",omitempty"`
}
type RunServiceResponse struct { type RunServiceResponse struct {
ID string ID string
Name string Name string
+107
View File
@@ -1,11 +1,20 @@
package api package api
import ( import (
"os"
"testing" "testing"
"github.com/docker/docker/api/types/container"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
// boolPtr is a convenience function to create a pointer to a uint64 value
// TODO: Make this a generic function that works for any type
func boolPtr(b bool) *bool {
return &b
}
func TestServiceSpec_Validate_CaddyAndPorts(t *testing.T) { func TestServiceSpec_Validate_CaddyAndPorts(t *testing.T) {
tests := []struct { tests := []struct {
name string name string
@@ -197,3 +206,101 @@ func TestServiceSpec_Validate_CaddyAndPorts(t *testing.T) {
}) })
} }
} }
func TestContainerSpec_Clone(t *testing.T) {
mode := os.FileMode(0o644)
original := ContainerSpec{
CapAdd: []string{"NET_ADMIN"},
CapDrop: []string{"ALL"},
Command: []string{"sh", "-c", "echo hello"},
Entrypoint: []string{"/bin/bash"},
Env: EnvVars{
"FOO": "bar",
"BAZ": "qux",
},
Image: "nginx:latest",
Init: boolPtr(true),
LogDriver: &LogDriver{
Name: "json-file",
Options: map[string]string{
"max-size": "10m",
},
},
Privileged: true,
PullPolicy: PullPolicyAlways,
Resources: ContainerResources{
CPU: 1234,
Memory: 2345,
MemoryReservation: 3456,
Devices: []DeviceMapping{
{HostPath: "/dev/sda", ContainerPath: "/dev/xvda", CgroupPermissions: "rwm"},
},
DeviceReservations: []container.DeviceRequest{
{Count: 1, Capabilities: [][]string{{"gpu"}}, Driver: "nvidia"},
},
},
Sysctls: map[string]string{
"net.ipv4.ip_forward": "1",
},
User: "1000:1000",
Volumes: []string{"/data", "/config"},
VolumeMounts: []VolumeMount{
{VolumeName: "data", ContainerPath: "/data"},
},
ConfigMounts: []ConfigMount{
{ConfigName: "app-config", ContainerPath: "/etc/config", Mode: &mode},
},
}
cloned := original.Clone()
// Check ContainerSpec equality
assert.True(t, original.Equals(cloned))
// Verify deep copy by modifying the original
stringModified := "modified"
original.CapAdd[0] = stringModified
original.CapDrop[0] = stringModified
original.Command[0] = stringModified
original.Entrypoint[0] = stringModified
original.Env["FOO"] = stringModified
original.LogDriver.Options["max-size"] = stringModified
original.Volumes[0] = stringModified
original.VolumeMounts[0].ContainerPath = stringModified
original.ConfigMounts[0].ContainerPath = stringModified
*original.ConfigMounts[0].Mode = 0o755 // Modify the Mode pointer value
original.Sysctls["net.ipv4.ip_forward"] = stringModified
original.Resources.Devices[0].HostPath = stringModified
original.Resources.DeviceReservations[0].Count = 2
original.Resources.DeviceReservations[0].Driver = stringModified
assert.False(t, original.Equals(cloned))
// Assert cloned values are unchanged
assert.Equal(t, "NET_ADMIN", cloned.CapAdd[0])
assert.Equal(t, "ALL", cloned.CapDrop[0])
assert.Equal(t, "sh", cloned.Command[0])
assert.Equal(t, "/bin/bash", cloned.Entrypoint[0])
assert.Equal(t, "bar", cloned.Env["FOO"])
assert.Equal(t, "qux", cloned.Env["BAZ"])
assert.Equal(t, "nginx:latest", cloned.Image)
assert.NotNil(t, cloned.Init)
assert.Equal(t, true, *cloned.Init)
assert.NotNil(t, cloned.LogDriver)
assert.Equal(t, "json-file", cloned.LogDriver.Name)
assert.Equal(t, "10m", cloned.LogDriver.Options["max-size"])
assert.Equal(t, true, cloned.Privileged)
assert.Equal(t, PullPolicyAlways, cloned.PullPolicy)
assert.Equal(t, int64(1234), cloned.Resources.CPU)
assert.Equal(t, int64(2345), cloned.Resources.Memory)
assert.Equal(t, int64(3456), cloned.Resources.MemoryReservation)
assert.Equal(t, "/dev/sda", cloned.Resources.Devices[0].HostPath)
assert.Equal(t, 1, cloned.Resources.DeviceReservations[0].Count)
assert.Equal(t, "nvidia", cloned.Resources.DeviceReservations[0].Driver)
assert.Equal(t, "1000:1000", cloned.User)
assert.Equal(t, "/data", cloned.Volumes[0])
assert.Equal(t, "/data", cloned.VolumeMounts[0].ContainerPath)
assert.Equal(t, "/etc/config", cloned.ConfigMounts[0].ContainerPath)
assert.NotNil(t, cloned.ConfigMounts[0].Mode)
assert.Equal(t, os.FileMode(0o644), *cloned.ConfigMounts[0].Mode, "Mode should be deep copied")
assert.Equal(t, "1", cloned.Sysctls["net.ipv4.ip_forward"])
}
+3
View File
@@ -83,6 +83,9 @@ func (v *VolumeSpec) SetDefaults() VolumeSpec {
if spec.VolumeOptions.Name == "" { if spec.VolumeOptions.Name == "" {
spec.VolumeOptions.Name = spec.Name spec.VolumeOptions.Name = spec.Name
} }
if spec.VolumeOptions.Driver != nil && spec.VolumeOptions.Driver.Name == "" {
spec.VolumeOptions.Driver.Name = VolumeDriverLocal
}
} }
// TODO: set explicit default values for Propagation and Recursive for bind mounts? // TODO: set explicit default values for Propagation and Recursive for bind mounts?
+142
View File
@@ -0,0 +1,142 @@
package api
import (
"testing"
"github.com/docker/docker/api/types/mount"
"github.com/docker/docker/api/types/volume"
"github.com/stretchr/testify/assert"
)
func TestVolumeSpec_MatchesDockerVolume(t *testing.T) {
t.Parallel()
tests := []struct {
name string
spec VolumeSpec
vol volume.Volume
expected bool
}{
{
name: "match with explicit local driver",
spec: VolumeSpec{
Type: VolumeTypeVolume,
VolumeOptions: &VolumeOptions{
Driver: &mount.Driver{
Name: "local",
Options: map[string]string{
"foo": "bar",
},
},
},
},
vol: volume.Volume{
Driver: "local",
Options: map[string]string{
"foo": "bar",
},
},
expected: true,
},
{
name: "match with empty driver name in spec (implicit local)",
spec: VolumeSpec{
Type: VolumeTypeVolume,
VolumeOptions: &VolumeOptions{
Driver: &mount.Driver{
Name: "", // Implicitly local
Options: map[string]string{
"foo": "bar",
},
},
},
},
vol: volume.Volume{
Driver: "local",
Options: map[string]string{
"foo": "bar",
},
},
expected: true,
},
{
name: "non-match with different driver options",
spec: VolumeSpec{
Type: VolumeTypeVolume,
VolumeOptions: &VolumeOptions{
Driver: &mount.Driver{
Name: "local",
Options: map[string]string{
"foo": "baz",
},
},
},
},
vol: volume.Volume{
Driver: "local",
Options: map[string]string{
"foo": "bar",
},
},
expected: false,
},
{
name: "non-match with different driver name",
spec: VolumeSpec{
Type: VolumeTypeVolume,
VolumeOptions: &VolumeOptions{
Driver: &mount.Driver{
Name: "custom",
Options: map[string]string{
"foo": "bar",
},
},
},
},
vol: volume.Volume{
Driver: "local",
Options: map[string]string{
"foo": "bar",
},
},
expected: false,
},
{
name: "match external volume without driver by name only",
spec: VolumeSpec{
Name: "external",
Type: VolumeTypeVolume,
},
vol: volume.Volume{
Name: "external",
Driver: "local",
Options: map[string]string{
"foo": "bar",
},
},
expected: true,
},
{
name: "non-match external volume by name",
spec: VolumeSpec{
Name: "unknown",
Type: VolumeTypeVolume,
},
vol: volume.Volume{
Name: "external",
Driver: "local",
Options: map[string]string{
"foo": "bar",
},
},
expected: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
matches := tt.spec.MatchesDockerVolume(tt.vol)
assert.Equal(t, tt.expected, matches)
})
}
}
+23 -10
View File
@@ -13,18 +13,27 @@ import (
func configSpecsFromCompose( func configSpecsFromCompose(
configs types.Configs, serviceConfigs []types.ServiceConfigObjConfig, workingDir string, configs types.Configs, serviceConfigs []types.ServiceConfigObjConfig, workingDir string,
) ([]api.ConfigSpec, []api.ConfigMount, error) { ) ([]api.ConfigSpec, []api.ConfigMount, error) {
var configSpecs []api.ConfigSpec
var configMounts []api.ConfigMount var configMounts []api.ConfigMount
// Temporary map to hold config specs
configSpecsMap := make(map[string]api.ConfigSpec)
// We iterate over all service config objects (config mounts)
for _, serviceConfig := range serviceConfigs { for _, serviceConfig := range serviceConfigs {
var spec api.ConfigSpec var spec api.ConfigSpec
if projectConfig, exists := configs[serviceConfig.Source]; exists { projectConfig, exists := configs[serviceConfig.Source]
if projectConfig.External { if !exists {
return nil, nil, fmt.Errorf("external configs are not supported: %s", return nil, nil, fmt.Errorf("config '%s' not found in project configs", serviceConfig.Source)
serviceConfig.Source) }
}
if projectConfig.External {
return nil, nil, fmt.Errorf("external configs are not supported: %s",
serviceConfig.Source)
}
spec, exists = configSpecsMap[serviceConfig.Source]
if !exists {
spec = api.ConfigSpec{ spec = api.ConfigSpec{
Name: serviceConfig.Source, Name: serviceConfig.Source,
Content: []byte(projectConfig.Content), Content: []byte(projectConfig.Content),
@@ -44,11 +53,9 @@ func configSpecsFromCompose(
} }
spec.Content = fileContent spec.Content = fileContent
} }
} else {
return nil, nil, fmt.Errorf("config '%s' not found in project configs", serviceConfig.Source)
}
configSpecs = append(configSpecs, spec) configSpecsMap[serviceConfig.Source] = spec
}
// Create config mount // Create config mount
target := serviceConfig.Target target := serviceConfig.Target
@@ -71,5 +78,11 @@ func configSpecsFromCompose(
configMounts = append(configMounts, mount) configMounts = append(configMounts, mount)
} }
var configSpecs []api.ConfigSpec
// Convert config spec map to slice
for _, spec := range configSpecsMap {
configSpecs = append(configSpecs, spec)
}
return configSpecs, configMounts, nil return configSpecs, configMounts, nil
} }
+141
View File
@@ -77,6 +77,147 @@ func TestConfigSpecsFromCompose(t *testing.T) {
}, },
}, },
}, },
{
name: "same source mounted to different targets",
configs: types.Configs{
"shared-config": types.ConfigObjConfig{
File: "testdata/config1.txt",
},
},
serviceConfigs: []types.ServiceConfigObjConfig{
{
Source: "shared-config",
Target: "/app/config.json",
UID: "1000",
GID: "1000",
},
{
Source: "shared-config",
Target: "/backup/config.json",
UID: "1001",
GID: "1001",
},
},
expectedSpecs: []api.ConfigSpec{
{
Name: "shared-config",
Content: []byte("test config content\n"),
},
},
expectedMounts: []api.ConfigMount{
{
ConfigName: "shared-config",
ContainerPath: "/app/config.json",
Uid: "1000",
Gid: "1000",
},
{
ConfigName: "shared-config",
ContainerPath: "/backup/config.json",
Uid: "1001",
Gid: "1001",
},
},
},
{
name: "config with default target path",
configs: types.Configs{
"default-config": types.ConfigObjConfig{
Content: "inline config content",
},
},
serviceConfigs: []types.ServiceConfigObjConfig{
{
Source: "default-config",
// No Target specified - should use default
},
},
expectedSpecs: []api.ConfigSpec{
{
Name: "default-config",
Content: []byte("inline config content"),
},
},
expectedMounts: []api.ConfigMount{
{
ConfigName: "default-config",
ContainerPath: "/default-config",
},
},
},
{
name: "config with inline content",
configs: types.Configs{
"inline-config": types.ConfigObjConfig{
Content: "server {\n listen 80;\n}",
},
},
serviceConfigs: []types.ServiceConfigObjConfig{
{
Source: "inline-config",
Target: "/etc/nginx/sites-available/default",
Mode: func() *types.FileMode { m := types.FileMode(0o755); return &m }(),
},
},
expectedSpecs: []api.ConfigSpec{
{
Name: "inline-config",
Content: []byte("server {\n listen 80;\n}"),
},
},
expectedMounts: []api.ConfigMount{
{
ConfigName: "inline-config",
ContainerPath: "/etc/nginx/sites-available/default",
Mode: func() *os.FileMode { m := os.FileMode(0o755); return &m }(),
},
},
},
{
name: "config not found error",
configs: types.Configs{
"existing-config": types.ConfigObjConfig{
Content: "some content",
},
},
serviceConfigs: []types.ServiceConfigObjConfig{
{
Source: "missing-config",
Target: "/app/config.json",
},
},
expectError: true,
},
{
name: "external config error",
configs: types.Configs{
"external-config": types.ConfigObjConfig{
External: true,
},
},
serviceConfigs: []types.ServiceConfigObjConfig{
{
Source: "external-config",
Target: "/app/config.json",
},
},
expectError: true,
},
{
name: "file not found error",
configs: types.Configs{
"missing-file-config": types.ConfigObjConfig{
File: "testdata/nonexistent.txt",
},
},
serviceConfigs: []types.ServiceConfigObjConfig{
{
Source: "missing-file-config",
Target: "/app/config.json",
},
},
expectError: true,
},
} }
for _, tt := range tests { for _, tt := range tests {
+7 -6
View File
@@ -13,6 +13,7 @@ import (
"github.com/docker/docker/api/types/volume" "github.com/docker/docker/api/types/volume"
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client/deploy" "github.com/psviderski/uncloud/pkg/client/deploy"
"github.com/psviderski/uncloud/pkg/client/deploy/operation"
"github.com/psviderski/uncloud/pkg/client/deploy/scheduler" "github.com/psviderski/uncloud/pkg/client/deploy/scheduler"
) )
@@ -27,7 +28,7 @@ type Deployment struct {
SpecResolver *deploy.ServiceSpecResolver SpecResolver *deploy.ServiceSpecResolver
Strategy deploy.Strategy Strategy deploy.Strategy
state *scheduler.ClusterState state *scheduler.ClusterState
plan *deploy.SequenceOperation plan *operation.SequenceOperation
} }
func NewDeployment(ctx context.Context, cli Client, project *types.Project) (*Deployment, error) { func NewDeployment(ctx context.Context, cli Client, project *types.Project) (*Deployment, error) {
@@ -58,11 +59,11 @@ func NewDeploymentWithStrategy(ctx context.Context, cli Client, project *types.P
}, nil }, nil
} }
func (d *Deployment) Plan(ctx context.Context) (deploy.SequenceOperation, error) { func (d *Deployment) Plan(ctx context.Context) (operation.SequenceOperation, error) {
if d.plan != nil { if d.plan != nil {
return *d.plan, nil return *d.plan, nil
} }
plan := deploy.SequenceOperation{} plan := operation.SequenceOperation{}
// Generate service specs for all services in the project. // Generate service specs for all services in the project.
var serviceSpecs []api.ServiceSpec var serviceSpecs []api.ServiceSpec
@@ -123,7 +124,7 @@ func (d *Deployment) ServiceSpec(name string) (api.ServiceSpec, error) {
} }
// PlanVolumes checks if the external volumes exist and plans the creation of missing volumes. // PlanVolumes checks if the external volumes exist and plans the creation of missing volumes.
func (d *Deployment) planVolumes(serviceSpecs []api.ServiceSpec) ([]*deploy.CreateVolumeOperation, error) { func (d *Deployment) planVolumes(serviceSpecs []api.ServiceSpec) ([]*operation.CreateVolumeOperation, error) {
if len(d.Project.Volumes) == 0 { if len(d.Project.Volumes) == 0 {
// No volumes to check or create. // No volumes to check or create.
return nil, nil return nil, nil
@@ -145,7 +146,7 @@ func (d *Deployment) planVolumes(serviceSpecs []api.ServiceSpec) ([]*deploy.Crea
} }
// Generate operations to create scheduled missing volumes. // Generate operations to create scheduled missing volumes.
var ops []*deploy.CreateVolumeOperation var ops []*operation.CreateVolumeOperation
for machineID, volumes := range scheduledVolumes { for machineID, volumes := range scheduledVolumes {
for _, v := range volumes { for _, v := range volumes {
machineName := machineID machineName := machineID
@@ -153,7 +154,7 @@ func (d *Deployment) planVolumes(serviceSpecs []api.ServiceSpec) ([]*deploy.Crea
machineName = m.Info.Name machineName = m.Info.Name
} }
ops = append(ops, &deploy.CreateVolumeOperation{ ops = append(ops, &operation.CreateVolumeOperation{
MachineID: machineID, MachineID: machineID,
MachineName: machineName, MachineName: machineName,
VolumeSpec: v, VolumeSpec: v,
+21
View File
@@ -6,13 +6,23 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"strings" "strings"
"sync"
composecli "github.com/compose-spec/compose-go/v2/cli" composecli "github.com/compose-spec/compose-go/v2/cli"
"github.com/compose-spec/compose-go/v2/transform"
"github.com/compose-spec/compose-go/v2/tree"
"github.com/compose-spec/compose-go/v2/types" "github.com/compose-spec/compose-go/v2/types"
"github.com/psviderski/uncloud/pkg/api"
) )
var registerComposeOverrides sync.Once
// LoadProject loads a Compose project from the default locations or the given paths. // LoadProject loads a Compose project from the default locations or the given paths.
func LoadProject(ctx context.Context, paths []string, opts ...composecli.ProjectOptionsFn) (*types.Project, error) { func LoadProject(ctx context.Context, paths []string, opts ...composecli.ProjectOptionsFn) (*types.Project, error) {
registerComposeOverrides.Do(func() {
transform.RegisterDefaultValue("services.*.deploy.update_config", setUpdateConfigDefaults)
})
defaultOpts := []composecli.ProjectOptionsFn{ defaultOpts := []composecli.ProjectOptionsFn{
// First apply os.Environment, always wins. // First apply os.Environment, always wins.
composecli.WithOsEnv, composecli.WithOsEnv,
@@ -92,3 +102,14 @@ func removeProjectPrefixFromNames(project *types.Project) {
project.Volumes[name] = vol project.Volumes[name] = vol
} }
} }
// setUpdateConfigDefaults sets default values for deploy.update_config attributes when not specified in the compose file.
func setUpdateConfigDefaults(data any, _ tree.Path, _ bool) (any, error) {
switch v := data.(type) {
case map[string]any:
if _, ok := v["monitor"]; !ok {
v["monitor"] = api.DefaultHealthMonitorPeriod.String()
}
}
return data, nil
}
+108 -9
View File
@@ -5,12 +5,14 @@ import (
"maps" "maps"
"os" "os"
"slices" "slices"
"time"
"github.com/compose-spec/compose-go/v2/types" "github.com/compose-spec/compose-go/v2/types"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/mount" "github.com/docker/docker/api/types/mount"
"github.com/opencontainers/go-digest" "github.com/opencontainers/go-digest"
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
cdi "tags.cncf.io/container-device-interface/pkg/parser"
) )
func ServiceSpecFromCompose(project *types.Project, serviceName string) (api.ServiceSpec, error) { func ServiceSpecFromCompose(project *types.Project, serviceName string) (api.ServiceSpec, error) {
@@ -43,15 +45,19 @@ func ServiceSpecFromCompose(project *types.Project, serviceName string) (api.Ser
spec := api.ServiceSpec{ spec := api.ServiceSpec{
Container: api.ContainerSpec{ Container: api.ContainerSpec{
Command: service.Command, CapAdd: service.CapAdd,
Entrypoint: service.Entrypoint, CapDrop: service.CapDrop,
Env: env, Command: service.Command,
Image: service.Image, Entrypoint: service.Entrypoint,
Init: service.Init, Env: env,
Privileged: service.Privileged, Healthcheck: healthcheckFromCompose(service.HealthCheck),
PullPolicy: pullPolicy, Image: service.Image,
Resources: resourcesFromCompose(service), Init: service.Init,
User: service.User, Privileged: service.Privileged,
PullPolicy: pullPolicy,
Resources: resourcesFromCompose(service),
Sysctls: service.Sysctls,
User: service.User,
}, },
Name: serviceName, Name: serviceName,
Mode: api.ServiceModeReplicated, Mode: api.ServiceModeReplicated,
@@ -94,6 +100,22 @@ func ServiceSpecFromCompose(project *types.Project, serviceName string) (api.Ser
default: default:
return spec, fmt.Errorf("unsupported deploy mode: '%s'", service.Deploy.Mode) return spec, fmt.Errorf("unsupported deploy mode: '%s'", service.Deploy.Mode)
} }
if cfg := service.Deploy.UpdateConfig; cfg != nil {
switch cfg.Order {
case "":
// No order specified, use default behavior.
case "start-first":
spec.UpdateConfig.Order = api.UpdateOrderStartFirst
case "stop-first":
spec.UpdateConfig.Order = api.UpdateOrderStopFirst
default:
return spec, fmt.Errorf("unsupported deploy.update_config.order: '%s'", cfg.Order)
}
d := time.Duration(cfg.Monitor)
spec.UpdateConfig.MonitorPeriod = &d
}
} }
// TODO: can service.tmpfs be handled as tmpfs volume mounts as well? // TODO: can service.tmpfs be handled as tmpfs volume mounts as well?
@@ -117,11 +139,61 @@ func ServiceSpecFromCompose(project *types.Project, serviceName string) (api.Ser
return spec, nil return spec, nil
} }
func healthcheckFromCompose(hc *types.HealthCheckConfig) *api.HealthcheckSpec {
if hc == nil {
return nil
}
if hc.Disable {
return &api.HealthcheckSpec{Disable: true}
}
spec := &api.HealthcheckSpec{Test: hc.Test}
if hc.Interval != nil {
spec.Interval = time.Duration(*hc.Interval)
}
if hc.Timeout != nil {
spec.Timeout = time.Duration(*hc.Timeout)
}
if hc.StartPeriod != nil {
spec.StartPeriod = time.Duration(*hc.StartPeriod)
}
if hc.StartInterval != nil {
spec.StartInterval = time.Duration(*hc.StartInterval)
}
if hc.Retries != nil {
spec.Retries = uint(*hc.Retries)
}
return spec
}
func resourcesFromCompose(service types.ServiceConfig) api.ContainerResources { func resourcesFromCompose(service types.ServiceConfig) api.ContainerResources {
resources := api.ContainerResources{ resources := api.ContainerResources{
CPU: int64(service.CPUS * 1e9), CPU: int64(service.CPUS * 1e9),
Memory: int64(service.MemLimit), Memory: int64(service.MemLimit),
MemoryReservation: int64(service.MemReservation), MemoryReservation: int64(service.MemReservation),
Ulimits: ulimitsFromCompose(service.Ulimits),
}
// Convert device mappings, separating CDI devices from regular device mappings.
// CDI devices are identified when Source == Target and the source is a qualified CDI name.
var cdiDeviceNames []string
for _, dev := range service.Devices {
if dev.Source == dev.Target && cdi.IsQualifiedName(dev.Source) {
cdiDeviceNames = append(cdiDeviceNames, dev.Source)
continue
}
resources.Devices = append(resources.Devices, api.DeviceMapping{
HostPath: dev.Source,
ContainerPath: dev.Target,
CgroupPermissions: dev.Permissions,
})
}
if len(cdiDeviceNames) > 0 {
resources.DeviceReservations = append(resources.DeviceReservations, container.DeviceRequest{
Driver: "cdi",
DeviceIDs: cdiDeviceNames,
})
} }
// Convert GPU device requests from compose format, appending "gpu" capability. // Convert GPU device requests from compose format, appending "gpu" capability.
@@ -290,6 +362,33 @@ func tmpfsVolumeSpecFromCompose(serviceVolume types.ServiceVolumeConfig) api.Vol
return spec return spec
} }
func ulimitsFromCompose(ulimits map[string]*types.UlimitsConfig) map[string]api.Ulimit {
if len(ulimits) == 0 {
return nil
}
res := make(map[string]api.Ulimit, len(ulimits))
for name, u := range ulimits {
soft := u.Soft
hard := u.Hard
if u.Single != 0 {
if soft == 0 {
soft = u.Single
}
if hard == 0 {
hard = u.Single
}
}
res[name] = api.Ulimit{
Soft: int64(soft),
Hard: int64(hard),
}
}
return res
}
// validateServicesExtensions validates extension combinations across all services in the project. // validateServicesExtensions validates extension combinations across all services in the project.
func validateServicesExtensions(project *types.Project) error { func validateServicesExtensions(project *types.Project) error {
for _, service := range project.Services { for _, service := range project.Services {
+431 -1
View File
@@ -8,6 +8,7 @@ import (
"slices" "slices"
"strings" "strings"
"testing" "testing"
"time"
composecli "github.com/compose-spec/compose-go/v2/cli" composecli "github.com/compose-spec/compose-go/v2/cli"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
@@ -93,13 +94,15 @@ func TestServiceSpecFromCompose(t *testing.T) {
}, },
}, },
{ {
name: "full-spec", name: "full spec",
filename: "compose-full-spec.yaml", filename: "compose-full-spec.yaml",
want: map[string]api.ServiceSpec{ want: map[string]api.ServiceSpec{
"test": { "test": {
Name: "test", Name: "test",
Mode: api.ServiceModeReplicated, Mode: api.ServiceModeReplicated,
Container: api.ContainerSpec{ Container: api.ContainerSpec{
CapAdd: []string{"NET_ADMIN"},
CapDrop: []string{"ALL"},
Command: []string{"nginx", "updated", "command"}, Command: []string{"nginx", "updated", "command"},
Entrypoint: []string{"/updated-docker-entrypoint.sh"}, Entrypoint: []string{"/updated-docker-entrypoint.sh"},
Env: map[string]string{ Env: map[string]string{
@@ -107,6 +110,14 @@ func TestServiceSpecFromCompose(t *testing.T) {
"EMPTY": "", "EMPTY": "",
"VAR": "value", "VAR": "value",
}, },
Healthcheck: &api.HealthcheckSpec{
Test: []string{"CMD", "curl", "-f", "http://localhost"},
Interval: 1*time.Minute + 30*time.Second,
Timeout: 10 * time.Second,
Retries: 5,
StartPeriod: 15 * time.Second,
StartInterval: 2 * time.Second,
},
Image: "nginx:latest", Image: "nginx:latest",
Init: &initTrue, Init: &initTrue,
LogDriver: &api.LogDriver{ LogDriver: &api.LogDriver{
@@ -122,6 +133,20 @@ func TestServiceSpecFromCompose(t *testing.T) {
CPU: 0.5 * api.Core, CPU: 0.5 * api.Core,
Memory: 100 * units.MiB, Memory: 100 * units.MiB,
MemoryReservation: 50 * units.MiB, MemoryReservation: 50 * units.MiB,
Ulimits: map[string]api.Ulimit{
"nofile": {Soft: 20000, Hard: 40000},
"nproc": {Soft: 65535, Hard: 65535},
},
Devices: []api.DeviceMapping{
{HostPath: "/dev/ttyUSB0", ContainerPath: "/dev/ttyUSB0", CgroupPermissions: "rw"},
{HostPath: "/dev/sda", ContainerPath: "/dev/xvda", CgroupPermissions: "rwm"},
},
DeviceReservations: []container.DeviceRequest{
{Count: -1, Capabilities: [][]string{{"gpu"}}},
},
},
Sysctls: map[string]string{
"net.ipv4.ip_forward": "1",
}, },
User: "nginx:nginx", User: "nginx:nginx",
VolumeMounts: []api.VolumeMount{ VolumeMounts: []api.VolumeMount{
@@ -134,6 +159,15 @@ func TestServiceSpecFromCompose(t *testing.T) {
VolumeName: "data1", VolumeName: "data1",
ContainerPath: "/data1", ContainerPath: "/data1",
}, },
{
VolumeName: "bind-53f1acbf1de61e9e608c93effca23791674e463d02bb7aaca7c625804aef1926",
ContainerPath: "/path/in/container",
ReadOnly: true,
},
{
VolumeName: "data3-labeled",
ContainerPath: "/data3",
},
{ {
VolumeName: "data2-alias", VolumeName: "data2-alias",
ContainerPath: "/data2/long/syntax", ContainerPath: "/data2/long/syntax",
@@ -168,8 +202,24 @@ func TestServiceSpecFromCompose(t *testing.T) {
Mode: api.PortModeHost, Mode: api.PortModeHost,
}, },
}, },
Placement: api.Placement{
Machines: []string{"machine-1", "machine-2"},
},
Replicas: 3, Replicas: 3,
UpdateConfig: api.UpdateConfig{
Order: api.UpdateOrderStopFirst,
MonitorPeriod: &api.DefaultHealthMonitorPeriod,
},
Volumes: []api.VolumeSpec{ Volumes: []api.VolumeSpec{
{
Name: "bind-53f1acbf1de61e9e608c93effca23791674e463d02bb7aaca7c625804aef1926",
Type: api.VolumeTypeBind,
BindOptions: &api.BindOptions{
HostPath: "/path/on/host",
CreateHostPath: true,
Propagation: mount.Propagation("rprivate"),
},
},
{ {
Name: "bind-bb6aed1683cea1e0a1ae5cd227aacd0734f2f87f7a78fcf1baeff978ce300b90", Name: "bind-bb6aed1683cea1e0a1ae5cd227aacd0734f2f87f7a78fcf1baeff978ce300b90",
Type: api.VolumeTypeBind, Type: api.VolumeTypeBind,
@@ -202,11 +252,22 @@ func TestServiceSpecFromCompose(t *testing.T) {
}, },
}, },
}, },
{
Name: "data3-labeled",
Type: api.VolumeTypeVolume,
VolumeOptions: &api.VolumeOptions{
Name: "data3-labeled",
NoCopy: true,
SubPath: "app/data",
Labels: map[string]string{"env": "test"},
},
},
{ {
Name: "tmpfs-efa57ba8b6a1779674ac438de3af8729e2d55900b79eb929431cf9c5b0179542", Name: "tmpfs-efa57ba8b6a1779674ac438de3af8729e2d55900b79eb929431cf9c5b0179542",
Type: api.VolumeTypeTmpfs, Type: api.VolumeTypeTmpfs,
TmpfsOptions: &mount.TmpfsOptions{ TmpfsOptions: &mount.TmpfsOptions{
SizeBytes: 10 * units.MiB, SizeBytes: 10 * units.MiB,
Mode: os.FileMode(1770),
}, },
}, },
}, },
@@ -252,6 +313,8 @@ func TestServiceSpecFromCompose(t *testing.T) {
} }
func TestServiceSpecFromCompose_Caddy(t *testing.T) { func TestServiceSpecFromCompose_Caddy(t *testing.T) {
t.Parallel()
tests := []struct { tests := []struct {
name string name string
composeYAML string composeYAML string
@@ -400,6 +463,8 @@ services:
} }
func TestServiceSpecFromCompose_GPUs(t *testing.T) { func TestServiceSpecFromCompose_GPUs(t *testing.T) {
t.Parallel()
tests := []struct { tests := []struct {
name string name string
composeYAML string composeYAML string
@@ -587,6 +652,8 @@ services:
} }
func TestServiceSpecFromCompose_VolumeDriverOpts(t *testing.T) { func TestServiceSpecFromCompose_VolumeDriverOpts(t *testing.T) {
t.Parallel()
tests := []struct { tests := []struct {
name string name string
composeYAML string composeYAML string
@@ -694,7 +761,269 @@ volumes:
} }
} }
func TestServiceSpecFromCompose_Ulimits(t *testing.T) {
t.Parallel()
tests := []struct {
name string
composeYAML string
expected map[string]api.Ulimit
}{
{
name: "single ulimit with soft and hard limits",
composeYAML: `
services:
db:
image: postgres
ulimits:
nofile:
soft: 20000
hard: 40000
`,
expected: map[string]api.Ulimit{
"nofile": {
Soft: 20000,
Hard: 40000,
},
},
},
{
name: "single ulimit with single value (soft=hard)",
composeYAML: `
services:
db:
image: postgres
ulimits:
nproc: 65535
`,
expected: map[string]api.Ulimit{
"nproc": {
Soft: 65535,
Hard: 65535,
},
},
},
{
name: "multiple ulimits",
composeYAML: `
services:
db:
image: postgres
ulimits:
nofile:
soft: 20000
hard: 40000
nproc: 65535
`,
expected: map[string]api.Ulimit{
"nofile": {
Soft: 20000,
Hard: 40000,
},
"nproc": {
Soft: 65535,
Hard: 65535,
},
},
},
{
name: "empty ulimits",
composeYAML: `
services:
db:
image: postgres
ulimits: {}
`,
expected: nil,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
project, err := LoadProjectFromContent(context.Background(), tt.composeYAML)
require.NoError(t, err)
spec, err := ServiceSpecFromCompose(project, "db")
require.NoError(t, err)
assert.Equal(t, tt.expected, spec.Container.Resources.Ulimits)
})
}
}
func TestServiceSpecFromCompose_UpdateConfig(t *testing.T) {
t.Parallel()
tests := []struct {
name string
composeYAML string
expected api.UpdateConfig
expectError bool
}{
{
name: "no update_config",
composeYAML: `
services:
test:
image: nginx
`,
expected: api.UpdateConfig{},
},
{
name: "empty update_config",
composeYAML: `
services:
test:
image: nginx
deploy:
update_config: {}
`,
expected: api.UpdateConfig{
MonitorPeriod: &api.DefaultHealthMonitorPeriod,
},
},
{
name: "update_config with unsupported attributes ignored",
composeYAML: `
services:
test:
image: nginx
deploy:
update_config:
parallelism: 1
`,
expected: api.UpdateConfig{
MonitorPeriod: &api.DefaultHealthMonitorPeriod,
},
},
{
name: "update_config with stop-first order",
composeYAML: `
services:
test:
image: postgres
deploy:
update_config:
order: stop-first
`,
expected: api.UpdateConfig{
Order: api.UpdateOrderStopFirst,
MonitorPeriod: &api.DefaultHealthMonitorPeriod,
},
},
{
name: "update_config with start-first order",
composeYAML: `
services:
test:
image: nginx
deploy:
update_config:
order: start-first
`,
expected: api.UpdateConfig{
Order: api.UpdateOrderStartFirst,
MonitorPeriod: &api.DefaultHealthMonitorPeriod,
},
},
{
name: "update_config with invalid order",
composeYAML: `
services:
test:
image: nginx
deploy:
update_config:
order: invalid-order
`,
expectError: true,
},
{
name: "update_config with replicas and order",
composeYAML: `
services:
test:
image: nginx
deploy:
replicas: 3
update_config:
order: stop-first
`,
expected: api.UpdateConfig{
Order: api.UpdateOrderStopFirst,
MonitorPeriod: &api.DefaultHealthMonitorPeriod,
},
},
{
name: "update_config with custom monitor",
composeYAML: `
services:
test:
image: nginx
deploy:
update_config:
monitor: 10s
`,
expected: api.UpdateConfig{
MonitorPeriod: api.AsPtr(10 * time.Second),
},
},
{
name: "update_config with monitor and order",
composeYAML: `
services:
test:
image: nginx
deploy:
update_config:
order: start-first
monitor: 30s
`,
expected: api.UpdateConfig{
Order: api.UpdateOrderStartFirst,
MonitorPeriod: api.AsPtr(30 * time.Second),
},
},
{
name: "update_config with zero monitor skips monitoring",
composeYAML: `
services:
test:
image: nginx
deploy:
update_config:
monitor: 0s
`,
expected: api.UpdateConfig{
MonitorPeriod: api.AsPtr(time.Duration(0)),
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
project, err := LoadProjectFromContent(context.Background(), tt.composeYAML)
if tt.expectError {
require.Error(t, err)
return
}
require.NoError(t, err)
spec, err := ServiceSpecFromCompose(project, "test")
if tt.expectError {
assert.Error(t, err)
return
}
require.NoError(t, err)
assert.Equal(t, tt.expected, spec.UpdateConfig)
})
}
}
func TestServiceSpecFromCompose_XMachinesPlacement(t *testing.T) { func TestServiceSpecFromCompose_XMachinesPlacement(t *testing.T) {
t.Parallel()
tests := []struct { tests := []struct {
name string name string
composeYAML string composeYAML string
@@ -851,3 +1180,104 @@ services:
}) })
} }
} }
func TestServiceSpecFromCompose_Devices(t *testing.T) {
t.Parallel()
tests := []struct {
name string
composeYAML string
expectedDevices []api.DeviceMapping
expectedReservations []container.DeviceRequest
}{
{
name: "simple device",
composeYAML: `
services:
test:
image: nginx
devices:
- /dev/dri
`,
expectedDevices: []api.DeviceMapping{
{HostPath: "/dev/dri", ContainerPath: "/dev/dri", CgroupPermissions: "rwm"},
},
},
{
name: "device with target and permissions",
composeYAML: `
services:
test:
image: nginx
devices:
- /dev/sda:/dev/xvda:r
`,
expectedDevices: []api.DeviceMapping{
{HostPath: "/dev/sda", ContainerPath: "/dev/xvda", CgroupPermissions: "r"},
},
},
{
name: "multiple devices",
composeYAML: `
services:
test:
image: nginx
devices:
- "/dev/ttyUSB0:/dev/ttyUSB0:rw"
- /dev/sda:/dev/xvda
- "/dev/dri"
`,
expectedDevices: []api.DeviceMapping{
{HostPath: "/dev/ttyUSB0", ContainerPath: "/dev/ttyUSB0", CgroupPermissions: "rw"},
{HostPath: "/dev/sda", ContainerPath: "/dev/xvda", CgroupPermissions: "rwm"},
{HostPath: "/dev/dri", ContainerPath: "/dev/dri", CgroupPermissions: "rwm"},
},
},
{
name: "CDI device",
composeYAML: `
services:
test:
image: nginx
devices:
- vendor.com/class=device1
`,
expectedReservations: []container.DeviceRequest{
{Driver: "cdi", DeviceIDs: []string{"vendor.com/class=device1"}},
},
},
{
name: "mixed CDI and regular devices",
composeYAML: `
services:
test:
image: nginx
devices:
- /dev/dri
- vendor.com/class=device1
- nvidia.com/gpu=0
- /dev/sda:/dev/xvda:r
`,
expectedDevices: []api.DeviceMapping{
{HostPath: "/dev/dri", ContainerPath: "/dev/dri", CgroupPermissions: "rwm"},
{HostPath: "/dev/sda", ContainerPath: "/dev/xvda", CgroupPermissions: "r"},
},
expectedReservations: []container.DeviceRequest{
{Driver: "cdi", DeviceIDs: []string{"vendor.com/class=device1", "nvidia.com/gpu=0"}},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
project, err := LoadProjectFromContent(context.Background(), tt.composeYAML)
require.NoError(t, err)
spec, err := ServiceSpecFromCompose(project, "test")
require.NoError(t, err)
assert.Equal(t, tt.expectedDevices, spec.Container.Resources.Devices)
assert.Equal(t, tt.expectedReservations, spec.Container.Resources.DeviceReservations)
})
}
}
+43
View File
@@ -1,12 +1,30 @@
services: services:
test: test:
cap_add:
- NET_ADMIN
cap_drop:
- ALL
command: ["nginx", "updated", "command"] command: ["nginx", "updated", "command"]
cpus: 0.5 cpus: 0.5
deploy:
update_config:
order: stop-first
devices:
- /dev/ttyUSB0:/dev/ttyUSB0:rw
- /dev/sda:/dev/xvda
entrypoint: ["/updated-docker-entrypoint.sh"] entrypoint: ["/updated-docker-entrypoint.sh"]
environment: environment:
BOOL: "true" BOOL: "true"
EMPTY: "" EMPTY: ""
VAR: value VAR: value
gpus: all
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost"]
interval: 1m30s
timeout: 10s
retries: 5
start_period: 15s
start_interval: 2s
image: nginx:latest image: nginx:latest
init: true init: true
logging: logging:
@@ -19,10 +37,30 @@ services:
privileged: true privileged: true
pull_policy: always pull_policy: always
scale: 3 scale: 3
sysctls:
- net.ipv4.ip_forward=1
ulimits:
nofile:
soft: 20000
hard: 40000
nproc: 65535
user: nginx:nginx user: nginx:nginx
volumes: volumes:
- /etc/passwd:/host/etc/passwd:ro - /etc/passwd:/host/etc/passwd:ro
- data1:/data1 - data1:/data1
- type: bind
source: /path/on/host
target: /path/in/container
read_only: true
bind:
create_host_path: true
propagation: rprivate
- type: volume
source: data3-labeled
target: /data3
volume:
nocopy: true
subpath: app/data
- type: volume - type: volume
source: data2-alias source: data2-alias
target: /data2/long/syntax target: /data2/long/syntax
@@ -31,6 +69,8 @@ services:
target: /tmpfs target: /tmpfs
tmpfs: tmpfs:
size: 10485760 size: 10485760
mode: 1770
x-machines: ["machine-1", "machine-2"]
x-ports: x-ports:
- test.example.com:80/https - test.example.com:80/https
- 8000/http - 8000/http
@@ -49,5 +89,8 @@ volumes:
data2-alias: data2-alias:
name: data2 name: data2
driver: local driver: local
data3-labeled:
labels:
env: test
data-external: data-external:
external: true external: true
+30
View File
@@ -0,0 +1,30 @@
package connector
import (
"encoding/json"
"fmt"
)
// defaultServiceConfig defines the default gRPC service configuration including retry policy for transient failures.
var defaultServiceConfig = mustMarshalJSON(map[string]any{
"methodConfig": []map[string]any{
{
"name": []map[string]string{{"service": ""}},
"retryPolicy": map[string]any{
"maxAttempts": 5, // 5 is the maximum allowed by gRPC
"initialBackoff": "0.5s",
"maxBackoff": "5s",
"backoffMultiplier": 2,
"retryableStatusCodes": []string{"UNAVAILABLE"},
},
},
},
})
func mustMarshalJSON(v any) string {
b, err := json.Marshal(v)
if err != nil {
panic(fmt.Sprintf("failed to marshal service config: %v", err))
}
return string(b)
}
+16 -1
View File
@@ -24,6 +24,16 @@ type SSHConnectorConfig struct {
SockPath string SockPath string
} }
// Destination returns the destination string that can be passed to ssh CLI in the format [user@]host.
func (cfg *SSHConnectorConfig) Destination() string {
dst := cfg.Host
if cfg.User != "" {
dst = fmt.Sprintf("%s@%s", cfg.User, dst)
}
return dst
}
// SSHConnector establishes a connection to the machine API through an SSH tunnel to the machine. // SSHConnector establishes a connection to the machine API through an SSH tunnel to the machine.
type SSHConnector struct { type SSHConnector struct {
config SSHConnectorConfig config SSHConnectorConfig
@@ -48,7 +58,11 @@ func (c *SSHConnector) Connect(ctx context.Context) (*grpc.ClientConn, error) {
var err error var err error
c.client, err = sshexec.Connect(c.config.User, c.config.Host, c.config.Port, c.config.KeyPath) c.client, err = sshexec.Connect(c.config.User, c.config.Host, c.config.Port, c.config.KeyPath)
if err != nil { if err != nil {
return nil, fmt.Errorf("SSH login to %s@%s:%d: %w", c.config.User, c.config.Host, c.config.Port, err) dst := c.config.Destination()
if c.config.Port != 0 {
dst = fmt.Sprintf("%s:%d", dst, c.config.Port)
}
return nil, fmt.Errorf("SSH login to %s: %w", dst, err)
} }
} }
@@ -59,6 +73,7 @@ func (c *SSHConnector) Connect(ctx context.Context) (*grpc.ClientConn, error) {
conn, err := grpc.NewClient( conn, err := grpc.NewClient(
"unix://"+sockPath, "unix://"+sockPath,
grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithDefaultServiceConfig(defaultServiceConfig),
grpc.WithContextDialer( grpc.WithContextDialer(
func(ctx context.Context, addr string) (net.Conn, error) { func(ctx context.Context, addr string) (net.Conn, error) {
addr = strings.TrimPrefix(addr, "unix://") addr = strings.TrimPrefix(addr, "unix://")
+128 -71
View File
@@ -4,6 +4,8 @@ import (
"context" "context"
"fmt" "fmt"
"net" "net"
"os"
"path/filepath"
"strconv" "strconv"
"github.com/docker/cli/cli/connhelper/commandconn" "github.com/docker/cli/cli/connhelper/commandconn"
@@ -17,100 +19,96 @@ import (
// and running `uncloudd dial-stdio` on the remote machine. // and running `uncloudd dial-stdio` on the remote machine.
type SSHCLIConnector struct { type SSHCLIConnector struct {
config SSHConnectorConfig config SSHConnectorConfig
conn net.Conn // Path to SSH control socket for connection reuse.
controlSockPath string
} }
func NewSSHCLIConnector(cfg *SSHConnectorConfig) *SSHCLIConnector { func NewSSHCLIConnector(cfg *SSHConnectorConfig) *SSHCLIConnector {
return &SSHCLIConnector{config: *cfg} return &SSHCLIConnector{
config: *cfg,
controlSockPath: controlSocketPath(),
}
} }
// sshCLIDialer implements proxy.ContextDialer by spawning SSH processes with -W flag. // controlSocketPath returns a unique control socket path for the SSH connection.
type sshCLIDialer struct { // Returns an empty string if unable to find or create a suitable path.
config SSHConnectorConfig func controlSocketPath() string {
} // %C is expanded by `ssh` to a hash of user, local and remote hostnames, port, and the contents
// of the ProxyJump option. This ensures that shared connections are uniquely identified.
sockName := fmt.Sprintf("uc_control_%%C.sock")
// buildDialArgs constructs SSH command arguments for -W flag dialing. // Prefer XDG_RUNTIME_DIR if set, fall back to ~/.ssh if it exists.
func (d *sshCLIDialer) buildDialArgs(address string) []string { if dir := os.Getenv("XDG_RUNTIME_DIR"); dir != "" {
args := []string{} return filepath.Join(dir, sockName)
}
// Add connection timeout to fail fast when node is down. if home, err := os.UserHomeDir(); err == nil {
args = append(args, "-o", "ConnectTimeout=5") sshDir := filepath.Join(home, ".ssh")
if fi, sErr := os.Stat(sshDir); sErr == nil && fi.IsDir() {
// Add port if non-standard. return filepath.Join(sshDir, sockName)
if d.config.Port != 0 && d.config.Port != 22 { }
args = append(args, "-p", strconv.Itoa(d.config.Port))
} }
// Add identity file if specified. // Last resort: create a subdirectory in temp with restricted permissions.
if d.config.KeyPath != "" { tmpDir := filepath.Join(os.TempDir(), fmt.Sprintf("uncloud-%d", os.Getuid()))
args = append(args, "-i", d.config.KeyPath) path := filepath.Join(tmpDir, sockName)
if len(path)-2+40 < 104 { // 40 chars for %C hash, 104 is typical UNIX socket path limit
if err := os.MkdirAll(tmpDir, 0o700); err == nil {
return path
}
} }
// Add -W flag for stdin/stdout forwarding to target address. return ""
args = append(args, "-W", address)
// Add user@host.
args = append(args, d.config.User+"@"+d.config.Host)
return args
}
// DialContext establishes a connection to the target address through an SSH tunnel using -W flag.
func (d *sshCLIDialer) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
// Only support TCP connections.
if network != "tcp" {
return nil, fmt.Errorf("unsupported network type: %s", network)
}
// Build SSH command arguments.
args := d.buildDialArgs(address)
// Create connection using commandconn.
conn, err := commandconn.New(ctx, "ssh", args...)
if err != nil {
return nil, fmt.Errorf("SSH connection to %s@%s for dialing %s: %w", d.config.User, d.config.Host, address, err)
}
return conn, nil
} }
func (c *SSHCLIConnector) Connect(ctx context.Context) (*grpc.ClientConn, error) { func (c *SSHCLIConnector) Connect(ctx context.Context) (*grpc.ClientConn, error) {
// Build SSH command arguments. // Create gRPC client with a dialer that spawns a new SSH connection on demand.
args := c.buildSSHArgs() // Each dial attempt runs `ssh ... uncloudd dial-stdio`, reusing the control socket if available.
// Create connection using commandconn.
conn, err := commandconn.New(ctx, "ssh", args...)
if err != nil {
return nil, fmt.Errorf("SSH connection to %s@%s: %w", c.config.User, c.config.Host, err)
}
c.conn = conn
// Create gRPC client over the connection.
// Use a custom dialer that returns our existing connection.
grpcConn, err := grpc.NewClient( grpcConn, err := grpc.NewClient(
"passthrough:///", // Dummy target since we're using a custom dialer. "passthrough:///", // Dummy target since we're using a custom dialer.
grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithDefaultServiceConfig(defaultServiceConfig),
grpc.WithContextDialer(func(ctx context.Context, _ string) (net.Conn, error) { grpc.WithContextDialer(func(ctx context.Context, _ string) (net.Conn, error) {
return c.conn, nil args := c.buildSSHArgs()
conn, err := commandconn.New(ctx, "ssh", args...)
if err != nil {
return nil, fmt.Errorf("SSH connection to %s: %w", c.config.Destination(), err)
}
return conn, nil
}), }),
) )
if err != nil { if err != nil {
c.conn.Close()
return nil, fmt.Errorf("create machine API client: %w", err) return nil, fmt.Errorf("create machine API client: %w", err)
} }
return grpcConn, nil return grpcConn, nil
} }
// buildSSHArgs constructs the SSH command arguments. // buildSSHArgs constructs the SSH command arguments to run `uncloudd dial-stdio` on the remote machine reusing
// the established connection via control socket.
func (c *SSHCLIConnector) buildSSHArgs() []string { func (c *SSHCLIConnector) buildSSHArgs() []string {
args := []string{} var args []string
// Add control socket options for connection reuse if available.
if c.controlSockPath != "" {
args = append(args, "-o", "ControlMaster=auto")
args = append(args, "-o", "ControlPath="+c.controlSockPath)
// Keep the established connection alive for a short duration after the last session closes to allow reuse.
controlPersist := "10m"
// Override the default duration with the UNCLOUD_SSH_CONTROL_PERSIST env variable.
if v := os.Getenv("UNCLOUD_SSH_CONTROL_PERSIST"); v != "" {
controlPersist = v
}
args = append(args, "-o", "ControlPersist="+controlPersist)
}
// Add connection timeout to fail fast when node is down. // Add connection timeout to fail fast when node is down.
args = append(args, "-o", "ConnectTimeout=5") args = append(args, "-o", "ConnectTimeout=5")
// Disable pseudo-terminal allocation to prevent SSH from executing as a login shell.
args = append(args, "-T")
// Add port if non-standard. // Add port if specified.
if c.config.Port != 0 && c.config.Port != 22 { if c.config.Port != 0 {
args = append(args, "-p", strconv.Itoa(c.config.Port)) args = append(args, "-p", strconv.Itoa(c.config.Port))
} }
@@ -119,8 +117,8 @@ func (c *SSHCLIConnector) buildSSHArgs() []string {
args = append(args, "-i", c.config.KeyPath) args = append(args, "-i", c.config.KeyPath)
} }
// Add user@host. // Add [user@]host destination.
args = append(args, c.config.User+"@"+c.config.Host) args = append(args, c.config.Destination())
// Add remote command: uncloudd dial-stdio // Add remote command: uncloudd dial-stdio
args = append(args, "uncloudd", "dial-stdio") args = append(args, "uncloudd", "dial-stdio")
@@ -140,15 +138,74 @@ func (c *SSHCLIConnector) Dialer() (proxy.ContextDialer, error) {
} }
return &sshCLIDialer{ return &sshCLIDialer{
config: c.config, config: c.config,
controlSockPath: c.controlSockPath,
}, nil }, nil
} }
func (c *SSHCLIConnector) Close() error { func (c *SSHCLIConnector) Close() error {
if c.conn != nil { // Individual connections are managed by gRPC and closed when the gRPC connection closes.
err := c.conn.Close() // The SSH control socket may persist for connection reuse across CLI invocations.
c.conn = nil
return err
}
return nil return nil
} }
// sshCLIDialer implements proxy.ContextDialer by spawning SSH processes with -W flag.
type sshCLIDialer struct {
config SSHConnectorConfig
// Shared control socket path from SSHCLIConnector for connection reuse.
controlSockPath string
}
// buildDialArgs constructs SSH command arguments for -W flag dialing.
func (d *sshCLIDialer) buildDialArgs(address string) []string {
var args []string
if d.controlSockPath != "" {
// Try to reuse the existing control connection without initiating a new one.
// Falls back to direct connection if the control socket is not available.
args = append(args, "-o", "ControlMaster=no")
args = append(args, "-o", "ControlPath="+d.controlSockPath)
}
// Add connection timeout to fail fast when node is down.
args = append(args, "-o", "ConnectTimeout=5")
// Disable pseudo-terminal allocation to prevent SSH from executing as a login shell.
args = append(args, "-T")
// Add port if specified.
if d.config.Port != 0 {
args = append(args, "-p", strconv.Itoa(d.config.Port))
}
// Add identity file if specified.
if d.config.KeyPath != "" {
args = append(args, "-i", d.config.KeyPath)
}
// Add -W flag for stdin/stdout forwarding to target address.
args = append(args, "-W", address)
// Add [user@]host destination.
args = append(args, d.config.Destination())
return args
}
// DialContext establishes a connection to the target address through an SSH tunnel using -W flag.
func (d *sshCLIDialer) DialContext(ctx context.Context, network, address string) (net.Conn, error) {
// Only support TCP connections.
if network != "tcp" {
return nil, fmt.Errorf("unsupported network type: %s", network)
}
// Build SSH command arguments.
args := d.buildDialArgs(address)
// Create connection using docker's commandconn.
conn, err := commandconn.New(ctx, "ssh", args...)
if err != nil {
return nil, fmt.Errorf("SSH connection to %s for dialing %s: %w", d.config.Destination(), address, err)
}
return conn, nil
}
+96 -33
View File
@@ -1,6 +1,7 @@
package connector package connector
import ( import (
"strings"
"testing" "testing"
"github.com/psviderski/uncloud/internal/machine" "github.com/psviderski/uncloud/internal/machine"
@@ -11,17 +12,28 @@ func TestSSHCLIConnector_buildSSHArgs(t *testing.T) {
t.Parallel() t.Parallel()
tests := []struct { tests := []struct {
name string name string
config SSHConnectorConfig config SSHConnectorConfig
expected []string controlSockPath string
expected []string
}{ }{
{ {
name: "basic connection", name: "basic connection with control socket",
config: SSHConnectorConfig{ config: SSHConnectorConfig{
User: "root", User: "root",
Host: "example.com", Host: "example.com",
}, },
expected: []string{"-o", "ConnectTimeout=5", "root@example.com", "uncloudd", "dial-stdio"}, controlSockPath: "/tmp/test.sock",
expected: []string{"-o", "ControlMaster=auto", "-o", "ControlPath=/tmp/test.sock", "-o", "ControlPersist=10m", "-o", "ConnectTimeout=5", "-T", "root@example.com", "uncloudd", "dial-stdio"},
},
{
name: "basic connection without control socket",
config: SSHConnectorConfig{
User: "root",
Host: "example.com",
},
controlSockPath: "",
expected: []string{"-o", "ConnectTimeout=5", "-T", "root@example.com", "uncloudd", "dial-stdio"},
}, },
{ {
name: "with custom port", name: "with custom port",
@@ -30,7 +42,8 @@ func TestSSHCLIConnector_buildSSHArgs(t *testing.T) {
Host: "example.com", Host: "example.com",
Port: 2222, Port: 2222,
}, },
expected: []string{"-o", "ConnectTimeout=5", "-p", "2222", "root@example.com", "uncloudd", "dial-stdio"}, controlSockPath: "/tmp/test.sock",
expected: []string{"-o", "ControlMaster=auto", "-o", "ControlPath=/tmp/test.sock", "-o", "ControlPersist=10m", "-o", "ConnectTimeout=5", "-T", "-p", "2222", "root@example.com", "uncloudd", "dial-stdio"},
}, },
{ {
name: "with identity file", name: "with identity file",
@@ -39,7 +52,8 @@ func TestSSHCLIConnector_buildSSHArgs(t *testing.T) {
Host: "example.com", Host: "example.com",
KeyPath: "/path/to/key", KeyPath: "/path/to/key",
}, },
expected: []string{"-o", "ConnectTimeout=5", "-i", "/path/to/key", "root@example.com", "uncloudd", "dial-stdio"}, controlSockPath: "/tmp/test.sock",
expected: []string{"-o", "ControlMaster=auto", "-o", "ControlPath=/tmp/test.sock", "-o", "ControlPersist=10m", "-o", "ConnectTimeout=5", "-T", "-i", "/path/to/key", "root@example.com", "uncloudd", "dial-stdio"},
}, },
{ {
name: "with custom socket path", name: "with custom socket path",
@@ -48,7 +62,8 @@ func TestSSHCLIConnector_buildSSHArgs(t *testing.T) {
Host: "example.com", Host: "example.com",
SockPath: "/custom/path/uncloud.sock", SockPath: "/custom/path/uncloud.sock",
}, },
expected: []string{"-o", "ConnectTimeout=5", "root@example.com", "uncloudd", "dial-stdio", "--socket", "/custom/path/uncloud.sock"}, controlSockPath: "/tmp/test.sock",
expected: []string{"-o", "ControlMaster=auto", "-o", "ControlPath=/tmp/test.sock", "-o", "ControlPersist=10m", "-o", "ConnectTimeout=5", "-T", "root@example.com", "uncloudd", "dial-stdio", "--socket", "/custom/path/uncloud.sock"},
}, },
{ {
name: "with default socket path (not included)", name: "with default socket path (not included)",
@@ -57,7 +72,8 @@ func TestSSHCLIConnector_buildSSHArgs(t *testing.T) {
Host: "example.com", Host: "example.com",
SockPath: machine.DefaultUncloudSockPath, SockPath: machine.DefaultUncloudSockPath,
}, },
expected: []string{"-o", "ConnectTimeout=5", "root@example.com", "uncloudd", "dial-stdio"}, controlSockPath: "/tmp/test.sock",
expected: []string{"-o", "ControlMaster=auto", "-o", "ControlPath=/tmp/test.sock", "-o", "ControlPersist=10m", "-o", "ConnectTimeout=5", "-T", "root@example.com", "uncloudd", "dial-stdio"},
}, },
{ {
name: "all options combined", name: "all options combined",
@@ -68,16 +84,28 @@ func TestSSHCLIConnector_buildSSHArgs(t *testing.T) {
KeyPath: "/path/to/key", KeyPath: "/path/to/key",
SockPath: "/custom/path/uncloud.sock", SockPath: "/custom/path/uncloud.sock",
}, },
expected: []string{"-o", "ConnectTimeout=5", "-p", "2222", "-i", "/path/to/key", "root@example.com", "uncloudd", "dial-stdio", "--socket", "/custom/path/uncloud.sock"}, controlSockPath: "/tmp/test.sock",
expected: []string{"-o", "ControlMaster=auto", "-o", "ControlPath=/tmp/test.sock", "-o", "ControlPersist=10m", "-o", "ConnectTimeout=5", "-T", "-p", "2222", "-i", "/path/to/key", "root@example.com", "uncloudd", "dial-stdio", "--socket", "/custom/path/uncloud.sock"},
}, },
{ {
name: "port 22 not included (default)", name: "port 0 not included",
config: SSHConnectorConfig{ config: SSHConnectorConfig{
User: "root", User: "root",
Host: "example.com", Host: "example.com",
Port: 0, Port: 0,
}, },
expected: []string{"-o", "ConnectTimeout=5", "root@example.com", "uncloudd", "dial-stdio"}, controlSockPath: "/tmp/test.sock",
expected: []string{"-o", "ControlMaster=auto", "-o", "ControlPath=/tmp/test.sock", "-o", "ControlPersist=10m", "-o", "ConnectTimeout=5", "-T", "root@example.com", "uncloudd", "dial-stdio"},
},
{
name: "port 22 included when explicit",
config: SSHConnectorConfig{
User: "root",
Host: "example.com",
Port: 22,
},
controlSockPath: "/tmp/test.sock",
expected: []string{"-o", "ControlMaster=auto", "-o", "ControlPath=/tmp/test.sock", "-o", "ControlPersist=10m", "-o", "ConnectTimeout=5", "-T", "-p", "22", "root@example.com", "uncloudd", "dial-stdio"},
}, },
} }
@@ -85,7 +113,7 @@ func TestSSHCLIConnector_buildSSHArgs(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
t.Parallel() t.Parallel()
c := &SSHCLIConnector{config: tt.config} c := &SSHCLIConnector{config: tt.config, controlSockPath: tt.controlSockPath}
got := c.buildSSHArgs() got := c.buildSSHArgs()
assert.Equal(t, tt.expected, got) assert.Equal(t, tt.expected, got)
}) })
@@ -96,61 +124,77 @@ func TestSSHCLIDialer_buildDialArgs(t *testing.T) {
t.Parallel() t.Parallel()
tests := []struct { tests := []struct {
name string name string
config SSHConnectorConfig config SSHConnectorConfig
address string controlSockPath string
expected []string address string
expected []string
}{ }{
{ {
name: "basic connection", name: "basic connection without control socket",
config: SSHConnectorConfig{ config: SSHConnectorConfig{
User: "root", User: "root",
Host: "example.com", Host: "example.com",
}, },
address: "10.210.1.1:5000", controlSockPath: "",
expected: []string{"-o", "ConnectTimeout=5", "-W", "10.210.1.1:5000", "root@example.com"}, address: "10.210.1.1:5000",
expected: []string{"-o", "ConnectTimeout=5", "-T", "-W", "10.210.1.1:5000", "root@example.com"},
}, },
{ {
name: "custom port", name: "basic connection with control socket",
config: SSHConnectorConfig{
User: "root",
Host: "example.com",
},
controlSockPath: "/tmp/test.sock",
address: "10.210.1.1:5000",
expected: []string{"-o", "ControlMaster=no", "-o", "ControlPath=/tmp/test.sock", "-o", "ConnectTimeout=5", "-T", "-W", "10.210.1.1:5000", "root@example.com"},
},
{
name: "custom port with control socket",
config: SSHConnectorConfig{ config: SSHConnectorConfig{
User: "root", User: "root",
Host: "example.com", Host: "example.com",
Port: 2222, Port: 2222,
}, },
address: "10.210.1.1:5000", controlSockPath: "/tmp/test.sock",
expected: []string{"-o", "ConnectTimeout=5", "-p", "2222", "-W", "10.210.1.1:5000", "root@example.com"}, address: "10.210.1.1:5000",
expected: []string{"-o", "ControlMaster=no", "-o", "ControlPath=/tmp/test.sock", "-o", "ConnectTimeout=5", "-T", "-p", "2222", "-W", "10.210.1.1:5000", "root@example.com"},
}, },
{ {
name: "with identity file", name: "with identity file and control socket",
config: SSHConnectorConfig{ config: SSHConnectorConfig{
User: "root", User: "root",
Host: "example.com", Host: "example.com",
Port: 22, Port: 22,
KeyPath: "/home/user/.ssh/id_rsa", KeyPath: "/home/user/.ssh/id_rsa",
}, },
address: "10.210.1.1:5000", controlSockPath: "/tmp/test.sock",
expected: []string{"-o", "ConnectTimeout=5", "-i", "/home/user/.ssh/id_rsa", "-W", "10.210.1.1:5000", "root@example.com"}, address: "10.210.1.1:5000",
expected: []string{"-o", "ControlMaster=no", "-o", "ControlPath=/tmp/test.sock", "-o", "ConnectTimeout=5", "-T", "-p", "22", "-i", "/home/user/.ssh/id_rsa", "-W", "10.210.1.1:5000", "root@example.com"},
}, },
{ {
name: "custom port with identity file", name: "custom port with identity file and control socket",
config: SSHConnectorConfig{ config: SSHConnectorConfig{
User: "root", User: "root",
Host: "example.com", Host: "example.com",
Port: 2222, Port: 2222,
KeyPath: "/home/user/.ssh/id_rsa", KeyPath: "/home/user/.ssh/id_rsa",
}, },
address: "10.210.1.1:5000", controlSockPath: "/tmp/test.sock",
expected: []string{"-o", "ConnectTimeout=5", "-p", "2222", "-i", "/home/user/.ssh/id_rsa", "-W", "10.210.1.1:5000", "root@example.com"}, address: "10.210.1.1:5000",
expected: []string{"-o", "ControlMaster=no", "-o", "ControlPath=/tmp/test.sock", "-o", "ConnectTimeout=5", "-T", "-p", "2222", "-i", "/home/user/.ssh/id_rsa", "-W", "10.210.1.1:5000", "root@example.com"},
}, },
{ {
name: "zero port defaults to 22", name: "port 0 not included",
config: SSHConnectorConfig{ config: SSHConnectorConfig{
User: "root", User: "root",
Host: "example.com", Host: "example.com",
Port: 0, Port: 0,
}, },
address: "10.210.1.1:5000", controlSockPath: "/tmp/test.sock",
expected: []string{"-o", "ConnectTimeout=5", "-W", "10.210.1.1:5000", "root@example.com"}, address: "10.210.1.1:5000",
expected: []string{"-o", "ControlMaster=no", "-o", "ControlPath=/tmp/test.sock", "-o", "ConnectTimeout=5", "-T", "-W", "10.210.1.1:5000", "root@example.com"},
}, },
} }
@@ -158,9 +202,28 @@ func TestSSHCLIDialer_buildDialArgs(t *testing.T) {
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
t.Parallel() t.Parallel()
d := &sshCLIDialer{config: tt.config} d := &sshCLIDialer{config: tt.config, controlSockPath: tt.controlSockPath}
got := d.buildDialArgs(tt.address) got := d.buildDialArgs(tt.address)
assert.Equal(t, tt.expected, got) assert.Equal(t, tt.expected, got)
}) })
} }
} }
func TestControlSocketPath(t *testing.T) {
// Note: Cannot use t.Parallel() because a subtest uses t.Setenv().
path1 := controlSocketPath()
path2 := controlSocketPath()
assert.Equal(t, path1, path2)
assert.True(t, strings.HasSuffix(path1, ".sock"))
assert.Contains(t, path1, "%C")
t.Run("uses XDG_RUNTIME_DIR when set", func(t *testing.T) {
runDir := "/user/runtime/dir"
t.Setenv("XDG_RUNTIME_DIR", runDir)
path := controlSocketPath()
assert.True(t, strings.HasPrefix(path, runDir))
})
}
+1
View File
@@ -23,6 +23,7 @@ func (c *TCPConnector) Connect(_ context.Context) (*grpc.ClientConn, error) {
conn, err := grpc.NewClient( conn, err := grpc.NewClient(
c.apiAddr.String(), c.apiAddr.String(),
grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithDefaultServiceConfig(defaultServiceConfig),
) )
if err != nil { if err != nil {
return nil, fmt.Errorf("create machine API client: %w", err) return nil, fmt.Errorf("create machine API client: %w", err)
+1
View File
@@ -25,6 +25,7 @@ func (c *UnixConnector) Connect(_ context.Context) (*grpc.ClientConn, error) {
conn, err := grpc.NewClient( conn, err := grpc.NewClient(
target, target,
grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithDefaultServiceConfig(defaultServiceConfig),
) )
if err != nil { if err != nil {
return nil, fmt.Errorf("create machine API client: %w", err) return nil, fmt.Errorf("create machine API client: %w", err)
+1
View File
@@ -66,6 +66,7 @@ func (c *WireGuardConnector) Connect(ctx context.Context) (*grpc.ClientConn, err
conn, err := grpc.NewClient( conn, err := grpc.NewClient(
machineAPIAddr, machineAPIAddr,
grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithDefaultServiceConfig(defaultServiceConfig),
grpc.WithContextDialer(func(ctx context.Context, addr string) (net.Conn, error) { grpc.WithContextDialer(func(ctx context.Context, addr string) (net.Conn, error) {
return c.tun.DialContext(ctx, "tcp", addr) return c.tun.DialContext(ctx, "tcp", addr)
}), }),
+151
View File
@@ -5,6 +5,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"strings" "strings"
"time"
"github.com/containerd/errdefs" "github.com/containerd/errdefs"
"github.com/docker/compose/v2/pkg/progress" "github.com/docker/compose/v2/pkg/progress"
@@ -17,6 +18,9 @@ import (
"google.golang.org/grpc/status" "google.golang.org/grpc/status"
) )
// TODO: format container and machine IDs in 'Container %s on %s' events as bold.
// Consider formatting containers as <service_name>/<short-container-id>.
// CreateContainer creates a new container for the given service on the specified machine. // CreateContainer creates a new container for the given service on the specified machine.
func (cli *Client) CreateContainer( func (cli *Client) CreateContainer(
ctx context.Context, serviceID string, spec api.ServiceSpec, machineID string, ctx context.Context, serviceID string, spec api.ServiceSpec, machineID string,
@@ -354,3 +358,150 @@ func (cli *Client) ExecContainer(
return exitCode, nil return exitCode, nil
} }
// WaitContainerHealthy polls the container until it is considered running and healthy.
//
// For containers without a health check, it waits for the monitor period and then verifies the container
// is still running and not restarting.
//
// For containers with a health check, it waits until Docker reports healthy or unhealthy. During the monitor period,
// unhealthy status is treated as retryable (the container may be recovering from a transient crash).
// After the monitor period, unhealthy becomes a permanent failure.
func (cli *Client) WaitContainerHealthy(
ctx context.Context, serviceNameOrID, containerNameOrID string, opts api.WaitContainerHealthyOptions,
) error {
// First inspect to get container info, machine name, and health check config.
mc, err := cli.InspectContainer(ctx, serviceNameOrID, containerNameOrID)
if err != nil {
return fmt.Errorf("inspect container: %w", err)
}
machine, err := cli.InspectMachine(ctx, mc.MachineID)
if err != nil {
return fmt.Errorf("inspect machine '%s': %w", mc.MachineID, err)
}
pw := progress.ContextWriter(ctx)
eventID := fmt.Sprintf("Container %s on %s", mc.Container.Name, machine.Machine.Name)
var monitor time.Duration
if opts.MonitorPeriod == nil {
monitor = api.DefaultHealthMonitorPeriod
} else {
monitor = *opts.MonitorPeriod
}
pw.Event(progress.NewEvent(eventID, progress.Working, fmt.Sprintf("Monitoring (%s)", monitor)))
// For containers without a health check, just wait for the monitor period and then check the container
// is still running and not restarting.
if !mc.Container.HasHealthcheck() {
select {
case <-ctx.Done():
return ctx.Err()
case <-time.After(monitor):
}
mc, err := cli.InspectContainer(ctx, serviceNameOrID, containerNameOrID)
if err != nil {
return fmt.Errorf("inspect container: %w", err)
}
if mc.Container.Healthy() {
pw.Event(progress.RunningEvent(eventID))
return nil
}
humanState, _ := mc.Container.HumanState()
pw.Event(progress.ErrorMessageEvent(eventID, fmt.Sprintf("Unhealthy (%s)", humanState)))
if mc.Container.State.Restarting {
return fmt.Errorf("container is restarting after monitor period (%s): exit_code=%d",
monitor, mc.Container.State.ExitCode)
}
return fmt.Errorf("container is unhealthy after monitor period (%s): %s", monitor, humanState)
}
// For containers with a health check, wait until Docker reports healthy or unhealthy.
mctx := proxyToMachine(ctx, machine.Machine)
mctx, cancel := context.WithTimeout(mctx, healthcheckTimeout(mc.Container.Config.Healthcheck))
defer cancel()
ticker := time.NewTicker(1 * time.Second)
defer ticker.Stop()
monitorDeadline := time.Now().Add(monitor)
for {
select {
case <-mctx.Done():
return mctx.Err()
case <-ticker.C:
ctr, err := cli.Docker.InspectServiceContainer(mctx, mc.Container.ID)
if err != nil {
pw.Event(progress.NewEvent(eventID, progress.Working,
fmt.Sprintf("Health checking (failed to inspect container: %v)", err)))
continue
}
// Reset the event status if previous inspect failed.
eventStatus := fmt.Sprintf("Monitoring (%s)", monitor)
if time.Now().After(monitorDeadline) {
// TODO: provide more details about running checks or waiting so the user can see what's going on.
eventStatus = "Health checking"
}
pw.Event(progress.NewEvent(eventID, progress.Working, eventStatus))
if ctr.Healthy() {
pw.Event(progress.Healthy(eventID))
return nil
}
if time.Now().Before(monitorDeadline) {
continue
}
if ctr.State.Health.Status == container.Unhealthy {
humanState, _ := ctr.HumanState()
pw.Event(progress.ErrorMessageEvent(eventID, fmt.Sprintf("Unhealthy (%s)", humanState)))
if ctr.State.Restarting {
return fmt.Errorf("container is restarting after monitor period (%s): exit_code=%d",
monitor, ctr.State.ExitCode)
}
return fmt.Errorf("container is unhealthy after monitor period (%s): %s", monitor, humanState)
}
}
}
}
const (
// defaultDockerHealthcheckInterval is the default Docker interval between health check runs.
defaultDockerHealthcheckInterval = 30 * time.Second
// defaultDockerHealthcheckTimeout is the default Docker timeout for each health check run.
defaultDockerHealthcheckTimeout = 30 * time.Second
// defaultDockerHealthcheckRetries is the default Docker number of consecutive failures needed
// to consider the container unhealthy.
defaultDockerHealthcheckRetries = 3
)
// healthcheckTimeout computes the maximum time to wait for a container to become healthy based on
// its health check config. This is the worst case timeout to stop polling in case something goes wrong and Docker
// doesn't report the container as unhealthy after it should.
func healthcheckTimeout(hc *container.HealthConfig) time.Duration {
if hc == nil {
return 0
}
interval := hc.Interval
if interval <= 0 {
interval = defaultDockerHealthcheckInterval
}
timeout := hc.Timeout
if timeout <= 0 {
timeout = defaultDockerHealthcheckTimeout
}
retries := hc.Retries
if retries <= 0 {
retries = defaultDockerHealthcheckRetries
}
// 5s is a buffer to account for scheduling delays.
return hc.StartPeriod + time.Duration(retries)*(interval+timeout) + 5*time.Second
}
+8 -1
View File
@@ -83,10 +83,17 @@ func EvalContainerSpecChange(current api.ServiceSpec, new api.ServiceSpec) Conta
} }
} }
// Device reservations are immutable, so we'll need to recreate if any have changed // Device reservations and mappings are immutable, so we'll need to recreate if any have changed
if !reflect.DeepEqual(current.Container.Resources.DeviceReservations, newResources.DeviceReservations) { if !reflect.DeepEqual(current.Container.Resources.DeviceReservations, newResources.DeviceReservations) {
return ContainerNeedsRecreate return ContainerNeedsRecreate
} }
if !reflect.DeepEqual(current.Container.Resources.Devices, newResources.Devices) {
return ContainerNeedsRecreate
}
// Ulimits are immutable, so we'll need to recreate if any have changed.
if !reflect.DeepEqual(current.Container.Resources.Ulimits, newResources.Ulimits) {
return ContainerNeedsRecreate
}
// Check if any mutable properties changed. // Check if any mutable properties changed.
if !current.Caddy.Equals(new.Caddy) { if !current.Caddy.Equals(new.Caddy) {
+200
View File
@@ -9,6 +9,44 @@ import (
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
) )
func TestEvalContainerSpecChange_ContainerCapAdd(t *testing.T) {
t.Parallel()
currentSpec := api.ServiceSpec{
Container: api.ContainerSpec{
Image: "nginx:latest",
},
}
newSpec := api.ServiceSpec{
Container: api.ContainerSpec{
Image: "nginx:latest",
CapAdd: []string{"NET_ADMIN"},
},
}
assert.Equal(t, ContainerNeedsRecreate, EvalContainerSpecChange(currentSpec, newSpec))
assert.Equal(t, ContainerNeedsRecreate, EvalContainerSpecChange(newSpec, currentSpec))
}
func TestEvalContainerSpecChange_ContainerCapDrop(t *testing.T) {
t.Parallel()
currentSpec := api.ServiceSpec{
Container: api.ContainerSpec{
Image: "nginx:latest",
},
}
newSpec := api.ServiceSpec{
Container: api.ContainerSpec{
Image: "nginx:latest",
CapDrop: []string{"ALL"},
},
}
assert.Equal(t, ContainerNeedsRecreate, EvalContainerSpecChange(currentSpec, newSpec))
assert.Equal(t, ContainerNeedsRecreate, EvalContainerSpecChange(newSpec, currentSpec))
}
func TestEvalContainerSpecChange_ContainerResources(t *testing.T) { func TestEvalContainerSpecChange_ContainerResources(t *testing.T) {
t.Parallel() t.Parallel()
@@ -325,6 +363,27 @@ func TestEvalContainerSpecChange_ContainerPrivileged(t *testing.T) {
assert.Equal(t, ContainerNeedsRecreate, EvalContainerSpecChange(newSpec, currentSpec)) assert.Equal(t, ContainerNeedsRecreate, EvalContainerSpecChange(newSpec, currentSpec))
} }
func TestEvalContainerSpecChange_ContainerSysctls(t *testing.T) {
t.Parallel()
currentSpec := api.ServiceSpec{
Container: api.ContainerSpec{
Image: "nginx:latest",
},
}
newSpec := api.ServiceSpec{
Container: api.ContainerSpec{
Image: "nginx:latest",
Sysctls: map[string]string{
"net.ipv4.ip_forward": "1",
},
},
}
assert.Equal(t, ContainerNeedsRecreate, EvalContainerSpecChange(currentSpec, newSpec))
assert.Equal(t, ContainerNeedsRecreate, EvalContainerSpecChange(newSpec, currentSpec))
}
func TestEvalContainerSpecChange_PullPolicy(t *testing.T) { func TestEvalContainerSpecChange_PullPolicy(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1351,6 +1410,68 @@ func TestEvalContainerSpecChange_Volumes(t *testing.T) {
} }
} }
func TestEvalContainerSpecChange_Devices(t *testing.T) {
t.Parallel()
tests := []struct {
name string
current api.ContainerResources
new api.ContainerResources
want ContainerSpecStatus
}{
{
name: "empty",
current: api.ContainerResources{},
new: api.ContainerResources{},
want: ContainerUpToDate,
},
{
name: "identical mapping",
current: api.ContainerResources{Devices: []api.DeviceMapping{{HostPath: "/dev/foo", ContainerPath: "/dev/foo", CgroupPermissions: "rwm"}}},
new: api.ContainerResources{Devices: []api.DeviceMapping{{HostPath: "/dev/foo", ContainerPath: "/dev/foo", CgroupPermissions: "rwm"}}},
want: ContainerUpToDate,
},
{
name: "add mapping",
current: api.ContainerResources{},
new: api.ContainerResources{Devices: []api.DeviceMapping{{HostPath: "/dev/foo", ContainerPath: "/dev/foo", CgroupPermissions: "rwm"}}},
want: ContainerNeedsRecreate,
},
{
name: "remove mapping",
current: api.ContainerResources{Devices: []api.DeviceMapping{{HostPath: "/dev/foo", ContainerPath: "/dev/foo", CgroupPermissions: "rwm"}}},
new: api.ContainerResources{},
want: ContainerNeedsRecreate,
},
{
name: "change mapping path",
current: api.ContainerResources{Devices: []api.DeviceMapping{{HostPath: "/dev/foo", ContainerPath: "/dev/foo", CgroupPermissions: "rwm"}}},
new: api.ContainerResources{Devices: []api.DeviceMapping{{HostPath: "/dev/foo", ContainerPath: "/dev/bar", CgroupPermissions: "rwm"}}},
want: ContainerNeedsRecreate,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
currentSpec := api.ServiceSpec{
Container: api.ContainerSpec{
Image: "nginx:latest",
Resources: tt.current,
},
}
newSpec := api.ServiceSpec{
Container: api.ContainerSpec{
Image: "nginx:latest",
Resources: tt.new,
},
}
result := EvalContainerSpecChange(currentSpec, newSpec)
assert.Equal(t, tt.want, result)
})
}
}
func TestEvalContainerSpecChange_DeviceReservations(t *testing.T) { func TestEvalContainerSpecChange_DeviceReservations(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1743,6 +1864,85 @@ func TestEvalContainerSpecChange_DeviceReservations(t *testing.T) {
} }
} }
func TestEvalContainerSpecChange_Ulimits(t *testing.T) {
t.Parallel()
tests := []struct {
name string
current api.ContainerResources
new api.ContainerResources
want ContainerSpecStatus
}{
{
name: "empty",
current: api.ContainerResources{},
new: api.ContainerResources{},
want: ContainerUpToDate,
},
{
name: "identical single ulimit",
current: api.ContainerResources{Ulimits: map[string]api.Ulimit{"nofile": {Soft: 20000, Hard: 40000}}},
new: api.ContainerResources{Ulimits: map[string]api.Ulimit{"nofile": {Soft: 20000, Hard: 40000}}},
want: ContainerUpToDate,
},
{
name: "set ulimit",
current: api.ContainerResources{},
new: api.ContainerResources{Ulimits: map[string]api.Ulimit{"nofile": {Soft: 20000, Hard: 40000}}},
want: ContainerNeedsRecreate,
},
{
name: "remove ulimit",
current: api.ContainerResources{Ulimits: map[string]api.Ulimit{"nofile": {Soft: 20000, Hard: 40000}}},
new: api.ContainerResources{},
want: ContainerNeedsRecreate,
},
{
name: "change ulimit soft value",
current: api.ContainerResources{Ulimits: map[string]api.Ulimit{"nofile": {Soft: 20000, Hard: 40000}}},
new: api.ContainerResources{Ulimits: map[string]api.Ulimit{"nofile": {Soft: 30000, Hard: 40000}}},
want: ContainerNeedsRecreate,
},
{
name: "change ulimit hard value",
current: api.ContainerResources{Ulimits: map[string]api.Ulimit{"nofile": {Soft: 20000, Hard: 40000}}},
new: api.ContainerResources{Ulimits: map[string]api.Ulimit{"nofile": {Soft: 20000, Hard: 80000}}},
want: ContainerNeedsRecreate,
},
{
name: "add ulimit",
current: api.ContainerResources{Ulimits: map[string]api.Ulimit{
"nofile": {Soft: 20000, Hard: 40000},
}},
new: api.ContainerResources{Ulimits: map[string]api.Ulimit{
"nofile": {Soft: 20000, Hard: 40000},
"nproc": {Soft: 65535, Hard: 65535},
}},
want: ContainerNeedsRecreate,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
currentSpec := api.ServiceSpec{
Container: api.ContainerSpec{
Image: "nginx:latest",
Resources: tt.current,
},
}
newSpec := api.ServiceSpec{
Container: api.ContainerSpec{
Image: "nginx:latest",
Resources: tt.new,
},
}
result := EvalContainerSpecChange(currentSpec, newSpec)
assert.Equal(t, tt.want, result)
})
}
}
func TestEvalContainerSpecChange_Mixed(t *testing.T) { func TestEvalContainerSpecChange_Mixed(t *testing.T) {
t.Parallel() t.Parallel()
+2 -1
View File
@@ -6,6 +6,7 @@ import (
"fmt" "fmt"
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client/deploy/operation"
"github.com/psviderski/uncloud/pkg/client/deploy/scheduler" "github.com/psviderski/uncloud/pkg/client/deploy/scheduler"
) )
@@ -33,7 +34,7 @@ type Deployment struct {
type Plan struct { type Plan struct {
ServiceID string ServiceID string
ServiceName string ServiceName string
SequenceOperation operation.SequenceOperation
} }
// NewDeployment creates a new deployment for the given service specification. // NewDeployment creates a new deployment for the given service specification.
-191
View File
@@ -1,191 +0,0 @@
package deploy
import (
"context"
"fmt"
"strings"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/volume"
"github.com/psviderski/uncloud/pkg/api"
)
// Operation represents a single atomic operation in a deployment process.
// Operations can be composed to form complex deployment strategies.
type Operation interface {
// Execute performs the operation using the provided client.
// TODO: Encapsulate the client in the operation as otherwise it gives an impression that different clients
// can be provided. But in reality, the operation is tightly coupled with the client that was used to create it.
Execute(ctx context.Context, cli Client) error
// Format returns a human-readable representation of the operation.
// TODO: get rid of the resolver and assign the required names for formatting in the operation itself.
Format(resolver NameResolver) string
String() string
}
// NameResolver resolves machine and container IDs to their names.
type NameResolver interface {
MachineName(machineID string) string
ContainerName(containerID string) string
}
// TODO: pass api.ServiceContainer to operations to simplify operation formatting in the plan.
// RunContainerOperation creates and starts a new container on a specific machine.
type RunContainerOperation struct {
ServiceID string
Spec api.ServiceSpec
MachineID string
}
func (o *RunContainerOperation) Execute(ctx context.Context, cli Client) error {
resp, err := cli.CreateContainer(ctx, o.ServiceID, o.Spec, o.MachineID)
if err != nil {
return fmt.Errorf("create container: %w", err)
}
if err = cli.StartContainer(ctx, o.ServiceID, resp.ID); err != nil {
return fmt.Errorf("start container: %w", err)
}
// TODO: wait for the container to become healthy
return nil
}
func (o *RunContainerOperation) Format(resolver NameResolver) string {
machineName := resolver.MachineName(o.MachineID)
return fmt.Sprintf("%s: Run container [image=%s]", machineName, o.Spec.Container.Image)
}
func (o *RunContainerOperation) String() string {
return fmt.Sprintf("RunContainerOperation[machine_id=%s service_id=%s image=%s]",
o.MachineID, o.ServiceID, o.Spec.Container.Image)
}
// StopContainerOperation stops a container on a specific machine.
type StopContainerOperation struct {
ServiceID string
ContainerID string
MachineID string
}
func (o *StopContainerOperation) Execute(ctx context.Context, cli Client) error {
if err := cli.StopContainer(ctx, o.ServiceID, o.ContainerID, container.StopOptions{}); err != nil {
return fmt.Errorf("stop container: %w", err)
}
return nil
}
func (o *StopContainerOperation) Format(resolver NameResolver) string {
machineName := resolver.MachineName(o.MachineID)
return fmt.Sprintf("%s: Stop container [id=%s name=%s]", machineName,
o.ContainerID[:12], resolver.ContainerName(o.ContainerID))
}
func (o *StopContainerOperation) String() string {
return fmt.Sprintf("StopContainerOperation[machine_id=%s service_id=%s container_id=%s]",
o.MachineID, o.ServiceID, o.ContainerID)
}
// RemoveContainerOperation stops and removes a container from a specific machine.
type RemoveContainerOperation struct {
MachineID string
Container api.ServiceContainer
}
func (o *RemoveContainerOperation) Execute(ctx context.Context, cli Client) error {
if err := cli.StopContainer(ctx, o.Container.ServiceID(), o.Container.ID, container.StopOptions{}); err != nil {
return fmt.Errorf("stop container: %w", err)
}
if err := cli.RemoveContainer(ctx, o.Container.ServiceID(), o.Container.ID, container.RemoveOptions{
// Remove anonymous volumes created by the container.
RemoveVolumes: true,
}); err != nil {
return fmt.Errorf("remove container: %w", err)
}
return nil
}
func (o *RemoveContainerOperation) Format(resolver NameResolver) string {
machineName := resolver.MachineName(o.MachineID)
return fmt.Sprintf("%s: Remove container [id=%s image=%s]",
machineName, o.Container.ShortID(), o.Container.Config.Image)
}
func (o *RemoveContainerOperation) String() string {
return fmt.Sprintf("RemoveContainerOperation[machine_id=%s service_id=%s container_id=%s]",
o.MachineID, o.Container.ServiceID(), o.Container.ID)
}
// CreateVolumeOperation creates a volume on a specific machine.
type CreateVolumeOperation struct {
VolumeSpec api.VolumeSpec
MachineID string
// MachineName is used for formatting the operation output only.
MachineName string
}
func (o *CreateVolumeOperation) Execute(ctx context.Context, cli Client) error {
if o.VolumeSpec.Type != api.VolumeTypeVolume {
return fmt.Errorf("invalid volume type: '%s', expected '%s'", o.VolumeSpec.Type, api.VolumeTypeVolume)
}
opts := volume.CreateOptions{
Name: o.VolumeSpec.DockerVolumeName(),
}
if o.VolumeSpec.VolumeOptions != nil {
if o.VolumeSpec.VolumeOptions.Driver != nil {
opts.Driver = o.VolumeSpec.VolumeOptions.Driver.Name
opts.DriverOpts = o.VolumeSpec.VolumeOptions.Driver.Options
}
opts.Labels = o.VolumeSpec.VolumeOptions.Labels
}
if _, err := cli.CreateVolume(ctx, o.MachineID, opts); err != nil {
return fmt.Errorf("create volume: %w", err)
}
return nil
}
func (o *CreateVolumeOperation) Format(_ NameResolver) string {
return fmt.Sprintf("%s: Create volume [name=%s]", o.MachineName, o.VolumeSpec.DockerVolumeName())
}
func (o *CreateVolumeOperation) String() string {
return fmt.Sprintf("CreateVolumeOperation[machine_id=%s volume=%s]",
o.MachineID, o.VolumeSpec.DockerVolumeName())
}
// SequenceOperation is a composite operation that executes a sequence of operations in order.
type SequenceOperation struct {
Operations []Operation
}
func (o *SequenceOperation) Execute(ctx context.Context, cli Client) error {
for _, op := range o.Operations {
if err := op.Execute(ctx, cli); err != nil {
return err
}
}
return nil
}
func (o *SequenceOperation) Format(resolver NameResolver) string {
ops := make([]string, len(o.Operations))
for i, op := range o.Operations {
ops[i] = "- " + op.Format(resolver)
}
return strings.Join(ops, "\n")
}
func (o *SequenceOperation) String() string {
ops := make([]string, len(o.Operations))
for i, op := range o.Operations {
ops[i] = op.String()
}
return fmt.Sprintf("SequenceOperation[%s]", strings.Join(ops, ", "))
}
+211
View File
@@ -0,0 +1,211 @@
package operation
import (
"context"
"fmt"
"github.com/docker/compose/v2/pkg/progress"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/pkg/stringid"
"github.com/psviderski/uncloud/pkg/api"
)
// RunContainerOperation creates and starts a new container on a specific machine.
type RunContainerOperation struct {
ServiceID string
Spec api.ServiceSpec
MachineID string
// SkipHealthMonitor skips the monitoring period and health checks after starting a container.
SkipHealthMonitor bool
}
func (o *RunContainerOperation) Execute(ctx context.Context, cli Client) error {
resp, err := cli.CreateContainer(ctx, o.ServiceID, o.Spec, o.MachineID)
if err != nil {
return fmt.Errorf("create container: %w", err)
}
if err = cli.StartContainer(ctx, o.ServiceID, resp.ID); err != nil {
return fmt.Errorf("start container: %w", err)
}
if o.SkipHealthMonitor {
return nil
}
opts := api.WaitContainerHealthyOptions{MonitorPeriod: o.Spec.UpdateConfig.MonitorPeriod}
if err = cli.WaitContainerHealthy(ctx, o.ServiceID, resp.ID, opts); err != nil {
return fmt.Errorf("container '%s/%s' failed to become healthy: %w",
o.Spec.Name, stringid.TruncateID(resp.ID), err)
}
return nil
}
func (o *RunContainerOperation) Format(resolver NameResolver) string {
machineName := resolver.MachineName(o.MachineID)
return fmt.Sprintf("%s: Run container [image=%s]", machineName, o.Spec.Container.Image)
}
func (o *RunContainerOperation) String() string {
return fmt.Sprintf("RunContainerOperation[machine_id=%s service_id=%s image=%s]",
o.MachineID, o.ServiceID, o.Spec.Container.Image)
}
// StopContainerOperation stops a container on a specific machine.
type StopContainerOperation struct {
ServiceID string
ContainerID string
MachineID string
}
func (o *StopContainerOperation) Execute(ctx context.Context, cli Client) error {
if err := cli.StopContainer(ctx, o.ServiceID, o.ContainerID, container.StopOptions{}); err != nil {
return fmt.Errorf("stop container: %w", err)
}
return nil
}
func (o *StopContainerOperation) Format(resolver NameResolver) string {
machineName := resolver.MachineName(o.MachineID)
return fmt.Sprintf("%s: Stop container [id=%s name=%s]", machineName,
o.ContainerID[:12], resolver.ContainerName(o.ContainerID))
}
func (o *StopContainerOperation) String() string {
return fmt.Sprintf("StopContainerOperation[machine_id=%s service_id=%s container_id=%s]",
o.MachineID, o.ServiceID, o.ContainerID)
}
// RemoveContainerOperation stops and removes a container from a specific machine.
type RemoveContainerOperation struct {
MachineID string
Container api.ServiceContainer
}
func (o *RemoveContainerOperation) Execute(ctx context.Context, cli Client) error {
if err := cli.StopContainer(ctx, o.Container.ServiceID(), o.Container.ID, container.StopOptions{}); err != nil {
return fmt.Errorf("stop container: %w", err)
}
if err := cli.RemoveContainer(ctx, o.Container.ServiceID(), o.Container.ID, container.RemoveOptions{
// Remove anonymous volumes created by the container.
RemoveVolumes: true,
}); err != nil {
return fmt.Errorf("remove container: %w", err)
}
return nil
}
func (o *RemoveContainerOperation) Format(resolver NameResolver) string {
machineName := resolver.MachineName(o.MachineID)
return fmt.Sprintf("%s: Remove container [id=%s image=%s]",
machineName, o.Container.ShortID(), o.Container.Config.Image)
}
func (o *RemoveContainerOperation) String() string {
return fmt.Sprintf("RemoveContainerOperation[machine_id=%s service_id=%s container_id=%s]",
o.MachineID, o.Container.ServiceID(), o.Container.ID)
}
// ReplaceContainerOperation replaces an old container with a new one based on the specified update order.
// For start-first: starts new container, then removes old container.
// For stop-first: stops old container, starts new container, then removes old container.
type ReplaceContainerOperation struct {
ServiceID string
Spec api.ServiceSpec
MachineID string
OldContainer api.ServiceContainer
// Order specifies the update order: "start-first" or "stop-first".
Order string
// SkipHealthMonitor skips the monitoring period and health checks after starting a new container.
SkipHealthMonitor bool
}
func (o *ReplaceContainerOperation) Execute(ctx context.Context, cli Client) error {
stopFirst := o.Order == api.UpdateOrderStopFirst
wasRunning := false
if stopFirst {
// Inspect the old container to remember its running state before stopping.
ctr, err := cli.InspectContainer(ctx, o.ServiceID, o.OldContainer.ID)
if err != nil {
return fmt.Errorf("inspect old container: %w", err)
}
wasRunning = ctr.Container.State.Running
if wasRunning {
if err = cli.StopContainer(ctx, o.ServiceID, o.OldContainer.ID, container.StopOptions{}); err != nil {
return fmt.Errorf("stop old container: %w", err)
}
}
}
resp, err := cli.CreateContainer(ctx, o.ServiceID, o.Spec, o.MachineID)
if err != nil {
return fmt.Errorf("create new container: %w", err)
}
if err = cli.StartContainer(ctx, o.ServiceID, resp.ID); err != nil {
return fmt.Errorf("start new container: %w", err)
}
if !o.SkipHealthMonitor {
opts := api.WaitContainerHealthyOptions{MonitorPeriod: o.Spec.UpdateConfig.MonitorPeriod}
if err = cli.WaitContainerHealthy(ctx, o.ServiceID, resp.ID, opts); err != nil {
// New container failed to become healthy. Stop it and roll back to the previous container.
// Don't remove the new stopped container to allow users to inspect logs and state.
// TODO: collect logs from the new container and include in the error message to speed up debugging.
// Use context without progress to not overwrite the container Unhealthy status with Stopped.
ctxWithoutProgress := progress.WithContextWriter(ctx, nil)
_ = cli.StopContainer(ctxWithoutProgress, o.ServiceID, resp.ID, container.StopOptions{})
newCtr := fmt.Sprintf("%s/%s", o.Spec.Name, stringid.TruncateID(resp.ID))
healthErr := fmt.Errorf(
"new container '%s' failed to become healthy: %w. "+
"It's stopped and available for inspection. Fetch logs with 'uc logs %s'",
newCtr, err, o.Spec.Name,
)
if stopFirst && wasRunning {
// Restart the old container only if it was running before we stopped it.
oldCtr := fmt.Sprintf("%s/%s", o.OldContainer.ServiceSpec.Name, o.OldContainer.ShortID())
if rollbackErr := cli.StartContainer(ctx, o.ServiceID, o.OldContainer.ID); rollbackErr != nil {
return fmt.Errorf("%w. Rolled back to old container '%s' but failed to restart it: %w",
healthErr, oldCtr, rollbackErr)
}
return fmt.Errorf("%w. Rolled back to old container '%s'", healthErr, oldCtr)
}
return healthErr
}
}
// For start-first, we need to stop before removing.
// For stop-first, the container is already stopped.
if !stopFirst {
// TODO: the new container is propagated to Caddy upstreams through the cluster store asynchronously.
// There still might be a brief downtime (for a 1 replica service) when Caddy doesn't know about
// the new container but we're stopping the old container. We should somehow ensure Caddy is updated
// with the new container before we stop the old one to avoid this downtime.
if err := cli.StopContainer(ctx, o.ServiceID, o.OldContainer.ID, container.StopOptions{}); err != nil {
return fmt.Errorf("stop old container: %w", err)
}
}
if err := cli.RemoveContainer(ctx, o.ServiceID, o.OldContainer.ID, container.RemoveOptions{
RemoveVolumes: true,
}); err != nil {
return fmt.Errorf("remove old container: %w", err)
}
return nil
}
func (o *ReplaceContainerOperation) Format(resolver NameResolver) string {
return fmt.Sprintf("%s: Replace container [id=%s image=%s order=%s]",
resolver.MachineName(o.MachineID), o.OldContainer.ShortID(), o.Spec.Container.Image, o.Order)
}
func (o *ReplaceContainerOperation) String() string {
return fmt.Sprintf("ReplaceContainerOperation[machine_id=%s service_id=%s old_container_id=%s order=%s]",
o.MachineID, o.ServiceID, o.OldContainer.ID, o.Order)
}
+30
View File
@@ -0,0 +1,30 @@
package operation
import (
"context"
"github.com/psviderski/uncloud/pkg/api"
)
// Operation represents a single atomic operation in a deployment process.
// Operations can be composed to form complex deployment strategies.
type Operation interface {
// Execute performs the operation using the provided client.
Execute(ctx context.Context, cli Client) error
// Format returns a human-readable representation of the operation.
// TODO: get rid of the resolver and assign the required names for formatting in the operation itself.
Format(resolver NameResolver) string
String() string
}
// Client defines the interface required to execute deployment operations.
type Client interface {
api.ContainerClient
api.VolumeClient
}
// NameResolver resolves machine and container IDs to their names.
type NameResolver interface {
MachineName(machineID string) string
ContainerName(containerID string) string
}
+39
View File
@@ -0,0 +1,39 @@
package operation
import (
"context"
"fmt"
"strings"
)
// SequenceOperation is a composite operation that executes a sequence of operations in order.
type SequenceOperation struct {
Operations []Operation
}
func (o *SequenceOperation) Execute(ctx context.Context, cli Client) error {
for _, op := range o.Operations {
if err := op.Execute(ctx, cli); err != nil {
return err
}
}
return nil
}
func (o *SequenceOperation) Format(resolver NameResolver) string {
ops := make([]string, len(o.Operations))
for i, op := range o.Operations {
ops[i] = "- " + op.Format(resolver)
}
return strings.Join(ops, "\n")
}
func (o *SequenceOperation) String() string {
ops := make([]string, len(o.Operations))
for i, op := range o.Operations {
ops[i] = op.String()
}
return fmt.Sprintf("SequenceOperation[%s]", strings.Join(ops, ", "))
}
+49
View File
@@ -0,0 +1,49 @@
package operation
import (
"context"
"fmt"
"github.com/docker/docker/api/types/volume"
"github.com/psviderski/uncloud/pkg/api"
)
// CreateVolumeOperation creates a volume on a specific machine.
type CreateVolumeOperation struct {
VolumeSpec api.VolumeSpec
MachineID string
// MachineName is used for formatting the operation output only.
MachineName string
}
func (o *CreateVolumeOperation) Execute(ctx context.Context, cli Client) error {
if o.VolumeSpec.Type != api.VolumeTypeVolume {
return fmt.Errorf("invalid volume type: '%s', expected '%s'", o.VolumeSpec.Type, api.VolumeTypeVolume)
}
opts := volume.CreateOptions{
Name: o.VolumeSpec.DockerVolumeName(),
}
if o.VolumeSpec.VolumeOptions != nil {
if o.VolumeSpec.VolumeOptions.Driver != nil {
opts.Driver = o.VolumeSpec.VolumeOptions.Driver.Name
opts.DriverOpts = o.VolumeSpec.VolumeOptions.Driver.Options
}
opts.Labels = o.VolumeSpec.VolumeOptions.Labels
}
if _, err := cli.CreateVolume(ctx, o.MachineID, opts); err != nil {
return fmt.Errorf("create volume: %w", err)
}
return nil
}
func (o *CreateVolumeOperation) Format(_ NameResolver) string {
return fmt.Sprintf("%s: Create volume [name=%s]", o.MachineName, o.VolumeSpec.DockerVolumeName())
}
func (o *CreateVolumeOperation) String() string {
return fmt.Sprintf("CreateVolumeOperation[machine_id=%s volume=%s]",
o.MachineID, o.VolumeSpec.DockerVolumeName())
}
+125 -29
View File
@@ -11,6 +11,7 @@ import (
// VolumeScheduler determines what missing volumes should be created and where for a multi-service deployment. // VolumeScheduler determines what missing volumes should be created and where for a multi-service deployment.
// It satisfies the following constraints: // It satisfies the following constraints:
// - Volumes used by global services will be created on all eligible machines.
// - Services that share a volume must be placed on the same machine where the volume is located. // - Services that share a volume must be placed on the same machine where the volume is located.
// If the volume is located on multiple machines, services can be placed on any of them. // If the volume is located on multiple machines, services can be placed on any of them.
// - Services must respect their individual placement constraints. // - Services must respect their individual placement constraints.
@@ -83,7 +84,10 @@ func NewVolumeScheduler(state *ClusterState, specs []api.ServiceSpec) (*VolumeSc
if spec, ok := volumeSpecs[vol.Name]; ok { if spec, ok := volumeSpecs[vol.Name]; ok {
if !spec.MatchesDockerVolume(vol) { if !spec.MatchesDockerVolume(vol) {
return nil, fmt.Errorf("volume '%s' specification does not match the existing volume "+ return nil, fmt.Errorf("volume '%s' specification does not match the existing volume "+
"on machine '%s'", vol.Name, machine.Info.Name) "on machine '%s'. Use a different volume name or adjust the volume options to match "+
"the existing volume. You can also remove the existing volume from the machine(s) with "+
"'uc volume rm' (WARNING: the data will be lost) and run the deployment again to create "+
"a new volume with the correct specification", vol.Name, machine.Info.Name)
} }
if _, setInitialised := existingVolumeMachines[vol.Name]; !setInitialised { if _, setInitialised := existingVolumeMachines[vol.Name]; !setInitialised {
@@ -125,12 +129,20 @@ func (s *VolumeScheduler) Schedule() (map[string][]api.VolumeSpec, error) {
serviceEligibleMachines[spec.Name] = machineIDs serviceEligibleMachines[spec.Name] = machineIDs
} }
// For each volume that exists on any machine(s) (which shouldn't be created), intersect each service's // For each volume that exists on any machine(s), intersect each non-global service's
// eligible machines that use the volume with the machines the volume is located on. // eligible machines that use the volume with the machines the volume is located on.
// Global services skip this constraint as they need the volume on ALL eligible machines,
// and the volume will be created on machines that don't have it.
// //
// Service name -> list of processed volume names (quoted) to format the error message. // Service name -> list of processed volume names (quoted) to format the error message.
quotedServiceVolumes := make(map[string][]string) quotedServiceVolumes := make(map[string][]string)
for volumeName, volumeMachines := range s.existingVolumeMachines { for volumeName, volumeMachines := range s.existingVolumeMachines {
// Skip constraint narrowing for global services - they don't need to be constrained
// to machines that already have the volume.
if s.isVolumeForGlobalService(volumeName) {
continue
}
for _, serviceName := range s.volumeServices[volumeName] { for _, serviceName := range s.volumeServices[volumeName] {
quotedServiceVolumes[serviceName] = append(quotedServiceVolumes[serviceName], quotedServiceVolumes[serviceName] = append(quotedServiceVolumes[serviceName],
fmt.Sprintf("'%s'", volumeName)) fmt.Sprintf("'%s'", volumeName))
@@ -144,52 +156,97 @@ func (s *VolumeScheduler) Schedule() (map[string][]api.VolumeSpec, error) {
} }
} }
// Skip constraints propagation for volumes that already exist on machines as the propagation only works // Skip constraints propagation for:
// for missing volumes. // 1. Volumes that already exist on machines (for replicated services) as the propagation only works
// for missing volumes. Global service volumes are NOT marked as placed here since they still need
// to be scheduled on machines that don't have them.
// 2. Volumes only used by global services - these need UNION of eligible machines, not intersection.
placedVolumes := make(map[string]struct{}) placedVolumes := make(map[string]struct{})
for volumeName := range s.existingVolumeMachines { for volumeName := range s.existingVolumeMachines {
placedVolumes[volumeName] = struct{}{} if !s.isVolumeForGlobalService(volumeName) {
placedVolumes[volumeName] = struct{}{}
}
}
// Skip constraint propagation for volumes used by global services.
// Also check for invalid configuration: volume shared between global and replicated services.
for volumeName := range s.volumeSpecs {
if s.isVolumeSharedBetweenGlobalAndReplicated(volumeName) {
return nil, fmt.Errorf("volume '%s' cannot be shared between global and replicated services: "+
"global services require the volume on all machines while replicated services require "+
"co-location with the volume", volumeName)
}
if s.isVolumeForGlobalService(volumeName) {
placedVolumes[volumeName] = struct{}{}
}
} }
if err := s.propagateConstraintsUntilConvergence(serviceEligibleMachines, placedVolumes); err != nil { if err := s.propagateConstraintsUntilConvergence(serviceEligibleMachines, placedVolumes); err != nil {
return nil, err return nil, err
} }
// Schedule each missing volume on one of its eligible machines. // Schedule each missing volume on eligible machines.
// For global services: schedule on ALL eligible machines that don't already have the volume.
// For replicated services: schedule on ONE eligible machine (skip if volume exists anywhere).
scheduledVolumes := make(map[string][]api.VolumeSpec) scheduledVolumes := make(map[string][]api.VolumeSpec)
for missingVolumeName, missingVolumeSpec := range s.volumeSpecs { for volumeName, volumeSpec := range s.volumeSpecs {
// Skip volumes that already exist on machines. existingMachines := s.existingVolumeMachines[volumeName]
if _, ok := s.existingVolumeMachines[missingVolumeName]; ok { serviceNames := s.volumeServices[volumeName]
continue
}
serviceNames := s.volumeServices[missingVolumeName]
if len(serviceNames) == 0 { if len(serviceNames) == 0 {
return nil, fmt.Errorf("bug detected: no services using volume '%s'", missingVolumeName) return nil, fmt.Errorf("bug detected: no services using volume '%s'", volumeName)
} }
// Get the current eligible machines (any service using the volume will have the same set after convergence). // Get the eligible machines for this volume.
eligibleMachines := serviceEligibleMachines[serviceNames[0]] // For volumes used by global services: compute UNION of all services' eligible machines.
// For other volumes: any service will have the same set after constraint convergence.
var eligibleMachines mapset.Set[string]
if s.isVolumeForGlobalService(volumeName) {
// Compute union of eligible machines for all global services using this volume.
eligibleMachines = mapset.NewSet[string]()
for _, serviceName := range serviceNames {
eligibleMachines = eligibleMachines.Union(serviceEligibleMachines[serviceName])
}
} else {
eligibleMachines = serviceEligibleMachines[serviceNames[0]]
}
if eligibleMachines.Cardinality() == 0 { if eligibleMachines.Cardinality() == 0 {
return nil, fmt.Errorf("bug detected: no eligible machines for volume '%s'", missingVolumeName) return nil, fmt.Errorf("bug detected: no eligible machines for volume '%s'", volumeName)
} }
// Choose the first machine in the sorted eligible machines to schedule the volume on.
// Sort the eligible machines to ensure deterministic behavior. // Sort the eligible machines to ensure deterministic behavior.
sortedEligibleMachines := eligibleMachines.ToSlice() sortedEligibleMachines := eligibleMachines.ToSlice()
slices.Sort(sortedEligibleMachines) slices.Sort(sortedEligibleMachines)
machineID := sortedEligibleMachines[0]
// Update constraints for all services that use this volume to be placed on the selected machine.
for _, serviceName := range serviceNames {
serviceEligibleMachines[serviceName] = mapset.NewSet(machineID)
}
placedVolumes[missingVolumeName] = struct{}{}
scheduledVolumes[machineID] = append(scheduledVolumes[machineID], missingVolumeSpec)
// Propagate the updated constraints. if s.isVolumeForGlobalService(volumeName) {
if err := s.propagateConstraintsUntilConvergence(serviceEligibleMachines, placedVolumes); err != nil { // Global service: schedule volume on eligible machines that don't already have it.
return nil, fmt.Errorf("unexpected error while propagating constraints after "+ for _, machineID := range sortedEligibleMachines {
"scheduling volume '%s' on machine '%s': %w", missingVolumeName, machineID, err) if existingMachines != nil && existingMachines.Contains(machineID) {
// Volume already exists on this machine, skip it.
continue
}
scheduledVolumes[machineID] = append(scheduledVolumes[machineID], volumeSpec)
}
// Mark volume as placed - no constraint propagation needed since volume will be on all machines.
placedVolumes[volumeName] = struct{}{}
} else {
// Replicated service: skip if volume already exists on any machine (services will use that location).
if existingMachines != nil && existingMachines.Cardinality() > 0 {
continue
}
// Schedule volume on ONE machine (first in sorted order).
machineID := sortedEligibleMachines[0]
// Update constraints for all services that use this volume to be placed on the selected machine.
for _, serviceName := range serviceNames {
serviceEligibleMachines[serviceName] = mapset.NewSet(machineID)
}
placedVolumes[volumeName] = struct{}{}
scheduledVolumes[machineID] = append(scheduledVolumes[machineID], volumeSpec)
// Propagate the updated constraints.
if err := s.propagateConstraintsUntilConvergence(serviceEligibleMachines, placedVolumes); err != nil {
return nil, fmt.Errorf("unexpected error while propagating constraints after "+
"scheduling volume '%s' on machine '%s': %w", volumeName, machineID, err)
}
} }
} }
@@ -295,3 +352,42 @@ func (s *VolumeScheduler) propagateConstraintsUntilConvergence(
return nil return nil
} }
// isVolumeForGlobalService returns true if any service using this volume is a global service.
func (s *VolumeScheduler) isVolumeForGlobalService(volumeName string) bool {
serviceNames := s.volumeServices[volumeName]
for _, serviceName := range serviceNames {
for _, spec := range s.serviceSpecs {
if spec.Name == serviceName && spec.Mode == api.ServiceModeGlobal {
return true
}
}
}
return false
}
// isVolumeSharedBetweenGlobalAndReplicated returns true if a volume is used by both
// global and replicated services, which is an invalid configuration.
func (s *VolumeScheduler) isVolumeSharedBetweenGlobalAndReplicated(volumeName string) bool {
serviceNames := s.volumeServices[volumeName]
hasGlobal := false
hasReplicated := false
for _, serviceName := range serviceNames {
for _, spec := range s.serviceSpecs {
if spec.Name == serviceName {
mode := spec.Mode
if mode == "" {
mode = api.ServiceModeReplicated
}
if mode == api.ServiceModeGlobal {
hasGlobal = true
} else {
hasReplicated = true
}
}
}
}
return hasGlobal && hasReplicated
}
+390
View File
@@ -802,6 +802,396 @@ func TestVolumeScheduler_Schedule(t *testing.T) {
}, },
}, },
}, },
{
name: "global service with missing volume schedules on all machines",
machines: []*Machine{
{
Info: &pb.MachineInfo{
Id: "machine1",
},
},
{
Info: &pb.MachineInfo{
Id: "machine2",
},
},
{
Info: &pb.MachineInfo{
Id: "machine3",
},
},
},
serviceSpecs: []api.ServiceSpec{
{
Name: "global-service",
Mode: api.ServiceModeGlobal,
Container: api.ContainerSpec{
Image: "portainer/pause:latest",
VolumeMounts: []api.VolumeMount{
{
VolumeName: "vol1",
ContainerPath: "/data",
},
},
},
Volumes: []api.VolumeSpec{
{
Name: "vol1",
Type: api.VolumeTypeVolume,
},
},
},
},
want: map[string][]api.VolumeSpec{
"machine1": {
{
Name: "vol1",
Type: api.VolumeTypeVolume,
},
},
"machine2": {
{
Name: "vol1",
Type: api.VolumeTypeVolume,
},
},
"machine3": {
{
Name: "vol1",
Type: api.VolumeTypeVolume,
},
},
},
},
{
name: "global service with volume on some machines schedules remaining",
machines: []*Machine{
{
Info: &pb.MachineInfo{
Id: "machine1",
},
Volumes: []volume.Volume{
{
Name: "vol1",
},
},
},
{
Info: &pb.MachineInfo{
Id: "machine2",
},
},
{
Info: &pb.MachineInfo{
Id: "machine3",
},
},
},
serviceSpecs: []api.ServiceSpec{
{
Name: "global-service",
Mode: api.ServiceModeGlobal,
Container: api.ContainerSpec{
Image: "portainer/pause:latest",
VolumeMounts: []api.VolumeMount{
{
VolumeName: "vol1",
ContainerPath: "/data",
},
},
},
Volumes: []api.VolumeSpec{
{
Name: "vol1",
Type: api.VolumeTypeVolume,
},
},
},
},
want: map[string][]api.VolumeSpec{
"machine2": {
{
Name: "vol1",
Type: api.VolumeTypeVolume,
},
},
"machine3": {
{
Name: "vol1",
Type: api.VolumeTypeVolume,
},
},
},
},
{
name: "global service with placement constraint",
machines: []*Machine{
{
Info: &pb.MachineInfo{
Id: "machine1",
},
},
{
Info: &pb.MachineInfo{
Id: "machine2",
},
},
{
Info: &pb.MachineInfo{
Id: "machine3",
},
},
},
serviceSpecs: []api.ServiceSpec{
{
Name: "global-service",
Mode: api.ServiceModeGlobal,
Placement: api.Placement{
Machines: []string{"machine1", "machine3"},
},
Container: api.ContainerSpec{
Image: "portainer/pause:latest",
VolumeMounts: []api.VolumeMount{
{
VolumeName: "vol1",
ContainerPath: "/data",
},
},
},
Volumes: []api.VolumeSpec{
{
Name: "vol1",
Type: api.VolumeTypeVolume,
},
},
},
},
want: map[string][]api.VolumeSpec{
"machine1": {
{
Name: "vol1",
Type: api.VolumeTypeVolume,
},
},
"machine3": {
{
Name: "vol1",
Type: api.VolumeTypeVolume,
},
},
},
},
{
name: "volume shared between global and replicated fails",
machines: []*Machine{
{
Info: &pb.MachineInfo{
Id: "machine1",
},
},
{
Info: &pb.MachineInfo{
Id: "machine2",
},
},
},
serviceSpecs: []api.ServiceSpec{
{
Name: "global-service",
Mode: api.ServiceModeGlobal,
Container: api.ContainerSpec{
Image: "portainer/pause:latest",
VolumeMounts: []api.VolumeMount{
{
VolumeName: "shared-vol",
ContainerPath: "/data",
},
},
},
Volumes: []api.VolumeSpec{
{
Name: "shared-vol",
Type: api.VolumeTypeVolume,
},
},
},
{
Name: "replicated-service",
Mode: api.ServiceModeReplicated,
Container: api.ContainerSpec{
Image: "portainer/pause:latest",
VolumeMounts: []api.VolumeMount{
{
VolumeName: "shared-vol",
ContainerPath: "/data",
},
},
},
Volumes: []api.VolumeSpec{
{
Name: "shared-vol",
Type: api.VolumeTypeVolume,
},
},
},
},
wantErr: "volume 'shared-vol' cannot be shared between global and replicated services",
},
{
name: "multiple global services sharing same volume",
machines: []*Machine{
{
Info: &pb.MachineInfo{
Id: "machine1",
},
},
{
Info: &pb.MachineInfo{
Id: "machine2",
},
},
},
serviceSpecs: []api.ServiceSpec{
{
Name: "global-service-1",
Mode: api.ServiceModeGlobal,
Container: api.ContainerSpec{
Image: "portainer/pause:latest",
VolumeMounts: []api.VolumeMount{
{
VolumeName: "shared-vol",
ContainerPath: "/data1",
},
},
},
Volumes: []api.VolumeSpec{
{
Name: "shared-vol",
Type: api.VolumeTypeVolume,
},
},
},
{
Name: "global-service-2",
Mode: api.ServiceModeGlobal,
Container: api.ContainerSpec{
Image: "portainer/pause:latest",
VolumeMounts: []api.VolumeMount{
{
VolumeName: "shared-vol",
ContainerPath: "/data2",
},
},
},
Volumes: []api.VolumeSpec{
{
Name: "shared-vol",
Type: api.VolumeTypeVolume,
},
},
},
},
want: map[string][]api.VolumeSpec{
"machine1": {
{
Name: "shared-vol",
Type: api.VolumeTypeVolume,
},
},
"machine2": {
{
Name: "shared-vol",
Type: api.VolumeTypeVolume,
},
},
},
},
{
name: "multiple global services sharing same volume with different placement constraints",
machines: []*Machine{
{
Info: &pb.MachineInfo{
Id: "machine1",
},
},
{
Info: &pb.MachineInfo{
Id: "machine2",
},
},
{
Info: &pb.MachineInfo{
Id: "machine3",
},
},
},
serviceSpecs: []api.ServiceSpec{
{
Name: "global-service-1",
Mode: api.ServiceModeGlobal,
Placement: api.Placement{
Machines: []string{"machine1", "machine2"},
},
Container: api.ContainerSpec{
Image: "portainer/pause:latest",
VolumeMounts: []api.VolumeMount{
{
VolumeName: "shared-vol",
ContainerPath: "/data1",
},
},
},
Volumes: []api.VolumeSpec{
{
Name: "shared-vol",
Type: api.VolumeTypeVolume,
},
},
},
{
Name: "global-service-2",
Mode: api.ServiceModeGlobal,
Placement: api.Placement{
Machines: []string{"machine2", "machine3"},
},
Container: api.ContainerSpec{
Image: "portainer/pause:latest",
VolumeMounts: []api.VolumeMount{
{
VolumeName: "shared-vol",
ContainerPath: "/data2",
},
},
},
Volumes: []api.VolumeSpec{
{
Name: "shared-vol",
Type: api.VolumeTypeVolume,
},
},
},
},
want: map[string][]api.VolumeSpec{
"machine1": {
{
Name: "shared-vol",
Type: api.VolumeTypeVolume,
},
},
"machine2": {
{
Name: "shared-vol",
Type: api.VolumeTypeVolume,
},
},
"machine3": {
{
Name: "shared-vol",
Type: api.VolumeTypeVolume,
},
},
},
},
} }
for _, tt := range tests { for _, tt := range tests {
+116 -62
View File
@@ -8,6 +8,7 @@ import (
"github.com/psviderski/uncloud/internal/machine/api/pb" "github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/secret" "github.com/psviderski/uncloud/internal/secret"
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client/deploy/operation"
"github.com/psviderski/uncloud/pkg/client/deploy/scheduler" "github.com/psviderski/uncloud/pkg/client/deploy/scheduler"
) )
@@ -28,6 +29,8 @@ type RollingStrategy struct {
// ForceRecreate indicates whether all containers should be recreated during the deployment, // ForceRecreate indicates whether all containers should be recreated during the deployment,
// regardless of whether their specifications have changed. // regardless of whether their specifications have changed.
ForceRecreate bool ForceRecreate bool
// SkipHealthMonitor skips the monitoring period and health checks for faster emergency deployments.
SkipHealthMonitor bool
// state is the current and planned state of the cluster used for scheduling decisions. // state is the current and planned state of the cluster used for scheduling decisions.
state *scheduler.ClusterState state *scheduler.ClusterState
@@ -144,10 +147,11 @@ func (s *RollingStrategy) planReplicated(svc *api.Service, spec api.ServiceSpec)
if len(containers) == 0 { if len(containers) == 0 {
// No more existing containers on this machine, create a new one. // No more existing containers on this machine, create a new one.
plan.Operations = append(plan.Operations, &RunContainerOperation{ plan.Operations = append(plan.Operations, &operation.RunContainerOperation{
ServiceID: plan.ServiceID, ServiceID: plan.ServiceID,
Spec: spec, Spec: spec,
MachineID: m.Id, MachineID: m.Id,
SkipHealthMonitor: s.SkipHealthMonitor,
}) })
continue continue
} }
@@ -160,36 +164,24 @@ func (s *RollingStrategy) planReplicated(svc *api.Service, spec api.ServiceSpec)
continue continue
} }
// TODO: handle ContainerNeedsUpdate when update of mutable fields on a container is supported. // TODO: handle ContainerNeedsUpdate when update of mutable fields on a container is supported.
conflictingPorts, portsErr := ctr.ConflictingServicePorts(spec.Ports)
if portsErr != nil || len(conflictingPorts) > 0 {
// Stop the malformed container or the container with conflicting ports.
plan.Operations = append(plan.Operations, &StopContainerOperation{
ServiceID: plan.ServiceID,
ContainerID: ctr.ID,
MachineID: m.Id,
})
}
} }
// Run a new container. // Replace the old container with a new one.
plan.Operations = append(plan.Operations, &RunContainerOperation{ order := determineUpdateOrder(ctr, spec)
ServiceID: plan.ServiceID, plan.Operations = append(plan.Operations, &operation.ReplaceContainerOperation{
Spec: spec, ServiceID: plan.ServiceID,
MachineID: m.Id, Spec: spec,
}) MachineID: m.Id,
OldContainer: ctr,
// Remove the old container. Order: order,
plan.Operations = append(plan.Operations, &RemoveContainerOperation{ SkipHealthMonitor: s.SkipHealthMonitor,
MachineID: m.Id,
Container: ctr,
}) })
} }
// Remove any remaining containers that are not needed. // Remove any remaining containers that are not needed.
for mid, containers := range containersOnMachine { for mid, containers := range containersOnMachine {
for _, c := range containers { for _, c := range containers {
plan.Operations = append(plan.Operations, &RemoveContainerOperation{ plan.Operations = append(plan.Operations, &operation.RemoveContainerOperation{
MachineID: mid, MachineID: mid,
Container: c, Container: c,
}) })
@@ -228,7 +220,8 @@ func (s *RollingStrategy) planGlobal(svc *api.Service, spec api.ServiceSpec) (Pl
for _, m := range availableMachines { for _, m := range availableMachines {
containers := containersOnMachine[m.Info.Id] containers := containersOnMachine[m.Info.Id]
ops, err := reconcileGlobalContainer(containers, spec, plan.ServiceID, m.Info.Id, s.ForceRecreate) ops, err := reconcileGlobalContainer(
containers, spec, plan.ServiceID, m.Info.Id, s.ForceRecreate, s.SkipHealthMonitor)
if err != nil { if err != nil {
return plan, err return plan, err
} }
@@ -240,7 +233,7 @@ func (s *RollingStrategy) planGlobal(svc *api.Service, spec api.ServiceSpec) (Pl
// Remove any remaining containers on machines that don't match the new placement constraints. // Remove any remaining containers on machines that don't match the new placement constraints.
for _, containers := range containersOnMachine { for _, containers := range containersOnMachine {
for _, c := range containers { for _, c := range containers {
plan.Operations = append(plan.Operations, &RemoveContainerOperation{ plan.Operations = append(plan.Operations, &operation.RemoveContainerOperation{
MachineID: c.MachineID, MachineID: c.MachineID,
Container: c.Container, Container: c.Container,
}) })
@@ -254,16 +247,18 @@ func (s *RollingStrategy) planGlobal(svc *api.Service, spec api.ServiceSpec) (Pl
// It ensures exactly one container with the desired spec is running on the machine by creating a new container and // It ensures exactly one container with the desired spec is running on the machine by creating a new container and
// removing old ones. If there is a host port conflict, it stops the old container before starting a new one. // removing old ones. If there is a host port conflict, it stops the old container before starting a new one.
func reconcileGlobalContainer( func reconcileGlobalContainer(
containers []api.MachineServiceContainer, spec api.ServiceSpec, serviceID, machineID string, forceRecreate bool, containers []api.MachineServiceContainer, spec api.ServiceSpec, serviceID, machineID string,
) ([]Operation, error) { forceRecreate, skipHealthCheck bool,
var ops []Operation ) ([]operation.Operation, error) {
var ops []operation.Operation
if len(containers) == 0 { if len(containers) == 0 {
// No containers on this machine, create a new one. // No containers on this machine, create a new one.
ops = append(ops, &RunContainerOperation{ ops = append(ops, &operation.RunContainerOperation{
ServiceID: serviceID, ServiceID: serviceID,
Spec: spec, Spec: spec,
MachineID: machineID, MachineID: machineID,
SkipHealthMonitor: skipHealthCheck,
}) })
return ops, nil return ops, nil
} }
@@ -290,7 +285,7 @@ func reconcileGlobalContainer(
if i == j { if i == j {
continue continue
} }
ops = append(ops, &RemoveContainerOperation{ ops = append(ops, &operation.RemoveContainerOperation{
MachineID: old.MachineID, MachineID: old.MachineID,
Container: old.Container, Container: old.Container,
}) })
@@ -304,43 +299,102 @@ func reconcileGlobalContainer(
} }
// The machine has containers but none of them match the new spec. // The machine has containers but none of them match the new spec.
// Stop the old running containers that have conflicting ports with the new spec before running a new one. // Find the first running container to replace (there should typically be only one).
for _, c := range containers { var containerToReplace *api.MachineServiceContainer
for i, c := range containers {
if c.Container.State.Running { if c.Container.State.Running {
conflictingPorts, err := c.Container.ConflictingServicePorts(spec.Ports) containerToReplace = &containers[i]
if err != nil { break
return nil, fmt.Errorf("check conflicting ports: %w", err)
}
if len(conflictingPorts) > 0 {
// Stop the running container with conflicting ports.
ops = append(ops, &StopContainerOperation{
ServiceID: serviceID,
ContainerID: c.Container.ID,
MachineID: c.MachineID,
})
}
} }
} }
// Run a new container. if containerToReplace != nil {
ops = append(ops, &RunContainerOperation{ // Stop any other running containers that have conflicting ports before replacing the container.
ServiceID: serviceID, // This handles the edge case where multiple running containers exist (due to bugs or interrupted deployments)
Spec: spec, // and more than one has ports that conflict with the new spec.
MachineID: machineID, for _, c := range containers {
}) if c.Container.ID == containerToReplace.Container.ID || !c.Container.State.Running {
continue
}
conflictingPorts, err := c.Container.ConflictingServicePorts(spec.Ports)
if err != nil || len(conflictingPorts) > 0 {
ops = append(ops, &operation.StopContainerOperation{
ServiceID: serviceID,
ContainerID: c.Container.ID,
MachineID: machineID,
})
}
}
// Remove the old containers. // Replace the running container with a new one.
for _, c := range containers { order := determineUpdateOrder(containerToReplace.Container, spec)
ops = append(ops, &RemoveContainerOperation{ ops = append(ops, &operation.ReplaceContainerOperation{
MachineID: c.MachineID, ServiceID: serviceID,
Container: c.Container, Spec: spec,
MachineID: machineID,
OldContainer: containerToReplace.Container,
Order: order,
SkipHealthMonitor: skipHealthCheck,
}) })
// Remove any other containers (there shouldn't be any in normal operation).
for _, c := range containers {
if c.Container.ID == containerToReplace.Container.ID {
continue
}
ops = append(ops, &operation.RemoveContainerOperation{
MachineID: c.MachineID,
Container: c.Container,
})
}
} else {
// No running containers, create a new one and remove all stopped containers.
ops = append(ops, &operation.RunContainerOperation{
ServiceID: serviceID,
Spec: spec,
MachineID: machineID,
SkipHealthMonitor: skipHealthCheck,
})
for _, c := range containers {
ops = append(ops, &operation.RemoveContainerOperation{
MachineID: c.MachineID,
Container: c.Container,
})
}
} }
return ops, nil return ops, nil
} }
// determineUpdateOrder determines the update order for replacing a container based on the service spec
// and current container state. The order can be explicitly set in UpdateConfig, or automatically determined:
// - If the user explicitly set order, respect it
// - Services with port conflicts require stop-first (ports must be freed first)
// - Single-replica services with data volumes default to stop-first (prevents data corruption)
// - Multi-replica services use start-first (concurrent access already happening)
// - All other services default to start-first (minimizes downtime)
func determineUpdateOrder(oldContainer api.ServiceContainer, spec api.ServiceSpec) string {
// User explicitly set order - respect it
if spec.UpdateConfig.Order != "" {
return spec.UpdateConfig.Order
}
// Port conflicts require stop-first
conflictingPorts, err := oldContainer.ConflictingServicePorts(spec.Ports)
if err != nil || len(conflictingPorts) > 0 {
return api.UpdateOrderStopFirst
}
// Single-replica services with data volumes default to stop-first to prevent data corruption.
// Multi-replica services already have concurrent access, so start-first is safe.
if spec.Replicas <= 1 && len(spec.MountedDockerVolumes()) > 0 {
return api.UpdateOrderStopFirst
}
// Default: start-first for minimal downtime
return api.UpdateOrderStartFirst
}
// newEmptyPlan creates a new empty plan for a service deployment with initialised service ID and name. // newEmptyPlan creates a new empty plan for a service deployment with initialised service ID and name.
func newEmptyPlan(svc *api.Service, spec api.ServiceSpec) (Plan, error) { func newEmptyPlan(svc *api.Service, spec api.ServiceSpec) (Plan, error) {
var plan Plan var plan Plan
+425
View File
@@ -0,0 +1,425 @@
package deploy
import (
"testing"
"github.com/docker/docker/api/types/container"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client/deploy/operation"
"github.com/stretchr/testify/assert"
)
func TestDetermineUpdateOrder(t *testing.T) {
tests := []struct {
name string
oldContainer api.ServiceContainer
spec api.ServiceSpec
expected string
}{
{
name: "explicit stop-first order",
oldContainer: api.ServiceContainer{
Container: api.Container{
InspectResponse: container.InspectResponse{
Config: &container.Config{Labels: map[string]string{}},
},
},
},
spec: api.ServiceSpec{
UpdateConfig: api.UpdateConfig{
Order: api.UpdateOrderStopFirst,
},
},
expected: api.UpdateOrderStopFirst,
},
{
name: "explicit start-first order",
oldContainer: api.ServiceContainer{
Container: api.Container{
InspectResponse: container.InspectResponse{
Config: &container.Config{Labels: map[string]string{}},
},
},
},
spec: api.ServiceSpec{
UpdateConfig: api.UpdateConfig{
Order: api.UpdateOrderStartFirst,
},
},
expected: api.UpdateOrderStartFirst,
},
{
name: "explicit start-first overrides volume default",
oldContainer: api.ServiceContainer{
Container: api.Container{
InspectResponse: container.InspectResponse{
Config: &container.Config{Labels: map[string]string{}},
},
},
},
spec: api.ServiceSpec{
UpdateConfig: api.UpdateConfig{
Order: api.UpdateOrderStartFirst,
},
Volumes: []api.VolumeSpec{
{
Name: "data",
Type: api.VolumeTypeVolume,
VolumeOptions: &api.VolumeOptions{
Name: "data",
},
},
},
Container: api.ContainerSpec{
VolumeMounts: []api.VolumeMount{
{
VolumeName: "data",
ContainerPath: "/data",
},
},
},
},
expected: api.UpdateOrderStartFirst,
},
{
name: "single-replica service with volume defaults to stop-first",
oldContainer: api.ServiceContainer{
Container: api.Container{
InspectResponse: container.InspectResponse{
Config: &container.Config{Labels: map[string]string{}},
},
},
},
spec: api.ServiceSpec{
Replicas: 1,
Volumes: []api.VolumeSpec{
{
Name: "db-data",
Type: api.VolumeTypeVolume,
VolumeOptions: &api.VolumeOptions{
Name: "db-data",
},
},
},
Container: api.ContainerSpec{
VolumeMounts: []api.VolumeMount{
{
VolumeName: "db-data",
ContainerPath: "/var/lib/postgresql/data",
},
},
},
},
expected: api.UpdateOrderStopFirst,
},
{
name: "multi-replica service with volume defaults to start-first",
oldContainer: api.ServiceContainer{
Container: api.Container{
InspectResponse: container.InspectResponse{
Config: &container.Config{Labels: map[string]string{}},
},
},
},
spec: api.ServiceSpec{
Replicas: 3,
Volumes: []api.VolumeSpec{
{
Name: "app-data",
Type: api.VolumeTypeVolume,
VolumeOptions: &api.VolumeOptions{
Name: "app-data",
},
},
},
Container: api.ContainerSpec{
VolumeMounts: []api.VolumeMount{
{
VolumeName: "app-data",
ContainerPath: "/data",
},
},
},
},
expected: api.UpdateOrderStartFirst,
},
{
name: "service with bind mount defaults to start-first",
oldContainer: api.ServiceContainer{
Container: api.Container{
InspectResponse: container.InspectResponse{
Config: &container.Config{Labels: map[string]string{}},
},
},
},
spec: api.ServiceSpec{
Volumes: []api.VolumeSpec{
{
Name: "config",
Type: api.VolumeTypeBind,
BindOptions: &api.BindOptions{
HostPath: "/etc/app/config",
},
},
},
Container: api.ContainerSpec{
VolumeMounts: []api.VolumeMount{
{
VolumeName: "config",
ContainerPath: "/config",
},
},
},
},
expected: api.UpdateOrderStartFirst,
},
{
name: "service with tmpfs mount defaults to start-first",
oldContainer: api.ServiceContainer{
Container: api.Container{
InspectResponse: container.InspectResponse{
Config: &container.Config{Labels: map[string]string{}},
},
},
},
spec: api.ServiceSpec{
Volumes: []api.VolumeSpec{
{
Name: "tmp",
Type: api.VolumeTypeTmpfs,
},
},
Container: api.ContainerSpec{
VolumeMounts: []api.VolumeMount{
{
VolumeName: "tmp",
ContainerPath: "/tmp",
},
},
},
},
expected: api.UpdateOrderStartFirst,
},
{
name: "stateless service defaults to start-first",
oldContainer: api.ServiceContainer{
Container: api.Container{
InspectResponse: container.InspectResponse{
Config: &container.Config{Labels: map[string]string{}},
},
},
},
spec: api.ServiceSpec{
Container: api.ContainerSpec{
Image: "nginx:latest",
},
},
expected: api.UpdateOrderStartFirst,
},
{
name: "port conflict forces stop-first",
oldContainer: api.ServiceContainer{
Container: api.Container{
InspectResponse: container.InspectResponse{
Config: &container.Config{
Labels: map[string]string{
api.LabelServicePorts: `[{"container_port":8080,"published_port":8080,"protocol":"tcp","mode":"host"}]`,
},
},
},
},
},
spec: api.ServiceSpec{
Ports: []api.PortSpec{
{
ContainerPort: 8080,
PublishedPort: 8080,
Protocol: "tcp",
Mode: api.PortModeHost,
},
},
},
expected: api.UpdateOrderStopFirst,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := determineUpdateOrder(tt.oldContainer, tt.spec)
assert.Equal(t, tt.expected, result)
})
}
}
func TestReconcileGlobalContainer(t *testing.T) {
container1 := api.ServiceContainer{
Container: api.Container{
InspectResponse: container.InspectResponse{
ContainerJSONBase: &container.ContainerJSONBase{
ID: "container-1",
State: &container.State{Running: true},
},
Config: &container.Config{
Labels: map[string]string{
api.LabelServicePorts: "8080:8080/tcp@host",
},
},
},
},
}
container2WithPort9090 := api.ServiceContainer{
Container: api.Container{
InspectResponse: container.InspectResponse{
ContainerJSONBase: &container.ContainerJSONBase{
ID: "container-2",
State: &container.State{Running: true},
},
Config: &container.Config{
Labels: map[string]string{
api.LabelServicePorts: "9090:9090/tcp@host",
},
},
},
},
}
container2WithPort3000 := api.ServiceContainer{
Container: api.Container{
InspectResponse: container.InspectResponse{
ContainerJSONBase: &container.ContainerJSONBase{
ID: "container-2",
State: &container.State{Running: true},
},
Config: &container.Config{
Labels: map[string]string{
api.LabelServicePorts: "3000:3000/tcp@host",
},
},
},
},
}
tests := []struct {
name string
containers []api.MachineServiceContainer
spec api.ServiceSpec
forceRecreate bool
expectedOps []operation.Operation
}{
{
name: "no containers creates new",
containers: nil,
spec: api.ServiceSpec{
Container: api.ContainerSpec{Image: "nginx:latest"},
},
expectedOps: []operation.Operation{
&operation.RunContainerOperation{
ServiceID: "service-1",
MachineID: "machine-1",
},
},
},
{
name: "single running container with port conflict uses replace",
containers: []api.MachineServiceContainer{
{MachineID: "machine-1", Container: container1},
},
spec: api.ServiceSpec{
Container: api.ContainerSpec{Image: "nginx:latest"},
Ports: []api.PortSpec{
{ContainerPort: 8080, PublishedPort: 8080, Protocol: "tcp", Mode: api.PortModeHost},
},
},
expectedOps: []operation.Operation{
&operation.ReplaceContainerOperation{
ServiceID: "service-1",
MachineID: "machine-1",
OldContainer: container1,
Order: api.UpdateOrderStopFirst,
},
},
},
{
name: "multiple running containers with different conflicting ports stops extras before replace",
containers: []api.MachineServiceContainer{
{MachineID: "machine-1", Container: container1},
{MachineID: "machine-1", Container: container2WithPort9090},
},
spec: api.ServiceSpec{
Container: api.ContainerSpec{Image: "nginx:latest"},
Ports: []api.PortSpec{
{ContainerPort: 8080, PublishedPort: 8080, Protocol: "tcp", Mode: api.PortModeHost},
{ContainerPort: 9090, PublishedPort: 9090, Protocol: "tcp", Mode: api.PortModeHost},
},
},
expectedOps: []operation.Operation{
&operation.StopContainerOperation{
ServiceID: "service-1",
ContainerID: "container-2",
MachineID: "machine-1",
},
&operation.ReplaceContainerOperation{
ServiceID: "service-1",
MachineID: "machine-1",
OldContainer: container1,
Order: api.UpdateOrderStopFirst,
},
&operation.RemoveContainerOperation{
MachineID: "machine-1",
Container: container2WithPort9090,
},
},
},
{
name: "multiple running containers but only one has port conflict",
containers: []api.MachineServiceContainer{
{MachineID: "machine-1", Container: container1},
{MachineID: "machine-1", Container: container2WithPort3000},
},
spec: api.ServiceSpec{
Container: api.ContainerSpec{Image: "nginx:latest"},
Ports: []api.PortSpec{
{ContainerPort: 8080, PublishedPort: 8080, Protocol: "tcp", Mode: api.PortModeHost},
},
},
// Container-2 has no conflicting ports, so no StopContainerOperation for it.
expectedOps: []operation.Operation{
&operation.ReplaceContainerOperation{
ServiceID: "service-1",
MachineID: "machine-1",
OldContainer: container1,
Order: api.UpdateOrderStopFirst,
},
&operation.RemoveContainerOperation{
MachineID: "machine-1",
Container: container2WithPort3000,
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ops, err := reconcileGlobalContainer(
tt.containers, tt.spec, "service-1", "machine-1", tt.forceRecreate, false,
)
assert.NoError(t, err)
assertOperationsEqual(t, tt.expectedOps, ops)
})
}
}
// assertOperationsEqual compares expected and actual operations, ignoring the Spec field
// which is passed separately to the function and not the focus of these tests.
func assertOperationsEqual(t *testing.T, expected, actual []operation.Operation) {
t.Helper()
opts := cmp.Options{
cmpopts.IgnoreFields(operation.RunContainerOperation{}, "Spec"),
cmpopts.IgnoreFields(operation.ReplaceContainerOperation{}, "Spec"),
cmpopts.IgnoreUnexported(api.Container{}),
}
if diff := cmp.Diff(expected, actual, opts); diff != "" {
t.Errorf("operations mismatch (-expected +actual):\n%s", diff)
}
}
+10 -7
View File
@@ -5,7 +5,9 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"net"
"net/http" "net/http"
"net/netip"
"sync" "sync"
"time" "time"
@@ -61,14 +63,11 @@ func (cli *Client) CreateIngressRecords(ctx context.Context, serviceID string) (
continue continue
} }
wg.Add(1) wg.Go(func() {
go func() {
defer wg.Done()
if err = verifyCaddyReachable(ctx, m.Machine); err == nil { if err = verifyCaddyReachable(ctx, m.Machine); err == nil {
reachableMachines <- m.Machine reachableMachines <- m.Machine
} }
}() })
} }
go func() { go func() {
@@ -111,8 +110,7 @@ func verifyCaddyReachable(ctx context.Context, m *pb.MachineInfo) error {
eventID := fmt.Sprintf("Machine %s (%s)", m.Name, publicIP) eventID := fmt.Sprintf("Machine %s (%s)", m.Name, publicIP)
pw.Event(progress.NewEvent(eventID, progress.Working, "Querying")) pw.Event(progress.NewEvent(eventID, progress.Working, "Querying"))
verifyURL := fmt.Sprintf("http://%s%s", publicIP, caddyconfig.VerifyPath) verifyURL := getVerifyURL(publicIP)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, verifyURL, nil) req, err := http.NewRequestWithContext(ctx, http.MethodGet, verifyURL, nil)
if err != nil { if err != nil {
pw.Event(progress.NewEvent(eventID, progress.Error, err.Error())) pw.Event(progress.NewEvent(eventID, progress.Error, err.Error()))
@@ -173,6 +171,11 @@ func verifyCaddyReachable(ctx context.Context, m *pb.MachineInfo) error {
} }
} }
func getVerifyURL(publicIP netip.Addr) string {
httpFormattedIP := net.JoinHostPort(publicIP.String(), "")
return fmt.Sprintf("http://%s%s", httpFormattedIP, caddyconfig.VerifyPath)
}
// unreachable creates a new Unreachable error event. // unreachable creates a new Unreachable error event.
func unreachable(id string) progress.Event { func unreachable(id string) progress.Event {
return progress.NewEvent( return progress.NewEvent(
+30
View File
@@ -0,0 +1,30 @@
package client
import (
"net/netip"
"testing"
"github.com/stretchr/testify/assert"
)
func TestGetVerifyURL(t *testing.T) {
tests := map[string]struct {
given netip.Addr
want string
}{
"IPv4": {
given: netip.MustParseAddr("93.184.216.34"),
want: "http://93.184.216.34:/.uncloud-verify",
},
"IPv6": {
given: netip.MustParseAddr("2001:db8::1"),
want: "http://[2001:db8::1]:/.uncloud-verify",
},
}
for name, tt := range tests {
t.Run(name, func(t *testing.T) {
got := getVerifyURL(tt.given)
assert.Equal(t, tt.want, got)
})
}
}
+17 -2
View File
@@ -6,6 +6,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"net" "net"
"runtime"
"strconv" "strconv"
"strings" "strings"
"sync" "sync"
@@ -363,14 +364,25 @@ func newUnregistryProxy(
return p, nil return p, nil
} }
// isDockerVirtualised checks if Docker is running in a virtualised environment like Docker Desktop on macOS. // isDockerVirtualised checks if Docker is running in a virtualised environment like Docker/Rancher Desktop or Colima.
// On macOS, Docker always requires a VM, so it returns true unless OrbStack is detected (which handles host networking
// natively). On other platforms, it checks for known virtualised Docker hostnames.
func isDockerVirtualised(ctx context.Context, dockerCli *docker.Client) (bool, error) { func isDockerVirtualised(ctx context.Context, dockerCli *docker.Client) (bool, error) {
info, err := dockerCli.Info(ctx) info, err := dockerCli.Info(ctx)
if err != nil { if err != nil {
return false, fmt.Errorf("get Docker info: %w", err) return false, fmt.Errorf("get Docker info: %w", err)
} }
virtualisedHostnames := []string{"docker-desktop", "colima"} // On macOS, Docker always runs in a VM. OrbStack is the only known exception that doesn't need a proxy.
if runtime.GOOS == "darwin" {
if info.Name == "orbstack" {
return false, nil
}
return true, nil
}
// On other platforms, check for known virtualised Docker environments.
virtualisedHostnames := []string{"docker-desktop", "rancher-desktop", "colima"}
for _, name := range virtualisedHostnames { for _, name := range virtualisedHostnames {
if strings.Contains(strings.ToLower(info.Name), name) { if strings.Contains(strings.ToLower(info.Name), name) {
return true, nil return true, nil
@@ -406,6 +418,9 @@ func runDockerVMProxyContainer(ctx context.Context, dockerCli *docker.Client, ta
ExposedPorts: nat.PortSet{ ExposedPorts: nat.PortSet{
containerPort: {}, containerPort: {},
}, },
Labels: map[string]string{
api.LabelManaged: "",
},
} }
// Get an available port on localhost to bind the container port to by creating a temporary listener and closing it. // Get an available port on localhost to bind the container port to by creating a temporary listener and closing it.
+48
View File
@@ -4,7 +4,9 @@ import (
"context" "context"
"fmt" "fmt"
"strings" "strings"
"time"
"github.com/cenkalti/backoff/v4"
"github.com/psviderski/uncloud/internal/machine/api/pb" "github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
"google.golang.org/grpc/codes" "google.golang.org/grpc/codes"
@@ -13,6 +15,7 @@ import (
) )
func (cli *Client) InspectMachine(ctx context.Context, nameOrID string) (*pb.MachineMember, error) { func (cli *Client) InspectMachine(ctx context.Context, nameOrID string) (*pb.MachineMember, error) {
// TODO: refactor to use MachineClient.InspectMachine.
machines, err := cli.ListMachines(ctx, nil) machines, err := cli.ListMachines(ctx, nil)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -99,3 +102,48 @@ func (cli *Client) RenameMachine(ctx context.Context, nameOrID, newName string)
return cli.UpdateMachine(ctx, req) return cli.UpdateMachine(ctx, req)
} }
// WaitMachineReady waits for the machine API on the connected machine to respond.
func (cli *Client) WaitMachineReady(ctx context.Context, timeout time.Duration) error {
boff := backoff.WithContext(backoff.NewExponentialBackOff(
backoff.WithInitialInterval(100*time.Millisecond),
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(timeout),
), ctx)
inspect := func() error {
if _, err := cli.Inspect(ctx, &emptypb.Empty{}); err != nil {
return fmt.Errorf("inspect machine: %w", err)
}
return nil
}
return backoff.Retry(inspect, boff)
}
// WaitClusterReady waits for the connected machine to be ready to server cluster requests.
func (cli *Client) WaitClusterReady(ctx context.Context, timeout time.Duration) error {
// Backoff is not really needed here as the default service config for the gRPC client is already
// doing retries with backoff for Unavailable errors. However, it's still convenient to use backoff
// to control the overall timeout for the operation.
boff := backoff.WithContext(backoff.NewExponentialBackOff(
backoff.WithInitialInterval(100*time.Millisecond),
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(timeout),
), ctx)
listMachines := func() error {
_, err := cli.ListMachines(ctx, nil)
if err != nil {
if s, ok := status.FromError(err); ok &&
// TODO: remove FailedPrecondition after releading 0.17.
(s.Code() == codes.Unavailable || s.Code() == codes.FailedPrecondition) {
// Machine is not ready yet, retry.
return err
}
// Other non-Unavailable errors should not be retried.
return backoff.Permanent(err)
}
return nil
}
return backoff.Retry(listMachines, boff)
}
+6 -18
View File
@@ -240,11 +240,7 @@ func (cli *Client) RemoveService(ctx context.Context, id string) error {
// Remove all containers on all machines that belong to the service. // Remove all containers on all machines that belong to the service.
for _, mc := range svc.Containers { for _, mc := range svc.Containers {
wg.Add(1) wg.Go(func() {
go func() {
defer wg.Done()
err := cli.StopContainer(ctx, svc.ID, mc.Container.ID, container.StopOptions{}) err := cli.StopContainer(ctx, svc.ID, mc.Container.ID, container.StopOptions{})
if err != nil { if err != nil {
errCh <- fmt.Errorf("stop container '%s': %w", mc.Container.ID, err) errCh <- fmt.Errorf("stop container '%s': %w", mc.Container.ID, err)
@@ -258,7 +254,7 @@ func (cli *Client) RemoveService(ctx context.Context, id string) error {
if err != nil && !errors.Is(err, api.ErrNotFound) { if err != nil && !errors.Is(err, api.ErrNotFound) {
errCh <- fmt.Errorf("remove container '%s': %w", mc.Container.ID, err) errCh <- fmt.Errorf("remove container '%s': %w", mc.Container.ID, err)
} }
}() })
} }
go func() { go func() {
@@ -286,16 +282,12 @@ func (cli *Client) StopService(ctx context.Context, id string, opts container.St
// Stop all containers on all machines that belong to the service. // Stop all containers on all machines that belong to the service.
for _, mc := range svc.Containers { for _, mc := range svc.Containers {
wg.Add(1) wg.Go(func() {
go func() {
defer wg.Done()
err := cli.StopContainer(ctx, svc.ID, mc.Container.ID, opts) err := cli.StopContainer(ctx, svc.ID, mc.Container.ID, opts)
if err != nil { if err != nil {
errCh <- fmt.Errorf("stop container '%s': %w", mc.Container.ID, err) errCh <- fmt.Errorf("stop container '%s': %w", mc.Container.ID, err)
} }
}() })
} }
go func() { go func() {
@@ -323,16 +315,12 @@ func (cli *Client) StartService(ctx context.Context, id string) error {
// Start all containers on all machines that belong to the service. // Start all containers on all machines that belong to the service.
for _, mc := range svc.Containers { for _, mc := range svc.Containers {
wg.Add(1) wg.Go(func() {
go func() {
defer wg.Done()
err := cli.StartContainer(ctx, svc.ID, mc.Container.ID) err := cli.StartContainer(ctx, svc.ID, mc.Container.ID)
if err != nil { if err != nil {
errCh <- fmt.Errorf("start container '%s': %w", mc.Container.ID, err) errCh <- fmt.Errorf("start container '%s': %w", mc.Container.ID, err)
} }
}() })
} }
go func() { go func() {
+1 -1
View File
@@ -338,7 +338,7 @@ if [ "$DOCKER_ALREADY_INSTALLED" = "true" ] && [ "$CONTAINERD_IMAGE_STORE_ENABLE
echo "" echo ""
warning "Docker was already installed on the machine but it doesn't use the containerd" warning "Docker was already installed on the machine but it doesn't use the containerd"
warning "image store. Uncloud works best with the containerd image store enabled in Docker." warning "image store. Uncloud works best with the containerd image store enabled in Docker."
warning "It allows Docker to directly use the images stores in containerd (pushed with" warning "It allows Docker to directly use the images stored in containerd (pushed with"
warning "'uc image push') without duplicating them in Docker. This saves disk space and" warning "'uc image push') without duplicating them in Docker. This saves disk space and"
warning "makes image management more efficient." warning "makes image management more efficient."
echo "" echo ""
+30
View File
@@ -58,6 +58,36 @@ func assertContainerMatchesSpec(t *testing.T, ctr api.ServiceContainer, spec api
assert.Contains(t, ctr.Config.Env, env) assert.Contains(t, ctr.Config.Env, env)
} }
// Healthcheck can only be compared if set in the spec. Otherwise, the container inherits it from the image.
if spec.Container.Healthcheck != nil {
hc := spec.Container.Healthcheck
require.NotNil(t, ctr.Config.Healthcheck)
if hc.Disable {
assert.Equal(t, []string{"NONE"}, ctr.Config.Healthcheck.Test)
} else {
// Only compare fields that are explicitly set in the spec as unset fields inherit their values
// from the image.
if hc.Test != nil {
assert.EqualValues(t, hc.Test, ctr.Config.Healthcheck.Test)
}
if hc.Interval != 0 {
assert.Equal(t, hc.Interval, ctr.Config.Healthcheck.Interval)
}
if hc.Timeout != 0 {
assert.Equal(t, hc.Timeout, ctr.Config.Healthcheck.Timeout)
}
if hc.StartPeriod != 0 {
assert.Equal(t, hc.StartPeriod, ctr.Config.Healthcheck.StartPeriod)
}
if hc.StartInterval != 0 {
assert.Equal(t, hc.StartInterval, ctr.Config.Healthcheck.StartInterval)
}
if hc.Retries != 0 {
assert.Equal(t, int(hc.Retries), ctr.Config.Healthcheck.Retries)
}
}
}
assert.Equal(t, spec.Container.Image, ctr.Config.Image) assert.Equal(t, spec.Container.Image, ctr.Config.Image)
assert.Equal(t, spec.Container.Init, ctr.HostConfig.Init) assert.Equal(t, spec.Container.Init, ctr.HostConfig.Init)
assert.True(t, strings.HasPrefix(ctr.Name, spec.Name+"-")) assert.True(t, strings.HasPrefix(ctr.Name, spec.Name+"-"))
+5 -7
View File
@@ -55,7 +55,7 @@ func createTestCluster(
}) })
if waitReady { if waitReady {
require.NoError(t, p.WaitClusterReady(ctx, c, 15*time.Second)) require.NoError(t, p.WaitClusterReady(ctx, c, 30*time.Second))
} }
return c, p return c, p
@@ -73,7 +73,6 @@ func TestClusterLifecycle(t *testing.T) {
// Create a client for each machine and wait for it to be ready. // Create a client for each machine and wait for it to be ready.
clients := make([]*client.Client, len(c.Machines)) clients := make([]*client.Client, len(c.Machines))
for i, m := range c.Machines { for i, m := range c.Machines {
require.NoError(t, ucind.WaitMachineReady(ctx, m, 5*time.Second))
clients[i], err = m.Connect(ctx) clients[i], err = m.Connect(ctx)
require.NoError(t, err) require.NoError(t, err)
//goland:noinspection GoDeferInLoop //goland:noinspection GoDeferInLoop
@@ -86,11 +85,10 @@ func TestClusterLifecycle(t *testing.T) {
require.Eventually(t, func() bool { require.Eventually(t, func() bool {
machines, err := cli.ListMachines(ctx, nil) machines, err := cli.ListMachines(ctx, nil)
if err != nil { if err != nil {
// FailedPrecondition "cluster is not initialised" is expected until the store is reconciled. // Unavailable "machine is not ready to serve cluster requests" is expected until
if s, ok := status.FromError(err); ok { // the store is reconciled.
if s.Code() == codes.FailedPrecondition { if s, ok := status.FromError(err); ok && s.Code() == codes.Unavailable {
return false return false
}
} }
require.NoError(t, err) require.NoError(t, err)
} }
+14
View File
@@ -66,6 +66,11 @@ func TestComposeConfigs(t *testing.T) {
Gid: "1000", Gid: "1000",
Mode: func() *os.FileMode { m := os.FileMode(0o600); return &m }(), Mode: func() *os.FileMode { m := os.FileMode(0o600); return &m }(),
}, },
{
ConfigName: "from-file",
ContainerPath: "/etc/new-dir/config-from-file.conf",
Mode: func() *os.FileMode { m := os.FileMode(0o644); return &m }(),
},
}, },
}, },
Configs: []api.ConfigSpec{ Configs: []api.ConfigSpec{
@@ -110,5 +115,14 @@ func TestComposeConfigs(t *testing.T) {
userId: 1000, userId: 1000,
groupId: 1000, groupId: 1000,
}, configContentSecond) }, configContentSecond)
configContentThird, err := readFileInfoInContainer(t, cli, name, containerName, "/etc/new-dir/config-from-file.conf")
require.NoError(t, err)
assert.Equal(t, fileInfo{
permissions: 0o644,
content: "this is file config\n",
userId: 0,
groupId: 0,
}, configContentThird, "Same config should be mountable to multiple paths, including nested directories")
}) })
} }
+41
View File
@@ -558,4 +558,45 @@ volumes:
assert.Len(t, plan.Operations, 2, "Expected 1 volume creation and 1 service to deploy") assert.Len(t, plan.Operations, 2, "Expected 1 volume creation and 1 service to deploy")
}) })
t.Run("global service auto-creates volumes on all machines", func(t *testing.T) {
t.Parallel()
serviceName := "test-compose-global-volume"
volumeName := serviceName
t.Cleanup(func() {
removeServices(t, cli, serviceName)
for _, machine := range c.Machines {
_ = cli.RemoveVolume(ctx, machine.Name, volumeName, false)
}
})
project, err := compose.LoadProject(ctx, []string{"fixtures/compose-global-volume.yaml"})
require.NoError(t, err)
deployment, err := compose.NewDeployment(ctx, cli, project)
require.NoError(t, err)
err = deployment.Run(ctx)
require.NoError(t, err, "Global deployment should auto-create volumes on all machines")
// Verify volumes were created on all machines.
volumes, err := cli.ListVolumes(ctx, &api.VolumeFilter{Names: []string{volumeName}})
require.NoError(t, err)
assert.Len(t, volumes, len(c.Machines), "Volume should be created on all machines")
// Verify containers are running on all machines.
svc, err := cli.InspectService(ctx, serviceName)
require.NoError(t, err)
assert.Equal(t, api.ServiceModeGlobal, svc.Mode)
assert.Len(t, svc.Containers, len(c.Machines), "Container should be running on all machines")
machines := serviceMachines(svc)
expectedMachines := make([]string, len(c.Machines))
for i, m := range c.Machines {
expectedMachines[i] = m.ID
}
assert.ElementsMatch(t, machines.ToSlice(), expectedMachines,
"Containers should be distributed across all machines")
})
} }
+4
View File
@@ -11,6 +11,10 @@ services:
uid: "1000" uid: "1000"
gid: "1000" gid: "1000"
mode: 0600 mode: 0600
# Writing a config to a new, non-existent directory
- source: from-file
target: /etc/new-dir/config-from-file.conf
mode: 0644
deploy: deploy:
replicas: 1 replicas: 1
configs: configs:
@@ -0,0 +1,10 @@
services:
test-compose-global-volume:
image: portainer/pause:latest
deploy:
mode: global
volumes:
- test-compose-global-volume:/data
volumes:
test-compose-global-volume:
-1
View File
@@ -71,7 +71,6 @@ func readFileInfoInContainer(t *testing.T, cli *client.Client, serviceNameOrID,
// Parse permissions // Parse permissions
permissions := strings.TrimSpace(permOutput) permissions := strings.TrimSpace(permOutput)
fmt.Printf("Permissions output: %s\n", permissions)
// Parse three numbers: permissions, uid, gid // Parse three numbers: permissions, uid, gid
var permissionsOctal, uid, gid int var permissionsOctal, uid, gid int
+15
View File
@@ -0,0 +1,15 @@
package e2e
import (
"os"
"testing"
"github.com/psviderski/uncloud/pkg/api"
)
func TestMain(m *testing.M) {
// Disable the default health monitor period to speed up tests. Tests that need a non-zero monitor period
// should set it explicitly via UpdateConfig.MonitorPeriod in their service spec.
api.DefaultHealthMonitorPeriod = 0
os.Exit(m.Run())
}
+234 -7
View File
@@ -56,11 +56,16 @@ func TestDeployment(t *testing.T) {
require.ErrorIs(t, err, api.ErrNotFound) require.ErrorIs(t, err, api.ErrNotFound)
}) })
// Explicit short period as the default 5s is disabled in tests for faster test execution.
monitorPeriod := 1 * time.Second
spec := api.ServiceSpec{ spec := api.ServiceSpec{
Mode: api.ServiceModeGlobal, Mode: api.ServiceModeGlobal,
Container: api.ContainerSpec{ Container: api.ContainerSpec{
Image: "portainer/pause:latest", Image: "portainer/pause:latest",
}, },
UpdateConfig: api.UpdateConfig{
MonitorPeriod: &monitorPeriod,
},
} }
deployment := cli.NewDeployment(spec, nil) deployment := cli.NewDeployment(spec, nil)
@@ -73,7 +78,9 @@ func TestDeployment(t *testing.T) {
assert.NotEmpty(t, plan.ServiceName) assert.NotEmpty(t, plan.ServiceName)
assert.Len(t, plan.SequenceOperation.Operations, 3) // 3 run assert.Len(t, plan.SequenceOperation.Operations, 3) // 3 run
start := time.Now()
runPlan, err := deployment.Run(ctx) runPlan, err := deployment.Run(ctx)
duration := time.Since(start)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, plan, runPlan) assert.Equal(t, plan, runPlan)
@@ -88,6 +95,13 @@ func TestDeployment(t *testing.T) {
machines := serviceMachines(svc) machines := serviceMachines(svc)
assert.Len(t, machines.ToSlice(), 3, "Expected 1 container on each machine") assert.Len(t, machines.ToSlice(), 3, "Expected 1 container on each machine")
for _, ctr := range svc.Containers {
assert.True(t, ctr.Container.Healthy(), "Expected deployed containers to be healthy")
}
assert.True(t, duration >= 3*monitorPeriod,
"Expected deployment to wait for at least the health monitor period for each container "+
"before checking health")
// Deploy a published port. // Deploy a published port.
initialContainers := serviceContainerIDs(svc) initialContainers := serviceContainerIDs(svc)
@@ -105,14 +119,19 @@ func TestDeployment(t *testing.T) {
Mode: api.PortModeHost, Mode: api.PortModeHost,
}, },
}, },
UpdateConfig: api.UpdateConfig{
MonitorPeriod: &monitorPeriod,
},
} }
deployment = cli.NewDeployment(specWithPort, nil) deployment = cli.NewDeployment(specWithPort, nil)
plan, err = deployment.Plan(ctx) plan, err = deployment.Plan(ctx)
require.NoError(t, err) require.NoError(t, err)
assert.Len(t, plan.SequenceOperation.Operations, 6) // 3 run + 3 remove assert.Len(t, plan.SequenceOperation.Operations, 3) // 3 replace
start = time.Now()
_, err = deployment.Run(ctx) _, err = deployment.Run(ctx)
duration = time.Since(start)
require.NoError(t, err) require.NoError(t, err)
svc, err = cli.InspectService(ctx, name) svc, err = cli.InspectService(ctx, name)
@@ -126,6 +145,13 @@ func TestDeployment(t *testing.T) {
assert.Empty(t, initialContainers.Intersect(containers).ToSlice(), assert.Empty(t, initialContainers.Intersect(containers).ToSlice(),
"All existing containers should be replaced") "All existing containers should be replaced")
for _, ctr := range svc.Containers {
assert.True(t, ctr.Container.Healthy(), "Expected redeployed containers to be healthy")
}
assert.True(t, duration >= 3*monitorPeriod,
"Expected deployment to wait for at least the health monitor period for each container "+
"before checking health")
// Deploy the same conflicting port but with container spec changes // Deploy the same conflicting port but with container spec changes
initialContainers = containers initialContainers = containers
@@ -150,7 +176,7 @@ func TestDeployment(t *testing.T) {
plan, err = deployment.Plan(ctx) plan, err = deployment.Plan(ctx)
require.NoError(t, err) require.NoError(t, err)
assert.Len(t, plan.SequenceOperation.Operations, 9) // 3 stop + 3 run + 3 remove assert.Len(t, plan.SequenceOperation.Operations, 3) // 3 replace (stop-first due to port conflicts)
_, err = deployment.Run(ctx) _, err = deployment.Run(ctx)
require.NoError(t, err) require.NoError(t, err)
@@ -293,7 +319,7 @@ func TestDeployment(t *testing.T) {
config, err := cli.Caddy.GetConfig(ctx, nil) config, err := cli.Caddy.GetConfig(ctx, nil)
require.NoError(t, err) require.NoError(t, err)
assert.Contains(t, config.Caddyfile, "# This file is autogenerated by Uncloud") assert.Contains(t, config.Caddyfile, "# Caddyfile autogenerated by Uncloud")
assert.Contains(t, config.Caddyfile, "handle /.uncloud-verify") assert.Contains(t, config.Caddyfile, "handle /.uncloud-verify")
}) })
@@ -423,7 +449,7 @@ myapp.example.com {
}, 5*time.Second, 100*time.Millisecond, }, 5*time.Second, 100*time.Millisecond,
"Expected both custom configs to be included in the Caddyfile") "Expected both custom configs to be included in the Caddyfile")
assert.Contains(t, config.Caddyfile, "# This file is autogenerated by Uncloud") assert.Contains(t, config.Caddyfile, "# Caddyfile autogenerated by Uncloud")
assert.Contains(t, config.Caddyfile, "handle /.uncloud-verify") assert.Contains(t, config.Caddyfile, "handle /.uncloud-verify")
assert.Contains(t, config.Caddyfile, caddyCaddyfile, assert.Contains(t, config.Caddyfile, caddyCaddyfile,
"Expected user-defined global Caddy config to be included in the Caddyfile") "Expected user-defined global Caddy config to be included in the Caddyfile")
@@ -480,7 +506,11 @@ myapp.example.com {
// Check that the Caddy config hasn't changed. // Check that the Caddy config hasn't changed.
newConfig, err := cli.Caddy.GetConfig(ctx, nil) newConfig, err := cli.Caddy.GetConfig(ctx, nil)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, validConfig, newConfig.Caddyfile,
// Compare stable parts of the Caddyfile only (skip autogenerated comment with timestamp).
_, stableValidConfig, _ := strings.Cut(validConfig, "\n")
_, stableNewConfig, _ := strings.Cut(newConfig.Caddyfile, "\n")
assert.Equal(t, stableValidConfig, stableNewConfig,
"Caddy config should not change when an invalid user-defined Caddy config is deployed") "Caddy config should not change when an invalid user-defined Caddy config is deployed")
}) })
@@ -496,6 +526,8 @@ myapp.example.com {
}) })
// 1. Create a basic replicated service with 2 replicas. // 1. Create a basic replicated service with 2 replicas.
// Explicit short period as the default 5s is disabled in tests for faster test execution.
monitorPeriod := 1 * time.Second
spec := api.ServiceSpec{ spec := api.ServiceSpec{
Name: name, Name: name,
Mode: api.ServiceModeReplicated, Mode: api.ServiceModeReplicated,
@@ -503,6 +535,9 @@ myapp.example.com {
Image: "portainer/pause:latest", Image: "portainer/pause:latest",
}, },
Replicas: 2, Replicas: 2,
UpdateConfig: api.UpdateConfig{
MonitorPeriod: &monitorPeriod,
},
} }
deployment := cli.NewDeployment(spec, nil) deployment := cli.NewDeployment(spec, nil)
@@ -515,7 +550,9 @@ myapp.example.com {
assert.Equal(t, name, plan.ServiceName) assert.Equal(t, name, plan.ServiceName)
assert.Len(t, plan.SequenceOperation.Operations, 2) // 2 run operations for 2 replicas assert.Len(t, plan.SequenceOperation.Operations, 2) // 2 run operations for 2 replicas
start := time.Now()
runPlan, err := deployment.Run(ctx) runPlan, err := deployment.Run(ctx)
duration := time.Since(start)
require.NoError(t, err) require.NoError(t, err)
assert.Equal(t, plan, runPlan) assert.Equal(t, plan, runPlan)
@@ -529,6 +566,13 @@ myapp.example.com {
assert.Len(t, initialMachines.ToSlice(), 2, "Expected 2 containers on 2 different machines") assert.Len(t, initialMachines.ToSlice(), 2, "Expected 2 containers on 2 different machines")
initialContainers := serviceContainerIDs(svc) initialContainers := serviceContainerIDs(svc)
for _, ctr := range svc.Containers {
assert.True(t, ctr.Container.Healthy(), "Expected deployed containers to be healthy")
}
assert.True(t, duration >= 2*monitorPeriod,
"Expected deployment to wait for at least the health monitor period for each container "+
"before checking health")
// 2. Update the service with a new configuration. // 2. Update the service with a new configuration.
init := true init := true
updatedSpec := spec updatedSpec := spec
@@ -537,9 +581,11 @@ myapp.example.com {
plan, err = deployment.Plan(ctx) plan, err = deployment.Plan(ctx)
require.NoError(t, err) require.NoError(t, err)
assert.Len(t, plan.Operations, 4, "Expected 2 run + 2 remove operations") assert.Len(t, plan.Operations, 2, "Expected 2 replace operations")
start = time.Now()
_, err = deployment.Run(ctx) _, err = deployment.Run(ctx)
duration = time.Since(start)
require.NoError(t, err) require.NoError(t, err)
svc, err = cli.InspectService(ctx, name) svc, err = cli.InspectService(ctx, name)
@@ -554,6 +600,13 @@ myapp.example.com {
assert.Empty(t, initialContainers.Intersect(containers).ToSlice(), assert.Empty(t, initialContainers.Intersect(containers).ToSlice(),
"All existing containers should be replaced") "All existing containers should be replaced")
for _, ctr := range svc.Containers {
assert.True(t, ctr.Container.Healthy(), "Expected deployed containers to be healthy")
}
assert.True(t, duration >= 2*monitorPeriod,
"Expected deployment to wait for at least the health monitor period for each container "+
"before checking health")
// 3. Scale to 3 replicas. // 3. Scale to 3 replicas.
initialMachines = machines initialMachines = machines
initialContainers = containers // Reset container tracking. initialContainers = containers // Reset container tracking.
@@ -589,7 +642,7 @@ myapp.example.com {
plan, err = deployment.Plan(ctx) plan, err = deployment.Plan(ctx)
require.NoError(t, err) require.NoError(t, err)
assert.Len(t, plan.Operations, 8, "Expected 5 run + 3 remove operations") assert.Len(t, plan.Operations, 5, "Expected 3 replace + 2 run operations")
_, err = deployment.Run(ctx) _, err = deployment.Run(ctx)
require.NoError(t, err) require.NoError(t, err)
@@ -1209,6 +1262,9 @@ myapp.example.com {
Container: api.ContainerSpec{ Container: api.ContainerSpec{
Image: uniqueImage, Image: uniqueImage,
}, },
Placement: api.Placement{
Machines: []string{c.Machines[0].Name, c.Machines[1].Name},
},
Replicas: 2, Replicas: 2,
} }
@@ -1260,6 +1316,168 @@ myapp.example.com {
} }
}) })
t.Run("healthcheck becomes healthy", func(t *testing.T) {
t.Parallel()
name := "test-health-ok"
t.Cleanup(func() {
err := cli.RemoveService(ctx, name)
if !errors.Is(err, api.ErrNotFound) {
require.NoError(t, err)
}
})
monitorPeriod := 60 * time.Second
spec := api.ServiceSpec{
Name: name,
Mode: api.ServiceModeReplicated,
Container: api.ContainerSpec{
Image: "busybox:1.37.0-musl",
Command: []string{"sh", "-c", "sleep 3600"},
Healthcheck: &api.HealthcheckSpec{
Test: []string{"CMD-SHELL", "exit 0"},
Interval: 1 * time.Second,
Retries: 2,
},
},
UpdateConfig: api.UpdateConfig{
MonitorPeriod: &monitorPeriod,
},
}
deployment := cli.NewDeployment(spec, nil)
start := time.Now()
_, err := deployment.Run(ctx)
duration := time.Since(start)
require.NoError(t, err)
svc, err := cli.InspectService(ctx, name)
require.NoError(t, err)
assertServiceMatchesSpec(t, svc, spec)
assert.True(t, svc.Containers[0].Container.Healthy())
assert.True(t, duration >= 1*time.Second,
"Deployment should wait for at least one health check interval before marking container as healthy")
assert.True(t, duration < monitorPeriod,
"Deployment should mark container as healthy after first successful health check "+
"and not wait for the entire monitor period")
})
t.Run("healthcheck becomes unhealthy", func(t *testing.T) {
t.Parallel()
name := "test-health-fail"
t.Cleanup(func() {
err := cli.RemoveService(ctx, name)
if !errors.Is(err, api.ErrNotFound) {
require.NoError(t, err)
}
})
monitorPeriod := 1 * time.Second
spec := api.ServiceSpec{
Name: name,
Mode: api.ServiceModeReplicated,
Container: api.ContainerSpec{
Image: "busybox:1.37.0-musl",
Command: []string{"sh", "-c", "sleep 3600"},
Healthcheck: &api.HealthcheckSpec{
Test: []string{"CMD-SHELL", "exit 1"},
Interval: 1 * time.Second,
Retries: 2,
},
},
UpdateConfig: api.UpdateConfig{
MonitorPeriod: &monitorPeriod,
},
}
deployment := cli.NewDeployment(spec, nil)
start := time.Now()
_, err := deployment.Run(ctx)
duration := time.Since(start)
require.Error(t, err)
assert.ErrorContains(t, err, "unhealthy")
assert.True(t, duration >= monitorPeriod,
"Deployment should wait for at least the monitor period before checking health status")
})
t.Run("container crashes on startup with healthcheck", func(t *testing.T) {
t.Parallel()
name := "test-crash-startup-healthcheck"
t.Cleanup(func() {
err := cli.RemoveService(ctx, name)
if !errors.Is(err, api.ErrNotFound) {
require.NoError(t, err)
}
})
monitorPeriod := 1 * time.Second
spec := api.ServiceSpec{
Name: name,
Mode: api.ServiceModeReplicated,
Container: api.ContainerSpec{
Image: "busybox:1.37.0-musl",
Command: []string{"false"},
Healthcheck: &api.HealthcheckSpec{
Test: []string{"CMD-SHELL", "exit 0"},
Interval: 1 * time.Second,
Retries: 2,
},
},
UpdateConfig: api.UpdateConfig{
MonitorPeriod: &monitorPeriod,
},
}
deployment := cli.NewDeployment(spec, nil)
start := time.Now()
_, err := deployment.Run(ctx)
duration := time.Since(start)
require.Error(t, err)
assert.ErrorContains(t, err, "restarting")
assert.True(t, duration >= monitorPeriod,
"Deployment should wait for at least the monitor period before marking container as unhealthy")
})
t.Run("container crashes on startup without healthcheck", func(t *testing.T) {
t.Parallel()
name := "test-crash-startup-no-healthcheck"
t.Cleanup(func() {
err := cli.RemoveService(ctx, name)
if !errors.Is(err, api.ErrNotFound) {
require.NoError(t, err)
}
})
monitorPeriod := 1 * time.Second
spec := api.ServiceSpec{
Name: name,
Mode: api.ServiceModeReplicated,
Container: api.ContainerSpec{
Image: "busybox:1.37.0-musl",
Command: []string{"false"},
},
UpdateConfig: api.UpdateConfig{
MonitorPeriod: &monitorPeriod,
},
}
deployment := cli.NewDeployment(spec, nil)
start := time.Now()
_, err := deployment.Run(ctx)
duration := time.Since(start)
require.Error(t, err)
assert.ErrorContains(t, err, "restarting")
assert.True(t, duration >= monitorPeriod,
"Deployment should wait for at least the monitor period before marking container as unhealthy")
})
// TODO: test deployments with unreachable machines. See https://github.com/psviderski/uncloud/issues/29. // TODO: test deployments with unreachable machines. See https://github.com/psviderski/uncloud/issues/29.
} }
@@ -1311,6 +1529,7 @@ func TestServiceLifecycle(t *testing.T) {
Name: "container-spec-full", Name: "container-spec-full",
Mode: api.ServiceModeGlobal, Mode: api.ServiceModeGlobal,
Container: api.ContainerSpec{ Container: api.ContainerSpec{
// TODO: Add the latest implemented fields to this spec and update assertContainerMatchesSpec.
Command: []string{"sleep", "infinity"}, Command: []string{"sleep", "infinity"},
// Extra slashes is not a typo, it changes the spec but Linux ignores them and uses the default /pause. // Extra slashes is not a typo, it changes the spec but Linux ignores them and uses the default /pause.
Entrypoint: []string{"///pause"}, Entrypoint: []string{"///pause"},
@@ -1320,6 +1539,14 @@ func TestServiceLifecycle(t *testing.T) {
"BOOL": "true", "BOOL": "true",
"": "ignored", "": "ignored",
}, },
Healthcheck: &api.HealthcheckSpec{
Test: []string{"CMD-SHELL", "exit 0"},
Interval: 1*time.Minute + 30*time.Second,
Timeout: 10 * time.Second,
Retries: 5,
StartPeriod: 15 * time.Second,
StartInterval: 2 * time.Second,
},
Image: "portainer/pause:latest", Image: "portainer/pause:latest",
Init: &init, Init: &init,
LogDriver: &api.LogDriver{ LogDriver: &api.LogDriver{
+3
View File
@@ -20,3 +20,6 @@ services:
log log
} }
scale: 2 scale: 2
x-machines:
- uc-prod-us2
- uc-prod-ap1
@@ -4,6 +4,14 @@ In this guide, we'll deploy [Excalidraw](https://excalidraw.com) — a popular s
Linux server. You'll learn the **basics of Uncloud** and see how simple it is to **run web apps** on your own Linux server. You'll learn the **basics of Uncloud** and see how simple it is to **run web apps** on your own
infrastructure with secure internet access. infrastructure with secure internet access.
:::info NOTE
To give you a chance to play with Uncloud without even leaving your browser or needing your own servers, we're providing interactive tutorials and playgrounds on the [iximiuz Labs](https://labs.iximiuz.com/) platform.
You can follow [this tutorial](https://labs.iximiuz.com/tutorials/uncloud-create-cluster-ebebf72b) which walks you through creating a new cluster with two machines and then deploying a simple web service to it.
You can also launch the [Uncloud playground](https://labs.iximiuz.com/playgrounds/uncloud-cluster-64523f7c) where you can play with an already initialized Uncloud cluster.
:::
## Prerequisites ## Prerequisites
Before you begin, you'll need: Before you begin, you'll need:
@@ -10,7 +10,7 @@ with [Let's Encrypt](https://letsencrypt.org/), and route requests to your servi
## How it works ## How it works
By default, Caddy runs as a global service `caddy` on every machine in your cluster, listening on the host ports 80 By default, Caddy runs as a global service `caddy` on every machine in your cluster, listening on the host ports 80
(HTTP) and 443 (HTTPS). (HTTP), 443 (HTTPS), and 443/UDP (HTTP/3).
It's deployed during cluster initialisation (`uc machine init`) unless you use the `--no-caddy` flag. It's deployed during cluster initialisation (`uc machine init`) unless you use the `--no-caddy` flag.
See [Managing Caddy](3-managing-caddy.md) for deployment and customisation instructions. See [Managing Caddy](3-managing-caddy.md) for deployment and customisation instructions.
@@ -197,8 +197,9 @@ debugging and verifying your `x-caddy` configs.
Example output: Example output:
```caddyfile ```caddyfile
# This file is autogenerated by Uncloud based on the configuration of running services. # Caddyfile autogenerated by Uncloud (DO NOT EDIT): 2025-12-20T22:43:56Z
# Do not edit manually. Any manual changes will be overwritten on the next update. # Automatically updated on service or health status changes.
# Docs: https://uncloud.run/docs/concepts/ingress/overview
# User-defined global config from service 'caddy'. # User-defined global config from service 'caddy'.
*.example.com { *.example.com {
@@ -94,6 +94,7 @@ services:
x-ports: x-ports:
- 80:80@host - 80:80@host
- 443:443@host - 443:443@host
- 443:443/udp@host
x-caddy: Caddyfile x-caddy: Caddyfile
deploy: deploy:
mode: global mode: global
@@ -138,7 +139,8 @@ internal.example.com {
:::info note :::info note
The specified `command`, `environment`, `volumes`, and `x-ports` properties are essential for Caddy to function The specified `command`, `environment`, `volumes`, and `x-ports` properties are essential for Caddy to function
correctly in the Uncloud cluster. correctly in the Uncloud cluster. Do not change the source paths of the volume mounts as the Uncloud daemon relies on
them to communicate with Caddy and update its configuration.
::: :::
@@ -160,8 +162,9 @@ uc caddy config
Example output: Example output:
```caddyfile ```caddyfile
# This file is autogenerated by Uncloud based on the configuration of running services. # Caddyfile autogenerated by Uncloud (DO NOT EDIT): 2025-12-22T10:30:12Z
# Do not edit manually. Any manual changes will be overwritten on the next update. # Automatically updated on service or health status changes.
# Docs: https://uncloud.run/docs/concepts/ingress/overview
# User-defined global config from service 'caddy'. # User-defined global config from service 'caddy'.
# Global options. # Global options.
@@ -78,7 +78,7 @@ See [Push local images to cluster machines](1-deploy-app.md#push-local-images-to
## See also ## See also
- [Deploy an app](1-deploy-app.md): Deploy from source code or prebuilt images - [Deploy an app](1-deploy-app.md): Build and deploy from source code or prebuilt images
- [Deploy a global service](3-deploy-global-services.md): Deploy one service replica on each cluster machine - [Deploy a global service](3-deploy-global-services.md): Deploy one service replica on each cluster machine
- [Compose support matrix](../../8-compose-file-reference/1-support-matrix.md): Supported Compose features and Uncloud - [Compose support matrix](../../8-compose-file-reference/1-support-matrix.md): Supported Compose features and Uncloud
extensions extensions
@@ -0,0 +1,173 @@
# Rolling deployments
How `uc deploy` updates your services without downtime and automatically rolls back on failure.
Uncloud uses a rolling deployment to update your service by replacing its containers **one at a time**. Before moving on
to the next container, Uncloud waits for the new one to pass [health monitoring](#health-monitoring). If it fails to
become healthy, Uncloud stops the deployment and rolls back that container to the old one. This keeps your service
available throughout the update.
For a service with three replicas and the default `start-first` [update order](#update-order), the deployment looks like
this:
1. Start new container #1, wait until healthy
2. Stop and remove old container #1
3. Start new container #2, wait until healthy
4. Stop and remove old container #2
5. Start new container #3, wait until healthy
6. Stop and remove old container #3
At every step, at least three containers are serving traffic.
## Update order
The **update order** controls whether Uncloud starts the new container before or after stopping the old one.
| Order | What happens | Best for |
|---------------|-----------------------------------------------------------------------------|-------------------------------------|
| `start-first` | Start new container, then stop old<br/>(running containers briefly overlap) | Stateless services (web apps, APIs) |
| `stop-first` | Stop old container, then start new | Stateful services (databases) |
The default is `start-first` so there's **no downtime**. But it automatically switches to `stop-first` in two cases:
- **Host port conflicts**: the old container must free the port before the new one can bind to it.
- **Single-replica service with a volume**: two containers simultaneously writing to the same volume can
**corrupt data**, so Uncloud stops the old container first to prevent this.
`stop-first` can cause a **brief downtime** while the old container stops and the new one starts in these cases. The
deployment plan printed by `uc deploy` indicates which containers will be replaced with `stop-first`.
A multiple-replica service with a volume doesn't automatically switch to `stop-first` as Uncloud assumes that the
concurrent access is desired and safe. Host path and tmpfs mounts don't trigger the switch either.
### Override update order
You can override the update order with `deploy.update_config.order`:
```yaml title="compose.yaml"
services:
app:
image: myapp
volumes:
- data:/data
deploy:
update_config:
order: start-first
volumes:
data:
```
This single-replica service uses a volume, so Uncloud would normally use `stop-first`. Setting `order: start-first`
overrides that.
This is useful if your app handles concurrent access to data safely and you want to avoid downtime. For example, the app
uses an SQLite database in WAL mode on the volume.
## Health monitoring
After starting each new container, Uncloud **monitors** it for failures for **5 seconds** to make sure it keeps running
and not crashing. If it keeps restarting after this period, the deployment fails and Uncloud
[rolls back](#rollback-on-failure) that container to the old one.
This is a safeguard to prevent you from deploying broken code or misconfiguration that would cause downtime. 5 seconds
is typically enough for a process in a container to initialise all its dependencies and start.
You can change the monitoring period for a service with `deploy.update_config.monitor`. For example, increase it if your
app takes longer to start or if you want to give it more time to recover from transient errors on startup.
Set it to `0s` to skip monitoring entirely if you are confident the new containers will start correctly and want to
speed up the deployment.
```yaml title="compose.yaml"
services:
app:
image: myapp
deploy:
update_config:
# Specified as duration: 500ms, 20s, 1m30s, 0s (skip)
monitor: 10s
```
You can also change the default monitoring period (`5s`) for all services globally with an environment variable
`UNCLOUD_HEALTH_MONITOR_PERIOD`:
```shell
export UNCLOUD_HEALTH_MONITOR_PERIOD=10s
# or skip monitoring for all services
export UNCLOUD_HEALTH_MONITOR_PERIOD=0s
```
`deploy.update_config.monitor` overrides the global default for each service.
### Health checks
If your container has a [`healthcheck`](https://github.com/compose-spec/compose-spec/blob/main/spec.md#healthcheck)
configured, Uncloud also checks its health status during and after the monitoring period.
If the container becomes `healthy` before the monitoring period ends, the deployment succeeds early and moves on to the
next container. If the container is `unhealthy` after the monitoring period, Uncloud
[rolls it back](#rollback-on-failure) and fails the deployment. Transient `unhealthy` states during the monitoring
period are tolerated to give the container time to recover from startup issues.
To make deployments **safer** and **faster**, it's recommended to configure a health check that can quickly notify
Uncloud when containers start successfully and become ready to serve traffic. You can configure it with
[`healthcheck`](https://github.com/compose-spec/compose-spec/blob/main/spec.md#healthcheck) in your Compose file or
[`HEALTHCHECK`](https://docs.docker.com/reference/dockerfile#healthcheck) in your image `Dockerfile`:
```yaml title="compose.yaml"
services:
app:
image: myapp
healthcheck:
test: curl -f http://localhost:8000/health
interval: 5s
retries: 3
start_period: 10s
start_interval: 1s
```
:::info important
If a health check fails after the deployment, Uncloud automatically removes the unhealthy container from the
[Caddy](../../3-concepts/1-ingress/1-overview.md) configuration to prevent routing traffic to that container. But it
doesn't automatically restart or roll it back.
Uncloud automatically adds it back to Caddy when it recovers and becomes healthy again. You can inspect the health
status of your containers with [`uc ps`](../../9-cli-reference/uc_ps.md) or
[`uc inspect`](../../9-cli-reference/uc_inspect.md) and check their logs with
[`uc logs`](../../9-cli-reference/uc_logs.md).
:::
### Skip health monitoring
To skip health monitoring for **faster emergency deployments**, use `uc deploy --skip-health`.
:::warning
`--skip-health` won't detect containers that crash on startup or become unhealthy so won't roll them back or stop the
deployment. Use this only for emergency deployments when you are confident the new containers will start correctly.
:::
## Rollback on failure
If a new container fails health monitoring during a deployment, Uncloud stops it but keeps it around so you can inspect
its state and logs. For `stop-first` order, Uncloud also restarts the old container. The deployment then stops and the
remaining containers are left untouched.
For example, if the first container in a rolling update succeeds but the second one fails, the first replacement stays
in place.
## Retry after failure
You can retry the deployment by running `uc deploy` again. Uncloud will skip the successfully deployed containers if the
configuration hasn't changed and only redeploy the remaining ones.
## See also
- [Deploy an app](1-deploy-app.md): Build and deploy from source code or pre-built images
- [Compose support matrix](../../8-compose-file-reference/1-support-matrix.md): Supported Compose features and Uncloud
extensions
@@ -3,63 +3,68 @@
Uncloud supports a subset of the [Compose specification](https://compose-spec.io/) with some extensions and limitations. Uncloud supports a subset of the [Compose specification](https://compose-spec.io/) with some extensions and limitations.
The following table shows the support status for main Compose features: The following table shows the support status for main Compose features:
| Feature | Support Status | Notes | | Feature | Support Status | Notes |
|--------------------|--------------------|---------------------------------------------------------------------------------------| |--------------------|--------------------|----------------------------------------------------------------------------------------------------------------|
| **Services** | | | | **Services** | | |
| `build` | ⚠️ Limited | Build context and Dockerfile | | `build` | ✅ Supported | Build context and Dockerfile |
| `command` | ✅ Supported | Override container command | | `cap_add` | ✅ Supported | Additional kernel [capabilities](https://man7.org/linux/man-pages/man7/capabilities.7.html) |
| `configs` | ✅ Supported | File-based and inline configs | | `cap_drop` | ✅ Supported | Which kernel [capabilities](https://man7.org/linux/man-pages/man7/capabilities.7.html) to drop |
| `cpus` | ✅ Supported | CPU limit | | `command` | ✅ Supported | Override container command |
| `depends_on` | ⚠️ Limited | Services deployed in order but conditions not checked | | `configs` | ✅ Supported | File-based and inline configs |
| `dns` | ❌ Not supported | Built-in service discovery | | `cpus` | ✅ Supported | CPU limit |
| `dns_search` | ❌ Not supported | Built-in service discovery | | `depends_on` | ⚠️ Limited | Services deployed in order but conditions not checked |
| `entrypoint` | ✅ Supported | Override container entrypoint | | `devices` | ✅ Supported | Device mappings |
| `env_file` | ✅ Supported | Environment file | | `dns` | ❌ Not supported | Built-in service discovery |
| `environment` | ✅ Supported | Environment variables | | `dns_search` | ❌ Not supported | Built-in service discovery |
| `gpus` | ✅ Supported | GPU device access | | `entrypoint` | ✅ Supported | Override container entrypoint |
| `image` | ✅ Supported | Container image specification | | `env_file` | ✅ Supported | Environment file |
| `init` | ✅ Supported | Run init process in container | | `environment` | ✅ Supported | Environment variables |
| `labels` | ❌ Not supported | | | `gpus` | ✅ Supported | GPU device access |
| `links` | ❌ Not supported | Use service names for communication | | `healthcheck` | ✅ Supported | Health check configuration |
| `logging` | ✅ Supported | Defaults to [local](https://docs.docker.com/engine/logging/drivers/local/) log driver | | `image` | ✅ Supported | Container image specification |
| `mem_limit` | ✅ Supported | Memory limit | | `init` | ✅ Supported | Run init process in container |
| `mem_reservation` | ✅ Supported | Memory reservation | | `labels` | ❌ Not supported | |
| `mem_swappiness` | ❌ Not supported | | | `links` | ❌ Not supported | Use service names for communication |
| `memswap_limit` | ❌ Not supported | | | `logging` | ✅ Supported | Defaults to [local](https://docs.docker.com/engine/logging/drivers/local/) log driver |
| `networks` | ❌ Not supported | All containers share cluster network | | `mem_limit` | ✅ Supported | Memory limit |
| `ports` | ⚠️ Limited | `mode: host` only, use `x-ports` for HTTP/HTTPS | | `mem_reservation` | ✅ Supported | Memory reservation |
| `privileged` | ✅ Supported | Run containers in privileged mode | | `mem_swappiness` | ❌ Not supported | |
| `pull_policy` | ✅ Supported | `always`, `missing`, `never` | | `memswap_limit` | ❌ Not supported | |
| `secrets` | ❌ Not supported | Use configs or environment variables | | `networks` | ❌ Not supported | All containers share cluster network |
| `security_opt` | ❌ Not supported | | | `ports` | ⚠️ Limited | `mode: host` only, use [`x-ports`](#x-ports) for HTTP/HTTPS |
| `storage_opt` | ❌ Not supported | | | `privileged` | ✅ Supported | Run containers in privileged mode |
| `user` | ✅ Supported | Set container user | | `pull_policy` | ✅ Supported | `always`, `missing`, `never` |
| `volumes` | ✅ Supported | Named volumes, bind mounts, tmpfs | | `secrets` | ❌ Not supported | Use configs or environment variables |
| **Deploy** | | | | `security_opt` | ❌ Not supported | |
| `labels` | ❌ Not supported | | | `storage_opt` | ❌ Not supported | |
| `mode` | ✅ Supported | Either `global` or `replicated` | | `sysctls` | ✅ Supported | Namespaced kernel parameters |
| `placement` | ❌ Not supported | Use `x-machines` extension | | `user` | ✅ Supported | Set container user |
| `replicas` | ✅ Supported | Number of container replicas | | `volumes` | ✅ Supported | Named volumes, bind mounts, tmpfs |
| `resources` | ⚠️ Limited | CPU, memory limits and device reservations | | **Deploy** | | |
| `restart_policy` | ❌ Not supported | Defaults to `unless-stopped` | | `labels` | ❌ Not supported | |
| `rollback_config` | ❌ Not supported | See [#151](https://github.com/psviderski/uncloud/issues/151) | | `mode` | ✅ Supported | Either `global` or `replicated` |
| `update_config` | ❌ Not supported | See [#151](https://github.com/psviderski/uncloud/issues/151) | | `placement` | ❌ Not supported | Use [`x-machines`](#x-machines) extension |
| **Volumes** | | | | `replicas` | ✅ Supported | Number of container replicas |
| Named volumes | ✅ Supported | Docker volumes | | `resources` | ⚠️ Limited | CPU, memory limits and device reservations |
| Bind mounts | ✅ Supported | Host path binding | | `restart_policy` | ❌ Not supported | Defaults to `unless-stopped` |
| Tmpfs mounts | ✅ Supported | In-memory filesystems | | `rollback_config` | ❌ Not supported | See [#151](https://github.com/psviderski/uncloud/issues/151) |
| Volume labels | ✅ Supported | Custom labels | | `update_config` | ⚠️ Limited | `order` and `monitor` supported. See [rolling deployments](../4-guides/1-deployments/4-rolling-deployments.md) |
| External volumes | ✅ Supported | Must exist before deployment | | **Volumes** | | |
| Volume drivers | ⚠️ Limited | Local driver only | | Named volumes | ✅ Supported | Docker volumes |
| **Configs** | | | | Bind mounts | ✅ Supported | Host path binding |
| File-based configs | ✅ Supported | Read from file | | Tmpfs mounts | ✅ Supported | In-memory filesystems |
| Inline configs | ✅ Supported | Defined in compose file | | Volume labels | ✅ Supported | Custom labels |
| External configs | ❌ Not supported | Not supported | | External volumes | ✅ Supported | Must exist before deployment |
| Short syntax | ❌ Not supported | Use long syntax only | | Volume drivers | ⚠️ Limited | Local driver only |
| **Extensions** | | | | **Configs** | | |
| `x-caddy` | ✅ Uncloud-specific | Custom Caddy configuration | | File-based configs | ✅ Supported | Read from file |
| `x-machines` | ✅ Uncloud-specific | Machine placement constraints | | Inline configs | ✅ Supported | Defined in compose file |
| `x-ports` | ✅ Uncloud-specific | Service port publishing | | External configs | ❌ Not supported | Not supported |
| Short syntax | ❌ Not supported | Use long syntax only |
| **Extensions** | | |
| `x-caddy` | ✅ Uncloud-specific | Custom Caddy configuration |
| `x-machines` | ✅ Uncloud-specific | Machine placement constraints |
| `x-ports` | ✅ Uncloud-specific | Service port publishing |
### Legend ### Legend
@@ -14,8 +14,8 @@ Git repository state. You can customise the image name and tag format for the bu
If you **don't specify** an `image` attribute for a service with a `build` section, Uncloud uses the following Go If you **don't specify** an `image` attribute for a service with a `build` section, Uncloud uses the following Go
template for tagging the built image: template for tagging the built image:
```go ```
{{.Project}}/{{.Service}}:{{if.Git.IsRepo}}{{gitdate "2006-01-02-150405"}}.{{gitsha 7}}{{if.Git.IsDirty}}.dirty{{end}}{{else}}{{date "2006-01-02-150405"}}{{end}} {{.Project}}/{{.Service}}:{{if .Git.IsRepo}}{{gitdate "2006-01-02-150405"}}.{{gitsha 7}}{{if .Git.IsDirty}}.dirty{{end}}{{else}}{{date "2006-01-02-150405"}}{{end}}
``` ```
```yaml title="compose.yaml" ```yaml title="compose.yaml"
@@ -92,8 +92,7 @@ image: myapp:{{date "20060102-150405" "Local"}} # → myapp:20251031-220651
### Date format reference ### Date format reference
Go uses a reference time (`Mon Jan 2 15:04:05 MST 2006`) for formatting. Replace reference components with desired Go uses a reference time `Mon Jan 2 15:04:05 MST 2006` for formatting. Replace reference components with desired format:
format:
| Component | Reference | Example | | Component | Reference | Example |
|-----------|-----------|-------------| |-----------|-----------|-------------|
+5 -2
View File
@@ -19,17 +19,20 @@ A CLI tool for managing Uncloud resources such as machines, services, and volume
* [uc ctx](uc_ctx.md) - Switch between different cluster contexts. Contains subcommands to manage contexts. * [uc ctx](uc_ctx.md) - Switch between different cluster contexts. Contains subcommands to manage contexts.
* [uc deploy](uc_deploy.md) - Deploy services from a Compose file. * [uc deploy](uc_deploy.md) - Deploy services from a Compose file.
* [uc dns](uc_dns.md) - Manage cluster domain in Uncloud DNS. * [uc dns](uc_dns.md) - Manage cluster domain in Uncloud DNS.
* [uc exec](uc_exec.md) - Execute a command in a running service container * [uc exec](uc_exec.md) - Execute a command in a running service container.
* [uc image](uc_image.md) - Manage images on machines in the cluster. * [uc image](uc_image.md) - Manage images on machines in the cluster.
* [uc images](uc_images.md) - List images on machines in the cluster. * [uc images](uc_images.md) - List images on machines in the cluster.
* [uc inspect](uc_inspect.md) - Display detailed information on a service. * [uc inspect](uc_inspect.md) - Display detailed information on a service.
* [uc logs](uc_logs.md) - View service logs. * [uc logs](uc_logs.md) - View service logs.
* [uc ls](uc_ls.md) - List services. * [uc ls](uc_ls.md) - List services.
* [uc machine](uc_machine.md) - Manage machines in the cluster. * [uc machine](uc_machine.md) - Manage machines in the cluster.
* [uc ps](uc_ps.md) - List all service containers in the cluster * [uc ps](uc_ps.md) - List all service containers.
* [uc rm](uc_rm.md) - Remove one or more services. * [uc rm](uc_rm.md) - Remove one or more services.
* [uc run](uc_run.md) - Run a service. * [uc run](uc_run.md) - Run a service.
* [uc scale](uc_scale.md) - Scale a replicated service by changing the number of replicas. * [uc scale](uc_scale.md) - Scale a replicated service by changing the number of replicas.
* [uc service](uc_service.md) - Manage services in the cluster. * [uc service](uc_service.md) - Manage services in the cluster.
* [uc start](uc_start.md) - Start one or more services.
* [uc stop](uc_stop.md) - Stop one or more services.
* [uc volume](uc_volume.md) - Manage volumes in the cluster. * [uc volume](uc_volume.md) - Manage volumes in the cluster.
* [uc wg](uc_wg.md) - Inspect WireGuard network
@@ -18,6 +18,8 @@ uc deploy [FLAGS] [SERVICE...] [flags]
--no-cache Do not use cache when building images. --no-cache Do not use cache when building images.
-p, --profile strings One or more Compose profiles to enable. -p, --profile strings One or more Compose profiles to enable.
--recreate Recreate containers even if their configuration and image haven't changed. --recreate Recreate containers even if their configuration and image haven't changed.
--skip-health Skip the monitoring period and health checks after starting new containers. Useful for faster emergency deployments.
Warning: This may cause downtime if new containers fail to start properly.
-y, --yes Auto-confirm deployment plan. Should be explicitly set when running non-interactively, -y, --yes Auto-confirm deployment plan. Should be explicitly set when running non-interactively,
e.g., in CI/CD pipelines. [$UNCLOUD_AUTO_CONFIRM] e.g., in CI/CD pipelines. [$UNCLOUD_AUTO_CONFIRM]
``` ```
+1 -1
View File
@@ -1,6 +1,6 @@
# uc exec # uc exec
Execute a command in a running service container Execute a command in a running service container.
## Synopsis ## Synopsis

Some files were not shown because too many files have changed in this diff Show More