Compare commits

...
679 Commits
Author SHA1 Message Date
Pasha Sviderski dc569175ef docs: add info that caddy could be deployed on specific machines 2025-10-13 20:40:11 +10:00
Pasha Sviderski 6ba98e4576 fix: 'uc image push' closed network connection error if takes >10s (fixes #141) 2025-10-13 20:19:46 +10:00
Pasha Sviderski 75b932b66e docs: regenerate CLI reference docs 2025-10-10 18:57:15 +10:00
Pasha Sviderski 5aa85ab501 fix(push): always provide encoded empty auth config to work around panic in Docker 2025-10-09 15:14:10 +10:00
Pasha Sviderski 521ccf2026 chore(push): run unregistry only if containerd image store enabled for docker 2025-10-09 12:48:51 +10:00
Pasha Sviderski e8b4dd0bd1 chore(push): fail 'image push' when containerd image store not used on target machine 2025-10-09 12:11:18 +10:00
Pasha Sviderski 265042e2c8 lint 2025-10-08 16:38:40 +10:00
Justin BradfordandGitHub 1c85112d4f docs: Rough notes for "services" docs with examples of internal DNS lookups and container ENV (#136) 2025-10-08 16:35:52 +10:00
Pasha Sviderski 0fc5032b27 feat(images): add IN USE column to highlight which images are used by any container 2025-10-08 15:43:54 +10:00
Pasha Sviderski 893ab1f9f5 fix: negotiate docker client API version in e2e test 2025-10-08 14:18:11 +10:00
Pasha Sviderski a30ed600b6 chore: update docker and compose Go dependencies to the latest versions 2025-10-08 13:56:58 +10:00
Pasha Sviderski a2c6cbe633 chore(images): filter images by name with wildcard pattern using Docker filter 2025-10-08 12:33:13 +10:00
Pasha Sviderski 1f9c2c873d feat(images): list images filtered by name 2025-10-07 20:39:31 +10:00
Pasha Sviderski 509c9eb7d3 chore(images): format platforms as pills, sort images by name 2025-10-07 19:14:40 +10:00
Pasha Sviderski fba03b3df7 feat(images): list image platforms and do not group images by machine 2025-10-07 17:06:52 +10:00
Pasha Sviderski c6d18f5d49 chore: NAME -> IMAGE column name 2025-10-07 15:12:05 +10:00
3e804d9ecc feat: set UNCLOUD_MACHINE_ID in service container ENV (#135)
* Set machine id in service container ENV

* Add machine id to clone of the container's env map to avoid exposing serialization format

* fix: panic on setting UNCLOUD_MACHINE_ID env var when Env is nil in the spec

---------

Co-authored-by: Pasha Sviderski <me@psviderski.name>
2025-10-07 14:12:00 +10:00
Pasha Sviderski 784d86d8e8 fix: test build for docker server 2025-10-03 21:10:02 +10:00
Pasha Sviderski 3bac1e61b9 chore: go mod tidy 2025-10-03 20:54:59 +10:00
Pasha Sviderski 6d2e100d63 feat(images): 'uc image ls' and 'uc images' (alias) commands to list images on machines 2025-10-03 20:53:30 +10:00
Pasha Sviderski cce81ecaad chore(images): simplifies ListImages service, server, and client to only list Docker images and not directly access containerd 2025-10-03 18:25:39 +10:00
Pasha Sviderski f8bf6bccd3 test: basic e2e test for ListImages 2025-10-03 16:55:20 +10:00
Pasha Sviderski 82fd9744bf chore: remove unused imports 2025-10-03 15:03:19 +10:00
Pasha Sviderski cdff036935 feat(images): gRPC client and server for ListImages 2025-10-03 14:23:18 +10:00
Pasha Sviderski 458d282357 chore: correct comment and add TODO to move machine resolution to grpc-proxy router 2025-10-02 20:12:45 +10:00
Pasha Sviderski 59074a275f chore(images): initialise Docker service with containerd client 2025-10-02 15:51:54 +10:00
Pasha Sviderski 92975eec51 feat(images): add gRPC proto ListImages 2025-10-02 12:50:53 +10:00
Pasha Sviderski e0d11a8abc chore: configure Docker to use containerd image store by default (closes #129) 2025-10-01 15:00:19 +10:00
Pasha Sviderski fa004d63bb chore: add firewall rule to allow cluster machines to push to other machines unregistry 2025-10-01 12:34:03 +10:00
Pasha Sviderski a217643ac9 chore: shorten socat proxy container progress message 2025-09-30 19:44:54 +10:00
Pasha Sviderski fc3297c7b7 chore: go mod tidy 2025-09-30 19:31:14 +10:00
Pasha Sviderski 684f0d38ee feat(push): '--platform all' to push image to all machines in cluster 2025-09-30 19:23:01 +10:00
Pasha Sviderski 332e7bca90 feat(push): --platform flag to push a specific platform of a multi-platform image 2025-09-30 18:59:03 +10:00
Pasha Sviderski ae458d260f chore: refactor duplication 2025-09-30 15:42:01 +10:00
Pasha Sviderski 6b758c810e docs: image push example 2025-09-30 15:24:40 +10:00
Pasha Sviderski ffed906cef feat(push): 'image push' to single or multiple machines via unregistry proxy 2025-09-30 15:18:26 +10:00
Pasha Sviderski bcc504b8cc chore: internal Docker client PushImage 2025-09-30 13:58:34 +10:00
Anton Ovchinnikov 782f4cb20b chore: Update AI.md with instructions around tests and docs 2025-09-29 00:02:58 +02:00
Pasha Sviderski aebd2ac232 docs: review and update Compose support matrix 2025-09-26 22:42:07 +10:00
Pasha Sviderski 2969b40ad4 chore: return a proxy dialer for SSH connections 2025-09-26 21:24:56 +10:00
Pasha Sviderski e0045c7570 chore: refactor gRPC docker client 2025-09-26 21:24:56 +10:00
Pasha Sviderski bc6f2fd49d chore: fix docker client CreateContainerWithImagePull, add WaitPortPublished 2025-09-26 21:24:56 +10:00
Pasha Sviderski cd25b973ea fix: docker client PullImage 2025-09-26 21:24:56 +10:00
Pasha Sviderski 27bbbe7186 chore: internal docker package with handy PullImage and CreateContainerWithImagePull methods 2025-09-26 21:24:56 +10:00
Pasha Sviderski 20cf98d42e fix: return an error if 'uc rm' tries to remove a service that doesn't exist 2025-09-26 21:24:56 +10:00
Anton OvchinnikovandGitHub 63c4de512e feat: Initial support for Compose configs (#116) 2025-09-26 21:24:10 +10:00
Anton Ovchinnikov 337c15de35 chore: Add node to Mise 2025-09-24 23:16:36 +02:00
Justin BradfordandGitHub 2d361d3ee6 Adding machine.service.internal dns entries (#123) 2025-09-24 20:06:54 +10:00
Pasha Sviderski 977fdf3cc3 chore: add Proxy for proxing local connections to a remote TCP address using the dialer 2025-09-24 19:52:37 +10:00
Pasha Sviderski 9f6880b701 feat(push): stub for 'image push' command and Dialer interface for cluster connectors 2025-09-23 16:54:47 +10:00
Pasha Sviderski 1a1afece7d fix: e2e test rebind the registry port to unoccupied 5001 2025-09-23 15:26:36 +10:00
Pasha Sviderski 08f233c5b1 fix: auto-detect containerd.sock path and conditionally start unregistry 2025-09-23 15:10:34 +10:00
Pasha Sviderski 9438654d99 docs: fix comment in schema.sql 2025-09-23 12:21:45 +10:00
Pasha Sviderski 60926f64cc feat: show IMAGE column when listing services (uc ls) #13 2025-09-22 14:03:45 +10:00
Pasha Sviderski b6b8286118 feat: run embedded unregistry on machine IP in uncloudd daemon 2025-09-19 20:43:20 +10:00
Pasha Sviderski 8f906cfd95 fix: do not auto-confirm deploy plan on CI (no TTY), require explicit --yes or UNCLOUD_AUTO_CONFIRM=true 2025-09-18 12:12:06 +10:00
Pasha Sviderski da03ed8ece docs: generate CLI reference docs 2025-09-17 15:11:36 +10:00
Pasha Sviderski 7239cd3a6b docs: clarify 'machine init' help, --context flag description, add examples 2025-09-17 15:10:59 +10:00
Pasha Sviderski 3db07ca2d4 feat: allow setting config and connection with UNCLOUD_CONFIG and UNCLOUD_CONNECT env vars 2025-09-16 10:17:44 +10:00
Pasha Sviderski f8fe3c7af3 feat: --no-install flag for 'machine init/add' to skip installing Uncloud daemon and dependencies #122 2025-09-15 20:44:35 +10:00
Pasha Sviderski b6e9766f40 lint: gRPC status.Error for static error messages 2025-09-15 17:11:12 +10:00
Pasha Sviderski 1f62e30a08 chore: update mise checksums for linux-x64 2025-09-13 22:02:11 +10:00
Pasha Sviderski 88f05ffd75 chore: update mise checksums for macos-arm64 2025-09-13 21:50:40 +10:00
Pasha Sviderski 375bb385a2 chore: bump golangci-lint to 2.4.0 on CI 2025-09-13 21:44:45 +10:00
Pasha Sviderski 51840e0366 chore: bump golangci-lint to 2.4.0 2025-09-13 21:20:48 +10:00
Pasha Sviderski 831c581e0f feat: try each cluster connection in order until one succeeds, add progress spinner 2025-09-13 21:12:20 +10:00
Pasha Sviderski e1734a1aa6 chore: bump Go to 1.25.1 2025-09-13 21:06:17 +10:00
Pasha Sviderski 84376bb34c chore: mise.lock include checksums for linux-x64 2025-09-12 23:18:45 +10:00
Pasha Sviderski 6d0b1dc1ee chore bump mise version on CI to 2025.9.6 2025-09-12 22:58:26 +10:00
Pasha Sviderski bc078472d4 feat: add -y|--yes flag for 'us deploy' to auto-confirm deployment plan 2025-09-12 22:52:06 +10:00
Pasha Sviderski 8a273dbf63 chore: rename cluster -> context option in commands for consistency 2025-09-11 20:21:07 +10:00
Pasha Sviderski 001acfc6a5 docs: remove --watch flag from compose file for deploying caddy 2025-09-10 17:32:15 +10:00
Pasha Sviderski 7b88b8b810 fix: write generated Caddyfile to disk only if succesfully loaded into local Caddy (means valid) 2025-09-10 16:59:31 +10:00
Pasha Sviderski 97bdb8eae1 fix: skip validation for user-defined Caddy configs if caddy not running locally 2025-09-10 15:04:45 +10:00
Pasha Sviderski 5baa8087e5 chore: load Caddy config via admin API instead of watching Caddyfile change on fs 2025-09-09 19:10:32 +10:00
Pasha Sviderski d25864e52f feat: automatically generate a unique 'default-N' context name for new cluster (fixes #113) 2025-09-08 20:35:17 +10:00
Anton Ovchinnikov ec2787c99c fix: Handle implicit relative path for config
Fixes #117
2025-09-07 15:58:09 +02:00
Anton Ovchinnikov 9963f9df2d chore: Update bug template 2025-09-07 15:13:15 +02:00
Anton Ovchinnikov 835834322b chore: Add GH bug report template 2025-09-07 15:02:59 +02:00
Pasha Sviderski 51ba3c7df8 fix: landing navbar on mobile 2025-09-04 18:15:52 +10:00
Pasha Sviderski 047f661462 docs: finish the Deploy demo app page 2025-09-03 20:42:26 +10:00
Pasha Sviderski 5a1e61ccff fix: broken links to completions docs in CLI reference 2025-08-28 20:15:54 +10:00
Pasha Sviderski db60a81b2d chore: lint 2025-08-28 20:09:48 +10:00
Pasha Sviderski 7c19323ea1 docs: 'uc docs' command to generate CLI reference docs 2025-08-28 20:09:18 +10:00
Pasha Sviderski 290e6db98e docs(ingress): Publishing service and Managing Caddy 2025-08-28 17:40:39 +10:00
Pasha Sviderski df29d9ba43 docs: Ingress & HTTP overview 2025-08-27 22:26:50 +10:00
Pasha Sviderski 64769081d9 fix: format for --publish flag 2025-08-27 22:25:32 +10:00
Pasha Sviderski 0107363d41 fix: allow host mode x-ports with x-caddy in compose 2025-08-26 19:15:09 +10:00
Pasha Sviderski 48dc1dd624 chore: delete unused image digest resolver 2025-08-25 20:36:56 +10:00
Pasha Sviderski 3a6eef410a feat: add --caddyfile flag to 'uc caddy deploy' and 'uc run' commands 2025-08-22 20:52:01 +10:00
Pasha Sviderski 9f5ca9a33d feat: add 'uc caddy config' command to show the current Caddyfile 2025-08-22 20:20:02 +10:00
Pasha Sviderski b046b78398 feat: migrate Caddy to generated Caddyfile, mount persistent data volume 2025-08-22 19:51:15 +10:00
Pasha Sviderski ff213e71d3 fix: unmarshaling of ServiceSpec in ServiceContainer struct 2025-08-22 17:25:28 +10:00
Pasha Sviderski 3cda5cc564 chore: caddy client to get caddy config 2025-08-22 15:58:21 +10:00
Pasha Sviderski 75fdbaf2f4 chore: relax ports+Caddy spec validation to allow host mode ports 2025-08-22 15:34:59 +10:00
Pasha Sviderski 81f4e3a67a chore: fix mockery for linux in .mise.lock 2025-08-21 17:31:43 +10:00
Pasha Sviderski e99e769455 chore: add Caddy gRPC service to retrieve Caddyfile config from machines 2025-08-21 17:04:42 +10:00
Pasha Sviderski 11949eeb3b chore: include validation errors for user-defined Caddy configs as a comment in Caddyfile 2025-08-21 14:01:24 +10:00
Pasha Sviderski b437659678 chore: update comments in generated Caddyfile 2025-08-21 13:14:54 +10:00
Pasha Sviderski c01365b416 chore: add header to generated Caddyfile that it's autogenerated 2025-08-21 08:54:06 +10:00
Pasha Sviderski 813c397644 chore: change reverse_proxy upstreams from 'to' to the directive arguments 2025-08-21 08:32:45 +10:00
Pasha Sviderski 5cc005a423 feat: validate and append custom per-service Caddy configs to generated Caddyfile 2025-08-21 08:18:59 +10:00
Pasha Sviderski 066d411367 chore: parse Created time on container with CreatedTime 2025-08-20 22:54:51 +10:00
Pasha Sviderski 03970862ab feat: concatenate custom Caddy configs for services into final Caddyfile (no upstream interpolation) 2025-08-19 22:01:48 +10:00
Pasha Sviderski 8bf9fc0c9c chore: introduce mockery for generating mocks, generate for CaddyfileValidator 2025-08-18 18:50:47 +10:00
Pasha Sviderski 93fef88fac chore: refactor Caddyfile generator to accept a validator 2025-08-18 18:48:32 +10:00
Pasha Sviderski 455174ccb0 chore: generate sites in Caddyfile from x-ports alongside caddy.json 2025-08-15 18:29:23 +10:00
Pasha Sviderski 1ce3e62dbb fix: use local and remote Docker credentials to pull image from private registry 2025-08-15 14:09:40 +10:00
Pasha Sviderski 4be8339c51 chore: generate a minimal Caddyfile with verify handler alongside caddy.json 2025-08-14 19:43:40 +10:00
Pasha Sviderski 8dd69b46da chore: refactor docker gRPC server to use docker service for inspecting and listing containers 2025-08-14 15:34:59 +10:00
Pasha Sviderski 4cc1e556dd chore: store ServiceContainer (includes service spec) instead of Container in Corrosion store 2025-08-14 14:53:33 +10:00
Pasha Sviderski 9186d31d12 chore: go mod tidy 2025-08-13 19:31:13 +10:00
Pasha Sviderski dd7bc6c982 chore: trim spaces for x-caddy, diff Caddy configs when comparing service specs 2025-08-13 19:27:40 +10:00
Pasha Sviderski 12c07812a2 chore: add Caddy config to ServiceSpec, load x-caddy to it 2025-08-13 18:44:38 +10:00
Pasha Sviderski ec73f9ecd8 chore: handle x-caddy: path/to/Caddyfile to read Caddy config in compose from file 2025-08-13 13:45:39 +10:00
Pasha Sviderski 879c7c1876 test: x-caddy extension parsing 2025-08-13 13:45:39 +10:00
Anton Ovchinnikov c67127f83f feat: Add basic LLM instruction files 2025-08-11 23:57:13 +02:00
Pasha Sviderski 5d3f1fe225 chore: x-caddy extension type in compose 2025-08-11 21:11:40 +10:00
Pasha Sviderski 2e585d0183 feat: add --recreate flag for deploy command to force container recreation 2025-08-07 18:09:35 +10:00
Pasha Sviderski ae9f943404 chore: change default restart policy for service containers always -> unless-stopped 2025-08-07 18:09:35 +10:00
Anton Ovchinnikov 8805178a58 doc: Add Sentry to sponsors 2025-08-07 00:03:38 +02:00
Pasha Sviderski ec3de3a099 feat: ask whether to reset already initialised machine on 'machine init/add' 2025-08-06 16:57:09 +10:00
Pasha Sviderski 2c02139369 fix: add ssh_key_path for connections in uncloud config only when using SSH key explicitly (not SSH agent) 2025-08-06 16:49:57 +10:00
Pasha Sviderski 6c244bb8f9 fix: do not try to reset machine when removing unreachable machine 2025-08-06 15:11:27 +10:00
Pasha Sviderski fc0bf4a91b chore: lint 2025-08-05 19:37:12 +10:00
Pasha Sviderski bc577fe405 docs: emphasize passwordless sudo in requirements 2025-08-05 19:32:11 +10:00
Pasha Sviderski 6cc0611d75 chore: meaningful error message when passwordless sudo required on machine provisioning 2025-08-05 18:56:14 +10:00
Pasha Sviderski 8beb9e2679 chore: lint 2025-08-04 19:20:52 +10:00
Pasha Sviderski 0c2ff499c6 fix: log message deleting iptables chain 2025-08-04 19:16:54 +10:00
Pasha Sviderski b39585df4c chore: do not masquerade container IPs when communicating via WG mesh 2025-08-04 19:09:38 +10:00
Pasha Sviderski 35d0a90125 fix: add machine with UFW firewall (allow ipv6 management traffic) fixes #65 2025-08-04 16:50:18 +10:00
Pasha Sviderski a7273c8c96 chore: fix wide svg rendering on landing 2025-08-04 15:22:05 +10:00
Pasha Sviderski 9b48003a3f blog: set diagram as og image for wireguard overlay post 2025-08-04 15:19:27 +10:00
Pasha Sviderski 1e750cc7e1 chore: add Blog links on landing page, fix header on small screens 2025-08-04 14:50:56 +10:00
Pasha Sviderski 2d73b541c5 blog: proof read WireGuard overlay post, update blog description tag 2025-08-04 14:21:48 +10:00
Pasha Sviderski cd9c0a1d2c docs: enable blog in docusaurus, add first wireguard overlay blog post 2025-08-01 21:45:50 +10:00
Pasha Sviderski 37a76809ec docs: wwizzle breadcrumbs and overwrite base url to /docs 2025-07-31 19:07:12 +10:00
Pasha Sviderski f6062cf343 docs: fix docs urls for local dev 2025-07-31 19:04:24 +10:00
Pasha Sviderski 4fe360c573 docs: combine landing page and docusaurus together and deploy as one image 2025-07-31 18:43:06 +10:00
Pasha Sviderski cac1550b7d docs: enable blog in docusaurus and change domain to uncloud.run 2025-07-30 21:28:06 +10:00
Pasha Sviderski 0bd42b2fca chore: do not allow to remove a machine the client is connected to, add --no-reset flag 2025-07-29 19:54:16 +10:00
Pasha Sviderski ad83d942e1 docs: manual cli upgrade 2025-07-28 21:45:33 +10:00
Pasha Sviderski 65f5a714dd feat: machine rm command to remove a machine from the cluster and reset it 2025-07-25 19:32:02 +10:00
Pasha Sviderski 2714587ec5 feat: add RemoveMachine API endpoint to remove a machine from the cluster 2025-07-25 19:30:59 +10:00
Pasha Sviderski 4166474ee8 fix: race on cluster init by waiting for corrosion service to become ready with schema applied 2025-07-25 16:28:32 +10:00
Pasha Sviderski 3faaac4da7 chore: run machine tests in parallel 2025-07-25 13:56:35 +10:00
Pasha Sviderski 1b6130447f chore: use a new uncancelled context for stopping corrosion service 2025-07-25 13:03:20 +10:00
Pasha Sviderski 614212a24c chore: clean up custom iptables chains on machine reset 2025-07-25 12:37:54 +10:00
Pasha Sviderski 3ab708a437 fix: graceful machine shutdown when not initialised 2025-07-25 10:50:23 +10:00
Pasha Sviderski 7a6c5bf6d7 feat: machine Reset endpoint with asynchronous resource and data cleanup 2025-07-24 19:50:25 +10:00
Pasha Sviderski 2acfafe218 chore: regenerate proto 2025-07-24 19:34:05 +10:00
Pasha Sviderski b24b55c4c7 chore: update api.MachineClient interface 2025-07-24 14:25:09 +10:00
Pasha Sviderski 9a88e914f4 chore: simplify cluster controller initialisation, prepare for reset 2025-07-24 14:23:33 +10:00
Pasha Sviderski f613d3ce6d chore: reformat code 2025-07-22 18:53:05 +10:00
Evgenii OrlovandGitHub 2c3bea64e5 feat: impl of functionality for renaming and updating machines (#91) 2025-07-22 18:45:19 +10:00
da3634b690 E2E tests use repeated strings for test data where constants add no value (#97)
---------

Co-authored-by: Pasha Sviderski <me@psviderski.name>
Co-authored-by: Anton Ovchinnikov <anton@tonyo.info>
2025-07-21 19:30:02 +02:00
Pasha Sviderski 6fb07db4b2 feat: follow-up compose 'ports' support: use ingress mode by default (closes #81) 2025-07-21 16:56:52 +10:00
Evgenii OrlovandGitHub fea7edcbc5 feat: add support for standard compose ports directive (#95) 2025-07-21 15:24:37 +10:00
Anton OvchinnikovandGitHub a54555cd13 feat(deploy): Recreate container when pull_policy set to Always (#100) 2025-07-21 14:47:38 +10:00
Pasha Sviderski 87c7889c2e docs: format debian installation section 2025-07-18 21:59:54 +10:00
Anton Ovchinnikov 05d0078451 chore: Enable more linters 2025-07-14 00:32:26 +02:00
Anton Ovchinnikov 053d73048c feat: Basic linting config 2025-07-13 23:50:27 +02:00
Matías García IsaíaandGitHub 4ec3e97a54 docs: add documentation regarding --no-dns flag (#93) 2025-07-11 11:59:25 +10:00
Evgenii OrlovandGitHub 10bbe9fbc5 fix: prevent race condition with Docker network creation (#89) 2025-07-10 17:51:08 +10:00
Evgenii OrlovandGitHub 31cd4c77e9 feat: support x-machines placement constraints in compose files (#90) 2025-07-10 16:46:29 +10:00
Pasha Sviderski 492a0af2b2 fix: migration script for uncloud Docker network to be compatible with Docker 28.2.0+ 2025-07-09 14:04:21 +10:00
Pasha Sviderski ff7bb25f2c fix: custom version handling in install.sh script 2025-07-09 13:22:21 +10:00
Pasha Sviderski a73cbfd691 chore: hide incomplete machine rm command from help output 2025-07-09 11:28:14 +10:00
Pasha Sviderski 55773e92ce fix: allow direct routing from WireGuard mesh to containers for Docker 28.2.0+ 2025-07-08 21:20:49 +10:00
Pasha SviderskiandGitHub 37f543b905 docs: fix suddenly exploded width of uc logo in README.md 2025-07-07 11:36:55 +10:00
Dario GriffoandGitHub e5f96509bf docs: add unofficial Debian package repository (#84) 2025-07-05 19:40:36 +10:00
Anton Ovchinnikov 2f35ac6498 chore: Add .editorconfig 2025-07-04 19:16:40 +02:00
Pasha Sviderski f75afa0cec Merge branch 'pasha/machine-rm' 2025-07-03 21:30:46 +10:00
Pasha Sviderski 0f38b128fa chore: fail 'machine rm' as not fully implemented 2025-07-03 21:30:09 +10:00
Pasha Sviderski 535b91fe52 feat: add stub for Reset method in machine API 2025-07-03 21:28:05 +10:00
Pasha Sviderski 5c8beb8bd3 chore: tidy up go.mod, remove toolchain 2025-07-03 20:57:56 +10:00
Pasha Sviderski 2b4da1e6fe chore: remove anonymous volumes created by service containers when removing them 2025-07-02 11:16:00 +10:00
Pasha Sviderski be8b4f079f feat: remove containers on machine before removing it 2025-07-02 11:14:31 +10:00
Pasha Sviderski 59b96cc01f feat: init 'machine rm' command that outputs service containers that will be removed 2025-07-01 19:51:57 +10:00
Pasha Sviderski 4c77fe2cfd chore: tidy up 2025-06-30 17:53:34 +10:00
Pavel Sviderski 3b89af4a11 fix: e2e compose build test on macOS with Docker Desktop 2025-06-27 17:54:02 +10:00
Zasda Yusuf MikailandGitHub 1abadaef50 feat: set default log driver for service containers to 'local' (#83) 2025-06-27 16:42:26 +10:00
Krishna SantoshandGitHub 3b1ce42dd7 fix(docs): add missing word to complete the sentence (#82) 2025-06-24 16:49:07 +10:00
Pavel Sviderski 1640b4c433 docs: mention github discussions for help in README 2025-06-24 11:40:06 +10:00
Anton OvchinnikovandGitHub 44549c00fd feat: Label new volumes as managed by uncloud (#79) 2025-06-23 22:05:05 +02:00
Anton Ovchinnikov 26f34df3e6 fix: Improve output for build operations 2025-06-22 19:26:25 +02:00
Anton Ovchinnikov 89fb9efcfa fix: Use GITHUB_TOKEN to increase the rate limit for mise action 2025-06-22 11:50:17 +02:00
Anton OvchinnikovandGitHub 638f320959 build: Install protobuf tools via mise (#77) 2025-06-22 11:23:56 +02:00
Pavel Sviderski 3f2e60ea74 docs: mention unregistry as comming soon feature 2025-06-22 09:38:15 +10:00
Pavel Sviderski 8e0a24e781 chore: add donate button in README 2025-06-20 13:37:50 +10:00
Pasha SviderskiandGitHub e3e39dde32 chore: create FUNDING.yml 2025-06-20 12:36:58 +10:00
Pavel Sviderski fc61e2fc07 feat: enable passive health checks and upstream retries in Caddy 2025-06-12 18:38:28 +10:00
Anton Ovchinnikov 2a0d31cec0 chore: Helper make target to install protobuf/grpc plugins 2025-06-11 23:57:50 +02:00
Anton OvchinnikovandGitHub 2f55af0cab test: Add an e2e test for build&push (#69)
* test: Add an e2e test for build&push

* test: Add image cleanup

* test: Switch to assert.NoError
2025-06-09 22:10:53 +02:00
Anton OvchinnikovandGitHub 971b21d6c1 feat: Implement basic build & push capabilities for services (#68)
* wip: Build experiments

* wip: overall structure for build/push

* ref: Split method, add test fixture

* feat: Basic push functionality

* ref: Better structure and output

* feat: Handle registry auth

* feat: Integrate with deploy

* ref: Split into files

* fix: Modernize push options
2025-06-03 21:35:54 +02:00
Pavel Sviderski b651d95183 docs: demo app - Compose and custom domain 2025-06-02 20:03:19 +10:00
Pavel Sviderski d45c1b6869 docs: add Deploy demo app page (partial), change prism theme to palenight 2025-05-30 20:00:04 +10:00
Pavel Sviderski a5fe9fbfb2 chore: update -p flag help for run command 2025-05-29 21:47:50 +10:00
Pavel Sviderski 4590a516ad chore: set default ssh key path to ~/.ssh/id_ed25519 and handle home expansion 2025-05-29 15:00:33 +10:00
Pavel Sviderski 5f9ed6dacf docs: fix links 2025-05-28 18:39:08 +10:00
Pavel Sviderski e97497d737 docs: refine overview 2025-05-28 18:35:23 +10:00
Pavel Sviderski b4bf1d17d0 docs: add note for Windows installation, enable tabs support 2025-05-28 11:14:27 +10:00
Pavel Sviderski cfb4693286 docs: installation page 2025-05-28 10:28:33 +10:00
Pavel Sviderski fd3facb6b0 chore: minor README change 2025-05-27 16:16:26 +10:00
Pavel Sviderski bd03bb9bfa chore: minor README change 2025-05-27 16:12:32 +10:00
Pavel Sviderski eec728620b chore: minor README change 2025-05-27 16:09:20 +10:00
Pavel Sviderski d2dd5e3c89 chore: add inspiration & acknowledgements section to README 2025-05-27 16:07:30 +10:00
Pavel Sviderski 2bfcce7803 chore: add uncloud-docs deploy from compose to demo section in README 2025-05-27 13:41:30 +10:00
Pavel Sviderski 1ec5184f21 chore: self-hosting focused front page (first screen) 2025-05-26 18:50:24 +10:00
Pavel Sviderski 23dfe45b2f chore: add docs badge to github README 2025-05-26 12:29:51 +10:00
Anton Ovchinnikov d9e906a77e test: Disable test caching 2025-05-25 19:53:43 +02:00
Anton Ovchinnikov 51f7ebf3bf test: Add cleanup target that removes dangling test containers 2025-05-25 19:46:13 +02:00
Pavel Sviderski ae04dea808 chore: add Docs to website navbar and footer, add other resources to footer 2025-05-23 20:11:11 +10:00
Pavel Sviderski 009c0e1b4c chore: scale docs service to 2 replicas in different regions 2025-05-23 18:09:54 +10:00
Pavel Sviderski 18d38e96b4 docs: minor overview polish 2025-05-23 16:31:29 +10:00
Pavel Sviderski 12c86f1dd4 fix: use existing cluster client when deploying caddy after adding new machine #65 2025-05-23 16:31:07 +10:00
Anton Ovchinnikov 4df1820f96 fix(doc): Tiny fixes 2025-05-22 22:19:02 +02:00
Pavel Sviderski 6f9ac1fcc7 chore: add Docker image for docs and Uncloud Compose deployment (dogfooding!) 2025-05-22 21:01:52 +10:00
Pavel Sviderski 6a98acbe79 docs: add Overview page and Getting started stubs 2025-05-22 18:37:10 +10:00
Pavel Sviderski b0059af9b1 docs: change color palette for links to blue 2025-05-22 18:36:35 +10:00
Pavel Sviderski 7edd8f2976 chore: add m alias for machine command 2025-05-21 15:46:31 +10:00
Pavel Sviderski 735e36012a chore: delete unused ucind-image-push target in Makefile 2025-05-21 15:46:09 +10:00
69ebce57e9 test: Add basic Github Actions config (#63)
* test: Add basic Github Actions config

* fix(e2e-tests): handle asynchronous port publishing for ucind container

* fix(e2e-tests): increase machine ready wait timeout 1->10s

* fix: Pin GHA versions

---------

Co-authored-by: Pavel Sviderski <me@psviderski.name>
2025-05-20 23:59:02 +02:00
Pavel Sviderski 365a5fc0b5 chore: build multiarch images for corrosion and ucind to support e2e tests on amd64 2025-05-20 12:47:27 +10:00
Pavel Sviderski 624caf037b chore(docs): init overview page 2025-05-19 22:42:04 +10:00
Pavel Sviderski ff1a8fbcf9 chore: fix comment 2025-05-16 21:35:21 +10:00
Pavel Sviderski 3aa8fead76 chore: configure docs custom styles, init intro page 2025-05-16 21:35:04 +10:00
Pavel Sviderski 4075f85b72 chore: minor website update 2025-05-15 20:46:02 +10:00
Pavel Sviderski 59f6fa75ca chore: init docusaurus docs 2025-05-15 20:45:46 +10:00
Pavel Sviderski 8b66a11394 chore: update hero 2025-05-14 22:06:31 +10:00
Pavel Sviderski fa3d992dbc chore: period 2025-05-13 18:27:55 +10:00
Pavel Sviderski a7d8afb052 chore: update value section wording 2025-05-13 18:26:37 +10:00
Pavel Sviderski 0d820bc75a chore: replace github discussions with discord in subscribe section 2025-05-13 18:23:36 +10:00
Pavel Sviderski 6a4e63dbd4 chore: add Discord to landing, remove crossed out Kubernetes 2025-05-13 18:21:33 +10:00
Pavel Sviderski 7699b530c2 chore: refine links 2025-05-13 17:59:41 +10:00
Pavel Sviderski f1abbbea6d chore: add discord and X badges to README 2025-05-13 17:53:57 +10:00
Pavel Sviderski 683ad22358 chore: remove extra [FLAGS] from volume commands help, add SERVICE for inspect 2025-05-13 10:24:16 +10:00
Pavel Sviderski 8f00e0082e chore: add social links in the footer 2025-05-10 14:36:05 +10:00
Pavel Sviderski c34542a9d8 chore: update website title 2025-05-10 09:11:16 +10:00
Pavel Sviderski 88db1cb6a9 fix: hero svg on safari 2025-05-10 07:48:20 +10:00
Pavel Sviderski dfa6e001c5 chore: cross over kubernetes on landing hero 2025-05-09 22:56:07 +10:00
Pavel Sviderski f49ec465d0 chore: bring terminal on landing back, update hero, move demo to bottom 2025-05-09 22:38:58 +10:00
Pavel Sviderski 1ffe9acdd7 chore: minor README rearrangement 2025-05-09 21:49:11 +10:00
Pavel Sviderski e35e69b4be fix: check latest Caddy image only if not specified for 'uc caddy deploy' 2025-05-07 16:03:07 +10:00
Pasha SviderskiandGitHub 11c2e641f1 Merge pull request #58 from Tova-Rozovsky/valid_DNS 2025-05-07 16:02:55 +10:00
Pasha SviderskiandGitHub bb97537df7 fix: format error message 2025-05-07 15:59:53 +10:00
Tova-Rozovsky 9f4b8179b5 valid service name 2025-05-06 17:43:05 +03:00
Pavel Sviderski f7b079c4f2 chore: collapse how it works section in readme 2025-05-06 19:00:54 +10:00
Pavel Sviderski 3648187219 chore: allow DNS queries from Uncloud containers to the embedded DNS server 2025-05-05 20:00:34 +10:00
Pavel Sviderski 93346301fb chore: clarify zero-downtime deployments feature 2025-05-05 17:15:41 +10:00
Pavel Sviderski cc86c8660c chore: rearrange features in README 2025-05-05 17:11:49 +10:00
Pavel Sviderski 2963436538 chore: update available features in README 2025-05-05 17:09:02 +10:00
Pavel Sviderski 6e326277bb chore: set up UNCLOUD-INPUT iptables chain and allow WireGuard traffic to the machine 2025-05-05 14:50:33 +10:00
Tova-Rozovsky 469d6fadcd Fix: restrict validation to RFC1123 label name 2025-05-04 16:14:17 +03:00
Tova-Rozovsky 5f214247ca added validition for service name 2025-05-02 10:00:16 +03:00
Pavel Sviderski 0b8495e964 feat: make all confirmation forms accessible 2025-05-01 14:12:12 +10:00
Pavel Sviderski 6068e24f4f fix(ctx): panic: index out of range when only one context in uncloud config (closes #52) 2025-05-01 14:01:29 +10:00
Pasha SviderskiandGitHub 7861d56dff Merge pull request #54 from cedws/profiles
feat: add support for Compose profiles
2025-05-01 13:54:25 +10:00
Connor Edwards 7bc69b9a9e feat: add support for Compose profiles 2025-05-01 12:23:56 +09:00
Pavel Sviderski 38800924ed fix(caddy): proxy to only healthy containers 2025-05-01 11:47:05 +10:00
Pavel Sviderski cf7e333579 feat(dns-server): use embedded DNS server for all service containers 2025-05-01 11:27:27 +10:00
Pavel Sviderski 769f5e47c3 feat(dns-server): check system prerequisites when init/add machine: DNS port is not in use by another DNS server 2025-04-30 20:46:10 +10:00
Pavel Sviderski 6e1165ec57 chore(dns-server): make TCP server non-critical; do not exit if it fails to start 2025-04-30 17:46:44 +10:00
Pavel Sviderski 8d16da596c feat(dns-server): run embedded internal DNS server on machine IP in uncloudd daemon process 2025-04-30 17:24:52 +10:00
Pavel Sviderski d6c06828b3 fix(dns-server): fix constructor and log messages 2025-04-30 17:20:48 +10:00
Pavel Sviderski fe93d91894 fix: attributes handling in SlogTextHandler for child loggers 2025-04-30 16:56:32 +10:00
Pavel Sviderski 3f70018e7c fix: import cycle in pkg/api 2025-04-30 14:18:28 +10:00
Pavel Sviderski 70e9385032 chore(dns-server): implement resolver for service records that watches container changes 2025-04-30 14:13:48 +10:00
Pavel Sviderski db3016fe89 fix: --file flag description for uc deploy cmd 2025-04-30 11:45:25 +10:00
Pavel Sviderski c56385602e chore(dns-server): authoritative responses for internal DNS queries and forwarding to ustream servers 2025-04-29 20:20:55 +10:00
Pavel Sviderski fbbf839258 chore: init embedded DNS server 2025-04-28 21:36:04 +10:00
Pavel Sviderski eee28e2ead feat: add version flag to uncloudd binary 2025-04-26 19:17:26 +10:00
Pasha SviderskiandGitHub ef57b62625 Merge pull request #50 from cedws/version
feat: add version flag to main cmd
2025-04-26 19:17:01 +10:00
Connor Edwards e854bbb3e1 feat: add version flag to main cmd 2025-04-25 19:55:37 +09:00
Pavel Sviderski 938e7d05c0 feat: add support for entrypoint, cpus, mem_limit, mem_reservation, priviliged, user to Compose deployment 2025-04-24 22:40:46 +10:00
Pavel Sviderski 6c2759f315 feat: add --cpu, --memory, --privileged, --user options for 'service run' 2025-04-24 20:30:48 +10:00
Pavel Sviderski 72bf34632f chore: map new container properties to Docker container configs 2025-04-24 17:24:58 +10:00
Pavel Sviderski 7db8bba6ea refactor: cpu and memory under Resources struct 2025-04-24 16:40:26 +10:00
Pavel Sviderski badb8ecd3c chore: implement EvalContainerSpecChange for new container properties 2025-04-24 15:51:47 +10:00
Pavel Sviderski 9413f1c0b1 feat: define essential container spec parameters 2025-04-24 13:37:25 +10:00
Pavel Sviderski d8818ba770 chore: remove unnecessary fields from containers table in machine.db 2025-04-23 16:11:29 +10:00
Pavel Sviderski 440215b811 fix: TestInstallCmd test 2025-04-23 16:10:40 +10:00
Pavel Sviderski 02a418aafb chore: add --version flag for 'machine init|add' to specify uncloud daemon version 2025-04-23 14:40:04 +10:00
Pavel Sviderski 33550be65e fix: compose deployment plan preview 2025-04-23 12:04:04 +10:00
Pavel Sviderski edad2429c5 chore: verify idempotency for compose deploy with volumes 2025-04-23 12:03:23 +10:00
Pavel Sviderski b605a64fbe chore: add e2e compose test with volumes 2025-04-23 10:41:24 +10:00
Pavel Sviderski 16c876d83f fix: compare container specs with equate empty option (nil map == {} map) 2025-04-22 23:31:45 +10:00
Pavel Sviderski d6ff62159d fix: compose deployment: use mutex to correctly collect service specs 2025-04-22 22:48:06 +10:00
Pavel Sviderski fcf164dda1 chore: refactor schedulers to work with cluster state snapshot, VolumeScheduler updates state with scheduled volumes 2025-04-22 20:34:54 +10:00
Pavel Sviderski e5957d9d6b chore: map compose volumes to server spec, initial volume scheduling 2025-04-21 21:25:26 +10:00
Pavel Sviderski 143e68cc95 chore: do not set default volume driver to local, distinguish behaviour when it's set or not 2025-04-21 16:53:43 +10:00
Pavel Sviderski 835c310987 chore: delete old basic-compose.yaml fixture 2025-04-21 12:50:13 +10:00
Pavel Sviderski 5fce84d33c chore: cleanup compose-basic test 2025-04-21 12:49:44 +10:00
Pavel Sviderski 45497fd20c chore: update api.ServiceClient interface to include service creation and removal 2025-04-21 12:48:21 +10:00
Pavel Sviderski f08d9fe405 refactor: 'service run' create missing volumes satisfying all placement constraints 2025-04-20 19:57:30 +10:00
Pavel Sviderski 565a8d8366 chore: implement VolumeScheduler to determine what missing volumes should be created and where 2025-04-20 18:57:26 +10:00
Pavel Sviderski 343357adf9 find optimal solution in VolumeScheduler 2025-04-20 18:53:18 +10:00
Pavel Sviderski 8ee551ca8f cleanup 2025-04-20 11:15:07 +10:00
Pavel Sviderski 948201cb6f add VolumeScheduler with one flaky test 2025-04-20 11:13:30 +10:00
Pavel Sviderski f71fc57f34 chore: e2e tests for replicated and global deployments with volumes 2025-04-18 17:27:34 +10:00
Pavel Sviderski 1acb0ff110 chore: implement VolumesConstraint to only schedule on machines with required volumes 2025-04-17 21:34:52 +10:00
Pavel Sviderski 03c2681792 chore: cleanup unused machine filter struct 2025-04-16 22:06:11 +10:00
Pavel Sviderski f3cb6657ae chore: replace machine filter with placement constraint in service spec 2025-04-16 22:04:37 +10:00
Pavel Sviderski 4532b985d4 chore: populate ucind cluster machines with IDs 2025-04-16 22:01:33 +10:00
Pavel Sviderski 000a8a1d3e chore: init scheduler entities 2025-04-15 22:30:03 +10:00
Pavel Sviderski 7705db2c4f refactor: machine listing for replicated plan 2025-04-14 22:04:39 +10:00
Pavel Sviderski 590a1b231f chore(volumes): e2e test when volume doesn't exist 2025-04-13 12:43:23 +10:00
Pavel Sviderski 8709ac0ed5 chore: fix Errorf arguments 2025-04-11 21:34:07 +10:00
Pavel Sviderski f8ebd94a79 feat(volumes): create and mount data volumes when running service with 'uc run' command 2025-04-11 21:28:36 +10:00
Pavel Sviderski 87331a9265 chore: add filter to ListMachines 2025-04-11 21:26:40 +10:00
Pavel Sviderski 2f5a2c6344 feat(volumes): new 'volume create|inspect|ls|rm' CLI commands to manage volumes 2025-04-10 20:33:03 +10:00
Pavel Sviderski 97e032cb7b chore: update ListVolumes to accept a filter 2025-04-10 20:31:52 +10:00
Pavel Sviderski 410a95ec88 chroe: add TODO to fix empty context in output message when adding machines 2025-04-10 20:29:52 +10:00
Pavel Sviderski df4ab40e12 chore(volumes): add client methods for volume management, e2e test 2025-04-10 13:50:08 +10:00
Pavel Sviderski c6205d4f57 chore(volumes): map volume specs to container config, diff specs with volumes 2025-04-09 21:51:20 +10:00
Pavel Sviderski 13dea92311 chore(volumes): gRPC methods for managing Docker volumes 2025-04-08 16:33:35 +10:00
Pavel Sviderski 9100c22e7f chore: new specs for defining service volumes 2025-04-07 21:00:26 +10:00
Pavel Sviderski 6125553a97 chore: introduce VolumeSpec supporting different volume types (bind, volume, tmpfs) 2025-04-04 21:22:54 +10:00
Pavel Sviderski c9452f1509 chore: ignore uc binary in the repo root 2025-04-03 18:19:06 +10:00
Pavel Sviderski 25b173128a feat(env): support env vars for services (both run command and compose) 2025-04-03 18:14:17 +10:00
Pavel Sviderski 4e40a93991 chore: remove anonymous volumes for ucind containers 2025-04-03 16:35:48 +10:00
Pavel Sviderski 9fc54c9d64 fix: format config.Path method 2025-04-03 16:35:00 +10:00
Pavel Sviderski cacfca7a88 chore: print message about changed context for 'ctx use' command 2025-04-03 16:21:04 +10:00
Pavel Sviderski b3885b0888 chore: activate the context of a newly initialised cluster 2025-04-03 13:38:48 +10:00
Pavel Sviderski 3ada89c4f8 fix: adding ucind cluster with tcp connections to config 2025-04-02 19:45:15 +10:00
Pavel Sviderski 20b2ca2ea5 feat(ctx): interactive ctx command to switch between cluster contexts 2025-04-02 16:31:26 +10:00
Pavel Sviderski 1bbefdd0d2 feat: add 'ctx ls' command to list available cluster contexts in config 2025-04-02 09:25:12 +10:00
Pavel Sviderski f467fad84c chore: rename cluster term to context in CLI and config, convert config toml -> yaml 2025-04-01 21:29:32 +10:00
Pavel Sviderski 0e80f2e5b2 chore: replace immutable hash with container spec comparison 2025-03-31 15:58:02 +10:00
Pavel Sviderski 64f6a4f3d3 chore: use original service spec instead of deriving from container, fixes diff for scale 2025-03-30 12:54:36 +10:00
Pavel Sviderski 0975cbab66 chore: handle RemoveContainer server-side that deletes container from machine db 2025-03-29 20:06:49 +10:00
Pavel Sviderski 615022c9ac feat(scale): set file permissions for machine.db database to 600 (may container secrets as env vars) 2025-03-29 15:19:37 +10:00
Pavel Sviderski ecbd4378e4 feat(scale): store container spec in local machine.db sqlite 2025-03-29 15:14:43 +10:00
Pavel Sviderski 1becd033e6 chore: rename CaddyConfigPath field 2025-03-29 11:39:45 +10:00
Pavel Sviderski 1cba1eef0a refactor: handle CreateContainer server-side 2025-03-28 19:01:19 +10:00
Pavel Sviderski 3893005866 fix(compose): remove ServiceSpec validation for 0 replicas, it's handled when resolving defaults 2025-03-27 20:40:11 +10:00
Pavel Sviderski 1911e03c82 feat(compose): handle image pull policy 2025-03-27 15:15:57 +10:00
Pavel Sviderski 641b11a92d feat: implement --pull policy for run command: always, missing, never 2025-03-27 15:06:32 +10:00
Pavel Sviderski 7c7530f977 chore: add comments and TODOs about the disabled image digest resolver 2025-03-27 14:12:20 +10:00
Pavel Sviderski 6e7fe405ef feat(compose): partially implement image resolver (pinner) but disable for now 2025-03-26 20:42:41 +10:00
Pavel Sviderski 03c5c054e1 feat(compose): client methods for inspecting images 2025-03-26 20:38:41 +10:00
Pavel Sviderski c42968c812 feat(compose): add InspectRemoteImage api endpoint to inspect images in registry 2025-03-26 20:37:53 +10:00
Pavel Sviderski ffef41081b feat(compose): add ContainerSpec.PullPolicy field to define when to pull image from registry 2025-03-25 11:03:19 +10:00
Pavel Sviderski 58b51b974c refactor: postpone spec resolver initialisation when creating Deployment 2025-03-24 21:56:30 +10:00
Pavel Sviderski 2d923380a3 chore: compare container ports to spec in assertContainerMatchesSpec 2025-03-24 20:24:40 +10:00
Pavel Sviderski 3910101a88 refactor: embed ServiceSpecResolver into Deployment to handle spec only there 2025-03-24 20:10:28 +10:00
Pavel Sviderski 7a9d0cf9db refactor: move compose and deploy packages under client 2025-03-24 14:41:50 +10:00
Pavel Sviderski 042dd594e0 refactor: decouple deploy and compose packages from client dependency 2025-03-24 14:38:16 +10:00
Pavel Sviderski 1471a94162 refactor(compose): move compose deployment from client to compose pkg 2025-03-24 11:11:36 +10:00
Pavel Sviderski b1abd07dde refactor: move api, client, compose packages to pkg 2025-03-22 18:43:00 +10:00
Pavel Sviderski 3757813b20 refactor: rename main module to github.com/psviderski/uncloud 2025-03-22 18:25:08 +10:00
Pavel Sviderski f1a150821c fix(compose): client.ImageInspect to handle broadcasted responses to multiple machines 2025-03-22 08:44:17 +10:00
Pavel Sviderski 874af1ba10 feat(compose): implement InspectImage rpc with broadcast support 2025-03-21 19:43:58 +10:00
Pavel Sviderski 2ad098c612 feat(deploy): introduce proto for InspectImage and InspectRemoteImage methods 2025-03-20 21:36:41 +10:00
Pavel Sviderski 4172cb826f refactor: compose deployment plan to not use PrepareDeploymentSpec 2025-03-19 16:54:00 +10:00
Pavel Sviderski 041d51b74f refactor: compose deployment with pluggable spec resolver, basic test 2025-03-19 16:28:17 +10:00
Pavel Sviderski fb01604d21 fix: recreate containers only if spec changed by comparing spec hash 2025-03-19 13:11:04 +10:00
Pavel SviderskiandGitHub bd8bcbd2f9 Merge pull request #40 from cedws/userguide 2025-03-19 09:24:50 +10:00
Connor Edwards 07501813c8 docs: s/uncloud/uc 2025-03-18 21:04:43 +09:00
Pavel Sviderski 1bbcfa8519 feat(entrypoint): allow to override service ENTRYPOINT for run command 2025-03-18 16:44:15 +10:00
Pavel Sviderski e8715fce38 fix: list containers if some of them concurrently removed, fixes #33 2025-03-18 15:39:15 +10:00
Pavel Sviderski a0a66f34c4 feat(connect): add --connect flag to connect to remote cluster machine without using config 2025-03-18 14:01:38 +10:00
Pavel Sviderski b799b7518e feat(compose): add TODO to address deployment idempotency 2025-03-17 20:32:14 +10:00
Connor Edwards 2571498ed2 docs: add basic user guide 2025-03-16 14:51:42 +09:00
Pavel Sviderski 5bcbfc332d feat(compose): deploy compose services without waiting dependencies 2025-03-15 20:45:02 +10:00
Pavel Sviderski 34574e41fc feat(compose): plan compose deployment 2025-03-15 20:43:30 +10:00
Pavel Sviderski c7355afa3d refactor: move Confirm logic to common place from caddy deployment 2025-03-15 20:42:44 +10:00
Pavel Sviderski a907fdb1af chore: update todo 2025-03-15 20:41:57 +10:00
Pavel Sviderski c9ba4adccb chore: add todo for ServiceSpec that it may return a spec not equal to original 2025-03-15 20:41:02 +10:00
Pavel Sviderski a0551d2d64 fix: print service endpoints for run service only when non-empty 2025-03-15 19:35:21 +10:00
Pavel Sviderski 76129ea056 refactor: include service name in deployment plan 2025-03-15 19:34:37 +10:00
Pavel Sviderski 6ffd35f596 feat(compose): parse Compose project and transform ports extension 2025-03-15 14:18:06 +10:00
Pavel Sviderski 0ef776e7f1 chore: add link to uncloud-dns repo in README 2025-03-14 22:01:19 +10:00
Pavel Sviderski 0b22b2136a chore: add deployments and managed DNS features to README 2025-03-14 21:59:40 +10:00
Pavel Sviderski e1530bb6f3 feat(deploy): load compose project 2025-03-14 21:41:06 +10:00
Pavel Sviderski 7fc07ce4bd feat(deploy): init a new deploy command to deploy services from Compose file 2025-03-12 20:37:01 +10:00
Pavel Sviderski eb42082955 fix(dns): update DNS records when reserving a new cluster domain, fixes #36 2025-03-11 21:12:00 +10:00
Pavel Sviderski 8719e37856 fix(dns): retry HTTP requests to verify Caddy reachability for up to 5 seconds 2025-03-10 21:26:02 +10:00
Pavel Sviderski e1cc39b0d5 lint: variable collides with imported package name 2025-03-09 19:26:06 +10:00
Pavel Sviderski efe646f54c chore: validate unsupported tcp/udp protocols for ingress port 2025-03-08 19:07:51 +10:00
Pavel Sviderski e130d803e4 fix(website): demo min height to not shrink before video is loaded 2025-03-06 17:30:16 +10:00
Pavel Sviderski 56e7e8baa2 chore(readme): cleanup coming soon features 2025-03-06 17:13:29 +10:00
Pavel Sviderski b963ef39d1 chore(makefile): demo-reset target to reset machines before recording a VHS demo 2025-03-06 16:55:16 +10:00
Pavel Sviderski 1df0fa290f chore(readme): add uc machine ls command 2025-03-06 16:54:35 +10:00
Pavel Sviderski c3bb0c9f49 chore(readme): add Quick demo section with video, update how it works outputs 2025-03-06 16:49:58 +10:00
Pavel Sviderski cf65cecd9e chore: replace terminal example with a demo video on landing 2025-03-06 15:49:06 +10:00
Pavel Sviderski fff17dbb60 add VHS tape script for the terminal demo 2025-03-05 22:17:54 +10:00
Pavel Sviderski 586a4fa129 feat(scale): basic service scale command 2025-03-05 20:24:54 +10:00
Pavel Sviderski 2a82f75cad fix: deduplicate discovered machine IPs when adding a new machine 2025-03-05 09:14:11 +10:00
Pavel Sviderski 805afd0d28 fix(dns): update cluster domain DNS records when adding a new machine to cluster 2025-03-04 20:49:41 +10:00
Pavel Sviderski 11098ea7d1 fix: url for uninstall.sh script in install script 2025-03-04 12:54:48 +10:00
Pavel Sviderski 90e357a1dc feat: install uncloud-uninstall script when provisioning machine 2025-03-04 12:50:29 +10:00
Pavel Sviderski 91d58bc701 chore: add uninstall.sh script to remove uncloud and managed containers and network from machine 2025-03-04 12:14:49 +10:00
Pavel Sviderski 69d3a511d6 feat(dns): reserve cluster domain when initialising cluster by default + update dns records 2025-03-03 19:12:21 +10:00
Pavel Sviderski c3123820b2 fix: install script tar flag 2025-03-03 18:51:41 +10:00
Pavel Sviderski 2e49d94fad fix: deduplicate discovered machine IPs 2025-03-03 18:42:00 +10:00
Pavel Sviderski bb51e01c5c fix: install script unpack uncloudd binary to tmp dir 2025-03-03 18:38:51 +10:00
Pavel Sviderski cdb2884e4e fix: check for dockerd command to determine if docker engine is installed 2025-03-03 18:05:52 +10:00
Pavel Sviderski 137abde808 feat: set container hostname equal to container name 2025-03-03 16:58:42 +10:00
Pavel Sviderski 342969b7f4 fix(caddy): remove debug prints 2025-03-03 15:05:57 +10:00
Pavel Sviderski d9e350c4dd fix(caddy): config generation for multiple https containers 2025-03-03 15:04:04 +10:00
Pavel Sviderski 3f4f263c05 test: increase test cluster wait timeout 5->30 seconds 2025-03-03 12:21:22 +10:00
Pavel Sviderski 0760ba3b18 test: replicated deployment with machine filter 2025-03-03 11:26:08 +10:00
Pavel Sviderski 0896bd27fb test: basic replicated deployment, prioritise host with the most up-to-date containers 2025-03-02 10:23:49 +10:00
Pavel Sviderski a16b2d1c74 feat: service ls command include ID column if there are multiple services with same name 2025-03-01 22:31:58 +10:00
Pavel Sviderski 52117954a8 fix: specs equals ignores replicas 2025-03-01 22:31:18 +10:00
Pavel Sviderski 435a794fa5 fix: e2e tests 2025-02-28 18:00:02 +10:00
Pavel Sviderski 9642863d7b feat(replicas): support --replicas and --machines for run command 2025-02-28 17:53:58 +10:00
Pavel Sviderski 6bb9bdeb5a feat(replicas): implement rolling strategy for replicated service with basic spread strategy 2025-02-28 17:51:38 +10:00
Pavel Sviderski 30f1e135f1 feat: print ENDPOINTS column when listing services 2025-02-27 20:37:02 +10:00
Pavel Sviderski 58ecc17ac1 feat: print service endpoints on service creation 2025-02-27 20:37:02 +10:00
Anton Ovchinnikov 4c239ed79a fix(smm): Update site description 2025-02-27 00:08:03 +01:00
Anton Ovchinnikov 968dd50705 fix(smm): Use PNG logo, add Twitter tags 2025-02-26 23:55:26 +01:00
Pavel Sviderski d7f437107d fix(dns): error message for ingress port without hostname and no cluster domain 2025-02-26 21:13:18 +10:00
Pavel Sviderski 72548fbd1b feat(dns): allow to specify http(s) ports without hostname to default to cluster domain 2025-02-26 21:10:55 +10:00
Pavel Sviderski a5f6ec4a68 feat(dns): expose service as service-name.cluster-domain if cluster domain reserved 2025-02-26 21:10:21 +10:00
Pavel Sviderski 4cc68e3133 include cloud vm in dev set in Makefile 2025-02-26 15:52:33 +10:00
Pavel Sviderski 2bacebe900 refactor(dns): remove leading dot from domain when interecting with Uncloud DNS 2025-02-26 15:45:28 +10:00
Pavel Sviderski 09bea1d12d feat(dns): update ingress records in Uncloud DNS when deploying Caddy 2025-02-26 14:23:00 +10:00
Pavel Sviderski 5e9b883ad5 fix: increase wait time for machine to 5 min, discard unknown when decoding json to proto 2025-02-26 12:19:04 +10:00
Pavel Sviderski 1046396edc feat(dns): configure and store public IP when adding new machine 2025-02-26 11:16:13 +10:00
Pavel SviderskiandGitHub 35804458b0 Merge pull request #28 from psviderski/tonyo/27-feat-social-tags
feat(smm): Add OpenGraph tags
2025-02-26 10:10:49 +10:00
Anton Ovchinnikov 6bb64ac5dc feat(smm): Add OpenGraph tags 2025-02-26 00:35:33 +01:00
Pavel Sviderski e5f68947f9 feat(dns): missed CLI changes 2025-02-25 21:22:32 +10:00
Pavel Sviderski 289d964b2a feat(dns): configure and store public IP when initialising machine 2025-02-25 21:21:37 +10:00
Pavel Sviderski d0fd9db258 feat(dns): serve machine IP at /.uncloud-verify to verify Caddy reachability 2025-02-25 17:36:44 +10:00
Pavel Sviderski 0ea997e826 feat(dns): update caddy deploy command to create ingress records (stubs for now) 2025-02-24 20:33:54 +10:00
Pavel Sviderski 2d84059d7f feat(dns): client for creating ingress records 2025-02-24 20:33:08 +10:00
Pavel Sviderski ecec42ea60 feat(dns): cluster API for creating DNS records for cluster domain 2025-02-24 20:32:30 +10:00
Pavel Sviderski af4629b679 feat(dns): add CLI commands to reserve, show, delete cluster domain 2025-02-24 17:37:43 +10:00
Pavel Sviderski f6cf5cc250 feat(dns): cluster API for reserving/releasing cluster domain in Uncloud DNS 2025-02-24 17:35:19 +10:00
Pavel Sviderski f37bfd01fe feat: init cluster API for updating domain records in Uncloud DNS 2025-02-22 21:20:16 +10:00
Pavel Sviderski d9897e5d1e feat: set restart policy for all services to always 2025-02-19 16:53:55 +10:00
Pavel Sviderski bcb67a4073 refactor: e2e tests for container config validation 2025-02-19 15:19:00 +10:00
Pavel Sviderski bcb5a2a173 feat: deploy caddy service when adding new machine to cluster 2025-02-18 21:35:36 +10:00
Pavel Sviderski c3b88077ac feat: deploy caddy service when initialising new cluster on remote machine 2025-02-18 18:38:38 +10:00
Pavel Sviderski 98e1bdc9b5 fix: fail global deployment if no matchine machines found, improve caddy deploy output in particular 2025-02-17 20:31:37 +10:00
Pavel Sviderski e13408d79f feat: allow to filter machines for global service deployments 2025-02-17 20:26:40 +10:00
Pavel Sviderski 8610322682 fix: panic when pulling non-existent image 2025-02-17 16:12:37 +10:00
Pavel Sviderski 0be42f1f59 feat: print deployment plan for caddy deployment 2025-02-17 14:29:56 +10:00
Pavel Sviderski dcbfcbd0fb fix: show user-friendly image name when inspecting service 2025-02-16 21:12:56 +10:00
Pavel Sviderski d0e46186ed feat: look up latest caddy version tag when deploying caddy service 2025-02-16 21:10:06 +10:00
Pavel Sviderski 625a8baed3 feat: implement 'caddy deploy' command to deploy global service using caddy:latest image 2025-02-15 10:51:02 +10:00
Pavel Sviderski 98db5ed440 feat: caddy deploy command and stub implementation 2025-02-15 08:52:14 +10:00
Pavel Sviderski c4398f9f06 refactor: deploy global service using Deployment with RollingStrategy 2025-02-14 20:45:23 +10:00
Pavel Sviderski a53fa2c1d8 feat: do not redeploy global service if spec hasn't changed 2025-02-14 17:49:46 +10:00
Pavel Sviderski 5e2b2ac836 refactor: api.Container to use the detailed ContainerJSON struct instead of Container (Summary) 2025-02-14 16:37:40 +10:00
Pavel Sviderski 43f28284f1 feat: rolling deploy strategy for global service, always recreate 2025-02-13 21:32:08 +10:00
Pavel Sviderski b17fd7531f feat: detect conflicing service port on a container 2025-02-13 11:50:34 +10:00
Pavel Sviderski 9859be50a6 feat: add Deployment, Strategy, Operation primitives to compose service deployments 2025-02-12 21:17:30 +10:00
Pavel Sviderski f209851986 feat: progress for removing service(s) 2025-02-11 21:21:50 +10:00
Pavel Sviderski 4da51636fd refactor: remove service with StopContainer and RemoveContainer 2025-02-11 20:43:31 +10:00
Pavel Sviderski 76a9f8e8b6 refactor: client StartContainer pass serviceID 2025-02-11 16:47:52 +10:00
Pavel Sviderski a477db6b16 refactor: client CreateContainer and StartContainer methods 2025-02-11 15:59:36 +10:00
Pavel Sviderski 04fd1b1425 feat: introduce client.Service entity for managing service containers 2025-02-10 20:21:42 +10:00
Pavel Sviderski c0bf1f2930 add contributors section to README, feature @cedws 2025-02-10 11:38:35 +10:00
Pavel Sviderski 0bf759333a fix: populate ssh_key_file in the config when adding machine to cluster 2025-02-10 09:14:54 +10:00
Pavel SviderskiandGitHub b482d836aa Merge pull request #21 from cedws/identityfile
feat: add ssh_key_file field to connections config
2025-02-10 09:03:13 +10:00
Connor Edwards 3d2d9b78e8 fix: rename IdentityFile to SSHKeyFile 2025-02-09 18:39:44 +00:00
Connor Edwards 99219dc376 fix: set identity_file in config after init 2025-02-08 15:59:05 +00:00
Connor Edwards 408703360c fix: use os.UserHomeDir() instead of $HOME 2025-02-08 15:56:18 +00:00
Connor Edwards ac48b8bee6 feat: add identity_file field to config connections config 2025-02-08 15:56:18 +00:00
Pavel SviderskiandGitHub e3a310397b Merge pull request #20 from cedws/init-19
fix: execute valid bash cmd when running init as root
2025-02-08 16:42:14 +10:00
Connor Edwards b2f7c6fb33 fix: execute valid bash cmd when running init as root 2025-02-07 21:14:27 +00:00
Pavel Sviderski 653f47f507 add clean up step to quick start 2025-02-07 14:53:30 +10:00
Pavel Sviderski 5b66cec627 verify checksum in install_cli.sh 2025-02-06 15:17:33 +10:00
Pavel Sviderski 0d9a2b6e07 add brew install command to README 2025-02-06 14:54:55 +10:00
Pavel Sviderski fb68303825 fix: cli install script, add manual instruction 2025-02-06 14:38:55 +10:00
Pavel Sviderski cfe5b018f3 fix: install.sh install command 2025-02-06 14:01:55 +10:00
Pavel Sviderski b451f2cf7c fix: install_cli.sh install command 2025-02-06 13:56:56 +10:00
Pavel Sviderski 7d25bbafdf fix: goreleaser format for homebrew 2025-02-06 13:51:15 +10:00
Pavel Sviderski 58e1c2eea3 goreleaser: specify id for brew 2025-02-06 13:21:52 +10:00
Pavel Sviderski 6c791840bd update token for brew release 2025-02-06 13:15:27 +10:00
Pavel Sviderski 3af97306e1 add homebrew release to goreleaser 2025-02-06 13:09:34 +10:00
Pavel Sviderski d645958631 make DNS A record creation a separate step in quick start 2025-02-06 08:42:39 +10:00
Pavel Sviderski 48bc94828e link design doc 2025-02-05 21:35:16 +10:00
Pavel Sviderski a43f4bf5df minor README 2025-02-05 21:01:55 +10:00
Pavel Sviderski 22f705d6c9 update image name on landing 2025-02-05 20:48:02 +10:00
Pavel Sviderski efcb9ee947 add How it works section to README 2025-02-05 20:47:53 +10:00
Pavel Sviderski a347b32b65 fix typo 2025-02-03 11:55:03 +10:00
Pavel Sviderski 7e865b61c2 update get started links on landing 2025-02-03 11:48:11 +10:00
Pavel Sviderski b959f5d9f3 remove duplicate feature 2025-02-02 22:17:51 +10:00
Pavel Sviderski df64a00813 header padding 2025-02-02 22:15:48 +10:00
Pavel Sviderski 07dc5c373d update README: header, features, why uncloud 2025-02-02 22:12:42 +10:00
Pavel SviderskiandGitHub a6e8616b36 Update curl url in README.md 2025-01-30 15:14:07 +10:00
Pavel Sviderski 7041244728 update curl | sh url 2025-01-24 22:49:57 +10:00
Pavel Sviderski bee101aa3e fix asset template in .goreleaser config 2025-01-24 19:45:38 +10:00
Pavel Sviderski 21e70dd846 update quick start links on landing 2025-01-24 16:57:57 +10:00
Pavel Sviderski 91f870989a update README with basic Quick start 2025-01-24 16:56:40 +10:00
Pavel Sviderski bda1123ae6 add install_cli.sh script to be donwloaded at get.uncloud.run 2025-01-24 16:27:07 +10:00
Pavel Sviderski 190d0de61f fix warnings on landing 2025-01-23 14:58:05 +10:00
Pavel Sviderski f2e730796f enable Kit email subscription form 2025-01-23 14:49:41 +10:00
Pavel Sviderski bb77ff9246 format 2025-01-22 21:37:39 +10:00
Pavel Sviderski 5061097dff add landing section to follow the journey and subscribe via email 2025-01-22 21:26:49 +10:00
Pavel Sviderski 911522c388 comment out alpinejs 2025-01-22 17:18:26 +10:00
Pavel Sviderski 3ee7cb6e8e describe features on landing 2025-01-22 16:36:35 +10:00
Pavel Sviderski 2e00632959 update landing CSS 2025-01-21 19:38:49 +10:00
Pavel Sviderski 8e53fd46fa add how it works section with diagram 2025-01-21 19:38:38 +10:00
Pavel Sviderski 2835edadeb minor landing wording update 2025-01-21 17:57:33 +10:00
Pavel Sviderski f7c4490c10 add diagram for landing page with drawio source 2025-01-21 16:30:45 +10:00
Pavel Sviderski b3102480c0 rename site -> website 2025-01-20 13:41:55 +10:00
Pavel Sviderski cbe7c563cd add value proposition section: Cloud-like experience you control 2025-01-20 13:40:58 +10:00
Pavel Sviderski b016252821 add terminal demo to hero section 2025-01-16 15:50:40 +10:00
Pavel Sviderski 6c197085d0 update hero 2025-01-15 17:10:38 +10:00
Pavel Sviderski c29fa571e1 add favicons 2025-01-10 19:07:55 +10:00
Pavel Sviderski 092fe254aa fix navbar on mobile 2025-01-10 15:22:26 +10:00
Pavel Sviderski 6951221b88 github and get started links in navbar 2025-01-10 15:04:47 +10:00
Pavel Sviderski fa54584128 add logo and project name 2025-01-10 14:07:44 +10:00
Pavel Sviderski d922f7bfed reformat index.html 2025-01-10 12:42:13 +10:00
Pavel Sviderski 18ec186e81 init landing page template 2025-01-10 12:39:21 +10:00
Pavel Sviderski 2a898b3eee add -v|--volume to bind mount host path into a service container 2024-12-27 21:25:33 +10:00
Pavel Sviderski 042b44cbb6 fix port binding for host ports 2024-12-27 21:23:56 +10:00
Pavel Sviderski 3f7726e5fb fix SlogTextHandler to respect minimum log level 2024-12-24 17:07:37 +10:00
Pavel Sviderski 2b3f325075 generate caddy config with routes from published http(s) ports 2024-12-23 20:41:04 +10:00
Pavel Sviderski be2a26e626 remove example from run command 2024-12-19 14:19:11 +10:00
Pavel Sviderski 4479490db2 create custom NewSlogTextHandler log handler to override the json one patched by caddy pkg 2024-12-19 13:29:58 +10:00
Pavel Sviderski 7556f8cb12 generate basic caddy.json with empty http and https servers 2024-12-18 22:14:58 +10:00
Pavel Sviderski bfbcec34e8 add caddy/v2 dependency to generate a config 2024-12-17 17:52:30 +11:00
Pavel Sviderski 8eea34d8dc remove wrong TODO 2024-12-11 16:50:15 +10:00
Pavel Sviderski 6b80ef9db8 subscribe to container changes in Caddyfile controller 2024-12-11 15:14:32 +10:00
Pavel Sviderski 2134f3c1d9 add a stub for Caddyfile controller 2024-12-11 12:17:15 +10:00
Pavel Sviderski 0652123bc7 encode and decode published ports using container labels 2024-12-10 19:21:52 +10:00
Pavel Sviderski 4de7ecd747 add String() method for PortSpec to convert to publish format 2024-12-10 19:14:26 +10:00
Pavel Sviderski e8908ced9e parse --publish ports into spec 2024-12-10 17:57:26 +10:00
Pavel Sviderski deedbcb8a9 minor validation for port 2024-12-10 17:37:42 +10:00
Pavel Sviderski 8bfb3b80ed basic validation for hostname in port publishing 2024-12-10 17:24:29 +10:00
Pavel Sviderski 848abbd2f8 add validation for PortSpec 2024-12-10 16:35:47 +10:00
Pavel Sviderski 9cb4b35546 implement PortSpec parsing 2024-12-10 15:24:34 +10:00
Pavel Sviderski 26df6740e3 define PortSpec type for publishing ports 2024-12-09 20:25:34 +10:00
Pavel Sviderski 537789902b update publish flag description with lb and host formats 2024-12-09 17:06:33 +10:00
Pavel Sviderski 4088fdfd84 print image pull progress when running a service 2024-12-06 18:27:26 +10:00
Pavel Sviderski 01a71b4b38 run each container in global service in parallel 2024-12-06 16:00:10 +10:00
Pavel Sviderski 73d41c75b3 add compose-like progress for running service 2024-12-06 15:44:23 +10:00
Pavel Sviderski f85b5dd01d add service rm CLI command 2024-12-05 17:32:30 +10:00
Pavel Sviderski 1fb8a3d7a0 add ListServices method and ls CLI command 2024-12-05 17:21:54 +10:00
Pavel Sviderski 3ce9c8be04 fix run command, support 1 replica and global mode 2024-12-05 16:24:04 +10:00
Pavel Sviderski 3ac166e6cb move common api types to api package 2024-12-05 15:17:36 +10:00
Pavel Sviderski 88ce2d5181 implement RunService for replicated mode with only 1 replica 2024-12-05 14:01:56 +10:00
Pavel Sviderski 8bddbec7bd wait for test cluster to be ready (all machines up) 2024-12-05 13:45:46 +10:00
Pavel Sviderski a40fd613e8 refactor RemoveService using InspectService 2024-12-05 12:27:51 +10:00
Pavel Sviderski 99c9aabdfd implement more consistent InspectService that broadcasts Docker requests to all machines 2024-12-05 12:13:17 +10:00
Pavel Sviderski 5504951a8a add e2e test for global service creation and removal 2024-12-04 18:28:22 +10:00
Pavel Sviderski 2727c3c97e implement RunService in global mode and RemoveService 2024-12-04 18:26:59 +10:00
Pavel Sviderski 7d47d274a0 minor 2024-12-04 18:24:47 +10:00
Pavel Sviderski a4247b0097 refactor grpc proxy backends to use machine management IP in metadata, including local responses 2024-12-04 18:23:51 +10:00
Pavel Sviderski 11612a802c forward NotFound errors from StartContainer and RemoveContainer 2024-12-04 16:29:04 +10:00
Pavel Sviderski d7bad5ed46 add todo 2024-12-04 15:39:13 +10:00
Pavel Sviderski c0dff56378 fix filter unmarshalling for list containers 2024-12-04 15:37:52 +10:00
Pavel Sviderski 00dbbec62b refactor grpc-proxy backends to enhance responses from local backends with metadata in one2many mode 2024-12-04 14:47:31 +10:00
Pavel Sviderski 9755a6973d allow one2many for ListContainers 2024-12-04 12:43:52 +10:00
Pavel Sviderski 16120b80ba add ListContainers and RemoveContainer methods to Docker service 2024-12-04 12:15:55 +10:00
Pavel Sviderski 386e4e25f3 fix context in PullImage 2024-12-03 18:33:48 +10:00
Pavel Sviderski 43ddb5c363 add PullImage method to gRPC Docker service 2024-12-03 18:07:00 +10:00
Pavel Sviderski 417427b4d5 remove duplicate ErrNotFound in client 2024-12-03 15:47:33 +10:00
Pavel Sviderski 5da1fc5387 support providing TEST_CLUSTER_NAME for e2e tests 2024-12-03 15:46:19 +10:00
Pavel Sviderski 5621a84e99 add Mode to Service entity 2024-12-03 11:37:13 +10:00
Pavel Sviderski 8db7ac84a0 add subprocess corrosion service runner 2024-12-03 11:07:09 +10:00
Pavel Sviderski 77890b8a8a strip one proto layer in ListMachines method 2024-12-03 11:05:58 +10:00
Pavel Sviderski daada3bbe9 add service inspect command, introduce squirrel for sql query building 2024-12-03 10:56:29 +10:00
Pavel Sviderski 21d55dc69a minor refactor e2e cluster test 2024-12-02 20:37:01 +10:00
Pavel Sviderski 6eb373e4de init InspectService method 2024-12-02 20:36:31 +10:00
Pavel Sviderski 6a4a7f2e6a update e2e lifecycle test to test store reconciliation on every node 2024-12-02 17:21:06 +10:00
Pavel Sviderski 349dd0d2ef add basic e2e cluster lifecycle test 2024-11-29 20:40:49 +10:00
Pavel Sviderski cf5818d963 merge ucind cluster configuration to the user’s uncloud config 2024-11-29 13:30:15 +10:00
Pavel Sviderski faf3378659 ucind: wait for machine to become ready through polling API 2024-11-29 12:08:57 +10:00
Pavel Sviderski bb967461d0 add logging for starting/restarting corrosion service 2024-11-29 12:08:05 +10:00
Pavel Sviderski 0b5dd2f7ba fix network peer configuration when just joining the cluster before store is synced 2024-11-29 11:27:51 +10:00
Pavel Sviderski d5f0bca4a0 ucind: call cleanup on exit as well, e.g. main process crash 2024-11-29 09:53:20 +10:00
Pavel Sviderski bdd034d169 proper terminate all background processes in ucind containers 2024-11-29 09:46:29 +10:00
Pavel Sviderski 48767125f4 ucind: init cluster with created machines 2024-11-28 19:57:51 +10:00
Pavel Sviderski 035f44896d refactor cluster network related constants 2024-11-28 19:57:26 +10:00
Pavel Sviderski 3d5fe0a824 add support for tcp cluster connections in uncloud config 2024-11-28 18:53:47 +10:00
Pavel Sviderski 6d1c3aa5b2 add basic client TCPConnector to directly connect to a gRPC endpoint 2024-11-28 18:53:03 +10:00
Pavel Sviderski cd60d5178f ucind: create machines in containers with published API ports 2024-11-28 16:15:57 +10:00
Pavel Sviderski 7aa7a9fea1 init ucind CLI commands 2024-11-28 13:54:37 +10:00
Pavel Sviderski 73785007cb init ucind (Uncloud in Docker) provisioner 2024-11-28 13:54:10 +10:00
Pavel Sviderski 24a824b890 add make targets to build and push ucind image 2024-11-27 17:14:41 +10:00
Pavel Sviderski 085ee285e8 add make targets to build and push corrosion image 2024-11-27 16:53:57 +10:00
Pavel Sviderski 6daa1291b5 update TODO 2024-11-27 16:41:20 +10:00
Pavel Sviderski c3d3a4242e run corrosion in Docker as uncloud user, bind mount data dir 2024-11-27 16:35:35 +10:00
Pavel Sviderski 871fd8c035 proxy uncloud.sock socket to tcp port when running machine in container 2024-11-27 16:33:05 +10:00
Pavel Sviderski a6ccd6055a move fs and file ownership related funcs to fs package 2024-11-27 14:30:08 +10:00
Pavel Sviderski ca607ee82d pull corrosion image if doesn't exist 2024-11-27 13:37:22 +10:00
Pavel Sviderski 1c8e60aac0 workaround cgroup race in dind 2024-11-27 13:18:16 +10:00
Pavel Sviderski d02fc4d981 update Makefile to remove built binary 2024-11-27 10:38:25 +10:00
Pavel Sviderski 410f74425c update dind machine image to run dockerd in background 2024-11-27 10:35:49 +10:00
Pavel Sviderski 6e0df53f13 add in docker service runner for corrosion 2024-11-27 10:34:42 +10:00
Pavel Sviderski e9d07687f2 build corrosion image based on wolfi] 2024-11-26 20:15:31 +10:00
Pavel Sviderski 95175d10c5 init Dockerfile to run machine in container 2024-11-25 19:49:59 +10:00
Pavel Sviderski df25b0d098 add --mode flag to run command 2024-11-23 15:56:03 +10:00
Pavel Sviderski 5c1a3dd694 move generic docker types to top level docker package 2024-11-23 15:35:26 +10:00
Pavel Sviderski 6d39174b51 add Healthy method to Container that derives it from status string 2024-11-21 20:50:55 +10:00
Pavel Sviderski e851f25b12 delete removed Docker containers from store, sync only uncloud labeled 2024-11-19 19:38:11 +10:00
Pavel Sviderski 3c1d2201ce add methods to create, update, list, delete containers in store 2024-11-19 19:35:38 +10:00
Pavel Sviderski b4c2ec3a2a sync new and updated containers to cluster store, no deletion yet 2024-11-19 16:06:16 +10:00
Pavel Sviderski bf10b21da5 minor comment 2024-11-18 20:07:51 +10:00
Pavel Sviderski 2819ba63aa add sync_status field to containers table 2024-11-18 19:50:40 +10:00
Pavel Sviderski 54a8b09c9f subscribe to Docker container events with backoff 2024-11-18 14:43:42 +10:00
Pavel Sviderski 24371eb401 move Docker manager to docker package 2024-11-17 19:41:47 +10:00
Pavel Sviderski 9c9e143dc8 minor rename 2024-11-16 20:28:32 +10:00
Pavel Sviderski 19d53c4f67 move Docker management from network controller to standalone DockerManager 2024-11-16 20:25:34 +10:00
Pavel Sviderski 40f929847c add updated_at field to containers table 2024-11-16 07:37:43 +10:00
Pavel Sviderski e73e38957a add index on containers.machine_id 2024-11-16 07:00:09 +10:00
Pavel Sviderski e82b78df49 comment DB schema 2024-11-16 06:43:24 +10:00
Pavel Sviderski 741378d0b7 create containers table in Corrosion store 2024-11-15 21:35:33 +10:00
Pavel Sviderski d31595820d run service container in uncloud Docker network 2024-11-14 19:19:50 +10:00
Pavel Sviderski 035601d366 generate container name and assign service labels to container 2024-11-14 12:05:48 +10:00
Pavel Sviderski 5c96f75958 generate a name for the service 2024-11-14 11:56:29 +10:00
Pavel Sviderski 2c62e1f6df refactor machine name generation using secrets package 2024-11-14 11:47:22 +10:00
Pavel Sviderski 22bca16226 include alpha-numeric random string generation in secret package 2024-11-14 11:45:28 +10:00
Pavel Sviderski a4be45186d find first available machine for running a service 2024-11-14 09:12:25 +10:00
Pavel Sviderski 5e79697505 move RunService from CLI to the client, print output in the command 2024-11-13 19:52:46 +10:00
Pavel Sviderski 6736839ccb refactor machine ls command: move formatter logic to the command 2024-11-13 17:12:02 +10:00
Pavel Sviderski 913fb2da0c update ListMachines to include machine state from Corrosion membership 2024-11-13 16:48:55 +10:00
Pavel Sviderski bd0e7f44b2 init corrosion admin client in machine 2024-11-12 15:23:02 +10:00
Pavel Sviderski 81cf5f5605 refactor corrosion admin client to establish a connection on every command 2024-11-12 15:22:47 +10:00
Pavel Sviderski cedafd069d implement ClusterMembershipStates for corrosion admin client 2024-11-12 14:29:29 +10:00
Pavel Sviderski 93dfa5ca9d implement admin client for Corrosion admin API 2024-11-11 20:51:11 +10:00
Pavel Sviderski c70c8a6ec1 validate machine info before returning in ListMachine 2024-11-10 19:38:55 +10:00
Pavel Sviderski 5c74b09f45 allow running a service on a specified machine 2024-11-10 19:38:19 +10:00
Pavel Sviderski 1316dd3ffc fix proxy Director usage and close on shutdown 2024-11-08 15:50:28 +10:00
Pavel Sviderski 4493bb3c27 support gRPC proxy to remote backends 2024-11-08 15:49:21 +10:00
Pavel Sviderski b23699470b add EmptyResponse common message 2024-11-08 11:48:58 +10:00
Pavel Sviderski 34316c77cf add helper Empty message 2024-11-08 11:32:42 +10:00
Pavel Sviderski c2b466b046 add common Metadata message to be injected in reply messages by gRPC proxy 2024-11-08 11:28:04 +10:00
Pavel Sviderski d36429e6e5 replace API server with transparent gRPC proxy that routes requests to local machine API 2024-11-05 20:05:31 +10:00
Pavel Sviderski 17b96a8744 move default uncloud socket to /run/uncloud/uncloud.sock 2024-11-04 20:02:48 +10:00
Pavel Sviderski a54666a7ac update run command to run a stub service container on the connected node 2024-11-01 15:05:07 +10:00
Pavel Sviderski 5cb765d286 add Docker gRPC service to create and start a Docker container on machine 2024-11-01 15:04:06 +10:00
Pavel Sviderski 5bf3cba2d6 sketch run command 2024-10-31 20:07:45 +10:00
Pavel Sviderski 8a3f0032ae format machine command flags 2024-10-31 20:07:25 +10:00
Pavel Sviderski 5fe77c688b exclude Docker bridge br-[0-9a-f]{12} interfaces from machine routable IPs 2024-10-31 18:35:59 +10:00
Pavel Sviderski 61dc97228d use ls command instead of list by default 2024-10-31 18:35:12 +10:00
Pavel Sviderski e6df331fb8 fix options for machine list command 2024-10-31 18:32:11 +10:00
Pavel Sviderski 9dbcf1e902 do not build and deploy uncloud binary to dev machines 2024-10-31 18:08:36 +10:00
Pavel Sviderski 2f6c988125 do not rotate endpoints on daemon restart, handle endpoint changes on established reverse peer connections 2024-10-13 01:15:32 -04:00
Pavel Sviderski 411c3398ce upgrade to Go 1.23 2024-10-10 23:06:24 -04:00
Pavel Sviderski 3a644771ca watch for peer endpoint changes and preserve in machine state 2024-10-08 16:51:40 +10:00
Pavel Sviderski 2eff7d1e22 format 2024-10-07 20:06:56 +10:00
Pavel Sviderski f06dd865fa update existing WG peers instead of replacing to not interrupt and lose track of handshakes 2024-10-07 20:06:15 +10:00
Pavel Sviderski ee580df6af automatically rotate WG peer endpoints if can't establish a connection 2024-10-07 17:24:18 +10:00
Pavel Sviderski 2c0cf28509 fix peer status persistence, update status every second 2024-10-07 15:35:15 +10:00
Pavel Sviderski 7b98ed34df calculate WG peer connection status 2024-10-07 14:58:21 +10:00
Pavel Sviderski 5094149406 improve error message for adding machine that already exists in the cluster 2024-10-04 20:45:54 +10:00
Pavel Sviderski 2fd1ee203a fix: access to /run/uncloud.sock for a non-root user when initialising a fresh machine 2024-10-04 20:31:23 +10:00
Pavel Sviderski 336bdf9f4b add minor log 2024-10-04 18:46:52 +10:00
Pavel Sviderski c80c75e4e1 install.sh: download corrosion binary from github release 2024-10-04 18:37:24 +10:00
Pavel Sviderski e5bda92ec6 add opptimistic sleep on corrosion service start to wait for schema initialisation 2024-10-04 16:19:07 +10:00
Pavel Sviderski fd3ad91136 update network controller to reconfigure network peers on machine changes 2024-10-04 16:18:08 +10:00
Pavel Sviderski 94186b2ae7 allow to subscribe to machine changes in the cluster store 2024-10-04 16:14:59 +10:00
Pavel Sviderski 1bcb8836ef auto resubscribe a corrosion subscription if an error occurs 2024-10-03 16:13:03 +10:00
Pavel Sviderski 840943b3b0 split corrosion client subscription in a separate file 2024-10-03 14:10:09 +10:00
Pavel Sviderski 518eae9bd2 split corrosion queries in a separate file, make subscription more consistent with queries 2024-10-03 14:07:25 +10:00
Pavel Sviderski 06a4001080 refactor corrosion client to accept custom http client, fix backoff for every request 2024-10-03 13:58:28 +10:00
Pavel Sviderski b73577e1a0 implement subscription in corrosion client 2024-10-02 18:28:54 +10:00
Pavel Sviderski c3ba43a8fa delete old non-distributed cluster state, fix storing endpoints in store 2024-10-02 13:03:17 +10:00
Pavel Sviderski c275273617 fix install.sh to start uncloud.service after installing all systemd units 2024-10-02 12:44:38 +10:00
Pavel Sviderski b688742e5a remove the concept of cluster user (at least the current implementation) 2024-10-02 11:50:40 +10:00
Pavel Sviderski 4ec87db8e8 add init machine to store when initialising a cluster 2024-10-02 11:34:54 +10:00
Pavel Sviderski c6358474ac add comments 2024-10-02 09:25:12 +10:00
Pavel Sviderski 8fbf406007 relax relationship between corrosion service, init cluster with new store 2024-10-01 23:37:07 +10:00
Pavel Sviderski e123f7214e add exponential backoff for network errors in corrosion client 2024-10-01 23:15:36 +10:00
Pavel Sviderski 3287691a31 start corrosion service even when machine is not initialised to be able to init cluster 2024-10-01 15:04:20 +10:00
Pavel Sviderski 6f9fc69c2e fix corrosion data dir, add reload to systemd unit 2024-10-01 13:33:54 +10:00
Pavel Sviderski 0effba0adb create corrosion store when initialising machine 2024-09-30 23:10:56 +10:00
Pavel Sviderski 667ecadd30 init cluster store backed by corrosion 2024-09-30 20:46:41 +10:00
Pavel Sviderski 68845e7607 refactor corrosion service init for machine 2024-09-30 20:46:29 +10:00
Pavel Sviderski 6d4698fc1d implement Query in corrosion API client 2024-09-30 19:00:51 +10:00
Pavel Sviderski 708e14a4e0 add Exec and ExecMulti methods to corrosion API client 2024-09-29 10:25:54 +10:00
Pavel Sviderski c80de16ebe init corrosion API client 2024-09-28 20:42:57 +10:00
Pavel Sviderski 3e9718f08c update store schema when configuring corrosion, create machines table 2024-09-27 20:10:05 +10:00
Pavel Sviderski 69c9167355 configure corrosion bootstrap peers when initialising a machine 2024-09-26 20:36:13 +10:00
Pavel Sviderski b25938acc1 clean up corrosion systemd service 2024-09-26 19:14:25 +10:00
Pavel Sviderski 1ec3376be5 remove indentations from corrosion config 2024-09-26 18:06:45 +10:00
Pavel Sviderski aa236aac10 do not init corrosion data dir in install.sh 2024-09-26 17:00:48 +10:00
Pavel Sviderski 9fcc319c3a create corrosion config 2024-09-26 17:00:30 +10:00
Pavel Sviderski 2f7463ebe3 refactor corrosion into interface service 2024-09-25 23:59:56 +10:00
Pavel Sviderski 498d35e866 install.sh: install corrosion systemd service 2024-09-25 22:10:01 +10:00
Pavel Sviderski ba67b96734 update install.sh to download uncloudd binary from the latest GitHub release 2024-09-23 21:39:51 +10:00
367 changed files with 67197 additions and 1999 deletions
+12
View File
@@ -0,0 +1,12 @@
# Ignore everything and use an allowlist of what is allowed to not package any secrets by accident.
*
# Allow files and directories.
!cmd/
!internal/
!pkg/
!scripts/docker/
!go.*
# Ignore unnecessary files inside allowed directories.
**/.DS_Store
+20
View File
@@ -0,0 +1,20 @@
# EditorConfig is awesome: https://editorconfig.org
root = true
# Default settings for all files
[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
# Settings for Go files
[*.go]
indent_style = tab
indent_size = 4
# Settings for Bash scripts
[*.sh]
indent_style = space
indent_size = 4
+1
View File
@@ -0,0 +1 @@
github: [psviderski]
+36
View File
@@ -0,0 +1,36 @@
---
name: Bug Report
about: Create a report to help us improve
title: "[BUG] "
labels: bug
assignees: ""
---
**Describe the bug**
<!-- A clear and concise description of what the bug is. -->
**How to reproduce**
<!-- Steps to reproduce the behavior:
1. Run ...
2. Do ...
-->
**Expected behavior**
<!-- A clear and concise description of what you expected to happen. -->
**Environment:**
- Uncloud versions:
- Control (client) node (`uc --version`):
- Uncloud daemon (from the server) (`uncloudd --version`):
- OS version (`uname -a`):
- Client (control node):
- Server:
**Additional context**
<!-- Add any other context about the problem here. -->
+1
View File
@@ -0,0 +1 @@
../AI.md
Binary file not shown.

After

Width:  |  Height:  |  Size: 525 KiB

+69
View File
@@ -0,0 +1,69 @@
name: Go Tests
on:
push:
branches:
- "main"
- "test/**"
- "release/**"
pull_request:
branches:
- main
paths:
- ".github/**"
- "**.go"
- "**.proto"
- "go.*"
- "Makefile"
- "scripts/**"
- "test/**"
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Set up Go
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with:
go-version: "1.25.1"
- name: Install dependencies
run: |
go mod tidy
git diff --exit-code ||
(echo "go.mod or go.sum has changed. Please run 'go mod tidy' and commit the changes." && exit 1)
- name: Run tests
run: |
make ucind-image
make test
timeout-minutes: 10
check-protobuf:
runs-on: ${{ matrix.os }}
timeout-minutes: 10
strategy:
matrix:
# Run on more platforms so that mise lockfile checksums are checked/generated for all platforms.
# See https://github.com/jdx/mise/issues/4276 for details.
os: [ubuntu-latest, macos-latest]
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Install Mise
uses: jdx/mise-action@13abe502c30c1559a5c37dff303831bab82c9402 # v2.2.3
with:
version: "2025.9.6"
env:
GITHUB_TOKEN: ${{ github.token }}
- name: Generate Protobuf files
run: |
make proto
# check if the generated files are up to date
git diff --exit-code ||
(echo "Generated protobuf files are not up to date. Please run 'make proto' and commit the changes." && exit 1)
+39
View File
@@ -0,0 +1,39 @@
name: Lint
on:
push:
branches:
- "main"
- "test/**"
- "release/**"
pull_request:
branches:
- main
paths:
- "**.go"
- "go.*"
permissions:
contents: read
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Set up Go
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with:
go-version: "1.25.1"
- name: golangci-lint
uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0
with:
version: v2.4.0
- name: Format code
run: |
make format
git diff --exit-code ||
(echo "Code is not formatted. Please run 'make format' and commit the changes." && exit 1)
timeout-minutes: 10
+13 -2
View File
@@ -8,6 +8,9 @@
*.so
*.dylib
# OS X
.DS_Store
# Test binary, built with `go test -c`
*.test
@@ -24,5 +27,13 @@ go.work.sum
.env
.idea/
data
dist/
/data/
/dist/
/uc
/tmp
# Web
node_modules/
# VS Code
.vscode
+42
View File
@@ -0,0 +1,42 @@
version: "2"
run:
concurrency: 4
tests: true
timeout: 5m
linters:
default: none
enable:
- bodyclose
# - dogsled
- dupl
# - errcheck
- gochecknoinits
- goconst
# - gocritic
# - gocyclo
# - godot
# - gosec
# - govet
- ineffassign
- misspell
- nakedret
# - prealloc
# - revive
# - staticcheck
- unconvert
# - unparam
# - unused
- whitespace
exclusions:
rules:
- path: ^test/e2e
linters:
- goconst # constants here add no value, so we skip goconst only for test/e2e.
formatters:
enable:
- gofumpt
- goimports
+27 -4
View File
@@ -24,6 +24,8 @@ builds:
goarch:
- amd64
- arm64
ldflags:
- -s -w -X github.com/psviderski/uncloud/internal/version.version={{ .Version }}
- id: uncloudd
main: ./cmd/uncloudd
@@ -35,28 +37,34 @@ builds:
goarch:
- amd64
- arm64
ldflags:
- -s -w -X github.com/psviderski/uncloud/internal/version.version={{ .Version }}
archives:
- id: uncloud
builds:
- uncloud
format: gz
formats:
# gz is not compatible with homebrew releases, so we use tar.gz
- tar.gz
name_template: >-
{{ .Binary }}_
{{- if eq .Os "darwin"}}macos
{{- else}}{{.Os}}{{end}}_
{{- .Arch }}"
{{- .Arch }}
# use zip for windows archives
format_overrides:
- goos: windows
format: zip
formats:
- zip
files:
- none*
- id: uncloudd
builds:
- uncloudd
format: gz
formats:
- tar.gz
name_template: "{{ .Binary }}_{{ .Os }}_{{ .Arch }}"
files:
- none*
@@ -70,3 +78,18 @@ changelog:
exclude:
- "^docs:"
- "^test:"
brews:
- name: uncloud
description: "Uncloud CLI"
homepage: "https://uncloud.run"
ids:
- uncloud
install: |
bin.install "uncloud"
bin.install_symlink "uncloud" => "uc"
skip_upload: false
repository:
owner: psviderski
name: homebrew-tap
token: "{{ .Env.GITHUB_TOKEN_HOMEBREW_TAP }}"
+97
View File
@@ -0,0 +1,97 @@
[[tools."aqua:vektra/mockery"]]
version = "3.5.3"
backend = "aqua:vektra/mockery"
[tools."aqua:vektra/mockery".platforms.linux-x64]
checksum = "sha256:ebce416b0175338525246c376885a1579ca6cd4d4015140ba0c70e6b5339a39c"
size = 5498859
url = "https://github.com/vektra/mockery/releases/download/v3.5.3/mockery_3.5.3_Linux_x86_64.tar.gz"
[tools."aqua:vektra/mockery".platforms.macos-arm64]
checksum = "sha256:a3a94b14c7414e148f2252199ffc4a0108d311358f3d336cbe05bb73cb203704"
size = 5238877
url = "https://github.com/vektra/mockery/releases/download/v3.5.3/mockery_3.5.3_Darwin_arm64.tar.gz"
[[tools.go]]
version = "1.25.1"
backend = "core:go"
[tools.go.platforms.linux-x64]
checksum = "sha256:7716a0d940a0f6ae8e1f3b3f4f36299dc53e31b16840dbd171254312c41ca12e"
size = 59660846
url = "https://dl.google.com/go/go1.25.1.linux-amd64.tar.gz"
[tools.go.platforms.macos-arm64]
checksum = "sha256:68deebb214f39d542e518ebb0598a406ab1b5a22bba8ec9ade9f55fb4dd94a6c"
size = 57906702
url = "https://dl.google.com/go/go1.25.1.darwin-arm64.tar.gz"
[[tools.golangci-lint]]
version = "2.4.0"
backend = "aqua:golangci/golangci-lint"
[tools.golangci-lint.platforms.linux-x64]
checksum = "sha256:fae792524c04424c0ac369f5b8076f04b45cf29fc945a370e55d369a8dc11840"
size = 14126779
url = "https://github.com/golangci/golangci-lint/releases/download/v2.4.0/golangci-lint-2.4.0-linux-amd64.tar.gz"
[tools.golangci-lint.platforms.macos-arm64]
checksum = "sha256:cd4dd53fa09b6646baff5fd22b8c64d91db02c21c7496df27992d75d34feec59"
size = 13542526
url = "https://github.com/golangci/golangci-lint/releases/download/v2.4.0/golangci-lint-2.4.0-darwin-arm64.tar.gz"
[[tools.node]]
version = "22.20.0"
backend = "core:node"
[tools.node.platforms.linux-x64]
checksum = "sha256:eeaccb0378b79406f2208e8b37a62479c70595e20be6b659125eb77dd1ab2a29"
size = 56645685
url = "https://nodejs.org/dist/v22.20.0/node-v22.20.0-linux-x64.tar.gz"
[tools.node.platforms.macos-arm64]
checksum = "sha256:cc04a76a09f79290194c0646f48fec40354d88969bec467789a5d55dd097f949"
size = 49838299
url = "https://nodejs.org/dist/v22.20.0/node-v22.20.0-darwin-arm64.tar.gz"
[[tools.protoc]]
version = "27.3"
backend = "aqua:protocolbuffers/protobuf/protoc"
[tools.protoc.platforms.linux-x64]
checksum = "blake3:8c2938accf7061164ea3968529a47ced1db4824ca553857f6903c68ae4347739"
size = 3235873
url = "https://github.com/protocolbuffers/protobuf/releases/download/v27.3/protoc-27.3-linux-x86_64.zip"
[tools.protoc.platforms.macos-arm64]
checksum = "blake3:650c2494c003e9fddd6f4164a0a36e31da51ae7e1e5208da8cb560a4fe886fcd"
size = 2372763
url = "https://github.com/protocolbuffers/protobuf/releases/download/v27.3/protoc-27.3-osx-aarch_64.zip"
[[tools.protoc-gen-go]]
version = "1.34.2"
backend = "aqua:protocolbuffers/protobuf-go/protoc-gen-go"
[tools.protoc-gen-go.platforms.linux-x64]
checksum = "blake3:f765e393e0f0b25070c13848d88f5e6303e66090132afa1b753e981507a4174e"
size = 1848331
url = "https://github.com/protocolbuffers/protobuf-go/releases/download/v1.34.2/protoc-gen-go.v1.34.2.linux.amd64.tar.gz"
[tools.protoc-gen-go.platforms.macos-arm64]
checksum = "blake3:c237256a7f718ac4ccdc037ee30025a0b7c3f79300d7831ec59d76d782443fd5"
size = 1784028
url = "https://github.com/protocolbuffers/protobuf-go/releases/download/v1.34.2/protoc-gen-go.v1.34.2.darwin.arm64.tar.gz"
[[tools.protoc-gen-go-grpc]]
version = "1.5.1"
backend = "aqua:grpc/grpc-go/protoc-gen-go-grpc"
[tools.protoc-gen-go-grpc.platforms.linux-x64]
checksum = "blake3:197739132f0408f017d0c83773ac1ace85c0572473fdd658ec6e4ccbf20e0552"
size = 4859466
url = "https://github.com/grpc/grpc-go/releases/download/cmd/protoc-gen-go-grpc/v1.5.1/protoc-gen-go-grpc.v1.5.1.linux.amd64.tar.gz"
[tools.protoc-gen-go-grpc.platforms.macos-arm64]
checksum = "blake3:3c668207565d23ea2cf006d699207e3a804ebb929a534bd38a3b4ff4e941e00c"
size = 4565944
url = "https://github.com/grpc/grpc-go/releases/download/cmd/protoc-gen-go-grpc/v1.5.1/protoc-gen-go-grpc.v1.5.1.darwin.arm64.tar.gz"
+10 -1
View File
@@ -1,5 +1,14 @@
[settings]
experimental = true
[tools]
go = "1.22"
"aqua:vektra/mockery" = "3.5.3"
go = "1.25.1"
golangci-lint = "2.4.0"
node = "22.20.0"
protoc = "27.3"
protoc-gen-go = "1.34.2"
protoc-gen-go-grpc = "1.5.1"
[env]
_.file = ".env"
+4
View File
@@ -0,0 +1,4 @@
packages:
github.com/psviderski/uncloud/internal/machine/caddyconfig:
interfaces:
CaddyfileValidator:
+258
View File
@@ -0,0 +1,258 @@
# AI.md - Uncloud Project Guide
This document provides comprehensive information about the Uncloud project for AI assistants to understand the codebase, architecture, and development practices.
## Project Overview
**Uncloud** is a lightweight clustering and container orchestration tool that enables deployment and management of web applications across cloud VMs and bare metal servers. It creates a secure WireGuard mesh network between Docker hosts and provides automatic service discovery, load balancing, HTTPS ingress, and simple CLI commands for application management.
### Key Characteristics
- **Language**: Go
- **Architecture**: Decentralized, no control plane
- **Target**: Self-hosted infrastructure without Kubernetes complexity
- **License**: View LICENSE file for details
- **Status**: Active development, not yet ready for production
## Core Features
### 🏗️ Infrastructure
- **Multi-machine deployment**: Combine cloud VMs, dedicated servers, and bare metal
- **Zero-config networking**: Automatic WireGuard mesh with NAT traversal
- **Decentralized design**: No central control plane, all machines are equal
- **Service discovery**: Built-in DNS server resolves service names to container IPs
### 🚀 Application Management
- **Docker Compose compatibility**: Uses familiar Docker Compose format
- **Zero-downtime deployments**: Rolling updates without service interruption
- **Automatic HTTPS**: Caddy reverse proxy with Let's Encrypt integration
- **Managed DNS**: Free `*.cluster.uncloud.run` subdomains via Uncloud DNS service
- **Cross-machine scaling**: Run containers across multiple machines
### 🔧 Developer Experience
- **Docker-like CLI**: Familiar commands (`uc` binary)
- **Imperative operations**: Direct commands vs. declarative state reconciliation
- **Remote management**: Control entire infrastructure via SSH to any machine
- **Minimal overhead**: ~150MB RAM footprint per machine
## Architecture
### Core Components
1. **CLI (`uc`)** - Main user interface for cluster management
2. **Daemon (`uncloudd`)** - Machine daemon running on each node
3. **Corrosion** - Distributed SQLite database for cluster state (Fly.io project)
4. **Caddy** - Reverse proxy for HTTPS termination and routing
5. **WireGuard** - Secure mesh networking between machines
### Network Architecture
- Each machine gets unique subnet (e.g., `10.210.0.0/24`, `10.210.1.0/24`)
- Containers get cluster-unique IPs for direct communication
- Automatic peer discovery and key management
- NAT traversal for machines behind firewalls
### State Management
- **CRDT-based distributed storage** using Corrosion
- **Eventually consistent** state across all machines
- **Gossip protocol** (Serf) for state propagation
- **No quorum requirements** - partial network splits remain functional
## Project Structure
### Key Directories
- **`cmd/`**: Contains main applications
- `uncloud/`: CLI tool with subcommands for machine, service, volume management
- `uncloudd/`: Daemon that runs on each machine
- `ucind/`: Development cluster management for testing
- **`internal/`**: Internal implementation packages
- `cli/`: Command-line interface logic
- `machine/`: Machine lifecycle and state management
- `daemon/`: Daemon implementation and gRPC services
- `dns/`: Internal DNS server for service discovery
- **`pkg/`**: Public API packages for external use
- `api/`: Core API types and definitions
- `client/`: Client libraries for interacting with Uncloud
- **`experiment/`**: Experimental features and prototypes
- **`scripts/`**: Installation and utility scripts
- **`test/`**: Test suites and test infrastructure
- **`website/`**: Documentation website (Docusaurus)
- `landing/`: Landing page
- `docs/`: User documentation
- **`misc/`**: Design documents and guides
## Key Technologies
### Core Dependencies
```go
// Networking and orchestration
github.com/docker/docker // Docker API client
github.com/docker/compose/v2 // Docker Compose integration
golang.zx2c4.com/wireguard // WireGuard implementation
github.com/hashicorp/serf // Gossip protocol
// State management
github.com/ipfs/go-ds-crdt // CRDT distributed storage
github.com/dgraph-io/badger/v3 // Embedded database
// Web proxy
github.com/caddyserver/caddy/v2 // HTTP server and reverse proxy
// CLI and UX
github.com/spf13/cobra // CLI framework
github.com/charmbracelet/huh // Interactive forms
// gRPC and networking
google.golang.org/grpc // gRPC framework
github.com/siderolabs/grpc-proxy // gRPC proxy for forwarding
```
## Development Workflow
### Build and Development
```bash
# Build binaries
go build -o uncloud ./cmd/uncloud
go build -o uncloudd ./cmd/uncloudd
```
### Key Make Targets
- `proto`: Generate protobuf code
- `ucind-cluster`: Create development cluster
- `update-dev`: Deploy to development machines
- `demo-reset`: Reset demo environment
- `fmt`: Format code
- `test`: Run all tests
- `lint`: Lint the code using golangci-lint
- `lint-and-fix`: Lint the code and fix issues whenever possible
## CLI Commands Structure
The `uc` CLI provides these main command groups:
### Machine Management
```bash
uc machine init <user@host> # Initialize new cluster
uc machine add <user@host> # Add machine to cluster
uc machine ls # List machines
uc machine rm <name> # Remove machine
```
### Service Management
```bash
uc run <image> # Run container from image
uc deploy # Deploy from compose.yaml
uc scale <service> <count> # Scale service replicas
uc ls # List services
uc rm <service> # Remove service
```
### Context and Connectivity
```bash
uc context ls # List available contexts
uc context use <name> # Switch context
```
### Global Flags
- `--connect`: Connect to remote machine directly, without a config file
- `--uncloud-config`: Override config file path
## Development Guidelines
### Code Organization
- **Package naming**: Use clear, descriptive names
- **Error handling**: Wrap errors with context using `fmt.Errorf`
- **Logging**: Use structured logging with levels
- **gRPC**: Services defined in `internal/machine/api/pb/`
### Testing
- Test files and locations
- Unit tests alongside source files (`*_test.go`)
- Integration tests in `test/e2e/`
- Test fixtures in `test/fixtures/`
- Use table driven tests whenever possible
### Dependencies
- Prefer standard library when possible
- Pin versions in `go.mod`
- Document rationale for external dependencies
### Configuration
- Support environment variables for key settings
- Validate configuration early
- Provide sensible defaults
## Troubleshooting and Debugging
### Common Issues
- **Networking**: Check WireGuard status, iptables rules
- **DNS**: Verify service discovery resolution
- **Containers**: Use standard Docker debugging tools
- **State sync**: Check Corrosion logs for replication issues
### Debugging Tools
- Standard Linux networking tools (`ping`, `traceroute`, `wireshark`)
- Docker commands (`docker ps`, `docker logs`)
- SSH access to machines for direct inspection
- gRPC debugging tools
### Logs and Monitoring
- Systemd services (getting logs via `journalctl -u SERVICE_NAME`)
- `uncloud` -- Uncloud daemon
- `uncloud-corrosion` -- Corrosion process
- Machine daemon logs
- Container logs via Docker
## File Patterns and Conventions
### Important Files to Understand
- `cmd/uncloud/main.go`: CLI entry point and command structure
- `internal/cli/cli.go`: CLI implementation and configuration
- `internal/machine/machine.go`: Core machine management
- `pkg/api/`: Public API definitions
- `misc/design.md`: Architecture and design philosophy
- `README.md`: Repository README
### Configuration Files
- `go.mod/go.sum`: Go dependency management
- `Makefile`: Build and development tasks
- `Dockerfile`: Container build instructions forUncloud-in-Docker (used for testing)
This document should help AI assistants understand the project structure, make informed suggestions, and contribute effectively to the Uncloud codebase.
## Documentation
Instructions when generating documentation:
- Use conversational language write as if you were speaking to a friend.
- Keep sentences simple, optimize for clarity and understanding.
- Place the subject before the action whenever possible. Example: prefer “The function loads data” over “Data is loaded by the function.”
Symlink
+1
View File
@@ -0,0 +1 @@
./AI.md
+60
View File
@@ -0,0 +1,60 @@
ARG ALPINE_VERSION=3.20.3
FROM golang:1.25.1-alpine AS uncloudd
ARG TARGETOS
ARG TARGETARCH
ENV CGO_ENABLED=0
WORKDIR /build
# Download and cache dependencies and only redownload them in subsequent builds if they change.
COPY go.mod go.sum ./
RUN go mod download && go mod verify
COPY . .
RUN GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build -o uncloudd cmd/uncloudd/main.go
FROM alpine:${ALPINE_VERSION} AS corrosion-download
ARG TARGETARCH
RUN CORROSION_ARCH=$(case "${TARGETARCH}" in \
"amd64") echo "x86_64" ;; \
"arm64") echo "aarch64" ;; \
*) echo "Architecture '${TARGETARCH}' not supported" >&2 && exit 1 ;; \
esac) \
&& wget -q -O /tmp/corrosion.tar.gz \
"https://github.com/psviderski/corrosion/releases/latest/download/corrosion-${CORROSION_ARCH}-unknown-linux-gnu.tar.gz" \
&& tar -xzf /tmp/corrosion.tar.gz -C /tmp \
&& install /tmp/corrosion /usr/local/bin/corrosion \
&& rm /tmp/corrosion.tar.gz /tmp/corrosion
# Beware that more modern images like chainguard/wolfi-base build glibc with flags that require newer CPU features,
# e.g. x86-64-v2. So such image may fail with "Fatal glibc error: CPU does not support x86-64-v2" on older CPUs.
FROM gcr.io/distroless/cc-debian12:latest AS corrosion
COPY --from=corrosion-download /usr/local/bin/corrosion /usr/local/bin/corrosion
CMD ["corrosion", "agent"]
FROM alpine:${ALPINE_VERSION} AS corrosion-image-tarball
ARG CORROSION_IMAGE="ghcr.io/psviderski/corrosion:latest"
RUN apk --no-cache add crane
RUN crane pull "${CORROSION_IMAGE}" /corrosion.tar
# Uncloud-in-Docker (ucind) image for running Uncloud test clusters using Docker.
FROM docker:27.3.1-dind AS ucind
# Create system group and user 'uncloud'.
RUN addgroup -S uncloud && adduser -SHD -h /nonexistent -G uncloud -g "" uncloud
RUN apk --no-cache add \
socat \
wireguard-tools
COPY --from=corrosion-image-tarball /corrosion.tar /images/corrosion.tar
COPY scripts/docker/dind scripts/docker/entrypoint.sh /usr/local/bin/
COPY --from=uncloudd /build/uncloudd /usr/local/bin/
ENTRYPOINT ["entrypoint.sh"]
CMD ["uncloudd"]
+113 -9
View File
@@ -1,16 +1,120 @@
.PHONY: build
uncloud-dev:
CORROSION_IMAGE ?= ghcr.io/psviderski/corrosion:latest
UCIND_IMAGE ?= ghcr.io/psviderski/ucind:latest
update-dev:
GOOS=linux GOARCH=amd64 go build -o uncloudd-linux-amd64 ./cmd/uncloudd && \
scp uncloudd-linux-amd64 spy@192.168.40.243:~/ && \
ssh spy@192.168.40.243 sudo install ./uncloudd-linux-amd64 /usr/local/bin/uncloudd
scp uncloudd-linux-amd64 spy@192.168.40.176:~/ && \
ssh spy@192.168.40.176 sudo install ./uncloudd-linux-amd64 /usr/local/bin/uncloudd
GOOS=linux GOARCH=amd64 go build -o uncloud-linux-amd64 ./cmd/uncloud && \
scp uncloud-linux-amd64 spy@192.168.40.243:~/ && \
ssh spy@192.168.40.243 sudo install ./uncloud-linux-amd64 /usr/local/bin/uncloud
scp uncloud-linux-amd64 spy@192.168.40.176:~/ && \
ssh spy@192.168.40.176 sudo install ./uncloud-linux-amd64 /usr/local/bin/uncloud
ssh spy@192.168.40.176 sudo install ./uncloudd-linux-amd64 /usr/local/bin/uncloudd && \
rm uncloudd-linux-amd64
# GOOS=linux GOARCH=arm64 go build -o uncloudd-linux-arm64 ./cmd/uncloudd && \
# scp uncloudd-linux-arm64 ubuntu@152.67.101.197:~/ && \
# ssh ubuntu@152.67.101.197 sudo install ./uncloudd-linux-arm64 /usr/local/bin/uncloudd && \
# rm uncloudd-linux-arm64
update-restart-dev:
GOOS=linux GOARCH=amd64 go build -o uncloudd-linux-amd64 ./cmd/uncloudd && \
scp uncloudd-linux-amd64 spy@192.168.40.243:~/ && \
ssh spy@192.168.40.243 "sudo install ./uncloudd-linux-amd64 /usr/local/bin/uncloudd && sudo systemctl restart uncloud" && \
scp uncloudd-linux-amd64 spy@192.168.40.176:~/ && \
ssh spy@192.168.40.176 "sudo install ./uncloudd-linux-amd64 /usr/local/bin/uncloudd && sudo systemctl restart uncloud" && \
rm uncloudd-linux-amd64
# GOOS=linux GOARCH=arm64 go build -o uncloudd-linux-arm64 ./cmd/uncloudd && \
# scp uncloudd-linux-arm64 ubuntu@152.67.101.197:~/ && \
# ssh ubuntu@152.67.101.197 "sudo install ./uncloudd-linux-arm64 /usr/local/bin/uncloudd && sudo systemctl restart uncloud" && \
# rm uncloudd-linux-arm64
reset-dev:
ssh spy@192.168.40.243 "sudo systemctl stop uncloud && sudo rm -rf /var/lib/uncloud"
ssh spy@192.168.40.176 "sudo systemctl stop uncloud && sudo rm -rf /var/lib/uncloud"
ssh ubuntu@152.67.101.197 "sudo systemctl stop uncloud && sudo rm -rf /var/lib/uncloud"
demo-reset:
rm -fv ~/.config/uncloud/config.yaml
ssh ubuntu@152.67.101.197 "AUTO_CONFIRM=true sudo -E uncloud-uninstall && docker rmi caddy:2.9.1"
ssh root@5.223.45.199 "AUTO_CONFIRM=true sudo -E uncloud-uninstall"
ssh spy@192.168.40.243 "AUTO_CONFIRM=true sudo -E uncloud-uninstall"
.PHONY: ucind-cluster
ucind-cluster:
go run ./cmd/ucind cluster rm && go run ./cmd/ucind cluster create -m $(if $(MACHINES_COUNT),$(MACHINES_COUNT),3)
.PHONY: proto
proto:
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative internal/machine/api/pb/*.proto
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative \
--proto_path=. --proto_path=internal/machine/api/vendor internal/machine/api/pb/*.proto
.PHONY: proto-mise
proto-mise:
mise exec -- protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative \
--proto_path=. --proto_path=internal/machine/api/vendor internal/machine/api/pb/*.proto
.PHONY: corrosion-image
corrosion-image:
docker build -t "$(CORROSION_IMAGE)" --target corrosion .
.PHONY: corrosion-multiarch-image-push
corrosion-multiarch-image-push:
docker buildx build --push --platform linux/amd64,linux/arm64 -t "$(CORROSION_IMAGE)" --target corrosion .
.PHONY: ucind-image
ucind-image:
docker build -t "$(UCIND_IMAGE)" --target ucind .
.PHONY: ucind-multiarch-image-push
ucind-multiarch-image-push:
docker buildx build --push --platform linux/amd64,linux/arm64 -t "$(UCIND_IMAGE)" --target ucind .
.PHONY: mocks
mocks:
@mockery
.PHONY: test
test:
ifeq ($(TEST_NAME),)
go test -count=1 -v ./...
else
go test -race -count=1 -v -run ^$(TEST_NAME)$$ ./...
endif
.PHONY: test-e2e
test-e2e:
go test -race -count=1 -v ./test/e2e
.PHONY: test-clean
test-clean:
@CONTAINERS=$$(docker ps --filter "name=ucind-test" -q); \
if [ -n "$$CONTAINERS" ]; then \
echo "Killing containers..."; \
docker kill $$CONTAINERS; \
fi; \
CONTAINERS_STOPPED=$$(docker ps -a --filter "name=ucind-test" -q); \
if [ -n "$$CONTAINERS_STOPPED" ]; then \
echo "Removing stopped containers..."; \
docker rm $$CONTAINERS_STOPPED; \
else \
echo "Nothing to clean."; \
fi
.PHONY: vet
vet:
go vet ./...
.PHONY: format fmt
format fmt:
GOOS=linux golangci-lint fmt
LINT_TARGETS := lint lint-and-fix
.PHONY: $(LINT_TARGETS) _lint
$(LINT_TARGETS): _lint
lint: ARGS=
lint-and-fix: ARGS=--fix
_lint:
# Explicitly set OS to Linux to not skip *_linux.go files when running on macOS.
# Uncloud daemon won't likely support OS other than Linux anytime soon, so for now we can rely on that.
GOOS=linux golangci-lint run $(ARGS)
.PHONY: cli-docs
cli-docs:
go run ./cmd/uncloud docs
+339 -7
View File
@@ -1,9 +1,341 @@
# Uncloud
<div align="center">
<img src="./website/landing/images/logo.svg" height="100" width="100" alt="Uncloud logo"/>
<h1>Uncloud</h1>
<p><strong>Docker simplicity. Multi-machine power.</strong></p>
Uncloud is a lightweight tool for deploying and managing containerised applications across a network of Docker hosts. It
bridges the gap between simple Docker deployments and complex cloud platforms and container orchestrators like
Kubernetes.
<p>
<a href="https://uncloud.run/docs"><img src="https://img.shields.io/badge/Docs-blue.svg?style=for-the-badge&logo=gitbook&logoColor=white" alt="Documentation"></a>
<a href="https://discord.gg/eR35KQJhPu"><img src="https://img.shields.io/badge/discord-5865F2.svg?style=for-the-badge&logo=discord&logoColor=white" alt="Join Discord"></a>
<a href="https://x.com/psviderski"><img src="https://img.shields.io/badge/follow-black?style=for-the-badge&logo=X&logoColor=while" alt="Follow on X"></a>
<a href="https://github.com/sponsors/psviderski"><img src="https://img.shields.io/badge/Donate-EA4AAA.svg?style=for-the-badge&logo=githubsponsors&logoColor=white" alt="Donate"></a>
</p>
</div>
Our goal is to become the go-to self-hosted alternative to platforms like [Fly.io](https://fly.io) and
[Render](https://render.com/). Whether you're using bare metal servers, Raspberry Pis, or cloud VMs, Uncloud transforms
your hardware into a personal cloud platform.
Uncloud is a lightweight clustering and container orchestration tool that lets you deploy and manage web apps across
cloud VMs and bare metal with minimised cluster management overhead. It creates a secure WireGuard mesh network between
your Docker hosts and provides automatic service discovery, load balancing, ingress with HTTPS, and simple CLI commands
to manage your apps.
Unlike traditional orchestrators, there's no central control plane and quorum to maintain. Each machine maintains a
synchronised copy of the cluster state through peer-to-peer communication, keeping cluster operations functional even if
some machines go offline.
Uncloud is the solution for developers who want the flexibility of self-hosted infrastructure without the operational
complexity of Kubernetes.
## ✨ Features
* **Deploy anywhere**: Combine cloud VMs, dedicated servers, and bare metal into a unified computing environment —
regardless of location or provider.
* **Docker Compose**: Familiar [Docker Compose](https://compose-spec.io/) format for defining services and volumes. No
need to learn a new bespoke DSL.
* **Zero-downtime deployments**: Rolling updates without service interruption. Automatic rollback on failure is coming
soon.
* **Service discovery**: Built-in DNS server resolves service names to container IPs.
* **Persistent storage**: Run stateful services with Docker volumes managed across machines.
* **Zero-config private network**: Automatic WireGuard mesh with peer discovery and NAT traversal. Containers get unique
IPs for direct cross-machine communication.
* **No control plane**: Fully decentralised design eliminates single points of failure and reduces operational overhead.
* **Imperative over declarative**: Favoring imperative operations over state reconciliation simplifies both the mental
model and troubleshooting.
* **Managed DNS**: Automatic DNS records `*.<id>.cluster.uncloud.run` for services with public access via managed
[Uncloud DNS](https://github.com/psviderski/uncloud-dns) service.
* **Automatic HTTPS**: Built-in Caddy reverse proxy handles TLS certificate provisioning and renewal using Let's
Encrypt.
* **Docker-like CLI**: Familiar commands for managing both infrastructure and applications.
* **Remote management**: Control your entire infrastructure through SSH access to any single machine in the cluster.
### 🚀 Coming soon
* **[Unregistry](https://github.com/psviderski/unregistry) integration**: Push your Docker images directly to your
machines without an external registry. It will transfer only the missing layers, making it fast and efficient.
## 🎬 Quick demo
The screenshot below demonstrates how I use Uncloud to deploy https://uncloud.run website to 2 remote machines from
the [`compose.yaml`](website/compose.yaml) file on my local machine.
It exposes the container port `8000/tcp` as HTTPS on the domain `uncloud.run`, served by the Caddy reverse proxy on the
remote machines. All managed by Uncloud.
![Uncloud compose deployment demo](.github/images/compose-deploy.jpg)
Here is a more advanced use case. Deploy a highly available web app with automatic HTTPS across multiple regions and
on-premises in just a couple minutes.
<a href="https://uncloud.wistia.com/medias/k47uwt9uau?wvideo=k47uwt9uau">
<img src="https://embed-ssl.wistia.com/deliveries/3cf7014a48b93afc556444bed3e39a8c.jpg?image_crop_resized=900x526&image_play_button_rounded=true&image_play_button_size=2x&image_play_button_color=18181Be0" alt="Uncloud demo" width="450" height="263" />
</a>
## 💫 Why Uncloud?
Modern cloud platforms like Heroku and Render offer amazing developer experiences but at a premium price. Traditional
container orchestrators like Kubernetes provide power and flexibility but require significant operational expertise. I
believe there's a sweet spot in between — a pragmatic solution for the majority of us who aren't running at Google
scale. You should be able to:
* **Own your infrastructure and data**: Whether driven by costs, compliance, or flexibility, run applications on any
combination of cloud VMs and personal hardware while controlling your data and maintaining the cloud-like experience
you love.
* **Stay simple as you grow**: Start with a single machine and add more whenever you need without changing your
workflow. No worrying about highly-available control planes or complex YAML configurations.
* **Build with proven primitives**: Get production-grade networking, deployment primitives, service discovery, load
balancing, and ingress with HTTPS out of the box without becoming a distributed systems expert.
* **Support sustainable computing** 🌿: Minimise system overhead to maximise resources available for your applications.
Uncloud's goal is to make deployment and management of containerised applications feel as seamless as using a cloud
platform, whether you're running on a $5 VPS, a spare Mac mini, or a rack of bare metal servers.
## 🚀 Quick start
1. Install Uncloud CLI:
```bash
brew install psviderski/tap/uncloud
# or using curl (macOS/Linux)
curl -fsS https://get.uncloud.run/install.sh | sh
```
See [Installation](https://uncloud.run/docs/getting-started/install-cli) for more options.
2. Initialise your first machine:
```bash
uc machine init root@your-server-ip
```
3. Deploy your app from a Docker image and publish its container port 8000 as HTTPS using `app.example.com` domain:
```bash
uc run -p app.example.com:8000/https image/my-app
```
4. Create a DNS A record in your DNS provider (Cloudflare, Namecheap, etc.) that points `app.example.com` to your
server's IP address. Allow a few minutes for DNS propagation.
That's it! Your app is now running and accessible at https://app.example.com ✨
5. Clean up when you're done:
```bash
uc ls
# Copy the service name from the output and run the rm command:
uc rm my-app-name
```
If you want to fully uninstall Uncloud on a machine, run:
```bash
uncloud-uninstall
```
View the [Documentation](https://uncloud.run/docs) for more information.
## ⚙️ How it works
Check out the [design document](misc/design.md) to understand Uncloud's design philosophy and goals.
Here is a diagram of an Uncloud multi-provider cluster of 3 machines:
![Diagram: multi-provider cluster of 3 machines](website/landing/images/diagram.webp)
<details>
<summary>Peek under the hood to see what happens when you run certain commands.</summary>
**When you initialise a new cluster on a machine:**
```bash
$ uc machine init --name oracle-vm ubuntu@152.67.101.197
Downloading Uncloud install script: https://raw.githubusercontent.com/psviderski/uncloud/refs/heads/main/scripts/install.sh
⏳ Running Uncloud install script...
✓ Docker is already installed.
⏳ Installing Docker...
...
✓ Docker installed successfully.
✓ Linux user and group 'uncloud' created.
✓ Linux user 'ubuntu' added to group 'uncloud'.
⏳ Installing Uncloud binaries...
⏳ Downloading uncloudd binary: https://github.com/psviderski/uncloud/releases/latest/download/uncloudd_linux_arm64.tar.gz
✓ uncloudd binary installed: /usr/local/bin/uncloudd
⏳ Downloading uninstall script: https://raw.githubusercontent.com/psviderski/uncloud/refs/heads/main/scripts/uninstall.sh
✓ uncloud-uninstall script installed: /usr/local/bin/uncloud-uninstall
✓ Systemd unit file created: /etc/systemd/system/uncloud.service
Created symlink /etc/systemd/system/multi-user.target.wants/uncloud.service → /etc/systemd/system/uncloud.service.
⏳ Downloading uncloud-corrosion binary: https://github.com/psviderski/corrosion/releases/latest/download/corrosion-aarch64-unknown-linux-gnu.tar.gz
✓ uncloud-corrosion binary installed: /usr/local/bin/uncloud-corrosion
✓ Systemd unit file created: /etc/systemd/system/uncloud-corrosion.service
⏳ Starting Uncloud machine daemon (uncloud.service)...
✓ Uncloud machine daemon started.
✓ Uncloud installed on the machine successfully! 🎉
Cluster "default" initialised with machine "oracle-vm"
Waiting for the machine to be ready...
Reserved cluster domain: xuw3xd.cluster.uncloud.run
[+] Deploying service caddy 1/1
✔ Container caddy-c47x on oracle-vm Started 0.9s
Updating cluster domain records in Uncloud DNS to point to machines running caddy service...
[+] Verifying internet access to caddy service 1/1
✔ Machine oracle-vm (152.67.101.197) Reachable 0.1s
DNS records updated to use only the internet-reachable machines running caddy service:
*.xuw3xd.cluster.uncloud.run A → 152.67.101.197
```
1. The CLI SSHs into the machine and installs Docker, the `uncloudd` machine daemon and
[corrosion](https://github.com/superfly/corrosion) service, managed by systemd.
2. Generates a unique WireGuard key pair, allocates a dedicated subnet `10.210.0.0/24` for the machine and its
containers, and configures `uncloudd` accordingly. All subsequent communication happens with `uncloudd`
through its gRPC API over SSH.
3. Configures and starts `corrosion`, a CRDT-based distributed SQLite database to share cluster state between machines.
4. Creates a Docker bridge network connected to the WireGuard interface.
5. This machine becomes an entry point for the newly created cluster which is stored in the cluster config under
`~/.config/uncloud` on your local machine.
**When you add another machine:**
```bash
$ uc machine add --name hetzner-server root@5.223.45.199
Downloading Uncloud install script: https://raw.githubusercontent.com/psviderski/uncloud/refs/heads/main/scripts/install.sh
⏳ Running Uncloud install script...
✓ Docker is already installed.
✓ Linux user and group 'uncloud' created.
⏳ Installing Uncloud binaries...
⏳ Downloading uncloudd binary: https://github.com/psviderski/uncloud/releases/latest/download/uncloudd_linux_amd64.tar.gz
✓ uncloudd binary installed: /usr/local/bin/uncloudd
⏳ Downloading uninstall script: https://raw.githubusercontent.com/psviderski/uncloud/refs/heads/main/scripts/uninstall.sh
✓ uncloud-uninstall script installed: /usr/local/bin/uncloud-uninstall
✓ Systemd unit file created: /etc/systemd/system/uncloud.service
Created symlink /etc/systemd/system/multi-user.target.wants/uncloud.service → /etc/systemd/system/uncloud.service.
⏳ Downloading uncloud-corrosion binary: https://github.com/psviderski/corrosion/releases/latest/download/corrosion-x86_64-unknown-linux-gnu.tar.gz
✓ uncloud-corrosion binary installed: /usr/local/bin/uncloud-corrosion
✓ Systemd unit file created: /etc/systemd/system/uncloud-corrosion.service
⏳ Starting Uncloud machine daemon (uncloud.service)...
✓ Uncloud machine daemon started.
✓ Uncloud installed on the machine successfully! 🎉
Machine "hetzner-server" added to cluster
Waiting for the machine to be ready...
[+] Deploying service caddy 1/1
✔ Container caddy-d36c on hetzner-server Started 1.0s
Updating cluster domain records in Uncloud DNS to point to machines running caddy service...
[+] Verifying internet access to caddy service 2/2
✔ Machine hetzner-server (5.223.45.199) Reachable 0.2s
✔ Machine oracle-vm (152.67.101.197) Reachable 0.1s
DNS records updated to use only the internet-reachable machines running caddy service:
*.xuw3xd.cluster.uncloud.run A → 152.67.101.197, 5.223.45.199
$ uc machine ls
NAME STATE ADDRESS PUBLIC IP WIREGUARD ENDPOINTS
oracle-vm Up 10.210.0.1/24 152.67.101.197 10.0.0.95:51820, 152.67.101.197:51820
hetzner-server Up 10.210.1.1/24 5.223.45.199 5.223.45.199:51820, [2a01:4ff:2f0:128b::1]:51820
```
1. The second machine gets provisioned just like the first. A non-root SSH user will need `sudo` access.
2. Allocates a new subnet `10.210.1.0/24` for the second machine and its containers.
3. Registers the second machine in the cluster state and exchanges WireGuard keys with the first machine.
4. Both machines establish a WireGuard tunnel between each other, allowing Docker containers connected to the bridge
network to communicate directly across machines.
5. Configures and starts `corrosion` on the second machine to sync the cluster state.
6. The second machine is added as an alternative entry point in the cluster config.
7. If one of the machines goes offline, the other machine can still serve cluster operations.
If one more machine is added, the process repeats with a new subnet. The new machine needs to establish a WireGuard
connection with only one of the existing machines. Other machines will learn about it through the shared cluster state
and automatically establish a WireGuard tunnel with it.
**When you run a service:**
```bash
$ uc run -p app.example.com:8000/https image/my-app
[+] Running service my-app-1b3b (replicated mode) 1/1
✔ Container my-app-1b3b-tcex on oracle-vm Started
my-app-1b3b endpoints:
• https://app.example.com → :8000
• https://my-app-1b3b.xuw3xd.cluster.uncloud.run → :8000
```
1. CLI picks a machine to run your container.
2. `uncloudd` that the CLI communicates with uses [`grpc-proxy`](https://github.com/siderolabs/grpc-proxy) to forward
the request to the target machine to launch a container there.
3. `uncloudd` on the target machine starts the Docker container in the bridge network and stores its info in the
cluster's distributed state.
4. The container gets a cluster-unique IP address from the bridge network (in the `10.210.X.2-254` range) and becomes
accessible from other machines in the cluster.
5. Caddy reverse proxy which runs in [`global`](https://github.com/compose-spec/compose-spec/blob/main/deploy.md#mode)
mode on each machine watches the cluster state for new services and updates its configuration to route traffic to the
new container.
Look ma, no control plane or master nodes to maintain! Just a simple overlay network and eventually consistent state
sync that lets machines work together. Want to check on things or make changes? Connect to any machine either implicitly
using the CLI or directly over SSH. They all have the complete cluster state and can control everything. It's like each
machine is a full backup of your control plane.
</details>
## 🏗 Project status
Uncloud is currently in active development and is **not ready for production use**. Features may change significantly
and there may be breaking changes between releases.
We'd love your input! Here's how you can contribute:
* 🐛 Found a bug? [Open an issue](https://github.com/psviderski/uncloud/issues)
* 💡 Have questions, ideas, or need help?
* Start a discussion or join an existing one in
the [Discussions](https://github.com/psviderski/uncloud/discussions).
* Join our [Discord community](https://discord.gg/eR35KQJhPu) where we discuss features, roadmap, implementation
details, and help each other out.
## 🙏 Inspiration & Acknowledgements
I'm grateful to the following projects that inspired Uncloud's design and implementation:
* [Kamal](https://kamal-deploy.org/) — for proving that even in the declarative era of Kubernetes there is a place for
simple deployment tools that use imperative commands without complex orchestration. Kamal powers the multi-billion
dollar company [37signals](https://37signals.com/) where it was created, and that's truly inspiring!
* [Fly.io](https://fly.io/) — for inspiring my vision for what self-hosted infrastructure should feel like, proving that
developer experience and powerful infrastructure can coexist beautifully.
* [Tailscale](https://tailscale.com/) — for pioneering the vision of decentralised flat mesh networking with an amazing
user experience that feels like magic.
* [Talos Linux](https://github.com/siderolabs/talos)
and [KubeSpan](https://www.talos.dev/v1.10/talos-guides/network/kubespan/) — for the machine API design using
[grpc-proxy](https://github.com/siderolabs/grpc-proxy) and for its elegant approach to secure WireGuard-based overlay
networking with zero configuration.
* [Docker Swarm Classic](https://github.com/docker-archive/classicswarm) and
[Rancher 1.x](http://rancher-com-website-main-elb-elb-1798790864.us-west-2.elb.amazonaws.com/docs/rancher/v1.6/en/)
— for showing the power of simplicity and pragmatism in container orchestration and that not every problem needs the
complexity of Kubernetes.
Special thanks to the [Corrosion](https://github.com/superfly/corrosion) project by Fly.io for providing the distributed
SQLite database used to share Uncloud's cluster state.
## 📫 Stay updated
* Join our [Discord server](https://discord.gg/eR35KQJhPu) for real-time discussions, support, and updates.
* Follow [@psviderski](https://x.com/psviderski) on X/Twitter.
* Subscribe to [my newsletter](https://uncloud.run/#subscribe) to follow the progress, get early insights into new
features, and be the first to know when it's ready for production use.
* Watch this repository for releases.
## 💖 Sponsors
These companies and projects are helping Uncloud with their generous sponsorship and/or services:
<!-- Sentry -->
<a href="https://sentry.io/welcome/">
<img height="100" alt="Sentry" src="https://github.com/user-attachments/assets/6c1439c0-d20d-40dc-a669-c9aa94651dfa" />
</a>
## ❤️ Contributors
Thank you [@cedws](https://github.com/cedws) for being the first contributor to Uncloud! 🎉
<a href="https://github.com/psviderski/uncloud/graphs/contributors">
<img src="https://contrib.rocks/image?repo=psviderski/uncloud" />
</a>
+35
View File
@@ -0,0 +1,35 @@
package cluster
import (
"fmt"
"github.com/psviderski/uncloud/internal/ucind"
"github.com/spf13/cobra"
)
func NewCreateCommand() *cobra.Command {
opts := ucind.CreateClusterOptions{}
cmd := &cobra.Command{
Use: "create [NAME]",
Short: "Create a new cluster.",
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
p := cmd.Context().Value("provisioner").(*ucind.Provisioner)
name := DefaultClusterName
if len(args) > 0 {
name = args[0]
}
if _, err := p.CreateCluster(cmd.Context(), name, opts); err != nil {
return fmt.Errorf("create cluster '%s': %w", name, err)
}
fmt.Printf("Cluster '%s' created.\n", name)
return nil
},
}
cmd.Flags().IntVarP(&opts.Machines, "machines", "m", 1, "Number of machines to create.")
return cmd
}
+31
View File
@@ -0,0 +1,31 @@
package cluster
import (
"fmt"
"github.com/psviderski/uncloud/internal/ucind"
"github.com/spf13/cobra"
)
func NewRemoveCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "rm [NAME]",
Short: "Remove a cluster.",
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
p := cmd.Context().Value("provisioner").(*ucind.Provisioner)
name := DefaultClusterName
if len(args) > 0 {
name = args[0]
}
if err := p.RemoveCluster(cmd.Context(), name); err != nil {
return fmt.Errorf("remove cluster '%s': %w", name, err)
}
fmt.Printf("Cluster '%s' removed.\n", name)
return nil
},
}
return cmd
}
+20
View File
@@ -0,0 +1,20 @@
package cluster
import (
"github.com/spf13/cobra"
)
const DefaultClusterName = "ucind-default"
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "cluster",
Short: "Manage local Docker-based clusters.",
}
cmd.AddCommand(
NewCreateCommand(),
// NewListCommand(),
NewRemoveCommand(),
)
return cmd
}
+56
View File
@@ -0,0 +1,56 @@
package main
import (
"context"
"fmt"
"os"
"strings"
"github.com/docker/docker/client"
"github.com/psviderski/uncloud/cmd/ucind/cluster"
"github.com/psviderski/uncloud/internal/ucind"
"github.com/spf13/cobra"
)
func main() {
var configPath string
cmd := &cobra.Command{
Use: "ucind",
Short: "A CLI tool for running Uncloud test clusters using Docker.",
Long: "A CLI tool for running Uncloud test clusters using Docker.\n" +
"Machines in a ucind cluster are Docker containers running a ucind image. All machines within a cluster " +
"are connected to the same Docker network.",
SilenceUsage: true,
SilenceErrors: true,
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
if err != nil {
return fmt.Errorf("create Docker client: %w", err)
}
if strings.HasPrefix(configPath, "~/") {
home, err := os.UserHomeDir()
if err != nil {
return fmt.Errorf("get user home directory to resolve %q: %w", configPath, err)
}
configPath = strings.Replace(configPath, "~", home, 1)
}
configUpdater := ucind.NewConfigUpdater(configPath)
p := ucind.NewProvisioner(cli, configUpdater)
// Persist the provisioner in the context so it can be used by subcommands.
cmd.SetContext(context.WithValue(cmd.Context(), "provisioner", p))
return nil
},
}
// TODO: allow to override using UNCLOUD_CONFIG env var.
cmd.PersistentFlags().StringVar(&configPath, "uncloud-config", "~/.config/uncloud/config.yaml",
"path to the Uncloud configuration file.")
_ = cmd.MarkPersistentFlagFilename("uncloud-config", "yaml", "yml")
cmd.AddCommand(
cluster.NewRootCommand(),
)
cobra.CheckErr(cmd.Execute())
}
+77
View File
@@ -0,0 +1,77 @@
package main
import (
"context"
"fmt"
composecli "github.com/compose-spec/compose-go/v2/cli"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/client/compose"
"github.com/spf13/cobra"
)
// NewBuildCommand creates a new command to build services from a Compose file.
func NewBuildCommand() *cobra.Command {
opts := cli.BuildOptions{}
cmd := &cobra.Command{
Use: "build [FLAGS] [SERVICE...]",
Short: "Build services from a Compose file.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
if len(args) > 0 {
opts.Services = args
}
return runBuild(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringSliceVarP(&opts.Files, "file", "f", nil,
"One or more Compose files to build (default compose.yaml)")
cmd.Flags().StringSliceVarP(&opts.Profiles, "profile", "p", nil,
"One or more Compose profiles to enable.")
cmd.Flags().BoolVarP(&opts.Push, "push", "P", false,
"Push built images to the registry after building. (default false)")
cmd.Flags().BoolVarP(&opts.NoCache, "no-cache", "n", false,
"Do not use cache when building images. (default false)")
return cmd
}
// TODO: deduplicate with a similar functino for deploy options
// projectOpts returns the project options for the Compose file(s).
func projectOptsFromBuildOpts(opts cli.BuildOptions) []composecli.ProjectOptionsFn {
projectOpts := []composecli.ProjectOptionsFn{}
if len(opts.Profiles) > 0 {
projectOpts = append(projectOpts, composecli.WithDefaultProfiles(opts.Profiles...))
}
return projectOpts
}
// runBuild parses the Compose file(s), builds the services, and pushes them if requested.
func runBuild(ctx context.Context, uncli *cli.CLI, opts cli.BuildOptions) error {
projectOpts := projectOptsFromBuildOpts(opts)
project, err := compose.LoadProject(ctx, opts.Files, projectOpts...)
if err != nil {
return fmt.Errorf("load compose file(s): %w", err)
}
if len(opts.Services) > 0 {
project, err = project.WithSelectedServices(opts.Services)
if err != nil {
return fmt.Errorf("select services: %w", err)
}
}
servicesToBuild := cli.GetServicesThatNeedBuild(project)
if len(servicesToBuild) == 0 {
fmt.Println("No services to build.")
return nil
}
return cli.BuildServices(ctx, servicesToBuild, opts)
}
+76
View File
@@ -0,0 +1,76 @@
package caddy
import (
"context"
"fmt"
"os"
"github.com/alecthomas/chroma/v2/quick"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type configOptions struct {
machine string
noColor bool
context string
}
func NewConfigCommand() *cobra.Command {
opts := configOptions{}
cmd := &cobra.Command{
Use: "config",
Short: "Show the current Caddy configuration (Caddyfile).",
Long: "Display the current Caddy configuration (Caddyfile) from the connected machine or a specified one.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return runConfig(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVarP(&opts.machine, "machine", "m", "",
"Name or ID of the machine to get the configuration from. (default is connected machine)")
cmd.Flags().BoolVar(&opts.noColor, "no-color", false,
"Disable syntax highlighting for the output.")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func runConfig(ctx context.Context, uncli *cli.CLI, opts configOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
if opts.machine != "" {
// If a specific machine is requested, use it to get the Caddy configuration.
ctx, _, err = api.ProxyMachinesContext(ctx, clusterClient, []string{opts.machine})
if err != nil {
return err
}
}
config, err := clusterClient.Caddy.GetConfig(ctx, nil)
if err != nil {
return fmt.Errorf("get Caddy config: %w", err)
}
// Print the Caddyfile with syntax highlighting.
if opts.noColor {
fmt.Print(config.Caddyfile)
} else {
if err = quick.Highlight(os.Stdout, config.Caddyfile, "caddy", "terminal256", "monokai"); err != nil {
// If highlighting fails, fall back to plain output.
fmt.Print(config.Caddyfile)
}
}
return nil
}
+213
View File
@@ -0,0 +1,213 @@
package caddy
import (
"context"
"errors"
"fmt"
"maps"
"os"
"slices"
"strings"
"github.com/docker/cli/cli/streams"
"github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client"
"github.com/spf13/cobra"
)
type deployOptions struct {
caddyfile string
image string
machines []string
context string
}
func NewDeployCommand() *cobra.Command {
opts := deployOptions{}
cmd := &cobra.Command{
Use: "deploy",
Short: "Deploy or upgrade Caddy reverse proxy across all machines in the cluster.",
Long: "Deploy or upgrade Caddy reverse proxy across all machines in the cluster.\n" +
"A rolling update is performed when updating existing containers to minimise disruption.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return runDeploy(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVar(&opts.caddyfile, "caddyfile", "",
"Path to a custom global Caddy config (Caddyfile) that will be prepended to the auto-generated Caddy config.")
cmd.Flags().StringVar(&opts.image, "image", "",
"Caddy Docker image to deploy. (default caddy:LATEST_VERSION)")
cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil,
"Machine names to deploy to. Can be specified multiple times or as a comma-separated "+
"list of machine names. (default is all machines)")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context to deploy to. (default is the current context)",
)
return cmd
}
func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
caddyfile := ""
if opts.caddyfile != "" {
data, err := os.ReadFile(opts.caddyfile)
if err != nil {
return fmt.Errorf("read Caddyfile: %w", err)
}
caddyfile = strings.TrimSpace(string(data))
}
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
svc, err := clusterClient.InspectService(ctx, client.CaddyServiceName)
if err != nil {
if !errors.Is(err, api.ErrNotFound) {
return fmt.Errorf("inspect caddy service: %w", err)
}
fmt.Printf("Service: %s (not running)\n", client.CaddyServiceName)
} else {
fmt.Printf("Service: %s (%s mode)\n", svc.Name, svc.Mode)
// Collect unique images of all containers in the running caddy service.
images := make(map[string]struct{}, len(svc.Containers))
for _, c := range svc.Containers {
images[c.Container.Config.Image] = struct{}{}
}
currentImages := slices.Collect(maps.Keys(images))
if len(currentImages) > 1 {
commaSeparatedImages := strings.Join(currentImages, ", ")
fmt.Printf("Current images (multiple versions detected): %s\n", commaSeparatedImages)
} else {
fmt.Printf("Current image: %s\n", currentImages[0])
}
}
if opts.image != "" {
fmt.Printf("Target image: %s\n", opts.image)
}
fmt.Println()
fmt.Println("Preparing a deployment plan...")
placement := api.Placement{
Machines: cli.ExpandCommaSeparatedValues(opts.machines),
}
d, err := clusterClient.NewCaddyDeployment(opts.image, caddyfile, placement)
if err != nil {
return fmt.Errorf("create caddy deployment: %w", err)
}
if opts.image == "" {
fmt.Printf("Target image: %s (latest stable)\n", d.Spec.Container.Image)
}
plan, err := d.Plan(ctx)
if err != nil {
return fmt.Errorf("plan caddy deployment: %w", err)
}
if len(plan.Operations) == 0 {
fmt.Printf("%s service is up to date.\n", client.CaddyServiceName)
} else {
if svc.ID == "" {
if len(opts.machines) > 0 {
fmt.Println("This will run a Caddy container on each selected machine.")
} else {
fmt.Println("This will run a Caddy container on each machine.")
}
}
// Initialise a machine and container name resolver to properly format the plan output.
resolver, err := clusterClient.ServiceOperationNameResolver(ctx, svc)
if err != nil {
return fmt.Errorf("create machine and container name resolver for service operations: %w", err)
}
fmt.Println()
fmt.Println("Deployment plan:")
fmt.Println(plan.Format(resolver))
fmt.Println()
confirmed, err := cli.Confirm()
if err != nil {
return fmt.Errorf("confirm deployment: %w", err)
}
if !confirmed {
fmt.Println("Cancelled. No changes were made.")
return nil
}
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
if _, err = d.Run(ctx); err != nil {
return fmt.Errorf("deploy caddy: %w", err)
}
return nil
}, uncli.ProgressOut(), fmt.Sprintf("Deploying service %s (%s mode)", d.Spec.Name, d.Spec.Mode))
if err != nil {
return err
}
}
fmt.Println()
return UpdateDomainRecords(ctx, clusterClient, uncli.ProgressOut())
}
func UpdateDomainRecords(ctx context.Context, clusterClient *client.Client, progressOut *streams.Out) error {
domain, err := clusterClient.GetDomain(ctx)
if err != nil {
if errors.Is(err, api.ErrNotFound) {
fmt.Println("Skipping DNS records update as no cluster domain is reserved (see 'uc dns').")
return nil
}
return fmt.Errorf("get cluster domain: %w", err)
}
fmt.Println("Updating cluster domain records in Uncloud DNS to point to machines running caddy service...")
// TODO: split the method into two: one to get the records and one to update them to ask for update confirmation.
var records []*pb.DNSRecord
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
var err error
records, err = clusterClient.CreateIngressRecords(ctx, client.CaddyServiceName)
return err
}, progressOut, "Verifying internet access to caddy service")
if err != nil {
if errors.Is(err, client.ErrNoReachableMachines) {
fmt.Println()
fmt.Printf("DNS records for domain '%s' could not be updated as there are no internet-reachable "+
"machines running caddy containers.\n", domain)
fmt.Println()
fmt.Println("Possible solutions:")
fmt.Println("- Ensure your machines have public IP addresses")
fmt.Println("- Use --public-ip flag when adding machines to override the automatically detected IPs")
fmt.Println("- Check firewall settings on your machines")
fmt.Println("- Configure port forwarding if behind NAT")
fmt.Println("- Retry Caddy deployment with 'uc caddy deploy' after resolving connectivity issues")
fmt.Println()
fmt.Println("Your services won't be accessible from the internet until at least one machine " +
"becomes reachable. If you aren't planning to expose any services publicly, you can release " +
"the domain by running 'uc dns release'.")
}
return fmt.Errorf("failed to update DNS records pointing to caddy service: %w", err)
}
fmt.Println()
fmt.Println("DNS records updated to use only the internet-reachable machines running caddy service:")
for _, r := range records {
fmt.Printf(" %s %s → %s\n", r.Name, r.Type, strings.Join(r.Values, ", "))
}
return nil
}
+17
View File
@@ -0,0 +1,17 @@
package caddy
import (
"github.com/spf13/cobra"
)
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "caddy",
Short: "Manage Caddy reverse proxy service.",
}
cmd.AddCommand(
NewConfigCommand(),
NewDeployCommand(),
)
return cmd
}
+54
View File
@@ -0,0 +1,54 @@
package context
import (
"fmt"
"maps"
"os"
"slices"
"text/tabwriter"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
func NewListCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "ls",
Aliases: []string{"list"},
Short: "List available cluster contexts.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return list(uncli)
},
}
return cmd
}
func list(uncli *cli.CLI) error {
if uncli.Config == nil {
return fmt.Errorf("context management is not available: Uncloud configuration file is not being used")
}
if len(uncli.Config.Contexts) == 0 {
fmt.Println("No contexts found")
return nil
}
contextNames := slices.Sorted(maps.Keys(uncli.Config.Contexts))
currentContext := uncli.Config.CurrentContext
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
fmt.Fprintln(tw, "NAME\tCURRENT\tCONNECTIONS")
for _, name := range contextNames {
current := ""
if name == currentContext {
current = "✓"
}
connCount := len(uncli.Config.Contexts[name].Connections)
fmt.Fprintf(tw, "%s\t%s\t%d\n", name, current, connCount)
}
return tw.Flush()
}
+25
View File
@@ -0,0 +1,25 @@
package context
import (
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "ctx",
Aliases: []string{"context"},
Short: "Switch between different cluster contexts. Contains subcommands to manage contexts.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return selectContext(uncli)
},
}
cmd.AddCommand(
NewListCommand(),
NewUseCommand(),
)
return cmd
}
+82
View File
@@ -0,0 +1,82 @@
package context
import (
"fmt"
"maps"
"slices"
"github.com/charmbracelet/huh"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
func NewUseCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "use [CONTEXT]",
Short: "Switch to a different cluster context.",
Long: "Switch to a different cluster context. If no context is provided, " +
"a list of available contexts will be displayed for selection.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
if len(args) == 1 {
if err := uncli.SetCurrentContext(args[0]); err != nil {
return fmt.Errorf("failed to set the current cluster context to '%s': %w", args[0], err)
}
fmt.Printf("Current cluster context is now '%s'.\n", args[0])
return nil
}
return selectContext(uncli)
},
}
return cmd
}
func selectContext(uncli *cli.CLI) error {
if uncli.Config == nil {
return fmt.Errorf("context management is not available: Uncloud configuration file is not being used")
}
if len(uncli.Config.Contexts) == 0 {
return fmt.Errorf("no contexts found in Uncloud config (%s)", uncli.Config.Path())
}
contextNames := slices.Sorted(maps.Keys(uncli.Config.Contexts))
var selected string
form := huh.NewForm(
huh.NewGroup(
huh.NewSelect[string]().
Title("Select a cluster context").
Options(buildContextOptions(contextNames, uncli.Config.CurrentContext)...).
Value(&selected),
),
)
if err := form.Run(); err != nil {
return fmt.Errorf("select cluster context: %w", err)
}
if err := uncli.SetCurrentContext(selected); err != nil {
return fmt.Errorf("set current cluster context: %w", err)
}
fmt.Printf("Current cluster context is now '%s'.\n", selected)
return nil
}
func buildContextOptions(contexts []string, current string) []huh.Option[string] {
options := make([]huh.Option[string], len(contexts))
for i, ctx := range contexts {
opt := huh.NewOption(ctx, ctx)
if ctx == current {
opt.Key += " (current)"
opt = opt.Selected(true)
}
options[i] = opt
}
return options
}
+201
View File
@@ -0,0 +1,201 @@
package main
import (
"context"
"errors"
"fmt"
"strings"
composecli "github.com/compose-spec/compose-go/v2/cli"
"github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client"
"github.com/psviderski/uncloud/pkg/client/compose"
"github.com/psviderski/uncloud/pkg/client/deploy"
"github.com/spf13/cobra"
)
type deployOptions struct {
files []string
profiles []string
services []string
noBuild bool
recreate bool
yes bool
context string
}
// NewDeployCommand creates a new command to deploy services from a Compose file.
func NewDeployCommand() *cobra.Command {
opts := deployOptions{}
cmd := &cobra.Command{
Use: "deploy [FLAGS] [SERVICE...]",
Short: "Deploy services from a Compose file.",
RunE: func(cmd *cobra.Command, args []string) error {
cli.BindEnvToFlag(cmd, "yes", "UNCLOUD_AUTO_CONFIRM")
uncli := cmd.Context().Value("cli").(*cli.CLI)
if len(args) > 0 {
opts.services = args
}
return runDeploy(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context to deploy to (default is the current context)")
cmd.Flags().StringSliceVarP(&opts.files, "file", "f", nil,
"One or more Compose files to deploy services from. (default compose.yaml)")
cmd.Flags().BoolVarP(&opts.noBuild, "no-build", "n", false,
"Do not build images before deploying services. (default false)")
cmd.Flags().StringSliceVarP(&opts.profiles, "profile", "p", nil,
"One or more Compose profiles to enable.")
cmd.Flags().BoolVar(&opts.recreate, "recreate", false,
"Recreate containers even if their configuration and image haven't changed.")
cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
"Auto-confirm deployment plan. Should be explicitly set when running non-interactively,\n"+
"e.g., in CI/CD pipelines. [$UNCLOUD_AUTO_CONFIRM]")
// TODO: Consider adding a filter flag to specify which machines to deploy to but keep the rest running.
// Could be useful to test a new version on a subset of machines before rolling out to all.
return cmd
}
// projectOpts returns the project options for the Compose file(s).
func projectOpts(opts deployOptions) []composecli.ProjectOptionsFn {
projectOpts := []composecli.ProjectOptionsFn{}
if len(opts.profiles) > 0 {
projectOpts = append(projectOpts, composecli.WithDefaultProfiles(opts.profiles...))
}
return projectOpts
}
// runDeploy parses the Compose file(s) and deploys the services.
func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
projectOpts := projectOpts(opts)
project, err := compose.LoadProject(ctx, opts.files, projectOpts...)
if err != nil {
return fmt.Errorf("load compose file(s): %w", err)
}
if len(opts.services) > 0 {
// Includes service dependencies by default. This is the default docker compose behavior.
project, err = project.WithSelectedServices(opts.services)
if err != nil {
return fmt.Errorf("select services: %w", err)
}
}
servicesToBuild := cli.GetServicesThatNeedBuild(project)
if len(servicesToBuild) > 0 {
if opts.noBuild {
fmt.Println("Not building services as requested.")
} else {
buildOpts := cli.BuildOptions{
Push: true,
NoCache: false,
}
if err := cli.BuildServices(ctx, servicesToBuild, buildOpts); err != nil {
return fmt.Errorf("build services: %w", err)
}
}
}
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
var strategy deploy.Strategy
if opts.recreate {
strategy = &deploy.RollingStrategy{ForceRecreate: true}
}
composeDeploy, err := compose.NewDeploymentWithStrategy(ctx, clusterClient, project, strategy)
if err != nil {
return fmt.Errorf("create compose deployment: %w", err)
}
plan, err := composeDeploy.Plan(ctx)
if err != nil {
return fmt.Errorf("plan deployment: %w", err)
}
if len(plan.Operations) == 0 {
fmt.Println("Services are up to date.")
return nil
}
fmt.Println("Deployment plan:")
if err = printPlan(ctx, clusterClient, plan); err != nil {
return fmt.Errorf("print deployment plan: %w", err)
}
fmt.Println()
// Ask for plan confirmation before proceeding with the deployment unless auto-confirmed with --yes.
if !opts.yes {
if !cli.IsStdinTerminal() {
return errors.New("cannot ask to confirm deployment plan in non-interactive mode, " +
"use --yes flag or set UNCLOUD_AUTO_CONFIRM=true to auto-confirm")
}
confirmed, err := cli.Confirm()
if err != nil {
return fmt.Errorf("confirm deployment: %w", err)
}
if !confirmed {
fmt.Println("Cancelled. No changes were made.")
return nil
}
}
return progress.RunWithTitle(ctx, func(ctx context.Context) error {
if err := plan.Execute(ctx, clusterClient); err != nil {
return fmt.Errorf("deploy services: %w", err)
}
return nil
}, uncli.ProgressOut(), "Deploying services")
}
func printPlan(ctx context.Context, cli *client.Client, plan deploy.SequenceOperation) error {
for _, op := range plan.Operations {
svcPlan, ok := op.(*deploy.Plan)
if !ok {
fmt.Println("- " + op.Format(nil))
continue
}
svc, err := cli.InspectService(ctx, svcPlan.ServiceID)
if err != nil && !errors.Is(err, api.ErrNotFound) {
return fmt.Errorf("inspect service: %w", err)
}
// Initialise a machine and container name resolver to properly format the service plan output.
resolver, err := cli.ServiceOperationNameResolver(ctx, svc)
if err != nil {
return fmt.Errorf("create machine and container name resolver for service operations: %w", err)
}
fmt.Printf("- Deploy service [name=%s]\n", svcPlan.ServiceName)
fmt.Println(indent(svcPlan.Format(resolver), " "))
}
return nil
}
func indent(text, prefix string) string {
lines := strings.Split(text, "\n")
for i, line := range lines {
lines[i] = prefix + line
}
return strings.Join(lines, "\n")
}
+55
View File
@@ -0,0 +1,55 @@
package dns
import (
"context"
"errors"
"fmt"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb"
)
type releaseOptions struct {
context string
}
func NewReleaseCommand() *cobra.Command {
opts := releaseOptions{}
cmd := &cobra.Command{
Use: "release",
Short: "Release the reserved cluster domain.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return release(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func release(ctx context.Context, uncli *cli.CLI, opts releaseOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
domain, err := client.ReleaseDomain(ctx, &emptypb.Empty{})
if err != nil {
if status.Convert(err).Code() == codes.NotFound {
return errors.New("no domain reserved")
}
}
fmt.Printf("Released cluster domain: %s\n", domain.Name)
return nil
}
+75
View File
@@ -0,0 +1,75 @@
package dns
import (
"context"
"errors"
"fmt"
"github.com/psviderski/uncloud/cmd/uncloud/caddy"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client"
"github.com/spf13/cobra"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
const DefaultUncloudDNSAPIEndpoint = "https://dns.uncloud.run/v1"
type reserveOptions struct {
endpoint string
context string
}
func NewReserveCommand() *cobra.Command {
opts := reserveOptions{}
cmd := &cobra.Command{
Use: "reserve",
Short: "Reserve a cluster domain in Uncloud DNS.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return reserve(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVar(&opts.endpoint, "endpoint", DefaultUncloudDNSAPIEndpoint,
"API endpoint for the Uncloud DNS service.")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func reserve(ctx context.Context, uncli *cli.CLI, opts reserveOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
domain, err := clusterClient.ReserveDomain(ctx, &pb.ReserveDomainRequest{Endpoint: opts.endpoint})
if err != nil {
if status.Convert(err).Code() == codes.AlreadyExists {
return errors.New("domain already reserved")
}
return err
}
fmt.Printf("Reserved cluster domain: %s\n", domain.Name)
// Update cluster domain records in Uncloud DNS to point to machines running caddy service if it has been deployed.
if _, err = clusterClient.InspectService(ctx, client.CaddyServiceName); err != nil {
if errors.Is(err, api.ErrNotFound) {
fmt.Println("Deploy the Caddy reverse proxy service ('uc caddy deploy') to enable internet access " +
"to your services via the reserved or your custom domain.")
return nil
}
return fmt.Errorf("inspect caddy service: %w", err)
}
return caddy.UpdateDomainRecords(ctx, clusterClient, uncli.ProgressOut())
}
+22
View File
@@ -0,0 +1,22 @@
package dns
import (
"github.com/spf13/cobra"
)
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "dns",
Short: "Manage cluster domain in Uncloud DNS.",
Long: "Manage cluster domain in Uncloud DNS.\n" +
"DNS commands allow you to reserve or release a unique '<id>.cluster.uncloud.run' domain for your " +
"cluster. When reserved, Caddy service deployments will automatically update DNS records to route " +
"traffic to the services in the cluster.",
}
cmd.AddCommand(
NewReleaseCommand(),
NewReserveCommand(),
NewShowCommand(),
)
return cmd
}
+54
View File
@@ -0,0 +1,54 @@
package dns
import (
"context"
"errors"
"fmt"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type showOptions struct {
context string
}
func NewShowCommand() *cobra.Command {
opts := showOptions{}
cmd := &cobra.Command{
Use: "show",
Short: "Print the cluster domain name.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return show(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func show(ctx context.Context, uncli *cli.CLI, opts showOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
domain, err := clusterClient.GetDomain(ctx)
if err != nil {
if errors.Is(err, api.ErrNotFound) {
return errors.New("no domain reserved")
}
return err
}
fmt.Println(domain)
return nil
}
+122
View File
@@ -0,0 +1,122 @@
package main
import (
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/spf13/cobra"
"github.com/spf13/cobra/doc"
)
const docsDir = "website/docs/9-cli-reference"
type cmdWrapper struct {
cmd *cobra.Command
}
// NewDocsCommand creates a new hidden command to generate CLI reference docs.
func NewDocsCommand() *cobra.Command {
wrapper := &cmdWrapper{}
cmd := &cobra.Command{
Use: "docs",
Short: "Generate Uncloud CLI reference docs",
SilenceUsage: true,
DisableFlagsInUseLine: true,
Hidden: true,
Args: cobra.NoArgs,
ValidArgsFunction: cobra.NoFileCompletions,
RunE: func(cmd *cobra.Command, _ []string) error {
// Remove existing markdown files.
mdFiles, err := filepath.Glob(filepath.Join(docsDir, "*.md"))
if err != nil {
return fmt.Errorf("list existing CLI docs: %w", err)
}
for _, f := range mdFiles {
if err = os.Remove(f); err != nil {
return fmt.Errorf("remove '%s': %w", f, err)
}
}
// Generate new CLI reference docs.
wrapper.cmd.Root().DisableAutoGenTag = true
if err := doc.GenMarkdownTree(cmd.Root(), docsDir); err != nil {
return fmt.Errorf("generate CLI docs: %w", err)
}
// Remove *completion*.md files that contain malformatted code blocks that break Docusaurus.
mdFiles, err = filepath.Glob(filepath.Join(docsDir, "*completion*.md"))
if err != nil {
return fmt.Errorf("list generated CLI docs: %w", err)
}
for _, f := range mdFiles {
if err = os.Remove(f); err != nil {
return fmt.Errorf("remove '%s': %w", f, err)
}
}
// Post-process generated markdown files.
mdFiles, err = filepath.Glob(filepath.Join(docsDir, "*.md"))
if err != nil {
return fmt.Errorf("list generated CLI docs: %w", err)
}
for _, f := range mdFiles {
if err = postProcessMarkdown(f); err != nil {
return fmt.Errorf("post-process '%s': %w", f, err)
}
}
return nil
},
}
wrapper.cmd = cmd
return cmd
}
// postProcessMarkdown applies transformations to generated markdown files.
func postProcessMarkdown(filename string) error {
data, err := os.ReadFile(filename)
if err != nil {
return err
}
content := string(data)
// Replace "SEE ALSO" with "See also".
content = strings.ReplaceAll(content, "SEE ALSO", "See also")
// Escape <id> to avoid Docusaurus treating it as an HTML tag.
content = strings.ReplaceAll(content, "<id>", "\\<id>")
// Remove broken links to completion docs.
if strings.Contains(content, "[uc completion") {
lines := strings.Split(content, "\n")
var filteredLines []string
for _, line := range lines {
if !strings.Contains(line, "[uc completion") {
filteredLines = append(filteredLines, line)
}
}
content = strings.Join(filteredLines, "\n")
}
// Adjust heading levels. Process from shortest to longest to avoid double replacements.
replacements := []struct {
old, new string
}{
{`(?m)^## `, `# `},
{`(?m)^### `, `## `},
{`(?m)^#### `, `### `},
{`(?m)^##### `, `#### `},
}
for _, r := range replacements {
re := regexp.MustCompile(r.old)
content = re.ReplaceAllString(content, r.new)
}
return os.WriteFile(filename, []byte(content), 0o644)
}
+312
View File
@@ -0,0 +1,312 @@
package image
import (
"context"
"fmt"
"slices"
"sort"
"strings"
"time"
"github.com/charmbracelet/lipgloss"
"github.com/charmbracelet/lipgloss/table"
"github.com/containerd/platforms"
"github.com/docker/docker/api/types/image"
"github.com/docker/go-units"
"github.com/muesli/termenv"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type listOptions struct {
machines []string
nameFilter string
context string
}
func NewListCommand() *cobra.Command {
opts := listOptions{}
cmd := &cobra.Command{
Use: "ls [REPO:[TAG]]",
Aliases: []string{"list"},
Short: "List images on machines in the cluster.",
Long: "List images on machines in the cluster. By default, on all machines. Optionally filter by image name.",
Example: ` # List all images on all machines.
uc image ls
# List images on specific machine.
uc image ls -m machine1
# List images on multiple machines.
uc image ls -m machine1,machine2
# List images filtered by name (with any tag) on all machines.
uc image ls myapp
# List images filtered by name pattern on specific machine.
uc image ls "myapp:1.*" -m machine1`,
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) > 0 {
opts.nameFilter = args[0]
}
uncli := cmd.Context().Value("cli").(*cli.CLI)
return list(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil,
"Filter images by machine name or ID. Can be specified multiple times or as a comma-separated list. "+
"(default is include all machines)")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
// imageRow represents a single image with its metadata for display.
type imageRow struct {
id string
name string
platforms string
createdHuman string
createdUnix int64
size string
inUse string
store string
machine string
}
func list(ctx context.Context, uncli *cli.CLI, opts listOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
// Get all machines to create ID to name mapping.
allMachines, err := clusterClient.ListMachines(ctx, nil)
if err != nil {
return fmt.Errorf("list machines: %w", err)
}
machineIDToName := make(map[string]string)
for _, machineMember := range allMachines {
if machineMember.Machine != nil && machineMember.Machine.Id != "" && machineMember.Machine.Name != "" {
machineIDToName[machineMember.Machine.Id] = machineMember.Machine.Name
}
}
machines := cli.ExpandCommaSeparatedValues(opts.machines)
clusterImages, err := clusterClient.ListImages(ctx, api.ImageFilter{
Machines: machines,
Name: opts.nameFilter,
})
if err != nil {
return fmt.Errorf("list images: %w", err)
}
// Collect all images from all machines.
var rows []imageRow
for _, machineImages := range clusterImages {
// Get machine name for better readability.
machineName := machineImages.Metadata.Machine
if m := allMachines.FindByNameOrID(machineName); m != nil {
machineName = m.Machine.Name
}
store := "docker"
if machineImages.ContainerdStore {
store = "containerd"
}
// Process each image for this machine.
for _, img := range machineImages.Images {
// Show the first 12 chars without 'sha256:' as the image ID like Docker does.
id := strings.TrimPrefix(img.ID, "sha256:")[:12]
name := "<none>"
if len(img.RepoTags) > 0 && img.RepoTags[0] != "<none>:<none>" {
name = img.RepoTags[0]
}
imgPlatforms, _ := imagePlatforms(img)
formattedPlatforms := formatPlatforms(imgPlatforms)
created := ""
createdAt := time.Unix(img.Created, 0)
if !createdAt.IsZero() {
created = units.HumanDuration(time.Now().UTC().Sub(createdAt)) + " ago"
}
size := units.HumanSizeWithPrecision(float64(img.Size), 3)
// Check if the image is in use by any containers. Only supported by Docker API >=1.51
inUse := "-"
if img.Containers != -1 { // -1 means the info is not available.
if img.Containers > 0 {
inUse = lipgloss.NewStyle().Foreground(lipgloss.Color("10")).Render("●")
} else {
inUse = lipgloss.NewStyle().Foreground(lipgloss.Color("8")).Render("○")
}
}
rows = append(rows, imageRow{
id: id,
name: name,
platforms: formattedPlatforms,
createdHuman: created,
createdUnix: img.Created,
size: size,
inUse: inUse,
store: store,
machine: machineName,
})
}
}
if len(rows) == 0 {
if opts.nameFilter != "" {
fmt.Printf("No images matching '%s' found.\n", opts.nameFilter)
} else {
fmt.Println("No images found.")
}
return nil
}
// Sort images by name, then by machine name.
sort.Slice(rows, func(i, j int) bool {
if rows[i].name != rows[j].name {
return rows[i].name < rows[j].name
}
return rows[i].machine < rows[j].machine
})
// Print the images in a table format.
fmt.Println(formatImageTable(rows))
return nil
}
// imagePlatforms returns a list of platforms supported by the image and a boolean indicating if it's multi-platform.
func imagePlatforms(img image.Summary) ([]string, bool) {
var formattedPlatforms []string
multiPlatform := false
for _, m := range img.Manifests {
if m.Kind != image.ManifestKindImage || !m.Available {
continue
}
if m.ID != img.ID {
// There is an image manifest that has digest different from the main image digest.
// This means the image manifest is an index or a manifest list (multi-platform image).
multiPlatform = true
}
formattedPlatforms = append(formattedPlatforms, platforms.Format(m.ImageData.Platform))
}
slices.Sort(formattedPlatforms)
return formattedPlatforms, multiPlatform
}
func formatPlatforms(platforms []string) string {
if len(platforms) == 0 {
return "-"
}
platformStyle := lipgloss.NewStyle().
BorderForeground(lipgloss.Color("152")).
Foreground(lipgloss.Color("0")).
Background(lipgloss.Color("152"))
// Use fancy pill borders only if the output is a terminal with color support.
if lipgloss.ColorProfile() != termenv.Ascii {
platformStyle = platformStyle.Border(lipgloss.Border{Left: "", Right: ""}, false, true, false, true)
}
styledPlatforms := make([]string, len(platforms))
for i, p := range platforms {
styledPlatforms[i] = platformStyle.Render(p)
}
return strings.Join(styledPlatforms, " ")
}
func formatImageTable(rows []imageRow) string {
columns := []struct {
name string
hide bool
}{
{name: "IMAGE ID"},
{name: "NAME"},
{name: "PLATFORMS"},
{name: "CREATED"},
{name: "SIZE"},
{name: "IN USE"},
{name: "STORE"},
{name: "MACHINE"},
}
// Hide the "IN USE" column if none of the images have that info available.
inUseInfoAvailable := slices.ContainsFunc(rows, func(r imageRow) bool {
return r.inUse != "-"
})
if !inUseInfoAvailable {
// Hide "IN USE" column.
columns[5].hide = true
}
t := table.New().
// Remove the default border.
Border(lipgloss.Border{}).
BorderTop(false).
BorderBottom(false).
BorderLeft(false).
BorderRight(false).
BorderHeader(false).
BorderColumn(false).
StyleFunc(func(row, col int) lipgloss.Style {
if row == table.HeaderRow {
return lipgloss.NewStyle().Bold(true).PaddingRight(3)
}
// Regular style for data rows with padding.
return lipgloss.NewStyle().PaddingRight(3)
})
var headers []string
for _, col := range columns {
if !col.hide {
headers = append(headers, col.name)
}
}
t.Headers(headers...)
for _, row := range rows {
values := []string{
row.id,
row.name,
row.platforms,
row.createdHuman,
row.size,
row.inUse,
row.store,
row.machine,
}
var filteredValues []string
for i, v := range values {
if !columns[i].hide {
filteredValues = append(filteredValues, v)
}
}
t.Row(filteredValues...)
}
return t.String()
}
+100
View File
@@ -0,0 +1,100 @@
package image
import (
"context"
"fmt"
"github.com/containerd/platforms"
"github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/client"
"github.com/spf13/cobra"
)
type pushOptions struct {
image string
machines []string
context string
platform string
}
func NewPushCommand() *cobra.Command {
opts := pushOptions{}
cmd := &cobra.Command{
Use: "push IMAGE",
Short: "Upload a local Docker image to the cluster.",
Long: `Upload a local Docker image to the cluster transferring only the missing layers.
The image is uploaded to the machine which CLI is connected to (default) or the specified machine(s).`,
Example: ` # Push image to the machine the CLI is connected to.
uc image push myapp:latest
# Push image to specific machine.
uc image push myapp:latest -m machine1
# Push image to multiple machines.
uc image push myapp:latest -m machine1,machine2,machine3
# Push image to all machines in the cluster.
uc image push myapp:latest -m all
# Push a specific platform of a multi-platform image.
uc image push myapp:latest --platform linux/amd64`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.image = args[0]
return push(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil,
"Machine names to push the image to. Can be specified multiple times or as a comma-separated "+
"list of machine names.\n"+
"Use 'all' to push to all machines. (default is connected machine)")
cmd.Flags().StringVar(
&opts.platform, "platform", "",
"Push a specific platform of a multi-platform image (e.g., linux/amd64, linux/arm64).\n"+
"Local Docker must be configured to use containerd image store to support multi-platform images.",
)
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func push(ctx context.Context, uncli *cli.CLI, opts pushOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
machines := cli.ExpandCommaSeparatedValues(opts.machines)
pushOpts := client.PushImageOptions{}
// Special handling for "all" keyword to push to all machines.
if len(machines) == 1 && machines[0] == "all" {
pushOpts.AllMachines = true
} else {
pushOpts.Machines = machines
}
if opts.platform != "" {
p, err := platforms.Parse(opts.platform)
if err != nil {
return fmt.Errorf("invalid platform '%s': %w", opts.platform, err)
}
pushOpts.Platform = &p
}
return progress.RunWithTitle(ctx, func(ctx context.Context) error {
if err = clusterClient.PushImage(ctx, opts.image, pushOpts); err != nil {
return fmt.Errorf("push image to cluster: %w", err)
}
return nil
}, uncli.ProgressOut(), fmt.Sprintf("Pushing image %s to cluster", opts.image))
}
+19
View File
@@ -0,0 +1,19 @@
package image
import (
"github.com/spf13/cobra"
)
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "image",
Short: "Manage images on machines in the cluster.",
}
cmd.AddCommand(
NewListCommand(),
NewPushCommand(),
)
return cmd
}
+19
View File
@@ -0,0 +1,19 @@
package main
import (
"strings"
"github.com/psviderski/uncloud/cmd/uncloud/image"
"github.com/spf13/cobra"
)
// NewImagesCommand returns the 'image ls' command modified to work as 'images'.
func NewImagesCommand() *cobra.Command {
listCmd := image.NewListCommand()
listCmd.Use = "images [IMAGE]"
// Remove 'list' alias since this command is already an alias.
listCmd.Aliases = nil
listCmd.Example = strings.ReplaceAll(listCmd.Example, "uc image ls", "uc images")
return listCmd
}
+157 -11
View File
@@ -1,16 +1,32 @@
package machine
import (
"context"
"errors"
"fmt"
"net/netip"
"time"
"github.com/cenkalti/backoff/v4"
"github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/cmd/uncloud/caddy"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/cli/config"
"github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client"
"github.com/spf13/cobra"
"uncloud/internal/cli"
"uncloud/internal/cli/config"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
type addOptions struct {
name string
sshKey string
cluster string
name string
noCaddy bool
noInstall bool
publicIP string
sshKey string
context string
version string
}
func NewAddCommand() *cobra.Command {
@@ -26,24 +42,154 @@ func NewAddCommand() *cobra.Command {
if err != nil {
return fmt.Errorf("parse remote machine: %w", err)
}
remoteMachine := cli.RemoteMachine{
remoteMachine := &cli.RemoteMachine{
User: user,
Host: host,
Port: port,
KeyPath: opts.sshKey,
}
return uncli.AddMachine(cmd.Context(), remoteMachine, opts.cluster, opts.name)
return add(cmd.Context(), uncli, remoteMachine, opts)
},
}
cmd.Flags().StringVarP(&opts.name, "name", "n", "", "Assign a name to the machine")
cmd.Flags().StringVarP(&opts.name, "name", "n", "", "Assign a name to the machine.")
cmd.Flags().BoolVar(
&opts.noCaddy, "no-caddy", false,
"Don't deploy Caddy reverse proxy service to the machine.",
)
cmd.Flags().BoolVar(
&opts.noInstall, "no-install", false,
"Skip installation of Docker, Uncloud daemon, and dependencies on the machine. "+
"Assumes they're already installed and running.",
)
cmd.Flags().StringVar(
&opts.publicIP, "public-ip", "auto",
"Public IP address of the machine for ingress configuration. Use 'auto' for automatic detection, "+
fmt.Sprintf("blank '' or '%s' to disable ingress on this machine, or specify an IP address.", PublicIPNone),
)
cmd.Flags().StringVarP(
&opts.sshKey, "ssh-key", "i", "",
"path to SSH private key for SSH remote login (default ~/.ssh/id_*)",
fmt.Sprintf("Path to SSH private key for remote login (if not already added to SSH agent). (default %q)",
cli.DefaultSSHKeyPath),
)
cmd.Flags().StringVar(
&opts.version, "version", "latest",
"Version of the Uncloud daemon to install on the machine.",
)
cmd.Flags().StringVarP(
&opts.cluster, "cluster", "c", "",
"Name of the cluster to add the machine to (default is the current cluster)",
&opts.context, "context", "c", "",
"Name of the cluster context to add the machine to. (default is the current context)",
)
return cmd
}
func add(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine, opts addOptions) error {
var publicIP *netip.Addr
switch opts.publicIP {
case "auto":
publicIP = &netip.Addr{}
case "", PublicIPNone:
publicIP = nil
default:
ip, err := netip.ParseAddr(opts.publicIP)
if err != nil {
return fmt.Errorf("parse public IP: %w", err)
}
publicIP = &ip
}
clusterClient, machineClient, err := uncli.AddMachine(ctx, cli.AddMachineOptions{
Context: opts.context,
MachineName: opts.name,
PublicIP: publicIP,
RemoteMachine: remoteMachine,
SkipInstall: opts.noInstall,
Version: opts.version,
})
if err != nil {
return err
}
defer clusterClient.Close()
defer machineClient.Close()
if opts.noCaddy {
return nil
}
// Wait for the cluster to be initialised to be able to deploy the Caddy service.
fmt.Println("Waiting for the machine to be ready...")
fmt.Println()
if err = waitClusterInitialised(ctx, machineClient); err != nil {
return fmt.Errorf("wait for cluster to be initialised on machine: %w", err)
}
// Deploy a Caddy service container to the added machine. If caddy service is already deployed on other machines,
// use the deployed image version. Otherwise, use the latest version.
// NOTE: We use the cluster client to inspect and scale the Caddy service because the newly added machine may have
// issues accessing the Machine API of existing machines in the cluster.
// See the issue for more details: https://github.com/psviderski/uncloud/issues/65.
caddyImage := ""
caddySvc, err := clusterClient.InspectService(ctx, client.CaddyServiceName)
if err != nil {
if !errors.Is(err, api.ErrNotFound) {
return fmt.Errorf("inspect caddy service: %w", err)
}
} else {
caddyImage = caddySvc.Containers[0].Container.Config.Image
// Find the latest created container and use its image.
var latestCreated time.Time
for _, c := range caddySvc.Containers[1:] {
created, err := time.Parse(time.RFC3339Nano, c.Container.Created)
if err != nil {
continue
}
if created.After(latestCreated) {
latestCreated = created
caddyImage = c.Container.Config.Image
}
}
}
// TODO: scale the existing Caddy service to the new machine instead of running a new deployment
// that may cause a small downtime.
d, err := clusterClient.NewCaddyDeployment(caddyImage, "", api.Placement{})
if err != nil {
return fmt.Errorf("create caddy deployment: %w", err)
}
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
if _, err = d.Run(ctx); err != nil {
return fmt.Errorf("deploy caddy: %w", err)
}
return nil
}, uncli.ProgressOut(), fmt.Sprintf("Deploying service %s", d.Spec.Name))
if err != nil {
return err
}
fmt.Println()
return caddy.UpdateDomainRecords(ctx, machineClient, uncli.ProgressOut())
}
func waitClusterInitialised(ctx context.Context, client *client.Client) error {
boff := backoff.WithContext(backoff.NewExponentialBackOff(
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(5*time.Minute),
), ctx)
check := func() error {
_, err := client.ListMachines(ctx, nil)
if err == nil {
return nil
}
statusErr := status.Convert(err)
if statusErr.Code() == codes.FailedPrecondition {
return err
}
return backoff.Permanent(err)
}
return backoff.Retry(check, boff)
}
+6
View File
@@ -0,0 +1,6 @@
package machine
const (
// PublicIPNone is the value used to indicate removal of public IP
PublicIPNone = "none"
)
+147 -27
View File
@@ -1,30 +1,55 @@
package machine
import (
"context"
"fmt"
"github.com/spf13/cobra"
"net/netip"
"uncloud/internal/cli"
"uncloud/internal/cli/config"
"uncloud/internal/machine/network"
"github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/cmd/uncloud/caddy"
"github.com/psviderski/uncloud/cmd/uncloud/dns"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/cli/config"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/machine/cluster"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type initOptions struct {
name string
network string
userPublicKey string
sshKey string
cluster string
dnsEndpoint string
name string
network string
noCaddy bool
noDNS bool
noInstall bool
publicIP string
sshKey string
version string
context string
}
func NewInitCommand() *cobra.Command {
opts := initOptions{}
cmd := &cobra.Command{
Use: "init [USER@HOST:PORT]",
Use: "init [USER@HOST:PORT]",
Short: "Initialise a new cluster with a remote machine as the first member.",
Long: "Initialise a new cluster by setting up a remote machine as the first member.\n" +
"This command creates a new context in your Uncloud config to manage the cluster.",
Example: ` # Initialise a new cluster with default settings.
uc machine init root@<your-server-ip>
# Initialise with a context name 'prod' in the Uncloud config (~/.config/uncloud/config.yaml) and machine name 'vps1'.
uc machine init root@<your-server-ip> -c prod -n vps1
# Initialise with a non-root user and custom SSH port and key.
uc machine init ubuntu@<your-server-ip>:2222 -i ~/.ssh/mykey
# Initialise without Caddy (no reverse proxy) and without an automatically managed domain name (xxxxxx.cluster.uncloud.run).
# You can deploy Caddy with 'uc caddy deploy' and reserve a domain with 'uc dns reserve' later.
uc machine init root@<your-server-ip> --no-caddy --no-dns`,
// TODO: support initialising a cluster on the local machine.
Args: cobra.MaximumNArgs(1),
// TODO: include usage examples of initialising a local and remote machine.
Short: "Initialise a new cluster that consists of the local or remote machine.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
@@ -41,30 +66,125 @@ func NewInitCommand() *cobra.Command {
KeyPath: opts.sshKey,
}
}
netPrefix, err := netip.ParsePrefix(opts.network)
if err != nil {
return fmt.Errorf("parse network CIDR: %w", err)
}
return uncli.InitCluster(cmd.Context(), remoteMachine, opts.cluster, opts.name, netPrefix)
return initCluster(cmd.Context(), uncli, remoteMachine, opts)
},
}
cmd.Flags().StringVarP(&opts.name, "name", "n", "", "Assign a name to the machine")
cmd.Flags().StringVar(&opts.dnsEndpoint, "dns-endpoint", dns.DefaultUncloudDNSAPIEndpoint,
"API endpoint for the Uncloud DNS service.")
cmd.Flags().StringVarP(
&opts.name, "name", "n", "",
"Assign a name to the machine.",
)
cmd.Flags().StringVar(
&opts.network, "network", network.DefaultNetwork.String(),
"IPv4 network CIDR to use for machines and services",
&opts.network, "network", cluster.DefaultNetwork.String(),
"IPv4 network CIDR to use for machines and services.",
)
cmd.Flags().BoolVar(
&opts.noCaddy, "no-caddy", false,
"Don't deploy Caddy reverse proxy service to the machine. You can deploy it later with 'uc caddy deploy'.",
)
cmd.Flags().BoolVar(
&opts.noDNS, "no-dns", false,
"Don't reserve a cluster domain in Uncloud DNS. You can reserve it later with 'uc dns reserve'.",
)
cmd.Flags().BoolVar(
&opts.noInstall, "no-install", false,
"Skip installation of Docker, Uncloud daemon, and dependencies on the machine. "+
"Assumes they're already installed and running.",
)
cmd.Flags().StringVar(
&opts.publicIP, "public-ip", "auto",
"Public IP address of the machine for ingress configuration. Use 'auto' for automatic detection, "+
fmt.Sprintf("blank '' or '%s' to disable ingress on this machine, or specify an IP address.", PublicIPNone),
)
//cmd.Flags().StringVar(&opts.userPublicKey, "user-pubkey", "",
// "User's public key which will be able to access the cluster (hex-encoded)")
cmd.Flags().StringVarP(
&opts.sshKey, "ssh-key", "i", "",
"path to SSH private key for SSH remote login (default ~/.ssh/id_*)",
fmt.Sprintf("Path to SSH private key for remote login (if not already added to SSH agent). (default %q)",
cli.DefaultSSHKeyPath),
)
cmd.Flags().StringVar(
&opts.version, "version", "latest",
"Version of the Uncloud daemon to install on the machine.",
)
cmd.Flags().StringVarP(
&opts.cluster, "cluster", "c", "",
"Name of the cluster in the local config if initialising a remote machine",
&opts.context, "context", "c", cli.DefaultContextName,
"Name of the new context to be created in the Uncloud config to manage the cluster.",
)
return cmd
}
func initCluster(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine, opts initOptions) error {
netPrefix, err := netip.ParsePrefix(opts.network)
if err != nil {
return fmt.Errorf("parse network CIDR: %w", err)
}
var publicIP *netip.Addr
switch opts.publicIP {
case "auto":
publicIP = &netip.Addr{}
case "", PublicIPNone:
publicIP = nil
default:
ip, err := netip.ParseAddr(opts.publicIP)
if err != nil {
return fmt.Errorf("parse public IP: %w", err)
}
publicIP = &ip
}
client, err := uncli.InitCluster(ctx, cli.InitClusterOptions{
Context: opts.context,
MachineName: opts.name,
Network: netPrefix,
PublicIP: publicIP,
RemoteMachine: remoteMachine,
SkipInstall: opts.noInstall,
Version: opts.version,
})
if err != nil {
return err
}
defer client.Close()
if opts.noCaddy && opts.noDNS {
return nil
}
// Deploy the Caddy service to the initialised machine.
// The creation of a deployment plan talks to cluster API. Since the API needs a few moments to become available
// after cluster initialisation, we keep the user informed during this wait.
fmt.Println("Waiting for the machine to be ready...")
fmt.Println()
if !opts.noDNS {
domain, err := client.ReserveDomain(ctx, &pb.ReserveDomainRequest{Endpoint: opts.dnsEndpoint})
if err != nil {
return fmt.Errorf("reserve cluster domain in Uncloud DNS: %w", err)
}
fmt.Printf("Reserved cluster domain: %s\n", domain.Name)
}
if !opts.noCaddy {
d, err := client.NewCaddyDeployment("", "", api.Placement{})
if err != nil {
return fmt.Errorf("create caddy deployment: %w", err)
}
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
if _, err = d.Run(ctx); err != nil {
return fmt.Errorf("deploy caddy: %w", err)
}
return nil
}, uncli.ProgressOut(), fmt.Sprintf("Deploying service %s", d.Spec.Name))
if err != nil {
return err
}
fmt.Println()
return caddy.UpdateDomainRecords(ctx, client, uncli.ProgressOut())
}
return nil
}
-24
View File
@@ -1,24 +0,0 @@
package machine
import (
"github.com/spf13/cobra"
"uncloud/internal/cli"
)
func NewListCommand() *cobra.Command {
opts := addOptions{}
cmd := &cobra.Command{
Use: "list",
Aliases: []string{"ls"},
Short: "List machines in a cluster",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return uncli.ListMachines(cmd.Context(), opts.cluster)
},
}
cmd.Flags().StringVarP(
&opts.cluster, "cluster", "c", "",
"Name of the cluster (default is the current cluster)",
)
return cmd
}
+86
View File
@@ -0,0 +1,86 @@
package machine
import (
"context"
"fmt"
"net/netip"
"os"
"strings"
"text/tabwriter"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/machine/network"
"github.com/spf13/cobra"
)
func NewListCommand() *cobra.Command {
var contextName string
cmd := &cobra.Command{
Use: "ls",
Aliases: []string{"list"},
Short: "List machines in a cluster.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return list(cmd.Context(), uncli, contextName)
},
}
cmd.Flags().StringVarP(
&contextName, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func list(ctx context.Context, uncli *cli.CLI, clusterName string) error {
client, err := uncli.ConnectCluster(ctx, clusterName)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
machines, err := client.ListMachines(ctx, nil)
if err != nil {
return fmt.Errorf("list machines: %w", err)
}
// Print the list of machines in a table format.
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
// Print header.
if _, err = fmt.Fprintln(tw, "NAME\tSTATE\tADDRESS\tPUBLIC IP\tWIREGUARD ENDPOINTS\tMACHINE ID"); err != nil {
return fmt.Errorf("write header: %w", err)
}
// Print rows.
for _, member := range machines {
m := member.Machine
subnet, _ := m.Network.Subnet.ToPrefix()
subnet = netip.PrefixFrom(network.MachineIP(subnet), subnet.Bits())
publicIP := "-"
if m.PublicIp != nil {
ip, _ := m.PublicIp.ToAddr()
publicIP = ip.String()
}
endpoints := make([]string, len(m.Network.Endpoints))
for i, ep := range m.Network.Endpoints {
addrPort, _ := ep.ToAddrPort()
endpoints[i] = addrPort.String()
}
if _, err = fmt.Fprintf(
tw, "%s\t%s\t%s\t%s\t%s\t%s\n", m.Name, capitalise(member.State.String()), subnet, publicIP,
strings.Join(endpoints, ", "), member.Machine.Id,
); err != nil {
return fmt.Errorf("write row: %w", err)
}
}
return tw.Flush()
}
// capitalise returns a string where the first character is upper case, and the rest is lower case.
func capitalise(s string) string {
if s == "" {
return ""
}
return strings.ToUpper(s[:1]) + strings.ToLower(s[1:])
}
+47
View File
@@ -0,0 +1,47 @@
package machine
import (
"context"
"fmt"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
func NewRenameCommand() *cobra.Command {
var contextName string
cmd := &cobra.Command{
Use: "rename OLD_NAME NEW_NAME",
Short: "Rename a machine in the cluster.",
Long: `Rename a machine in the cluster.
This command changes the name of an existing machine while preserving all other
configuration including network settings, public IP, and cluster membership.`,
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return rename(cmd.Context(), uncli, contextName, args[0], args[1])
},
}
cmd.Flags().StringVarP(
&contextName, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func rename(ctx context.Context, uncli *cli.CLI, contextName, oldName, newName string) error {
client, err := uncli.ConnectCluster(ctx, contextName)
if err != nil {
return err
}
defer client.Close()
machine, err := client.RenameMachine(ctx, oldName, newName)
if err != nil {
return fmt.Errorf("rename machine: %w", err)
}
fmt.Printf("Machine %q renamed to %q (ID: %s)\n", oldName, machine.Name, machine.Id)
return nil
}
+249
View File
@@ -0,0 +1,249 @@
package machine
import (
"context"
"errors"
"fmt"
"maps"
"slices"
"strings"
"sync"
"github.com/charmbracelet/lipgloss"
"github.com/charmbracelet/lipgloss/tree"
"github.com/docker/compose/v2/pkg/progress"
"github.com/docker/docker/api/types/container"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type removeOptions struct {
noReset bool
yes bool
context string
}
func NewRmCommand() *cobra.Command {
opts := removeOptions{}
cmd := &cobra.Command{
Use: "rm MACHINE",
Aliases: []string{"remove", "delete"},
Short: "Remove a machine from a cluster and reset it.",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return remove(cmd.Context(), uncli, args[0], opts)
},
}
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
"Do not prompt for confirmation before removing the machine.")
cmd.Flags().BoolVar(&opts.noReset, "no-reset", false,
"Do not reset the machine after removing it from the cluster. This will leave all containers and data intact.")
return cmd
}
func remove(ctx context.Context, uncli *cli.CLI, nameOrID string, opts removeOptions) error {
// TODO: automatically choose a connection to the machine that is not being removed.
client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
// Verify the machine exists and list all service containers on it including stopped ones.
mctx, machines, err := api.ProxyMachinesContext(ctx, client, []string{nameOrID})
if err != nil {
return err
}
if len(machines) == 0 {
return fmt.Errorf("machine '%s' not found in the cluster", nameOrID)
}
m := machines[0].Machine
// Verify if the machine being removed is the proxy machine we're connected to.
proxyMachine, err := client.MachineClient.Inspect(ctx, nil)
if err != nil {
return fmt.Errorf("inspect proxy machine: %w", err)
}
if proxyMachine.Id == m.Id {
allMachines, err := client.ListMachines(ctx, nil)
if err != nil {
return fmt.Errorf("list machines: %w", err)
}
if len(allMachines) > 1 {
return errors.New("cannot remove the machine you are currently connected to. " +
"Please connect to another machine in the cluster and try again. " +
"Use --connect flag or update 'connections' for the cluster context in your Uncloud config")
// It's ok to remove the proxy machine if it's the last one in the cluster.
}
}
// TODO: mark the machine as being removed and unschedulable when this is possible to prevent new containers
// from being scheduled on it while the removal is in progress.
reset := !opts.noReset
var containers []api.ServiceContainer
reachable := false
if reset {
// Check if the machine is up and has service containers.
listOpts := container.ListOptions{All: true}
machineContainers, err := client.Docker.ListServiceContainers(mctx, "", listOpts)
if err == nil {
reachable = true
containers = machineContainers[0].Containers
if len(containers) > 0 {
plural := ""
if len(containers) > 1 {
plural = "s"
}
fmt.Printf("Found %d service container%s on machine '%s':\n", len(containers), plural, m.Name)
fmt.Println(formatContainerTree(containers))
fmt.Println()
fmt.Println("This will remove all service containers from the machine, remove it from the cluster, " +
"and reset it to the uninitialised state.")
} else {
fmt.Printf("No service containers found on machine '%s'.\n", m.Name)
fmt.Println("This will remove the machine from the cluster and reset it to the uninitialised state.")
}
} else {
fmt.Printf("This will remove machine '%s' from the cluster without resetting it as it's unreachable.\n",
m.Name)
}
} else {
fmt.Printf("This will remove machine '%s' from the cluster without resetting it.\n", m.Name)
}
if !opts.yes {
confirmed, err := cli.Confirm()
if err != nil {
return fmt.Errorf("confirm removal: %w", err)
}
if !confirmed {
fmt.Println("Cancelled. Machine was not removed.")
return nil
}
}
if reset && len(containers) > 0 {
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
return removeContainers(ctx, client, containers)
}, uncli.ProgressOut(), "Removing containers")
if err != nil {
return fmt.Errorf("remove containers: %w", err)
}
fmt.Println()
}
if _, err = client.RemoveMachine(ctx, &pb.RemoveMachineRequest{Id: m.Id}); err != nil {
return fmt.Errorf("remove machine from cluster: %w", err)
}
fmt.Printf("Machine '%s' removed from the cluster.\n", m.Name)
if reset && reachable {
_, err = client.MachineClient.Reset(mctx, &pb.ResetRequest{})
if err != nil {
fmt.Printf("WARNING: Failed to reset machine: %v\n", err)
} else {
fmt.Println("Machine reset initiated and will complete in the background.")
}
}
// TODO: remove the connection to the machine from the uncloud config if it exists. We need a way to associate
// the machine with its connection in the config, e.g. by storing the machine name in the connection metadata.
// TODO: If Caddy was running on this machine and a cluster domain is reserved,
// let the user know that the DNS records should be updated.
return nil
}
// formatContainerTree formats a list of containers grouped by service as a tree structure.
func formatContainerTree(containers []api.ServiceContainer) string {
if len(containers) == 0 {
return ""
}
// Group containers by service.
serviceContainers := make(map[string][]api.ServiceContainer)
for _, ctr := range containers {
serviceName := ctr.ServiceName()
serviceContainers[serviceName] = append(serviceContainers[serviceName], ctr)
}
// Build tree output.
var output []string
serviceNames := slices.Sorted(maps.Keys(serviceContainers))
for _, serviceName := range serviceNames {
ctrs := serviceContainers[serviceName]
mode := ctrs[0].ServiceMode()
// Format a tree for the service with its containers.
plural := ""
if len(ctrs) > 1 {
plural = "s"
}
t := tree.Root(fmt.Sprintf("• %s (%s, %d container%s)", serviceName, mode, len(ctrs), plural)).
EnumeratorStyle(lipgloss.NewStyle().MarginLeft(2).MarginRight(1))
// Add containers as children.
for _, ctr := range ctrs {
state, _ := ctr.HumanState()
info := fmt.Sprintf("%s • %s • %s", ctr.Name, ctr.Config.Image, state)
t.Child(info)
}
output = append(output, t.String())
}
return strings.Join(output, "\n")
}
// removeContainers removes the given service containers from the machine.
func removeContainers(ctx context.Context, client api.Client, containers []api.ServiceContainer) error {
if len(containers) == 0 {
return nil
}
wg := sync.WaitGroup{}
errCh := make(chan error)
for _, ctr := range containers {
wg.Add(1)
go func(c api.ServiceContainer) {
defer wg.Done()
// Gracefully stop the container before removing it.
err := client.StopContainer(ctx, c.ServiceID(), c.ID, container.StopOptions{})
if err != nil && !errors.Is(err, api.ErrNotFound) {
errCh <- fmt.Errorf("stop container '%s': %w", c.ID, err)
}
err = client.RemoveContainer(ctx, c.ServiceID(), c.ID, container.RemoveOptions{
// Remove anonymous volumes created by the container.
RemoveVolumes: true,
})
if err != nil && !errors.Is(err, api.ErrNotFound) {
errCh <- fmt.Errorf("remove container '%s': %w", c.ID, err)
}
}(ctr)
}
go func() {
wg.Wait()
close(errCh)
}()
var err error
for e := range errCh {
err = errors.Join(err, e)
}
return err
}
+6 -2
View File
@@ -6,13 +6,17 @@ import (
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "machine",
Short: "Manage machines in an Uncloud cluster.",
Use: "machine",
Aliases: []string{"m"},
Short: "Manage machines in an Uncloud cluster.",
}
cmd.AddCommand(
NewAddCommand(),
NewInitCommand(),
NewListCommand(),
NewRenameCommand(),
NewRmCommand(),
NewUpdateCommand(),
NewTokenCommand(),
)
return cmd
+4 -3
View File
@@ -2,9 +2,10 @@ package machine
import (
"fmt"
"github.com/psviderski/uncloud/internal/daemon"
"github.com/psviderski/uncloud/internal/machine"
"github.com/spf13/cobra"
"uncloud/internal/daemon"
"uncloud/internal/machine"
)
type tokenOptions struct {
@@ -31,7 +32,7 @@ func NewTokenCommand() *cobra.Command {
}
cmd.Flags().StringVarP(&opts.dataDir, "data-dir", "d", machine.DefaultDataDir,
"Directory for storing persistent machine state")
"Directory for storing persistent machine state.")
_ = cmd.MarkFlagDirname("data-dir")
return cmd
+128
View File
@@ -0,0 +1,128 @@
package machine
import (
"context"
"fmt"
"net/netip"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/spf13/cobra"
)
type updateOptions struct {
name string
publicIP string
context string
}
func NewUpdateCommand() *cobra.Command {
opts := updateOptions{}
cmd := &cobra.Command{
Use: "update",
Short: "Update machine configuration in the cluster.",
Long: `Update machine configuration in the cluster.
This command allows setting various machine properties including:
- Machine name (--name)
- Public IP address (--public-ip)
At least one flag must be specified to perform an update operation.`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return update(cmd.Context(), uncli, cmd, opts, args[0])
},
}
cmd.Flags().StringVar(
&opts.name, "name", "",
"New name for the machine",
)
cmd.Flags().StringVar(
&opts.publicIP, "public-ip", "",
fmt.Sprintf("Public IP address of the machine for ingress configuration. Use '%s' or '' to remove the public IP.", PublicIPNone),
)
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func update(ctx context.Context, uncli *cli.CLI, cmd *cobra.Command, opts updateOptions, machineNameOrID string) error {
// Check if at least one flag was explicitly set
if !cmd.Flags().Changed("name") && !cmd.Flags().Changed("public-ip") {
return fmt.Errorf("at least one update flag must be specified (--name, --public-ip)")
}
client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return err
}
defer client.Close()
// First, resolve the machine to get its ID
machine, err := client.InspectMachine(ctx, machineNameOrID)
if err != nil {
return fmt.Errorf("find machine: %w", err)
}
// Build the update request
req := &pb.UpdateMachineRequest{
MachineId: machine.Machine.Id,
}
if opts.name != "" {
req.Name = &opts.name
}
// Check if --public-ip flag was explicitly provided
if cmd.Flags().Changed("public-ip") {
if opts.publicIP == "" || opts.publicIP == PublicIPNone {
req.PublicIp = &pb.IP{} // Empty IP to signal removal
} else {
// Parse and validate the public IP
ip, err := netip.ParseAddr(opts.publicIP)
if err != nil {
return fmt.Errorf("invalid public IP address %q: %w", opts.publicIP, err)
}
req.PublicIp = pb.NewIP(ip)
}
}
// Perform the update operation
updatedMachine, err := client.UpdateMachine(ctx, req)
if err != nil {
return fmt.Errorf("update machine: %w", err)
}
// Report what was changed
changes := make([]string, 0)
if opts.name != "" {
changes = append(changes, fmt.Sprintf("name: %q -> %q", machine.Machine.Name, updatedMachine.Name))
}
if cmd.Flags().Changed("public-ip") {
oldIP := PublicIPNone
if machine.Machine.PublicIp != nil {
if addr, err := machine.Machine.PublicIp.ToAddr(); err == nil {
oldIP = addr.String()
}
}
newIP := PublicIPNone
if updatedMachine.PublicIp != nil {
if addr, err := updatedMachine.PublicIp.ToAddr(); err == nil {
newIP = addr.String()
}
}
changes = append(changes, fmt.Sprintf("public IP: %s -> %s", oldIP, newIP))
}
fmt.Printf("Machine %q (ID: %s) configuration updated:\n", updatedMachine.Name, updatedMachine.Id)
for _, change := range changes {
fmt.Printf(" %s\n", change)
}
return nil
}
+70 -18
View File
@@ -3,44 +3,96 @@ package main
import (
"context"
"fmt"
"github.com/spf13/cobra"
"os"
"net/netip"
"strings"
"uncloud/cmd/uncloud/machine"
"uncloud/internal/cli"
"github.com/psviderski/uncloud/cmd/uncloud/caddy"
cmdcontext "github.com/psviderski/uncloud/cmd/uncloud/context"
"github.com/psviderski/uncloud/cmd/uncloud/dns"
"github.com/psviderski/uncloud/cmd/uncloud/image"
"github.com/psviderski/uncloud/cmd/uncloud/machine"
"github.com/psviderski/uncloud/cmd/uncloud/service"
"github.com/psviderski/uncloud/cmd/uncloud/volume"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/cli/config"
"github.com/psviderski/uncloud/internal/fs"
"github.com/psviderski/uncloud/internal/version"
"github.com/spf13/cobra"
)
type globalOptions struct {
configPath string
connect string
}
func main() {
var configPath string
opts := globalOptions{}
cmd := &cobra.Command{
Use: "uncloud",
Short: "A CLI tool for managing Uncloud resources such as clusters, machines, and services.",
Use: "uc",
Short: "A CLI tool for managing Uncloud resources such as machines, services, and volumes.",
Version: version.String(),
SilenceUsage: true,
SilenceErrors: true,
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
if strings.HasPrefix(configPath, "~/") {
home, err := os.UserHomeDir()
if err != nil {
return fmt.Errorf("get user home directory to resolve %q: %w", configPath, err)
cli.BindEnvToFlag(cmd, "connect", "UNCLOUD_CONNECT")
cli.BindEnvToFlag(cmd, "uncloud-config", "UNCLOUD_CONFIG")
var conn *config.MachineConnection
if opts.connect != "" {
if strings.HasPrefix(opts.connect, "tcp://") {
addrPort, err := netip.ParseAddrPort(opts.connect[len("tcp://"):])
if err != nil {
return fmt.Errorf("parse TCP address: %w", err)
}
conn = &config.MachineConnection{
TCP: &addrPort,
}
} else {
dest := opts.connect
if strings.HasPrefix(dest, "ssh://") {
dest = dest[len("ssh://"):]
}
conn = &config.MachineConnection{
SSH: config.SSHDestination(dest),
}
}
configPath = strings.Replace(configPath, "~", home, 1)
}
uncli, err := cli.New(configPath)
configPath := fs.ExpandHomeDir(opts.configPath)
uncli, err := cli.New(configPath, conn)
if err != nil {
return fmt.Errorf("initialize CLI: %w", err)
return fmt.Errorf("initialise CLI: %w", err)
}
cmd.SetContext(context.WithValue(cmd.Context(), "cli", uncli))
return nil
},
}
// TODO: allow to override using UNCLOUD_CONFIG env var.
cmd.PersistentFlags().StringVar(&configPath, "uncloud-config", "~/.config/uncloud/config.toml",
"path to the Uncloud configuration file")
_ = cmd.MarkPersistentFlagFilename("uncloud-config", "toml")
cmd.PersistentFlags().StringVar(&opts.connect, "connect", "",
"Connect to a remote cluster machine without using the Uncloud configuration file. [$UNCLOUD_CONNECT]\n"+
"Format: [ssh://]user@host[:port] or tcp://host:port")
cmd.PersistentFlags().StringVar(&opts.configPath, "uncloud-config", "~/.config/uncloud/config.yaml",
"Path to the Uncloud configuration file. [$UNCLOUD_CONFIG]")
_ = cmd.MarkPersistentFlagFilename("uncloud-config", "yaml", "yml")
// TODO: make --context a global flag and pass it as a value of the command context.
cmd.AddCommand(
NewDeployCommand(),
NewDocsCommand(),
NewBuildCommand(),
NewImagesCommand(),
caddy.NewRootCommand(),
cmdcontext.NewRootCommand(),
dns.NewRootCommand(),
image.NewRootCommand(),
machine.NewRootCommand(),
service.NewRootCommand(),
service.NewInspectCommand(),
service.NewListCommand(),
service.NewRmCommand(),
service.NewRunCommand(),
service.NewScaleCommand(),
volume.NewRootCommand(),
)
cobra.CheckErr(cmd.Execute())
}
+102
View File
@@ -0,0 +1,102 @@
package service
import (
"context"
"fmt"
"os"
"text/tabwriter"
"time"
"github.com/docker/docker/pkg/stringid"
"github.com/docker/go-units"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
type inspectOptions struct {
service string
context string
}
func NewInspectCommand() *cobra.Command {
opts := inspectOptions{}
cmd := &cobra.Command{
Use: "inspect SERVICE",
Short: "Display detailed information on a service.",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.service = args[0]
return inspect(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func inspect(ctx context.Context, uncli *cli.CLI, opts inspectOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
svc, err := client.InspectService(ctx, opts.service)
if err != nil {
return fmt.Errorf("inspect service: %w", err)
}
machines, err := client.ListMachines(ctx, nil)
if err != nil {
return fmt.Errorf("list machines: %w", err)
}
machinesNamesByID := make(map[string]string)
for _, m := range machines {
machinesNamesByID[m.Machine.Id] = m.Machine.Name
}
fmt.Printf("ID: %s\n", svc.ID)
fmt.Printf("Name: %s\n", svc.Name)
fmt.Printf("Mode: %s\n", svc.Mode)
fmt.Println()
// Print the list of containers in a table format.
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
if _, err = fmt.Fprintln(tw, "CONTAINER ID\tIMAGE\tCREATED\tSTATUS\tMACHINE"); err != nil {
return fmt.Errorf("write header: %w", err)
}
for _, ctr := range svc.Containers {
createdAt, err := time.Parse(time.RFC3339Nano, ctr.Container.Created)
if err != nil {
return fmt.Errorf("parse created time: %w", err)
}
created := units.HumanDuration(time.Now().UTC().Sub(createdAt)) + " ago"
machine := machinesNamesByID[ctr.MachineID]
if machine == "" {
machine = ctr.MachineID
}
state, err := ctr.Container.HumanState()
if err != nil {
return fmt.Errorf("get human state: %w", err)
}
_, err = fmt.Fprintf(
tw,
"%s\t%s\t%s\t%s\t%s\n",
stringid.TruncateID(ctr.Container.ID),
ctr.Container.Config.Image,
created,
state,
machine,
)
if err != nil {
return fmt.Errorf("write row: %w", err)
}
}
return tw.Flush()
}
+81
View File
@@ -0,0 +1,81 @@
package service
import (
"context"
"fmt"
"os"
"strings"
"text/tabwriter"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
func NewListCommand() *cobra.Command {
var contextName string
cmd := &cobra.Command{
Use: "ls",
Aliases: []string{"list"},
Short: "List services.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return list(cmd.Context(), uncli, contextName)
},
}
cmd.Flags().StringVarP(
&contextName, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func list(ctx context.Context, uncli *cli.CLI, contextName string) error {
client, err := uncli.ConnectCluster(ctx, contextName)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
services, err := client.ListServices(ctx)
if err != nil {
return fmt.Errorf("list services: %w", err)
}
serviceNames := make(map[string]struct{}, len(services))
haveDuplicateNames := false
for _, svc := range services {
if _, exists := serviceNames[svc.Name]; exists {
haveDuplicateNames = true
break
}
serviceNames[svc.Name] = struct{}{}
}
// Print the list of services in a table format.
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
// Include the ID column if there are duplicate service names to differentiate them.
if haveDuplicateNames {
if _, err = fmt.Fprintf(tw, "ID\t"); err != nil {
return fmt.Errorf("write header: %w", err)
}
}
if _, err = fmt.Fprintln(tw, "NAME\tMODE\tREPLICAS\tIMAGE\tENDPOINTS"); err != nil {
return fmt.Errorf("write header: %w", err)
}
for _, s := range services {
images := strings.Join(s.Images(), ", ")
endpoints := strings.Join(s.Endpoints(), ", ")
if haveDuplicateNames {
if _, err = fmt.Fprintf(tw, "%s\t", s.ID); err != nil {
return fmt.Errorf("write row: %w", err)
}
}
if _, err = fmt.Fprintf(tw, "%s\t%s\t%d\t%s\t%s\n",
s.Name, s.Mode, len(s.Containers), images, endpoints); err != nil {
return fmt.Errorf("write row: %w", err)
}
}
return tw.Flush()
}
+54
View File
@@ -0,0 +1,54 @@
package service
import (
"context"
"fmt"
"github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
type rmOptions struct {
services []string
context string
}
func NewRmCommand() *cobra.Command {
opts := rmOptions{}
cmd := &cobra.Command{
Use: "rm SERVICE [SERVICE...]",
Aliases: []string{"remove", "delete"},
Short: "Remove one or more services.",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.services = args
return rm(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func rm(ctx context.Context, uncli *cli.CLI, opts rmOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
for _, s := range opts.services {
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
if err = client.RemoveService(ctx, s); err != nil {
return fmt.Errorf("remove service '%s': %w", s, err)
}
return nil
}, uncli.ProgressOut(), "Removing service "+s)
}
return err
}
+21
View File
@@ -0,0 +1,21 @@
package service
import (
"github.com/spf13/cobra"
)
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "service",
Aliases: []string{"svc"},
Short: "Manage services in an Uncloud cluster.",
}
cmd.AddCommand(
NewInspectCommand(),
NewListCommand(),
NewRmCommand(),
NewRunCommand(),
NewScaleCommand(),
)
return cmd
}
+382
View File
@@ -0,0 +1,382 @@
package service
import (
"context"
"fmt"
"os"
"strings"
dockeropts "github.com/docker/cli/opts"
"github.com/docker/compose/v2/pkg/progress"
"github.com/docker/docker/daemon/names"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/secret"
"github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client/deploy"
"github.com/spf13/cobra"
)
type runOptions struct {
caddyfile string
command []string
cpu dockeropts.NanoCPUs
entrypoint string
entrypointChanged bool
env []string
image string
machines []string
memory dockeropts.MemBytes
mode string
name string
privileged bool
publish []string
pull string
replicas uint
user string
volumes []string
context string
}
func NewRunCommand() *cobra.Command {
opts := runOptions{}
cmd := &cobra.Command{
Use: "run IMAGE [COMMAND...]",
Short: "Run a service.",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.entrypointChanged = cmd.Flag("entrypoint").Changed
opts.image = args[0]
if len(args) > 1 {
opts.command = args[1:]
}
return run(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVar(&opts.caddyfile, "caddyfile", "",
"Path to a custom Caddy config (Caddyfile) for the service. "+
"Cannot be used together with non-@host published ports.")
cmd.Flags().VarP(&opts.cpu, "cpu", "",
"Maximum number of CPU cores a service container can use. Fractional values are allowed: "+
"0.5 for half a core or 2.25 for two and a quarter cores.")
cmd.Flags().StringVar(&opts.entrypoint, "entrypoint", "",
"Overwrite the default ENTRYPOINT of the image. Pass an empty string \"\" to reset it.")
cmd.Flags().StringSliceVarP(&opts.env, "env", "e", nil,
"Set an environment variable for service containers. Can be specified multiple times.\n"+
"Format: VAR=value or just VAR to use the value from the local environment.")
cmd.Flags().StringVar(&opts.mode, "mode", api.ServiceModeReplicated,
fmt.Sprintf("Replication mode of the service: either '%s' (a specified number of containers across "+
"the machines) or '%s' (one container on every machine).",
api.ServiceModeReplicated, api.ServiceModeGlobal))
cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil,
"Placement constraint by machine names, limiting which machines the service can run on. Can be specified "+
"multiple times or as a comma-separated list of machine names. (default is any suitable machine)")
cmd.Flags().VarP(&opts.memory, "memory", "",
"Maximum amount of memory a service container can use. Value is a positive integer with optional unit suffix "+
"(b, k, m, g). Default unit is bytes if no suffix specified.\n"+
"Examples: 1073741824, 1024m, 1g (all equal 1 gibibyte)")
cmd.Flags().StringVarP(&opts.name, "name", "n", "",
"Assign a name to the service. A random name is generated if not specified.")
cmd.Flags().BoolVar(&opts.privileged, "privileged", false,
"Give extended privileges to service containers. This is a security risk and should be used with caution.")
cmd.Flags().StringSliceVarP(&opts.publish, "publish", "p", nil,
"Publish a service port to make it accessible outside the cluster. Can be specified multiple times.\n"+
"Format: [hostname:]container_port[/protocol] or [host_ip:]host_port:container_port[/protocol]@host\n"+
"Supported protocols: tcp, udp, http, https (default is tcp). If a hostname for http(s) port is not specified\n"+
"and a cluster domain is reserved, service-name.cluster-domain will be used as the hostname.\n"+
"Examples:\n"+
" -p 8080/https Publish port 8080 as HTTPS via reverse proxy with default service-name.cluster-domain hostname\n"+
" -p app.example.com:8080/https Publish port 8080 as HTTPS via reverse proxy with custom hostname\n"+
// TODO: add support for publishing L4 tcp/udp ports.
//" -p 9000:8080 Publish port 8080 as TCP port 9000 via reverse proxy\n"+
" -p 53:5353/udp@host Bind UDP port 5353 to host port 53")
cmd.Flags().StringVar(&opts.pull, "pull", api.PullPolicyMissing,
fmt.Sprintf("Pull image from the registry before running service containers ('%s', '%s', '%s').",
api.PullPolicyAlways, api.PullPolicyMissing, api.PullPolicyNever))
cmd.Flags().UintVar(&opts.replicas, "replicas", 1,
"Number of containers to run for the service. Only valid for a replicated service.")
cmd.Flags().StringVarP(&opts.user, "user", "u", "",
"User name or UID and optionally group name or GID used for running the command inside service containers.\n"+
"Format: USER[:GROUP] or UID[:GID]. If not specified, the user is set to the default user of the image.")
cmd.Flags().StringSliceVarP(&opts.volumes, "volume", "v", nil,
"Mount a data volume or host path into service containers. Service containers will be scheduled on the machine(s) where\n"+
"the volume is located. Can be specified multiple times.\n"+
"Format: volume_name:/container/path[:ro|volume-nocopy] or /host/path:/container/path[:ro]\n"+
"Examples:\n"+
" -v postgres-data:/var/lib/postgresql/data Mount volume 'postgres-data' to /var/lib/postgresql/data in container\n"+
" -v /data/uploads:/app/uploads Bind mount /data/uploads host directory to /app/uploads in container\n"+
" -v /host/path:/container/path:ro Bind mount a host directory or file as read-only")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context to run the service in. (default is the current context)",
)
return cmd
}
func run(ctx context.Context, uncli *cli.CLI, opts runOptions) error {
spec, err := prepareServiceSpec(opts)
if err != nil {
return err
}
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
var resp api.RunServiceResponse
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
resp, err = clusterClient.RunService(ctx, spec)
if err != nil {
return fmt.Errorf("run service: %w", err)
}
return nil
}, uncli.ProgressOut(), fmt.Sprintf("Running service %s (%s mode)", spec.Name, spec.Mode))
if err != nil {
return err
}
svc, err := clusterClient.InspectService(ctx, resp.ID)
if err != nil {
return fmt.Errorf("inspect service: %w", err)
}
endpoints := svc.Endpoints()
if len(endpoints) > 0 {
fmt.Println()
fmt.Printf("%s endpoints:\n", svc.Name)
for _, endpoint := range endpoints {
fmt.Printf(" • %s\n", endpoint)
}
}
return nil
}
func prepareServiceSpec(opts runOptions) (api.ServiceSpec, error) {
var spec api.ServiceSpec
caddyfile := ""
if opts.caddyfile != "" {
data, err := os.ReadFile(opts.caddyfile)
if err != nil {
return spec, fmt.Errorf("read Caddyfile: %w", err)
}
caddyfile = strings.TrimSpace(string(data))
}
env, err := parseEnv(opts.env)
if err != nil {
return spec, err
}
switch opts.mode {
case api.ServiceModeReplicated, api.ServiceModeGlobal:
default:
return spec, fmt.Errorf("invalid replication mode: '%s'", opts.mode)
}
switch opts.pull {
case api.PullPolicyAlways, api.PullPolicyMissing, api.PullPolicyNever:
default:
return spec, fmt.Errorf("invalid pull policy: '%s'", opts.pull)
}
ports := make([]api.PortSpec, len(opts.publish))
for i, publishPort := range opts.publish {
port, err := api.ParsePortSpec(publishPort)
if err != nil {
return spec, fmt.Errorf("invalid service port '%s': %w", publishPort, err)
}
ports[i] = port
}
volumes, mounts, err := parseVolumeFlags(opts.volumes)
if err != nil {
return spec, err
}
placement := api.Placement{
Machines: cli.ExpandCommaSeparatedValues(opts.machines),
}
spec = api.ServiceSpec{
Container: api.ContainerSpec{
Command: opts.command,
Env: env,
Image: opts.image,
Privileged: opts.privileged,
PullPolicy: opts.pull,
Resources: api.ContainerResources{
CPU: opts.cpu.Value(),
Memory: opts.memory.Value(),
},
User: opts.user,
VolumeMounts: mounts,
},
Mode: opts.mode,
Name: opts.name,
Placement: placement,
Ports: ports,
Replicas: opts.replicas,
Volumes: volumes,
}
if caddyfile != "" {
spec.Caddy = &api.CaddySpec{
Config: caddyfile,
}
}
// Overwrite the default ENTRYPOINT of the image or reset it if an empty string is passed.
if opts.entrypoint != "" {
spec.Container.Entrypoint = []string{opts.entrypoint}
} else if opts.entrypointChanged {
spec.Container.Entrypoint = []string{""}
}
if err = spec.Validate(); err != nil {
return spec, fmt.Errorf("invalid service configuration: %w", err)
}
// Generate a service name if not specified to be able to include it in the progress title.
if spec.Name == "" {
spec.Name, err = deploy.GenerateServiceName(spec.Container.Image)
if err != nil {
return spec, fmt.Errorf("generate service name: %w", err)
}
}
return spec, err
}
// parseEnv parses the environment variables from the command line arguments.
// It supports two formats: "VAR=value" or just "VAR" to use the value from the local environment.
func parseEnv(env []string) (api.EnvVars, error) {
envVars := make(api.EnvVars)
for _, e := range env {
key, value, hasValue := strings.Cut(e, "=")
if key == "" {
return nil, fmt.Errorf("invalid environment variable: '%s'", e)
}
if hasValue {
envVars[key] = value
} else {
if localEnvValue, ok := os.LookupEnv(key); ok {
envVars[key] = localEnvValue
}
}
}
return envVars, nil
}
// parseVolumeFlags parses volume flag values in Docker CLI format and returns VolumeSpecs and VolumeMounts.
// It handles both named volumes (volume_name:/container/path[:ro|volume-nocopy])
// and bind mounts (/host/path:/container/path[:ro]).
func parseVolumeFlags(volumes []string) ([]api.VolumeSpec, []api.VolumeMount, error) {
specs := make([]api.VolumeSpec, 0, len(volumes))
mounts := make([]api.VolumeMount, 0, len(volumes))
// Track volume names to avoid duplicate specs.
seenVolumes := make(map[string]struct{})
for _, vol := range volumes {
spec, mount, err := parseVolumeFlagValue(vol)
if err != nil {
return nil, nil, fmt.Errorf("invalid volume mount '%s': %w", vol, err)
}
if _, ok := seenVolumes[spec.Name]; !ok {
specs = append(specs, spec)
seenVolumes[spec.Name] = struct{}{}
}
mounts = append(mounts, mount)
}
return specs, mounts, nil
}
func parseVolumeFlagValue(volume string) (api.VolumeSpec, api.VolumeMount, error) {
var spec api.VolumeSpec
var mount api.VolumeMount
parts := strings.Split(volume, ":")
switch len(parts) {
case 1:
return spec, mount, fmt.Errorf("invalid format, must contain at least one separator ':'")
case 2, 3:
// Format: (volume_name|/host/path):/container/path[:opts]
if !strings.HasPrefix(parts[1], "/") {
return spec, mount, fmt.Errorf("invalid container mount path: '%s', must be absolute path", parts[1])
}
mount.ContainerPath = parts[1]
volumeNoCopy := false
if len(parts) == 3 {
opts := strings.Split(parts[2], ",")
for _, opt := range opts {
switch opt {
case "ro", "readonly":
mount.ReadOnly = true
case "volume-nocopy":
volumeNoCopy = true
default:
return spec, mount, fmt.Errorf("invalid option: '%s'", opt)
}
}
}
if strings.HasPrefix(parts[0], "/") {
// Host path bind mount: /host/path:/container/path
suffix, err := secret.RandomAlphaNumeric(4)
if err != nil {
return spec, mount, fmt.Errorf("generate random suffix: %w", err)
}
spec = api.VolumeSpec{
Name: "bind-" + suffix,
Type: api.VolumeTypeBind,
BindOptions: &api.BindOptions{
HostPath: parts[0],
CreateHostPath: true,
},
}
} else {
// Named volume mount: volume_name:/container/path
volumeName := parts[0]
if !names.RestrictedNamePattern.MatchString(volumeName) {
return spec, mount, fmt.Errorf("volume name '%s' includes invalid characters, only '%s' are allowed. "+
"If you intended to pass a host directory or file, use absolute path",
volumeName, names.RestrictedNameChars)
}
spec = api.VolumeSpec{
Name: volumeName,
Type: api.VolumeTypeVolume,
VolumeOptions: &api.VolumeOptions{
Name: volumeName,
NoCopy: volumeNoCopy,
},
}
}
mount.VolumeName = spec.Name
default:
return spec, mount, fmt.Errorf("invalid format, must container at most 2 separators ':'")
}
return spec, mount, nil
}
+152
View File
@@ -0,0 +1,152 @@
package service
import (
"context"
"fmt"
"strconv"
"github.com/charmbracelet/huh"
"github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type scaleOptions struct {
service string
replicas uint
context string
}
func NewScaleCommand() *cobra.Command {
opts := scaleOptions{}
cmd := &cobra.Command{
Use: "scale SERVICE REPLICAS",
Short: "Scale a replicated service by changing the number of replicas.",
Long: "Scale a replicated service by changing the number of replicas. Scaling down requires confirmation.",
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.service = args[0]
replicas, err := strconv.ParseUint(args[1], 10, 0)
if err != nil {
return fmt.Errorf("invalid number of replicas: %w", err)
}
opts.replicas = uint(replicas)
return scale(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func scale(ctx context.Context, uncli *cli.CLI, opts scaleOptions) error {
if opts.replicas == 0 {
return fmt.Errorf(
"scaling to zero replicas is not supported. This would effectively remove the service without preserving "+
"its configuration, making it impossible to scale back up. Uncloud derives the service configuration "+
"from existing containers. Use 'uc rm %s' instead if you want to remove the service",
opts.service,
)
}
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
svc, err := clusterClient.InspectService(ctx, opts.service)
if err != nil {
return fmt.Errorf("inspect service '%s': %w", opts.service, err)
}
if svc.Mode != api.ServiceModeReplicated {
return fmt.Errorf("scaling is only supported for services in %s mode, service '%s' is in %s mode",
api.ServiceModeReplicated, svc.Name, svc.Mode)
}
currentReplicas := uint(len(svc.Containers))
if currentReplicas == opts.replicas {
fmt.Printf("Service '%s' already has %d replicas. No changes required.\n", svc.Name, currentReplicas)
return nil
}
// TODO: Check if all containers have the same spec. If not, prompt user to choose which one to scale.
// This can happen if a service deployment failed midway and some containers were not updated.
spec := svc.Containers[0].Container.ServiceSpec
spec.Replicas = opts.replicas
deployment := clusterClient.NewDeployment(spec, nil)
plan, err := deployment.Plan(ctx)
if err != nil {
return fmt.Errorf("plan deployment: %w", err)
}
if len(plan.Operations) == 0 {
fmt.Printf("Service '%s' is already scaled to %d replicas.\n", svc.Name, opts.replicas)
return nil
}
if opts.replicas < currentReplicas {
// Initialise a machine and container name resolver to properly format the plan output.
resolver, err := clusterClient.ServiceOperationNameResolver(ctx, svc)
if err != nil {
return fmt.Errorf("create machine and container name resolver for service operations: %w", err)
}
fmt.Printf("Scaling plan for service %s (%d → %d replicas):\n", svc.Name, currentReplicas, opts.replicas)
fmt.Println(plan.Format(resolver))
fmt.Println()
// Ask for confirmation before scaling down as it may cause data loss.
confirmed, err := confirm()
if err != nil {
return fmt.Errorf("confirm scaling: %w", err)
}
if !confirmed {
fmt.Println("Cancelled. No changes were made.")
return nil
}
}
title := fmt.Sprintf("Scaling service %s (%d → %d replicas)", svc.Name, currentReplicas, opts.replicas)
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
if _, err = deployment.Run(ctx); err != nil {
return fmt.Errorf("deploy service: %w", err)
}
return nil
}, uncli.ProgressOut(), title)
if err != nil {
return err
}
return nil
}
func confirm() (bool, error) {
var confirmed bool
form := huh.NewForm(
huh.NewGroup(
huh.NewConfirm().
Title(
"Do you want to continue?",
).
Affirmative("Yes!").
Negative("No").
Value(&confirmed),
),
).WithAccessible(true)
if err := form.Run(); err != nil {
return false, err
}
return confirmed, nil
}
+148
View File
@@ -0,0 +1,148 @@
package volume
import (
"context"
"fmt"
"slices"
"strings"
"github.com/charmbracelet/huh"
"github.com/docker/docker/api/types/volume"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type createOptions struct {
driver string
driverOpts []string
labels []string
machine string
context string
}
func NewCreateCommand() *cobra.Command {
opts := createOptions{}
cmd := &cobra.Command{
Use: "create VOLUME_NAME",
Short: "Create a volume on a specific machine.",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.driver = strings.TrimSpace(opts.driver)
volumeName := args[0]
if volumeName == "" {
return fmt.Errorf("volume name is required")
}
return create(cmd.Context(), uncli, volumeName, opts)
},
}
cmd.Flags().StringVarP(&opts.driver, "driver", "d", "local",
"Volume driver to use.")
cmd.Flags().StringSliceVarP(&opts.driverOpts, "opt", "o", nil,
"Driver specific options in the form of 'key=value' pairs. Can be specified multiple times.")
cmd.Flags().StringSliceVarP(&opts.labels, "label", "l", nil,
"Labels to assign to the volume in the form of 'key=value' pairs. Can be specified multiple times.")
cmd.Flags().StringVarP(&opts.machine, "machine", "m", "",
"Name or ID of the machine to create the volume on.")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd
}
func create(ctx context.Context, uncli *cli.CLI, name string, opts createOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
// Parse driver options.
driverOpts := make(map[string]string)
for _, opt := range opts.driverOpts {
k, v, ok := strings.Cut(opt, "=")
if !ok {
return fmt.Errorf("invalid driver option format: '%s' (expected key=value)", opt)
}
driverOpts[k] = v
}
// Parse labels.
labels := make(map[string]string)
for _, label := range opts.labels {
k, v, ok := strings.Cut(label, "=")
if !ok {
return fmt.Errorf("invalid label format: '%s' (expected key=value)", label)
}
labels[k] = v
}
// List machines and filter by the specified machine name or ID.
// If no machine is specified, prompt the user to select one.
machines, err := client.ListMachines(ctx, nil)
if err != nil {
return fmt.Errorf("list machines: %w", err)
}
var selectedMachine *pb.MachineInfo
if opts.machine == "" {
if len(machines) == 1 {
selectedMachine = machines[0].Machine
} else {
if selectedMachine, err = promptSelectMachine(ctx, machines); err != nil {
return fmt.Errorf("select machine: %w", err)
}
}
} else {
m := machines.FindByNameOrID(opts.machine)
if m == nil {
return fmt.Errorf("machine '%s' not found", opts.machine)
}
selectedMachine = m.Machine
}
createOpts := volume.CreateOptions{
Name: name,
Driver: opts.driver,
DriverOpts: driverOpts,
Labels: labels,
}
vol, err := client.CreateVolume(ctx, selectedMachine.Id, createOpts)
if err != nil {
return fmt.Errorf("create volume '%s' on machine '%s': %w", name, selectedMachine.Name, err)
}
fmt.Printf("Volume '%s' created on machine '%s'.\n", vol.Volume.Name, vol.MachineName)
return nil
}
func promptSelectMachine(ctx context.Context, machines api.MachineMembersList) (*pb.MachineInfo, error) {
options := make([]huh.Option[*pb.MachineInfo], len(machines))
for i, m := range machines {
options[i] = huh.NewOption(m.Machine.Name, m.Machine)
}
slices.SortFunc(options, func(a, b huh.Option[*pb.MachineInfo]) int {
return strings.Compare(a.Key, b.Key)
})
var selected *pb.MachineInfo
form := huh.NewForm(
huh.NewGroup(
huh.NewSelect[*pb.MachineInfo]().
Title("Select a machine to create the volume on (or specify with --machine flag)").
Options(options...).
Value(&selected),
),
)
if err := form.RunWithContext(ctx); err != nil {
return nil, err
}
return selected, nil
}
+81
View File
@@ -0,0 +1,81 @@
package volume
import (
"context"
"encoding/json"
"errors"
"fmt"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type inspectOptions struct {
machine string
context string
}
func NewInspectCommand() *cobra.Command {
opts := inspectOptions{}
cmd := &cobra.Command{
Use: "inspect VOLUME_NAME",
Short: "Display detailed information on a volume.",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return inspect(cmd.Context(), uncli, args[0], opts)
},
}
cmd.Flags().StringVarP(&opts.machine, "machine", "m", "",
"Name or ID of the machine where the volume is located. "+
"If not specified, the volume will be searched across all machines.")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd
}
func inspect(ctx context.Context, uncli *cli.CLI, name string, opts inspectOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
filter := &api.VolumeFilter{
Names: []string{name},
}
if opts.machine != "" {
filter.Machines = []string{opts.machine}
}
volumes, err := client.ListVolumes(ctx, filter)
if err != nil {
return fmt.Errorf("list volumes: %w", err)
}
if len(volumes) == 0 {
if opts.machine != "" {
return fmt.Errorf("volume '%s' not found on machine '%s'", name, opts.machine)
}
return fmt.Errorf("volume '%s' not found on any machine", name)
}
if len(volumes) > 1 {
fmt.Printf("Volume '%s' found on multiple machines:\n", name)
for _, v := range volumes {
fmt.Printf(" • %s\n", v.MachineName)
}
return errors.New("specify --machine flag to choose which machine to use")
}
data, err := json.MarshalIndent(volumes[0], "", " ")
if err != nil {
return fmt.Errorf("marshal volume: %w", err)
}
fmt.Println(string(data))
return nil
}
+105
View File
@@ -0,0 +1,105 @@
package volume
import (
"context"
"fmt"
"os"
"slices"
"strings"
"text/tabwriter"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type listOptions struct {
machines []string
quiet bool
context string
}
func NewListCommand() *cobra.Command {
opts := listOptions{}
cmd := &cobra.Command{
Use: "ls",
Aliases: []string{"list"},
Short: "List volumes across all machines in the cluster.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return list(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil,
"Filter volumes by machine name or ID. Can be specified multiple times or as a comma-separated list. "+
"(default is include all machines)")
cmd.Flags().BoolVarP(&opts.quiet, "quiet", "q", false,
"Only display volume names.")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd
}
func list(ctx context.Context, uncli *cli.CLI, opts listOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
// Apply machine filter if specified.
var filter *api.VolumeFilter
if len(opts.machines) > 0 {
machines := cli.ExpandCommaSeparatedValues(opts.machines)
filter = &api.VolumeFilter{
Machines: machines,
}
}
volumes, err := client.ListVolumes(ctx, filter)
if err != nil {
return fmt.Errorf("list volumes: %w", err)
}
if len(volumes) == 0 {
if !opts.quiet {
fmt.Println("No volumes found.")
}
return nil
}
// Sort the volumes by name first, then by machine name.
slices.SortFunc(volumes, func(a, b api.MachineVolume) int {
cmp := strings.Compare(a.Volume.Name, b.Volume.Name)
if cmp != 0 {
return cmp
}
return strings.Compare(a.MachineName, b.MachineName)
})
// If quiet mode, just print volume names.
if opts.quiet {
for _, v := range volumes {
fmt.Println(v.Volume.Name)
}
return nil
}
// Print the volumes in a table format.
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
fmt.Fprintln(tw, "NAME\tDRIVER\tMACHINE")
for _, v := range volumes {
fmt.Fprintf(tw, "%s\t%s\t%s\n",
v.Volume.Name,
v.Volume.Driver,
v.MachineName,
)
}
return tw.Flush()
}
+111
View File
@@ -0,0 +1,111 @@
package volume
import (
"context"
"errors"
"fmt"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type removeOptions struct {
force bool
machines []string
yes bool
context string
}
func NewRemoveCommand() *cobra.Command {
opts := removeOptions{}
cmd := &cobra.Command{
Use: "rm VOLUME_NAME [VOLUME_NAME...]",
Aliases: []string{"remove", "delete"},
Short: "Remove one or more volumes.",
Long: "Remove one or more volumes. You cannot remove a volume that is in use by a container.",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return remove(cmd.Context(), uncli, args, opts)
},
}
cmd.Flags().BoolVarP(&opts.force, "force", "f", false,
"Force the removal of one or more volumes.")
cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil,
"Name or ID of the machine to remove one or more volumes from. "+
"Can be specified multiple times or as a comma-separated list.\n"+
"If not specified, the found volume(s) will be removed from all machines.")
cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
"Do not prompt for confirmation before removing the volume(s).")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd
}
func remove(ctx context.Context, uncli *cli.CLI, names []string, opts removeOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
filter := &api.VolumeFilter{
Names: names,
}
if len(opts.machines) > 0 {
machines := cli.ExpandCommaSeparatedValues(opts.machines)
filter.Machines = machines
}
volumes, err := client.ListVolumes(ctx, filter)
if err != nil {
return fmt.Errorf("list volumes: %w", err)
}
if len(volumes) == 0 {
if len(names) == 1 {
return fmt.Errorf("volume '%s' not found", names[0])
}
return fmt.Errorf("no volumes found matching the specified names")
}
// Confirm removal if not using --yes flag.
if !opts.yes {
fmt.Println("The following volumes will be removed:")
for _, v := range volumes {
fmt.Printf(" • '%s' on machine '%s'\n", v.Volume.Name, v.MachineName)
}
fmt.Println()
confirmed, err := cli.Confirm()
if err != nil {
return fmt.Errorf("confirm removal: %w", err)
}
if !confirmed {
fmt.Println("Cancelled. No volumes were removed.")
return nil
}
}
// Remove the volumes one by one collecting errors.
var removeErr error
for _, v := range volumes {
if err = client.RemoveVolume(ctx, v.MachineID, v.Volume.Name, opts.force); err != nil {
if !errors.Is(err, api.ErrNotFound) {
removeErr = errors.Join(removeErr, fmt.Errorf("failed to remove volume '%s' on machine '%s': %w",
v.Volume.Name, v.MachineName, err))
}
continue
}
fmt.Printf("Volume '%s' removed from machine '%s'.\n", v.Volume.Name, v.MachineName)
}
return removeErr
}
+19
View File
@@ -0,0 +1,19 @@
package volume
import (
"github.com/spf13/cobra"
)
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "volume",
Short: "Manage volumes in an Uncloud cluster.",
}
cmd.AddCommand(
NewCreateCommand(),
NewInspectCommand(),
NewListCommand(),
NewRemoveCommand(),
)
return cmd
}
+11 -5
View File
@@ -2,22 +2,29 @@ package main
import (
"context"
"github.com/spf13/cobra"
"log/slog"
"os"
"os/signal"
"syscall"
"uncloud/internal/daemon"
"uncloud/internal/machine"
"github.com/psviderski/uncloud/internal/daemon"
"github.com/psviderski/uncloud/internal/log"
"github.com/psviderski/uncloud/internal/machine"
"github.com/psviderski/uncloud/internal/version"
"github.com/spf13/cobra"
)
func main() {
slog.SetLogLoggerLevel(slog.LevelDebug)
logger := slog.New(log.NewSlogTextHandler(os.Stderr, &slog.HandlerOptions{
Level: slog.LevelDebug,
}))
slog.SetDefault(logger)
var dataDir string
cmd := &cobra.Command{
Use: "uncloudd",
Short: "Uncloud machine daemon.",
Version: version.String(),
SilenceUsage: true,
SilenceErrors: true,
RunE: func(cmd *cobra.Command, args []string) error {
@@ -48,5 +55,4 @@ func main() {
}()
cobra.CheckErr(cmd.ExecuteContext(ctx))
}
+3 -2
View File
@@ -3,10 +3,11 @@ package main
import (
"context"
"fmt"
"github.com/hashicorp/serf/serf"
crdt "github.com/ipfs/go-ds-crdt"
"log/slog"
"time"
"github.com/hashicorp/serf/serf"
crdt "github.com/ipfs/go-ds-crdt"
)
// Implements the Broadcaster interface.
+2 -1
View File
@@ -2,9 +2,10 @@ package main
import (
"fmt"
"github.com/dgraph-io/badger/v3"
"log"
"time"
"github.com/dgraph-io/badger/v3"
)
func customTimeEncoder(t time.Time) string {
+2 -1
View File
@@ -2,9 +2,10 @@ package main
import (
"fmt"
"github.com/ipfs/go-log/v2"
"log/slog"
"os"
"github.com/ipfs/go-log/v2"
)
// ipfsLogger is an slog logger that implements the IPFS go-log StandardLogger interface.
+2 -1
View File
@@ -2,12 +2,13 @@ package main
import (
"fmt"
"github.com/docker/docker/libnetwork/networkdb"
"log/slog"
"os"
"os/signal"
"syscall"
"time"
"github.com/docker/docker/libnetwork/networkdb"
)
func main() {
+13 -12
View File
@@ -4,6 +4,13 @@ import (
"context"
"flag"
"fmt"
"log/slog"
"net"
"os"
"os/signal"
"syscall"
"time"
"github.com/hashicorp/memberlist"
"github.com/hashicorp/serf/cmd/serf/command/agent"
"github.com/hashicorp/serf/serf"
@@ -11,12 +18,6 @@ import (
badger "github.com/ipfs/go-ds-badger3"
crdt "github.com/ipfs/go-ds-crdt"
"github.com/lmittmann/tint"
"log/slog"
"net"
"os"
"os/signal"
"syscall"
"time"
)
func createSerfAgentConfig(name, bindAddr, rpcAddr, profile string) *agent.Config {
@@ -49,9 +50,9 @@ func createSerfAgent(config *agent.Config) (*agent.Agent, error) {
serfConfig.MemberlistConfig.BindAddr = bindIP
serfConfig.MemberlistConfig.BindPort = bindPort
//serfConfig.MemberlistConfig.AdvertiseAddr = advertiseIP
//serfConfig.MemberlistConfig.AdvertisePort = advertisePort
//serfConfig.MemberlistConfig.SecretKey = encryptKey
// serfConfig.MemberlistConfig.AdvertiseAddr = advertiseIP
// serfConfig.MemberlistConfig.AdvertisePort = advertisePort
// serfConfig.MemberlistConfig.SecretKey = encryptKey
serfConfig.NodeName = config.NodeName
serfConfig.Tags = config.Tags
serfConfig.SnapshotPath = config.SnapshotPath
@@ -129,7 +130,7 @@ func main() {
logger := slog.New(tint.NewHandler(os.Stdout, &tint.Options{
AddSource: true,
Level: slog.LevelDebug,
//Level: slog.LevelInfo,
// Level: slog.LevelInfo,
TimeFormat: time.RFC3339Nano,
}))
slog.SetDefault(logger)
@@ -164,7 +165,7 @@ func main() {
opts := crdt.DefaultOptions()
opts.Logger = newIPFSLogger(logger)
//opts.MultiHeadProcessing = true
// opts.MultiHeadProcessing = true
// TODO: debug why the heads count may grow on the receiving side if the event backlog is huge and the processing
// is slow.
store, err := crdt.New(localStore, ds.NewKey("/"), syncer, broadcaster, opts)
@@ -172,7 +173,7 @@ func main() {
panic(err)
}
//ticker := time.NewTicker(10 * time.Millisecond)
// ticker := time.NewTicker(10 * time.Millisecond)
ticker := time.NewTicker(3 * time.Second)
go func() {
for {
+4 -5
View File
@@ -4,18 +4,19 @@ import (
"context"
"errors"
"fmt"
"log/slog"
"github.com/hashicorp/serf/serf"
"github.com/ipfs/boxo/datastore/dshelp"
dag "github.com/ipfs/boxo/ipld/merkledag"
"github.com/ipfs/go-cid"
ds "github.com/ipfs/go-datastore"
ipld "github.com/ipfs/go-ipld-format"
"log/slog"
)
// Implements the DAGService interface.
// TODO: implement SessionDAGService to optimize node fetching.
// TOOD: persistentSerfDAG?
// TODO: persistentSerfDAG?
type dagSyncer struct {
// Persistent storage for the nodes.
store ds.Datastore
@@ -50,8 +51,7 @@ func (d *dagSyncer) Get(ctx context.Context, cid cid.Cid) (ipld.Node, error) {
}
slog.Debug("Queried node from peers", "cid", cid, "deadline", query.Deadline())
responded := false
for !responded {
for {
select {
case resp, ok := <-query.ResponseCh():
if !ok {
@@ -63,7 +63,6 @@ func (d *dagSyncer) Get(ctx context.Context, cid cid.Cid) (ipld.Node, error) {
continue
}
slog.Debug("Received node from peer", "cid", cid, "peer", resp.From)
responded = true
query.Close()
node, err = nodeFromBytes(resp.Payload)
+4 -3
View File
@@ -6,12 +6,13 @@ import (
"crypto/cipher"
"encoding/hex"
"fmt"
"net/netip"
"time"
"github.com/psviderski/uncloud/internal/machine/network"
"github.com/siderolabs/discovery-api/api/v1alpha1/client/pb"
discovery "github.com/siderolabs/discovery-client/pkg/client"
"go.uber.org/zap"
"net/netip"
"time"
"uncloud/internal/machine/network"
)
const (
+228 -49
View File
@@ -1,16 +1,33 @@
module uncloud
module github.com/psviderski/uncloud
go 1.22.2
toolchain go1.22.5
go 1.25
require (
github.com/BurntSushi/toml v1.4.0
github.com/BurntSushi/toml v1.5.0
github.com/Masterminds/semver v1.5.0
github.com/Masterminds/squirrel v1.5.4
github.com/alecthomas/chroma/v2 v2.20.0
github.com/caddyserver/caddy/v2 v2.8.4
github.com/cenkalti/backoff/v4 v4.3.0
github.com/charmbracelet/bubbles v0.20.0
github.com/charmbracelet/bubbletea v1.3.9
github.com/charmbracelet/huh v0.6.0
github.com/charmbracelet/lipgloss v1.1.0
github.com/compose-spec/compose-go/v2 v2.9.0
github.com/containerd/errdefs v1.0.0
github.com/containerd/platforms v1.0.0-rc.1
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf
github.com/deckarep/golang-set/v2 v2.8.0
github.com/dgraph-io/badger/v3 v3.2103.5
github.com/docker/docker v27.3.0+incompatible
github.com/docker/go-connections v0.5.0
github.com/distribution/reference v0.6.0
github.com/docker/cli v28.5.0+incompatible
github.com/docker/compose/v2 v2.40.0
github.com/docker/docker v28.5.0+incompatible
github.com/docker/go-connections v0.6.0
github.com/docker/go-units v0.5.0
github.com/goccy/go-yaml v1.17.1
github.com/google/go-cmp v0.7.0
github.com/google/go-containerregistry v0.20.2
github.com/hashicorp/memberlist v0.5.1
github.com/hashicorp/serf v0.10.1
github.com/ipfs/boxo v0.21.0
@@ -20,78 +37,143 @@ require (
github.com/ipfs/go-ds-crdt v0.5.2
github.com/ipfs/go-ipld-format v0.6.0
github.com/ipfs/go-log/v2 v2.5.1
github.com/jmoiron/sqlx v1.4.0
github.com/lmittmann/tint v1.0.5
github.com/miekg/dns v1.1.65
github.com/mitchellh/mapstructure v1.5.0
github.com/moby/term v0.5.2
github.com/muesli/termenv v0.16.0
github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.1.1
github.com/psviderski/unregistry v0.3.1
github.com/siderolabs/discovery-api v0.1.4
github.com/siderolabs/discovery-client v0.1.9
github.com/spf13/cobra v1.8.1
github.com/vishvananda/netlink v1.3.0
github.com/siderolabs/grpc-proxy v0.5.1
github.com/spf13/cobra v1.10.1
github.com/stretchr/testify v1.11.1
github.com/vishvananda/netlink v1.3.1
go.uber.org/zap v1.27.0
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba
golang.org/x/crypto v0.27.0
golang.org/x/sync v0.8.0
golang.org/x/sys v0.25.0
golang.org/x/crypto v0.41.0
golang.org/x/net v0.43.0
golang.org/x/sync v0.17.0
golang.org/x/sys v0.36.0
golang.org/x/term v0.34.0
golang.zx2c4.com/wireguard v0.0.0-20231211153847-12269c276173
golang.zx2c4.com/wireguard/wgctrl v0.0.0-20230429144221-925a1e7659e6
google.golang.org/grpc v1.66.0
google.golang.org/protobuf v1.34.2
google.golang.org/genproto/googleapis/rpc v0.0.0-20250528174236-200df99c418a
google.golang.org/grpc v1.74.2
google.golang.org/protobuf v1.36.9
modernc.org/sqlite v1.36.3
)
require (
cloud.google.com/go/kms v1.19.0 // indirect
cloud.google.com/go/longrunning v0.6.0 // indirect
filippo.io/edwards25519 v1.1.0 // indirect
github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 // indirect
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.2.1 // indirect
github.com/Masterminds/semver/v3 v3.4.0 // indirect
github.com/Masterminds/sprig/v3 v3.2.3 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/Microsoft/hcsshim v0.13.0 // indirect
github.com/antlr4-go/antlr/v4 v4.13.0 // indirect
github.com/armon/circbuf v0.0.0-20190214190532-5111143e8da2 // indirect
github.com/armon/go-metrics v0.4.1 // indirect
github.com/armon/go-radix v1.0.0 // indirect
github.com/aryann/difflib v0.0.0-20210328193216-ff5ff6dc229b // indirect
github.com/atotto/clipboard v0.1.4 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bgentry/speakeasy v0.2.0 // indirect
github.com/buger/goterm v1.0.4 // indirect
github.com/caddyserver/certmagic v0.21.4 // indirect
github.com/caddyserver/zerossl v0.1.3 // indirect
github.com/catppuccin/go v0.2.0 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cespare/xxhash v1.1.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/charmbracelet/bubbles v0.20.0 // indirect
github.com/charmbracelet/bubbletea v1.1.1 // indirect
github.com/charmbracelet/lipgloss v0.13.0 // indirect
github.com/charmbracelet/x/ansi v0.3.2 // indirect
github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc // indirect
github.com/charmbracelet/x/ansi v0.10.1 // indirect
github.com/charmbracelet/x/cellbuf v0.0.13-0.20250311204145-2c3ea96c31dd // indirect
github.com/charmbracelet/x/exp/strings v0.0.0-20240919170804-a4978c8e603a // indirect
github.com/charmbracelet/x/term v0.2.0 // indirect
github.com/charmbracelet/x/term v0.2.1 // indirect
github.com/chzyer/readline v1.5.1 // indirect
github.com/containerd/cgroups/v3 v3.0.5 // indirect
github.com/containerd/console v1.0.5 // indirect
github.com/containerd/containerd/api v1.9.0 // indirect
github.com/containerd/containerd/v2 v2.1.4 // indirect
github.com/containerd/continuity v0.4.5 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/containerd/fifo v1.1.0 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/plugin v1.0.0 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.16.3 // indirect
github.com/containerd/ttrpc v1.2.7 // indirect
github.com/containerd/typeurl/v2 v2.2.3 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.7 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.3.0 // indirect
github.com/dgraph-io/badger v1.6.2 // indirect
github.com/dgraph-io/badger/v2 v2.2007.4 // indirect
github.com/dgraph-io/ristretto v0.1.1 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/distribution/distribution/v3 v3.0.0 // indirect
github.com/dlclark/regexp2 v1.11.5 // indirect
github.com/docker/buildx v0.29.1 // indirect
github.com/docker/distribution v2.8.3+incompatible // indirect
github.com/docker/docker-credential-helpers v0.9.3 // indirect
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/docker/go-metrics v0.0.1 // indirect
github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
github.com/fatih/color v1.17.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/fvbommel/sortorder v1.1.0 // indirect
github.com/go-jose/go-jose/v3 v3.0.3 // indirect
github.com/go-kit/kit v0.13.0 // indirect
github.com/go-kit/log v0.2.1 // indirect
github.com/go-logfmt/logfmt v0.6.0 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-sql-driver/mysql v1.8.1 // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/godbus/dbus/v5 v5.1.0 // indirect
github.com/gofrs/flock v0.12.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/glog v1.2.2 // indirect
github.com/golang/glog v1.2.5 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/btree v1.1.2 // indirect
github.com/google/cel-go v0.20.1 // indirect
github.com/google/flatbuffers v24.3.25+incompatible // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/handlers v1.5.2 // indirect
github.com/gorilla/mux v1.8.1 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
github.com/hashicorp/go-immutable-radix/v2 v2.1.0 // indirect
github.com/hashicorp/go-msgpack v0.5.5 // indirect
github.com/hashicorp/go-msgpack/v2 v2.1.2 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-sockaddr v1.0.6 // indirect
github.com/hashicorp/go-syslog v1.0.0 // indirect
github.com/hashicorp/go-version v1.7.0 // indirect
github.com/hashicorp/golang-lru v1.0.2 // indirect
github.com/hashicorp/golang-lru/arc/v2 v2.0.7 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/hashicorp/logutils v1.0.0 // indirect
github.com/hashicorp/mdns v1.0.5 // indirect
github.com/huandu/xstrings v1.5.0 // indirect
github.com/imdario/mergo v0.3.16 // indirect
github.com/in-toto/in-toto-golang v0.9.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/ipfs/bbloom v0.0.4 // indirect
github.com/ipfs/go-block-format v0.2.0 // indirect
@@ -101,36 +183,59 @@ require (
github.com/ipld/go-codec-dagpb v1.6.0 // indirect
github.com/ipld/go-ipld-prime v0.21.0 // indirect
github.com/ishidawataru/sctp v0.0.0-20230406120618-7ff4192f6ff2 // indirect
github.com/jackc/chunkreader/v2 v2.0.1 // indirect
github.com/jackc/pgconn v1.14.3 // indirect
github.com/jackc/pgio v1.0.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgproto3/v2 v2.3.3 // indirect
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
github.com/jackc/pgtype v1.14.0 // indirect
github.com/jackc/pgx/v4 v4.18.3 // indirect
github.com/jbenet/goprocess v0.1.4 // indirect
github.com/josharian/native v1.1.0 // indirect
github.com/klauspost/compress v1.17.9 // indirect
github.com/klauspost/cpuid/v2 v2.2.8 // indirect
github.com/klauspost/compress v1.18.0 // indirect
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
github.com/lann/builder v0.0.0-20180802200727-47ae307949d0 // indirect
github.com/lann/ps v0.0.0-20150810152359-62de8c46ede0 // indirect
github.com/libdns/libdns v0.2.2 // indirect
github.com/libp2p/go-buffer-pool v0.1.0 // indirect
github.com/libp2p/go-libp2p v0.35.4 // indirect
github.com/libp2p/go-libp2p-pubsub v0.11.0 // indirect
github.com/libp2p/go-msgio v0.3.0 // indirect
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
github.com/manifoldco/promptui v0.9.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-localereader v0.0.1 // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/mattn/go-shellwords v1.0.12 // indirect
github.com/mdlayher/genetlink v1.3.2 // indirect
github.com/mdlayher/netlink v1.7.2 // indirect
github.com/mdlayher/socket v0.5.1 // indirect
github.com/miekg/dns v1.1.62 // indirect
github.com/mgutz/ansi v0.0.0-20200706080929-d51e80ef957d // indirect
github.com/mholt/acmez/v2 v2.0.3 // indirect
github.com/minio/sha256-simd v1.0.1 // indirect
github.com/mitchellh/cli v1.1.5 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-ps v1.0.0 // indirect
github.com/mitchellh/hashstructure/v2 v2.0.2 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/moby/buildkit v0.25.0 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/term v0.5.0 // indirect
github.com/moby/go-archive v0.1.0 // indirect
github.com/moby/locker v1.0.1 // indirect
github.com/moby/patternmatcher v0.6.0 // indirect
github.com/moby/sys/atomicwriter v0.1.0 // indirect
github.com/moby/sys/mountinfo v0.7.2 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect
github.com/moby/sys/signal v0.7.1 // indirect
github.com/moby/sys/user v0.4.0 // indirect
github.com/moby/sys/userns v0.1.0 // indirect
github.com/morikuni/aec v1.0.0 // indirect
github.com/mr-tron/base58 v1.2.0 // indirect
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/termenv v0.15.3-0.20240618155329-98d742f6907a // indirect
github.com/multiformats/go-base32 v0.1.0 // indirect
github.com/multiformats/go-base36 v0.2.0 // indirect
github.com/multiformats/go-multiaddr v0.13.0 // indirect
@@ -140,37 +245,111 @@ require (
github.com/multiformats/go-multihash v0.2.3 // indirect
github.com/multiformats/go-multistream v0.5.0 // indirect
github.com/multiformats/go-varint v0.0.7 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/onsi/ginkgo/v2 v2.22.0 // indirect
github.com/opencontainers/runtime-spec v1.2.1 // indirect
github.com/opencontainers/selinux v1.12.0 // indirect
github.com/pelletier/go-toml v1.9.5 // indirect
github.com/pires/go-proxyproto v0.7.0 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/polydawn/refmt v0.89.0 // indirect
github.com/posener/complete v1.2.3 // indirect
github.com/prometheus/client_golang v1.22.0 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.62.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/quic-go/qpack v0.5.1 // indirect
github.com/quic-go/quic-go v0.48.2 // indirect
github.com/redis/go-redis/extra/rediscmd/v9 v9.0.5 // indirect
github.com/redis/go-redis/extra/redisotel/v9 v9.0.5 // indirect
github.com/redis/go-redis/v9 v9.7.3 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/rs/xid v1.5.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/santhosh-tekuri/jsonschema/v6 v6.0.1 // indirect
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 // indirect
github.com/secure-systems-lab/go-securesystemslib v0.6.0 // indirect
github.com/shibumi/go-pathspec v1.3.0 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
github.com/shurcooL/sanitized_anchor_name v1.0.0 // indirect
github.com/siderolabs/gen v0.4.8 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/slackhq/nebula v1.6.1 // indirect
github.com/smallstep/certificates v0.26.1 // indirect
github.com/smallstep/nosql v0.6.1 // indirect
github.com/smallstep/pkcs7 v0.1.1 // indirect
github.com/smallstep/scep v0.0.0-20231024192529-aee96d7ad34d // indirect
github.com/smallstep/truststore v0.13.0 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect
github.com/spf13/cast v1.7.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/vishvananda/netns v0.0.4 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/stoewer/go-strcase v1.2.0 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/tailscale/tscert v0.0.0-20240517230440-bbccfbf48933 // indirect
github.com/tonistiigi/dchapes-mode v0.0.0-20250318174251-73d941a28323 // indirect
github.com/tonistiigi/fsutil v0.0.0-20250605211040-586307ad452f // indirect
github.com/tonistiigi/go-csvvalue v0.0.0-20240814133006-030d3b2625d0 // indirect
github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea // indirect
github.com/tonistiigi/vt100 v0.0.0-20240514184818-90bafcd6abab // indirect
github.com/urfave/cli v1.22.17 // indirect
github.com/vbatts/tar-split v0.12.1 // indirect
github.com/vishvananda/netns v0.0.5 // indirect
github.com/xhit/go-str2duration/v2 v2.1.0 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
github.com/zeebo/blake3 v0.2.4 // indirect
go.etcd.io/bbolt v1.4.3 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.52.0 // indirect
go.opentelemetry.io/otel v1.30.0 // indirect
go.opentelemetry.io/otel/metric v1.30.0 // indirect
go.opentelemetry.io/otel/trace v1.30.0 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/bridges/prometheus v0.57.0 // indirect
go.opentelemetry.io/contrib/exporters/autoexport v0.57.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.60.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 // indirect
go.opentelemetry.io/otel v1.36.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc v0.8.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp v0.8.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.35.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.35.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.35.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.35.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.35.0 // indirect
go.opentelemetry.io/otel/exporters/prometheus v0.54.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdoutlog v0.8.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.32.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.32.0 // indirect
go.opentelemetry.io/otel/log v0.8.0 // indirect
go.opentelemetry.io/otel/metric v1.36.0 // indirect
go.opentelemetry.io/otel/sdk v1.36.0 // indirect
go.opentelemetry.io/otel/sdk/log v0.8.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.36.0 // indirect
go.opentelemetry.io/otel/trace v1.36.0 // indirect
go.opentelemetry.io/proto/otlp v1.5.0 // indirect
go.step.sm/cli-utils v0.9.0 // indirect
go.step.sm/crypto v0.45.0 // indirect
go.step.sm/linkedca v0.20.1 // indirect
go.uber.org/automaxprocs v1.5.3 // indirect
go.uber.org/mock v0.6.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect
golang.org/x/mod v0.21.0 // indirect
golang.org/x/net v0.29.0 // indirect
golang.org/x/text v0.18.0 // indirect
golang.org/x/time v0.0.0-20220210224613-90d013bbcef8 // indirect
golang.org/x/tools v0.25.0 // indirect
go.uber.org/zap/exp v0.2.0 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto/x509roots/fallback v0.0.0-20240507223354-67b13616a595 // indirect
golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0 // indirect
golang.org/x/mod v0.27.0 // indirect
golang.org/x/text v0.28.0 // indirect
golang.org/x/time v0.11.0 // indirect
golang.org/x/tools v0.36.0 // indirect
golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240827150818-7e3bb234dfed // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240827150818-7e3bb234dfed // indirect
gotest.tools/v3 v3.5.1 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250528174236-200df99c418a // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
gvisor.dev/gvisor v0.0.0-20230927004350-cbd86285d259 // indirect
howett.net/plist v1.0.0 // indirect
lukechampine.com/blake3 v1.3.0 // indirect
modernc.org/libc v1.61.13 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.8.2 // indirect
)
+729 -117
View File
File diff suppressed because it is too large Load Diff
+177
View File
@@ -0,0 +1,177 @@
package cli
import (
"context"
"encoding/base64"
"encoding/json"
"fmt"
"os"
composetypes "github.com/compose-spec/compose-go/v2/types"
"github.com/distribution/reference"
"github.com/docker/cli/cli/config"
"github.com/docker/docker/api/types/build"
"github.com/docker/docker/api/types/image"
dockerclient "github.com/docker/docker/client"
"github.com/docker/docker/pkg/archive"
"github.com/docker/docker/pkg/jsonmessage"
"github.com/docker/docker/registry"
"github.com/moby/term"
)
type BuildOptions struct {
Files []string
Profiles []string
Services []string
Push bool
NoCache bool
}
// GetServicesThatNeedBuild returns a map of services that require building
func GetServicesThatNeedBuild(project *composetypes.Project) map[string]composetypes.ServiceConfig {
servicesToBuild := make(map[string]composetypes.ServiceConfig, len(project.Services))
for serviceName, service := range project.Services {
if service.Build == nil {
continue
}
servicesToBuild[serviceName] = service
}
return servicesToBuild
}
// BuildServices builds the services defined in the provided map.
func BuildServices(ctx context.Context, servicesToBuild map[string]composetypes.ServiceConfig, opts BuildOptions) error {
fmt.Println("Building services...")
// Init docker client (can be local or remote, depending on DOCKER_HOST environment variable)
dockerCli, err := dockerclient.NewClientWithOpts(dockerclient.FromEnv, dockerclient.WithAPIVersionNegotiation())
if err != nil {
return err
}
defer dockerCli.Close()
serviceImages := make(map[string]string, len(servicesToBuild))
// Build the services using the local docker client and compose libraries
for _, service := range servicesToBuild {
fmt.Printf("Building service: %s\n", service.Name)
imageName, err := buildSingleService(ctx, dockerCli, service, opts)
if err != nil {
return fmt.Errorf("build service %s: %w", service.Name, err)
}
serviceImages[service.Name] = imageName
}
fmt.Printf("Service images are built.\n")
if opts.Push {
err = pushServiceImages(ctx, dockerCli, serviceImages)
}
return err
}
// buildSingleService builds a single service using the Docker client and Compose libraries.
func buildSingleService(ctx context.Context, dockerCli *dockerclient.Client, service composetypes.ServiceConfig, opts BuildOptions) (string, error) {
if service.Build == nil {
return "", fmt.Errorf("service %s has no build configuration", service.Name)
}
if service.Image == "" {
return "", fmt.Errorf("service %s has no image specified; building services without image is not supported yet",
service.Name)
}
buildContextPath := service.Build.Context
imageName := service.Image
// Create a tar archive of the build context
buildContext, err := archive.TarWithOptions(buildContextPath, &archive.TarOptions{})
if err != nil {
return "", fmt.Errorf("failed to create build context for service %s: %w", service.Name, err)
}
buildOptions := build.ImageBuildOptions{
// TODO: Support Dockerfiles outside the build context
// See https://github.com/docker/compose/blob/cf89fd1aa1328d5af77658ccc5a1e1b29981ae80/pkg/compose/build_classic.go#L92
Dockerfile: service.Build.Dockerfile,
Tags: []string{imageName},
Remove: true, // Remove intermediate containers
NoCache: opts.NoCache,
}
buildResponse, err := dockerCli.ImageBuild(ctx, buildContext, buildOptions)
if err != nil {
return "", fmt.Errorf("failed to build image for service %s: %w", service.Name, err)
}
defer buildResponse.Body.Close()
// Display the build response
fd, isTerminal := term.GetFdInfo(os.Stdout)
if err := jsonmessage.DisplayJSONMessagesStream(buildResponse.Body, os.Stdout, fd, isTerminal, nil); err != nil {
return "", fmt.Errorf("failed to display build response for service %s: %w", service.Name, err)
}
return imageName, nil
}
// pushSingleServiceImage pushes a single service image.
func pushSingleServiceImage(ctx context.Context, dockerCli *dockerclient.Client, serviceName string, imageName string) error {
ref, err := reference.ParseNormalizedNamed(imageName)
if err != nil {
return err
}
repoInfo, err := registry.ParseRepositoryInfo(ref)
if err != nil {
return err
}
registryKey := repoInfo.Index.Name
if repoInfo.Index.Official {
registryKey = registry.IndexServer
}
// Load the Docker config file with auth details, if available
configFile := config.LoadDefaultConfigFile(os.Stderr)
authConfig, err := configFile.GetAuthConfig(registryKey)
if err != nil {
return err
}
authJSON, err := json.Marshal(authConfig)
if err != nil {
return fmt.Errorf("failed to marshal auth config for registry %s: %w", registryKey, err)
}
authStr := base64.URLEncoding.EncodeToString(authJSON)
pushOptions := image.PushOptions{
RegistryAuth: authStr,
}
pushResponse, err := dockerCli.ImagePush(ctx, imageName, pushOptions)
if err != nil {
return fmt.Errorf("failed to push image %s: %w", imageName, err)
}
defer pushResponse.Close()
fmt.Printf("Pushing image %s for service %s...\n", imageName, serviceName)
fd, isTerminal := term.GetFdInfo(os.Stdout)
if err := jsonmessage.DisplayJSONMessagesStream(pushResponse, os.Stdout, fd, isTerminal, nil); err != nil {
return fmt.Errorf("failed to display push response for image %s: %w", imageName, err)
}
fmt.Printf("Image %s pushed successfully.\n", imageName)
return nil
}
// pushServiceImages pushes all built service images to the registry.
func pushServiceImages(ctx context.Context, dockerCli *dockerclient.Client, serviceImages map[string]string) error {
fmt.Printf("Pushing images...\n")
for serviceName, imageName := range serviceImages {
if err := pushSingleServiceImage(ctx, dockerCli, serviceName, imageName); err != nil {
return fmt.Errorf("push image for service %s: %w", serviceName, err)
}
}
return nil
}
+341 -252
View File
@@ -2,264 +2,346 @@ package cli
import (
"context"
"errors"
"fmt"
"github.com/charmbracelet/huh"
"google.golang.org/protobuf/types/known/emptypb"
"net/netip"
"os"
"strings"
"text/tabwriter"
"uncloud/internal/cli/client"
"uncloud/internal/cli/client/connector"
"uncloud/internal/cli/config"
"uncloud/internal/machine"
"uncloud/internal/machine/api/pb"
"uncloud/internal/secret"
"uncloud/internal/sshexec"
"slices"
"github.com/docker/cli/cli/streams"
"github.com/psviderski/uncloud/internal/cli/config"
"github.com/psviderski/uncloud/internal/machine"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/sshexec"
"github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client"
"github.com/psviderski/uncloud/pkg/client/connector"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb"
)
const defaultClusterName = "default"
var (
ErrNotFound = errors.New("not found")
const (
// DefaultSSHKeyPath is the fallback location for the SSH private key when provisioning remote machines.
// Used when no key is explicitly provided and SSH agent authentication fails.
DefaultSSHKeyPath = "~/.ssh/id_ed25519"
DefaultContextName = "default"
)
type CLI struct {
config *config.Config
Config *config.Config
conn *config.MachineConnection
}
func New(configPath string) (*CLI, error) {
// New creates a new CLI instance with the given config path or remote machine connection.
// If the connection is provided, the config is ignored for all operations which is useful for interacting with
// a cluster without creating a config.
func New(configPath string, conn *config.MachineConnection) (*CLI, error) {
if conn != nil {
return &CLI{conn: conn}, nil
}
cfg, err := config.NewFromFile(configPath)
if err != nil {
return nil, fmt.Errorf("read Uncloud config: %w", err)
}
return &CLI{
config: cfg,
Config: cfg,
}, nil
}
func (cli *CLI) CreateCluster(name string, userPrivateKey secret.Secret) error {
if _, ok := cli.config.Clusters[name]; ok {
return fmt.Errorf("cluster %q already exists", name)
func (cli *CLI) CreateContext(name string) error {
if _, ok := cli.Config.Contexts[name]; ok {
return fmt.Errorf("context '%s' already exists", name)
}
if userPrivateKey == nil {
user, err := client.NewUser(nil)
if err != nil {
return fmt.Errorf("generate user: %w", err)
}
userPrivateKey = user.PrivateKey()
cli.Config.Contexts[name] = &config.Context{
Name: name,
}
cli.config.Clusters[name] = &config.Cluster{
Name: name,
UserPrivateKey: userPrivateKey,
}
return cli.config.Save()
return cli.Config.Save()
}
func (cli *CLI) CreateDefaultCluster() error {
if err := cli.CreateCluster(defaultClusterName, nil); err != nil {
return err
func (cli *CLI) SetCurrentContext(name string) error {
if _, ok := cli.Config.Contexts[name]; !ok {
return api.ErrNotFound
}
return cli.SetCurrentCluster(defaultClusterName)
cli.Config.CurrentContext = name
return cli.Config.Save()
}
func (cli *CLI) SetCurrentCluster(name string) error {
if _, ok := cli.config.Clusters[name]; !ok {
return ErrNotFound
}
cli.config.CurrentCluster = name
return cli.config.Save()
// ConnectCluster connects to a cluster using the given context name or the current context if not specified.
// If the CLI was initialised with a machine connection, the config is ignored and the connection is used instead.
func (cli *CLI) ConnectCluster(ctx context.Context, contextName string) (*client.Client, error) {
return cli.ConnectClusterWithOptions(ctx, contextName, ConnectOptions{
// Default to showing progress for CLI usage.
ShowProgress: true,
})
}
func (cli *CLI) ConnectCluster(ctx context.Context, clusterName string) (*client.Client, error) {
if len(cli.config.Clusters) == 0 {
return nil, errors.New(
"no clusters found in the Uncloud config. " +
"Please initialise a cluster with `uncloud machine init` first",
// ConnectClusterWithOptions connects to a cluster using the given context name and options.
// If the CLI was initialised with a machine connection, the config is ignored and the connection is used instead.
// Options are useful when using the CLI as a library where you may want to disable visual feedback.
func (cli *CLI) ConnectClusterWithOptions(ctx context.Context, contextName string, opts ConnectOptions) (*client.Client, error) {
if cli.conn != nil {
return ConnectCluster(ctx, *cli.conn, opts)
}
if len(cli.Config.Contexts) == 0 {
return nil, fmt.Errorf(
"no cluster contexts found in the Uncloud config (%s). "+
"Please initialise a cluster with 'uncloud machine init' first",
cli.Config.Path(),
)
}
if clusterName == "" {
if contextName == "" {
// If the cluster is not specified, use the current cluster if set.
if cli.config.CurrentCluster == "" {
return nil, errors.New(
"the current cluster is not set in the Uncloud config. " +
"Please specify a cluster with the --cluster flag or set current_cluster in the config",
)
}
if _, ok := cli.config.Clusters[cli.config.CurrentCluster]; !ok {
if cli.Config.CurrentContext == "" {
return nil, fmt.Errorf(
"current cluster %q not found in the config. "+
"Please specify a cluster with the --cluster flag or update current_cluster in the config",
cli.config.CurrentCluster,
"the current cluster context is not set in the Uncloud config (%s). "+
"Please specify the context with the '--context' flag or set 'current_context' in the config",
cli.Config.Path(),
)
}
clusterName = cli.config.CurrentCluster
if _, ok := cli.Config.Contexts[cli.Config.CurrentContext]; !ok {
return nil, fmt.Errorf(
"current cluster context '%s' not found in the Uncloud config (%s). "+
"Please specify the context with the '--context' flag or update 'current_context' in the config",
cli.Config.CurrentContext,
cli.Config.Path(),
)
}
contextName = cli.Config.CurrentContext
}
cfg, ok := cli.config.Clusters[clusterName]
cfg, ok := cli.Config.Contexts[contextName]
if !ok {
return nil, fmt.Errorf("cluster %q not found in the config", clusterName)
return nil, fmt.Errorf("cluster context '%s' not found in the Uncloud config (%s)",
contextName, cli.Config.Path())
}
if len(cfg.Connections) == 0 {
return nil, fmt.Errorf("no connection configurations found for cluster %q in the config", clusterName)
}
// TODO: iterate over all connections and try to connect to the cluster using the first successful connection.
conn := cfg.Connections[0]
user, host, port, err := conn.SSH.Parse()
if err != nil {
return nil, fmt.Errorf("parse SSH connection %q: %w", conn.SSH, err)
}
sshConfig := &connector.SSHConnectorConfig{
User: user,
Host: host,
Port: port,
}
return client.New(ctx, connector.NewSSHConnector(sshConfig))
}
func (cli *CLI) InitCluster(
ctx context.Context, remoteMachine *RemoteMachine, clusterName, machineName string, netPrefix netip.Prefix,
) error {
if remoteMachine != nil {
return cli.initRemoteMachine(ctx, *remoteMachine, clusterName, machineName, netPrefix)
}
// TODO: implement local machine initialisation
return fmt.Errorf("local machine initialisation is not implemented yet")
}
func (cli *CLI) initRemoteMachine(
ctx context.Context, remoteMachine RemoteMachine, clusterName, machineName string, netPrefix netip.Prefix,
) error {
if clusterName == "" {
clusterName = defaultClusterName
}
if _, ok := cli.config.Clusters[clusterName]; ok {
return fmt.Errorf("cluster %q already exists", clusterName)
}
user, err := client.NewUser(nil)
if err != nil {
return fmt.Errorf("generate cluster user: %w", err)
}
// Provision the remote machine by installing the Uncloud daemon and dependencies over SSH.
sshClient, err := sshexec.Connect(remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath)
if err != nil {
return fmt.Errorf(
"SSH login to remote machine %s: %w",
config.NewSSHDestination(remoteMachine.User, remoteMachine.Host, remoteMachine.Port), err,
return nil, fmt.Errorf(
"no connection configurations found for cluster context '%s' in the Uncloud config (%s)",
contextName, cli.Config.Path(),
)
}
exec := sshexec.NewRemote(sshClient)
// Install and run the Uncloud daemon and dependencies on the remote machine.
if err = provisionMachine(ctx, exec); err != nil {
return fmt.Errorf("provision machine: %w", err)
// Try each connection in order until one succeeds.
var lastErr error
for _, conn := range cfg.Connections {
c, err := ConnectCluster(ctx, conn, opts)
if err == nil {
return c, nil
}
lastErr = err
}
// Create a machine API client over the SSH connection to the remote machine.
machineClient, err := client.New(ctx, connector.NewSSHConnectorFromClient(sshClient))
if err != nil {
return fmt.Errorf("connect to remote machine: %w", err)
return nil, fmt.Errorf("failed to connect to cluster context '%s': "+
"all connections (%d) in the Uncloud config (%s) failed; last error: %w",
contextName, len(cfg.Connections), cli.Config.Path(), lastErr)
}
type InitClusterOptions struct {
Context string
MachineName string
Network netip.Prefix
PublicIP *netip.Addr
RemoteMachine *RemoteMachine
SkipInstall bool
Version string
}
// InitCluster initialises a new cluster on a remote machine and returns a client to interact with the cluster.
// The client should be closed after use by the caller.
func (cli *CLI) InitCluster(ctx context.Context, opts InitClusterOptions) (*client.Client, error) {
if opts.RemoteMachine != nil {
return cli.initRemoteMachine(ctx, opts)
}
defer machineClient.Close()
// TODO: implement local machine initialisation
return nil, fmt.Errorf("local machine initialisation is not implemented yet")
}
func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions) (*client.Client, error) {
contextName, err := cli.newContextName(opts.Context)
if err != nil {
return nil, err
}
machineClient, err := provisionOrConnectRemoteMachine(ctx, opts.RemoteMachine, opts.SkipInstall, opts.Version)
if err != nil {
return nil, err
}
// Ensure machineClient is closed on error.
defer func() {
if err != nil {
machineClient.Close()
}
}()
// Check if the machine is already initialised as a cluster member and prompt the user to reset it first.
minfo, err := machineClient.Inspect(ctx, &emptypb.Empty{})
if err != nil {
return fmt.Errorf("inspect machine: %w", err)
return nil, fmt.Errorf("inspect machine: %w", err)
}
if minfo.Id != "" {
if err = cli.promptResetMachine(); err != nil {
return err
if err = promptResetMachine(ctx, machineClient.MachineClient); err != nil {
return nil, err
}
}
// Check machine meets all necessary system requirements before proceeding.
checkResp, err := machineClient.CheckPrerequisites(ctx, &emptypb.Empty{})
// TODO(lhf): remove Unimplemented check when v0.9.0 is released.
if err != nil {
if status.Convert(err).Code() != codes.Unimplemented {
return nil, fmt.Errorf("check machine prerequisites: %w", err)
}
} else if !checkResp.Satisfied {
return nil, fmt.Errorf("machine prerequisites not satisfied: %s", checkResp.Error)
}
req := &pb.InitClusterRequest{
MachineName: machineName,
Network: pb.NewIPPrefix(netPrefix),
User: &pb.User{
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(user.ManagementIP()),
PublicKey: user.PublicKey(),
},
},
MachineName: opts.MachineName,
Network: pb.NewIPPrefix(opts.Network),
}
resp, err := machineClient.InitCluster(ctx, req)
if err != nil {
return fmt.Errorf("init cluster: %w", err)
}
fmt.Printf("Cluster %q initialised with machine %q\n", clusterName, resp.Machine.Name)
if err = cli.CreateCluster(clusterName, user.PrivateKey()); err != nil {
return fmt.Errorf("save cluster to config: %w", err)
}
// Set the current cluster to the just created one if it is the only cluster in the config.
if len(cli.config.Clusters) == 1 {
if err = cli.SetCurrentCluster(clusterName); err != nil {
return fmt.Errorf("set current cluster: %w", err)
if opts.PublicIP != nil {
if opts.PublicIP.IsValid() {
req.PublicIpConfig = &pb.InitClusterRequest_PublicIp{PublicIp: pb.NewIP(*opts.PublicIP)}
} else {
// Invalid or in other words zero IP means to automatically detect the public IP.
req.PublicIpConfig = &pb.InitClusterRequest_PublicIpAuto{PublicIpAuto: true}
}
}
// Save the machine's SSH connection details in the cluster config.
resp, err := machineClient.InitCluster(ctx, req)
if err != nil {
return nil, fmt.Errorf("init cluster: %w", err)
}
fmt.Printf("Cluster initialised with machine '%s' and saved as context '%s' in your local config (%s)\n",
resp.Machine.Name, contextName, cli.Config.Path())
if err = cli.CreateContext(contextName); err != nil {
return nil, fmt.Errorf("save cluster context to config: %w", err)
}
if err = cli.SetCurrentContext(contextName); err != nil {
return nil, fmt.Errorf("set current cluster context: %w", err)
}
fmt.Printf("Current cluster context is now '%s'.\n", contextName)
// Save the machine's SSH connection details in the context config.
connCfg := config.MachineConnection{
SSH: config.NewSSHDestination(remoteMachine.User, remoteMachine.Host, remoteMachine.Port),
SSH: config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port),
SSHKeyFile: opts.RemoteMachine.KeyPath,
}
cli.config.Clusters[clusterName].Connections = append(cli.config.Clusters[clusterName].Connections, connCfg)
if err = cli.config.Save(); err != nil {
return fmt.Errorf("save config: %w", err)
cli.Config.Contexts[contextName].Connections = append(cli.Config.Contexts[contextName].Connections, connCfg)
if err = cli.Config.Save(); err != nil {
return nil, fmt.Errorf("save config: %w", err)
}
return nil
return machineClient, nil
}
func (cli *CLI) AddMachine(ctx context.Context, remoteMachine RemoteMachine, clusterName, machineName string) error {
c, err := cli.ConnectCluster(ctx, clusterName)
// newContextName returns a unique name for a new cluster context. If the provided name is not DefaultContextName,
// and it's already taken, an error is returned. If the name is not provided or is DefaultContextName, the first
// available name "default[-N]" is returned.
func (cli *CLI) newContextName(name string) (string, error) {
if name == "" {
name = DefaultContextName
}
if _, exists := cli.Config.Contexts[name]; !exists {
return name, nil
}
// If non-default context already exists, error out.
if name != DefaultContextName {
return "", fmt.Errorf("cluster context '%s' already exists", name)
}
// The default context already exists, generate a numbered suffix to make it unique.
for i := 1; ; i++ {
name = fmt.Sprintf("%s-%d", DefaultContextName, i)
if _, exists := cli.Config.Contexts[name]; !exists {
return name, nil
}
}
}
type AddMachineOptions struct {
Context string
MachineName string
PublicIP *netip.Addr
RemoteMachine *RemoteMachine
SkipInstall bool
Version string
}
// AddMachine provisions a remote machine and adds it to the cluster. It returns a cluster client and a machine client.
// The cluster client is connected to the existing machine in the cluster. It was used to add the new machine to the
// cluster. The machine client is connected to the new machine and can be used to interact with it.
// Both client should be closed after use by the caller.
func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client.Client, *client.Client, error) {
contextName := opts.Context
if contextName == "" {
contextName = cli.Config.CurrentContext
}
c, err := cli.ConnectCluster(ctx, contextName)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
return nil, nil, fmt.Errorf("connect to cluster (context '%s'): %w", contextName, err)
}
defer func() {
_ = c.Close()
if err != nil {
c.Close()
}
}()
// Provision the remote machine by installing the Uncloud daemon and dependencies over SSH.
sshClient, err := sshexec.Connect(remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath)
machineClient, err := provisionOrConnectRemoteMachine(ctx, opts.RemoteMachine, opts.SkipInstall, opts.Version)
if err != nil {
return fmt.Errorf(
"SSH login to remote machine %s: %w",
config.NewSSHDestination(remoteMachine.User, remoteMachine.Host, remoteMachine.Port), err,
)
return nil, nil, err
}
exec := sshexec.NewRemote(sshClient)
// Install and run the Uncloud daemon and dependencies on the remote machine.
if err = provisionMachine(ctx, exec); err != nil {
return fmt.Errorf("provision machine: %w", err)
}
// Create a machine API client over the SSH connection to the remote machine.
machineClient, err := client.New(ctx, connector.NewSSHConnectorFromClient(sshClient))
if err != nil {
return fmt.Errorf("connect to remote machine: %w", err)
}
defer machineClient.Close()
defer func() {
if err != nil {
machineClient.Close()
}
}()
// Check if the machine is already initialised as a cluster member and prompt the user to reset it first.
minfo, err := machineClient.Inspect(ctx, &emptypb.Empty{})
if err != nil {
return fmt.Errorf("inspect machine: %w", err)
return nil, nil, fmt.Errorf("inspect machine: %w", err)
}
if minfo.Id != "" {
if err = cli.promptResetMachine(); err != nil {
return err
// Check if the machine is already a member of this cluster.
machines, err := c.ListMachines(ctx, nil)
if err != nil {
return nil, nil, fmt.Errorf("list cluster machines: %w", err)
}
if slices.ContainsFunc(machines, func(m *pb.MachineMember) bool {
return m.Machine.Id == minfo.Id
}) {
return nil, nil, fmt.Errorf("machine is already a member of this cluster (%s)", minfo.Name)
}
if err = promptResetMachine(ctx, machineClient.MachineClient); err != nil {
return nil, nil, err
}
}
// Check machine meets all necessary system requirements before proceeding.
checkResp, err := machineClient.CheckPrerequisites(ctx, &emptypb.Empty{})
// TODO(lhf): remove Unimplemented check when v0.9.0 is released.
if err != nil {
if status.Convert(err).Code() != codes.Unimplemented {
return nil, nil, fmt.Errorf("check machine prerequisites: %w", err)
}
} else if !checkResp.Satisfied {
return nil, nil, fmt.Errorf("machine prerequisites not satisfied: %s", checkResp.Error)
}
tokenResp, err := machineClient.Token(ctx, &emptypb.Empty{})
if err != nil {
return fmt.Errorf("get remote machine token: %w", err)
return nil, nil, fmt.Errorf("get remote machine token: %w", err)
}
token, err := machine.ParseToken(tokenResp.Token)
if err != nil {
return fmt.Errorf("parse remote machine token: %w", err)
return nil, nil, fmt.Errorf("parse remote machine token: %w", err)
}
// Register the machine in the cluster using its public key and endpoints from the token.
@@ -268,26 +350,35 @@ func (cli *CLI) AddMachine(ctx context.Context, remoteMachine RemoteMachine, clu
endpoints[i] = pb.NewIPPort(addrPort)
}
addReq := &pb.AddMachineRequest{
Name: machineName,
Name: opts.MachineName,
Network: &pb.NetworkConfig{
Endpoints: endpoints,
PublicKey: token.PublicKey,
},
}
addResp, err := c.AddMachine(ctx, addReq)
if err != nil {
return fmt.Errorf("add machine to cluster: %w", err)
if opts.PublicIP != nil {
if opts.PublicIP.IsValid() {
addReq.PublicIp = pb.NewIP(*opts.PublicIP)
} else if token.PublicIP.IsValid() {
// Invalid or in other words zero IP means to use an automatically detected public IP from the token.
addReq.PublicIp = pb.NewIP(token.PublicIP)
}
}
// List other machines in the cluster to include them in the join request.
listResp, err := c.ListMachines(ctx, &emptypb.Empty{})
addResp, err := c.AddMachine(ctx, addReq)
if err != nil {
return fmt.Errorf("list cluster machines: %w", err)
return nil, nil, fmt.Errorf("add machine to cluster (context '%s'): %w", contextName, err)
}
otherMachines := make([]*pb.MachineInfo, 0, len(listResp.Machines)-1)
for _, m := range listResp.Machines {
if m.Id != addResp.Machine.Id {
otherMachines = append(otherMachines, m)
// Get the most up-to-date list of other machines in the cluster to include them in the join request.
machines, err := c.ListMachines(ctx, nil)
if err != nil {
return nil, nil, fmt.Errorf("list cluster machines: %w", err)
}
otherMachines := make([]*pb.MachineInfo, 0, len(machines)-1)
for _, m := range machines {
if m.Machine.Id != addResp.Machine.Id {
otherMachines = append(otherMachines, m.Machine)
}
}
@@ -297,85 +388,83 @@ func (cli *CLI) AddMachine(ctx context.Context, remoteMachine RemoteMachine, clu
OtherMachines: otherMachines,
}
if _, err = machineClient.JoinCluster(ctx, joinReq); err != nil {
return fmt.Errorf("join cluster: %w", err)
return nil, nil, fmt.Errorf("join cluster: %w", err)
}
fmt.Printf("Machine %q added to cluster\n", addResp.Machine.Name)
// TODO: fix empty context name when using the current context (contextName == "").
fmt.Printf("Machine '%s' added to the cluster (context '%s').\n", addResp.Machine.Name, contextName)
// Save the machine's SSH connection details in the cluster config.
// Save the machine's SSH connection details in the context config.
connCfg := config.MachineConnection{
SSH: config.NewSSHDestination(remoteMachine.User, remoteMachine.Host, remoteMachine.Port),
SSH: config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port),
SSHKeyFile: opts.RemoteMachine.KeyPath,
}
if clusterName == "" {
clusterName = cli.config.CurrentCluster
if contextName == "" {
contextName = cli.Config.CurrentContext
}
cli.config.Clusters[clusterName].Connections = append(cli.config.Clusters[clusterName].Connections, connCfg)
if err = cli.config.Save(); err != nil {
return fmt.Errorf("save config: %w", err)
cli.Config.Contexts[contextName].Connections = append(cli.Config.Contexts[contextName].Connections, connCfg)
if err = cli.Config.Save(); err != nil {
return nil, nil, fmt.Errorf("save config: %w", err)
}
return nil
return c, machineClient, nil
}
func (cli *CLI) promptResetMachine() error {
var confirm bool
form := huh.NewForm(
huh.NewGroup(
huh.NewConfirm().
Title(
"The remote machine is already initialised as a cluster member. Do you want to reset it first?",
).
Affirmative("Yes!").
Negative("No").
Value(&confirm),
),
)
if err := form.Run(); err != nil {
return fmt.Errorf("prompt user to confirm: %w", err)
// provisionOrConnectRemoteMachine installs the Uncloud daemon and dependencies on the remote machine over SSH and
// returns a machine API client to interact with the machine. The client should be closed after use by the caller.
// The version parameter specifies the version of the Uncloud daemon to install. If empty, the latest version is used.
// If skipInstall is true, the installation step is skipped, and it is assumed that the Uncloud daemon and dependencies
// are already installed and running.
// The remoteMachine.SSHKeyPath could be updated to the default SSH key path if it is not set and the SSH agent
// authentication fails.
func provisionOrConnectRemoteMachine(
ctx context.Context, remoteMachine *RemoteMachine, skipInstall bool, version string,
) (*client.Client, error) {
sshClient, err := sshexec.Connect(remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath)
// If the SSH connection using SSH agent fails and no key path is provided, try to use the default SSH key.
if err != nil && remoteMachine.KeyPath == "" {
remoteMachine.KeyPath = DefaultSSHKeyPath
sshClient, err = sshexec.Connect(
remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath,
)
}
if !confirm {
return fmt.Errorf("remote machine is already initialised as a cluster member")
}
// TODO: implement resetting the remote machine.
return fmt.Errorf("resetting the remote machine is not implemented yet")
}
func (cli *CLI) ListMachines(ctx context.Context, clusterName string) error {
c, err := cli.ConnectCluster(ctx, clusterName)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer func() {
_ = c.Close()
}()
listResp, err := c.ListMachines(ctx, &emptypb.Empty{})
if err != nil {
return fmt.Errorf("list machines: %w", err)
return nil, fmt.Errorf(
"SSH login to remote machine %s: %w",
config.NewSSHDestination(remoteMachine.User, remoteMachine.Host, remoteMachine.Port), err,
)
}
// Print the list of machines in a table format.
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
// Print header.
// TODO: print ADDRESS instead of SUBNET which is a machine IP with prefix.
if _, err = fmt.Fprintln(tw, "NAME\tSUBNET\tPUBLIC KEY\tENDPOINTS"); err != nil {
return fmt.Errorf("write header: %w", err)
}
// Print rows.
for _, m := range listResp.Machines {
subnet, _ := m.Network.Subnet.ToPrefix()
endpoints := make([]string, len(m.Network.Endpoints))
for i, ep := range m.Network.Endpoints {
addrPort, _ := ep.ToAddrPort()
endpoints[i] = addrPort.String()
}
publicKey := secret.Secret(m.Network.PublicKey)
if _, err = fmt.Fprintf(
tw, "%s\t%s\t%s\t%s\n", m.Name, subnet, publicKey, strings.Join(endpoints, ", "),
); err != nil {
return fmt.Errorf("write row: %w", err)
if !skipInstall {
// Provision the remote machine by installing the Uncloud daemon and dependencies over SSH.
exec := sshexec.NewRemote(sshClient)
if err = provisionMachine(ctx, exec, version); err != nil {
return nil, fmt.Errorf("provision machine: %w", err)
}
}
return tw.Flush()
var machineClient *client.Client
if remoteMachine.User == "root" || skipInstall {
// Create a machine API client over the established SSH connection to the remote machine.
machineClient, err = client.New(ctx, connector.NewSSHConnectorFromClient(sshClient))
} else {
// Since the user is not root, we need to establish a new SSH connection to make the user's addition
// to the uncloud group effective, thus allowing access to the Uncloud daemon Unix socket.
sshConfig := &connector.SSHConnectorConfig{
User: remoteMachine.User,
Host: remoteMachine.Host,
Port: remoteMachine.Port,
KeyPath: remoteMachine.KeyPath,
}
machineClient, err = client.New(ctx, connector.NewSSHConnector(sshConfig))
}
if err != nil {
return nil, fmt.Errorf("connect to remote machine: %w", err)
}
return machineClient, nil
}
// ProgressOut returns an output stream for progress writer.
func (cli *CLI) ProgressOut() *streams.Out {
return streams.NewOut(os.Stdout)
}
-46
View File
@@ -1,46 +0,0 @@
package client
import (
"context"
"errors"
"fmt"
"google.golang.org/grpc"
"uncloud/internal/machine/api/pb"
)
// Client is a client for the machine API.
type Client struct {
connector Connector
conn *grpc.ClientConn
pb.MachineClient
pb.ClusterClient
}
// Connector is an interface for establishing a connection to the machine API.
type Connector interface {
Connect(ctx context.Context) (*grpc.ClientConn, error)
Close() error
}
// New creates a new client for the machine API. The connector is used to establish the connection
// either locally or remotely. The client is responsible for closing the connector.
func New(ctx context.Context, connector Connector) (*Client, error) {
c := &Client{
connector: connector,
}
var err error
c.conn, err = connector.Connect(ctx)
if err != nil {
return nil, fmt.Errorf("connect to machine: %w", err)
}
c.MachineClient = pb.NewMachineClient(c.conn)
c.ClusterClient = pb.NewClusterClient(c.conn)
return c, nil
}
func (c *Client) Close() error {
err := c.conn.Close()
return errors.Join(err, c.connector.Close())
}
-10
View File
@@ -1,10 +0,0 @@
package config
import "uncloud/internal/secret"
type Cluster struct {
Name string `toml:"-"`
Connections []MachineConnection `toml:"connections"`
// UserPrivateKey is the user's WireGuard private key used to connect to cluster machines.
UserPrivateKey secret.Secret `toml:"user_private_key"`
}
+26 -14
View File
@@ -2,14 +2,15 @@ package config
import (
"fmt"
"github.com/BurntSushi/toml"
"os"
"path/filepath"
"github.com/goccy/go-yaml"
)
type Config struct {
Clusters map[string]*Cluster `toml:"clusters"`
CurrentCluster string `toml:"current_cluster"`
CurrentContext string `yaml:"current_context"`
Contexts map[string]*Context `yaml:"contexts"`
// path is the file path config is read from.
path string
@@ -18,10 +19,10 @@ type Config struct {
func NewFromFile(path string) (*Config, error) {
_, err := os.Stat(path)
if err != nil && !os.IsNotExist(err) {
return nil, fmt.Errorf("check file permissions %q: %w", path, err)
return nil, fmt.Errorf("check file permissions '%s': %w", path, err)
}
c := &Config{
Clusters: map[string]*Cluster{},
Contexts: map[string]*Context{},
path: path,
}
if os.IsNotExist(err) {
@@ -34,30 +35,41 @@ func NewFromFile(path string) (*Config, error) {
return c, nil
}
func (c *Config) Path() string {
return c.path
}
func (c *Config) Read() error {
_, err := toml.DecodeFile(c.path, c)
data, err := os.ReadFile(c.path)
if err != nil {
return fmt.Errorf("read config file %q: %w", c.path, err)
return fmt.Errorf("read config file '%s': %w", c.path, err)
}
if err = yaml.Unmarshal(data, c); err != nil {
return fmt.Errorf("parse config file '%s': %s", c.path, yaml.FormatError(err, true, true))
}
return nil
}
func (c *Config) Save() error {
dir, _ := filepath.Split(c.path)
if err := os.MkdirAll(dir, 0700); err != nil {
return fmt.Errorf("create config directory %q: %w", dir, err)
// If dir is empty (e.g., when path is just a filename), use current directory
if dir == "" {
dir = "."
}
if err := os.MkdirAll(dir, 0o700); err != nil {
return fmt.Errorf("create config directory '%s': %w", dir, err)
}
f, err := os.OpenFile(c.path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
f, err := os.OpenFile(c.path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0o600)
if err != nil {
return fmt.Errorf("write config file %q: %w", c.path, err)
return fmt.Errorf("write config file '%s': %w", c.path, err)
}
encoder := toml.NewEncoder(f)
encoder.Indent = ""
encoder := yaml.NewEncoder(f, yaml.Indent(2), yaml.IndentSequence(true))
if err = encoder.Encode(c); err != nil {
_ = f.Close()
return fmt.Errorf("encode config file %q: %w", c.path, err)
return fmt.Errorf("encode config file '%s': %w", c.path, err)
}
return f.Close()
}
+80
View File
@@ -0,0 +1,80 @@
package config
import (
"os"
"path/filepath"
"testing"
)
func TestConfig_Save(t *testing.T) {
t.Parallel()
// Create a temporary directory for the test
tmpDir := t.TempDir()
// Change to temp directory so relative paths resolve correctly
originalDir, err := os.Getwd()
if err != nil {
t.Fatalf("Failed to get current directory: %v", err)
}
defer func() {
if err := os.Chdir(originalDir); err != nil {
t.Logf("Failed to restore original directory: %v", err)
}
}()
tests := []struct {
name string
configPath string
contextName string
expectFileAt string // Expected file location for verification
useAbsolutePath bool // Whether to use absolute path for expectFileAt
}{
{
name: "relative path without prefix",
configPath: "test-config.yaml",
contextName: "test",
},
{
name: "relative path with prefix",
configPath: "./test-config-2.yaml",
contextName: "test2",
},
{
name: "absolute path",
configPath: filepath.Join(tmpDir, "absolute-config.yaml"),
contextName: "test3",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
if err := os.Chdir(tmpDir); err != nil {
t.Fatalf("Failed to change to temp directory: %v", err)
}
cfg := &Config{
CurrentContext: tt.contextName,
Contexts: map[string]*Context{
tt.contextName: {
Name: tt.contextName,
},
},
path: tt.configPath,
}
// This should not fail when saving the config
err := cfg.Save()
if err != nil {
t.Errorf("Expected no error when saving config, got: %v", err)
}
// Verify the file was created
if _, err := os.Stat(tt.configPath); os.IsNotExist(err) {
t.Errorf("Config file was not created at expected path: %s", tt.configPath)
}
})
}
}
+20 -4
View File
@@ -1,10 +1,13 @@
package config
import (
"fmt"
"net"
"net/netip"
"strconv"
"strings"
"uncloud/internal/secret"
"github.com/psviderski/uncloud/internal/secret"
)
const (
@@ -13,9 +16,22 @@ const (
)
type MachineConnection struct {
SSH SSHDestination `toml:"ssh,omitempty"`
Host string `toml:"host,omitempty"`
PublicKey secret.Secret `toml:"public_key,omitempty"`
SSH SSHDestination `yaml:"ssh,omitempty"`
SSHKeyFile string `yaml:"ssh_key_file,omitempty"`
// TCP is the address and port of the machine's API server.
// The pointer is used to omit the field when not set. Otherwise, yaml marshalling includes an empty object.
TCP *netip.AddrPort `yaml:"tcp,omitempty"`
Host string `yaml:"host,omitempty"`
PublicKey secret.Secret `yaml:"public_key,omitempty"`
}
func (c MachineConnection) String() string {
if c.SSH != "" {
return string(c.SSH)
} else if c.TCP != nil && c.TCP.IsValid() {
return fmt.Sprintf("tcp://%s", c.TCP)
}
return "unknown connection"
}
// SSHDestination represents an SSH destination string in the canonical form of "user@host:port".
+6
View File
@@ -0,0 +1,6 @@
package config
type Context struct {
Name string `yaml:"-"`
Connections []MachineConnection `yaml:"connections"`
}
+189
View File
@@ -0,0 +1,189 @@
package cli
import (
"context"
"errors"
"fmt"
"os"
"time"
"github.com/charmbracelet/bubbles/spinner"
tea "github.com/charmbracelet/bubbletea"
"github.com/charmbracelet/lipgloss"
"github.com/psviderski/uncloud/internal/cli/config"
"github.com/psviderski/uncloud/internal/fs"
"github.com/psviderski/uncloud/pkg/client"
"github.com/psviderski/uncloud/pkg/client/connector"
)
// ConnectOptions configures the behavior of cluster connection attempts.
type ConnectOptions struct {
// Whether to show connection progress spinner if stdout is a terminal or progress logs if not.
ShowProgress bool
}
func ConnectCluster(ctx context.Context, conn config.MachineConnection, opts ConnectOptions) (*client.Client, error) {
if opts.ShowProgress {
return connectClusterWithProgress(ctx, conn)
}
return connectCluster(ctx, conn)
}
// connectClusterWithProgress connects to the cluster while displaying a progress spinner.
// If the stdout is not a terminal, it falls back to simple progress logs to stderr.
func connectClusterWithProgress(ctx context.Context, conn config.MachineConnection) (*client.Client, error) {
// If stdout is not a terminal, fall back to simple progress logs.
if !IsStdoutTerminal() {
fmt.Fprintln(os.Stderr, "Connecting to", conn.String())
cli, err := connectCluster(ctx, conn)
if err != nil {
fmt.Fprintln(os.Stderr, "Connection failed:", err)
} else {
fmt.Fprintln(os.Stderr, "Connected to cluster.")
}
return cli, err
}
// Run the connection TUI model.
p := tea.NewProgram(newConnectModel(ctx, conn))
model, err := p.Run()
if err != nil {
return nil, fmt.Errorf("run connection TUI: %w", err)
}
m := model.(connectModel)
return m.result.client, m.result.err
}
func connectCluster(ctx context.Context, conn config.MachineConnection) (*client.Client, error) {
if conn.SSH != "" {
user, host, port, err := conn.SSH.Parse()
if err != nil {
return nil, fmt.Errorf("parse SSH connection %q: %w", conn.SSH, err)
}
keyPath := fs.ExpandHomeDir(conn.SSHKeyFile)
sshConfig := &connector.SSHConnectorConfig{
User: user,
Host: host,
Port: port,
KeyPath: keyPath,
}
return client.New(ctx, connector.NewSSHConnector(sshConfig))
} else if conn.TCP != nil && conn.TCP.IsValid() {
return client.New(ctx, connector.NewTCPConnector(*conn.TCP))
}
return nil, errors.New("connection configuration is invalid")
}
// connectModel is a TUI model for connecting to a cluster with a progress spinner.
type connectModel struct {
ctx context.Context
conn config.MachineConnection
spinner spinner.Model
// showSpinner controls whether the spinner is visible (delayed to avoid flashing).
showSpinner bool
// done indicates whether the connection attempt has completed (successfully or with error).
done bool
// result holds the result of the connection attempt.
result connectResultMsg
}
type connectResultMsg struct {
client *client.Client
err error
}
// showSpinnerMsg is sent after a delay to show the spinner.
type showSpinnerMsg struct{}
func newConnectModel(ctx context.Context, conn config.MachineConnection) connectModel {
s := spinner.New()
s.Spinner = spinner.MiniDot
s.Style = lipgloss.NewStyle().Foreground(lipgloss.Color("3")) // the same yellow as in compose progress
return connectModel{
ctx: ctx,
conn: conn,
spinner: s,
}
}
func (m connectModel) Init() tea.Cmd {
return tea.Batch(
m.spinner.Tick,
m.connect(),
m.delayShowSpinner(),
)
}
func (m connectModel) connect() tea.Cmd {
return func() tea.Msg {
cli, err := connectCluster(m.ctx, m.conn)
return connectResultMsg{
client: cli,
err: err,
}
}
}
// delayShowSpinner returns a command that sends a message to show the spinner after a delay.
// This avoids flashing the spinner if the connection is fast.
func (m connectModel) delayShowSpinner() tea.Cmd {
return tea.Tick(500*time.Millisecond, func(t time.Time) tea.Msg {
return showSpinnerMsg{}
})
}
func (m connectModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
select {
case <-m.ctx.Done():
m.result.err = m.ctx.Err()
m.done = true
return m, tea.Quit
default:
}
switch msg := msg.(type) {
case connectResultMsg:
m.result = msg
m.done = true
return m, tea.Quit
case showSpinnerMsg:
// Only show spinner if connection hasn't completed yet.
if !m.done {
m.showSpinner = true
}
return m, nil
case spinner.TickMsg:
var cmd tea.Cmd
m.spinner, cmd = m.spinner.Update(msg)
return m, cmd
case tea.KeyMsg:
if msg.Type == tea.KeyCtrlC {
m.result.err = fmt.Errorf("connection cancelled")
m.done = true
return m, tea.Quit
}
}
return m, nil
}
func (m connectModel) View() string {
// Don't show anything if done or spinner not yet visible.
if m.done || !m.showSpinner {
return ""
}
style := lipgloss.NewStyle().Foreground(lipgloss.Color("153"))
return fmt.Sprintf("%s %s\n",
m.spinner.View(),
fmt.Sprintf("Connecting to %s", style.Render(m.conn.String())),
)
}
+36
View File
@@ -0,0 +1,36 @@
package cli
import (
"log"
"os"
"strings"
"github.com/spf13/cobra"
)
// ExpandCommaSeparatedValues takes a slice of strings and expands any comma-separated values into individual elements.
func ExpandCommaSeparatedValues(values []string) []string {
if len(values) == 0 {
return nil
}
var expanded []string
for _, value := range values {
for _, v := range strings.Split(value, ",") {
if v = strings.TrimSpace(v); v != "" {
expanded = append(expanded, v)
}
}
}
return expanded
}
// BindEnvToFlag assigns the value of an environment variable to the given command flag if the flag has not been set.
func BindEnvToFlag(cmd *cobra.Command, flagName, envVar string) {
if value := os.Getenv(envVar); value != "" && !cmd.Flags().Changed(flagName) {
if err := cmd.Flags().Set(flagName, value); err != nil {
log.Fatalf("Failed to bind environment variable '%s' to flag '%s': %v", envVar, flagName, err)
}
}
}
+110 -13
View File
@@ -4,11 +4,21 @@ import (
"context"
"fmt"
"os"
"uncloud/internal/sshexec"
"strings"
"time"
"github.com/cenkalti/backoff/v4"
"github.com/charmbracelet/huh"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/sshexec"
"google.golang.org/protobuf/types/known/emptypb"
)
// TODO: support pinning the script version to the CLI version.
const installScriptURL = "https://raw.githubusercontent.com/psviderski/uncloud/refs/heads/main/scripts/install.sh"
const (
// TODO: support pinning the script version to the CLI version.
installScriptURL = "https://raw.githubusercontent.com/psviderski/uncloud/refs/heads/main/scripts/install.sh"
rootUser = "root"
)
type RemoteMachine struct {
User string
@@ -17,26 +27,113 @@ type RemoteMachine struct {
KeyPath string
}
func installCmd(user string, version string) string {
sudoPrefix := ""
var env []string
// Add the SSH user (non-root) to the uncloud group to allow access to the Uncloud daemon unix socket.
if user != rootUser {
sudoPrefix = "sudo"
env = append(env, "UNCLOUD_GROUP_ADD_USER="+sshexec.Quote(user))
}
if version != "" {
env = append(env, "UNCLOUD_VERSION="+sshexec.Quote(version))
}
envCmd := strings.Join(env, " ")
curlBashCmd := fmt.Sprintf("curl -fsSL %s | %s %s bash", sshexec.Quote(installScriptURL), sudoPrefix, envCmd)
return curlBashCmd
}
// provisionMachine provisions the remote machine by downloading the Uncloud install script from GitHub and running it.
func provisionMachine(ctx context.Context, exec sshexec.Executor) error {
// If version is specified, it will be passed to the install script as UNCLOUD_VERSION environment variable.
func provisionMachine(ctx context.Context, exec sshexec.Executor, version string) error {
user, err := exec.Run(ctx, "whoami")
if err != nil {
return fmt.Errorf("run whoami: %w", err)
}
sudoPrefix, env := "", ""
if user != "root" {
sudoPrefix = "sudo"
// Add the SSH user (non-root) to the uncloud group to allow access to the Uncloud daemon unix socket.
env = "UNCLOUD_GROUP_ADD_USER=" + user
if user != rootUser {
// 'sudo -n' is not used because it fails with 'sudo: a password is required' when the user has no password
// in /etc/shadow even though it may have valid sudo access.
out, err := exec.Run(ctx, "sudo true")
if err != nil {
if strings.Contains(out, "password is required") {
return fmt.Errorf(
"user '%[1]s' requires a password for sudo, but Uncloud needs passwordless sudo or root access "+
"to install and configure the uncloudd daemon on the remote machine.\n\n"+
"Possible solutions:\n"+
"1. Use root user or a user with passwordless sudo instead.\n"+
"2. Configure passwordless sudo for the user '%[1]s' by running on the remote machine:\n"+
" echo '%[1]s ALL=(ALL) NOPASSWD:ALL' | sudo tee /etc/sudoers.d/%[1]s",
user)
}
return fmt.Errorf("sudo command failed for user '%s': %w. "+
"Please ensure the user has sudo privileges or use root user instead", user, err)
}
}
cmd := installCmd(user, version)
fmt.Println("Downloading Uncloud install script:", installScriptURL)
curlBashCmd := fmt.Sprintf(
"curl -fsSL %s | %s %s bash", sshexec.Quote(installScriptURL), sudoPrefix, sshexec.Quote(env),
)
cmd := sshexec.QuoteCommand("bash", "-c", "set -o pipefail; "+curlBashCmd)
cmd = sshexec.QuoteCommand("bash", "-c", "set -o pipefail; "+cmd)
if err = exec.Stream(ctx, cmd, os.Stdout, os.Stderr); err != nil {
return fmt.Errorf("download and run install script: %w", err)
}
return nil
}
func promptResetMachine(ctx context.Context, machineClient pb.MachineClient) error {
var confirm bool
form := huh.NewForm(
huh.NewGroup(
huh.NewConfirm().
Title(
"The remote machine is already initialised as a cluster member. Do you want to reset it first?\n" +
"This will:\n" +
"- Remove all service containers from the machine\n" +
"- Reset the machine to the uninitialised state",
).
Affirmative("Yes!").
Negative("No").
Value(&confirm),
),
).WithAccessible(true)
if err := form.Run(); err != nil {
return fmt.Errorf("prompt user to confirm: %w", err)
}
if !confirm {
return fmt.Errorf("remote machine is already initialised as a cluster member")
}
if _, err := machineClient.Reset(ctx, &pb.ResetRequest{}); err != nil {
return fmt.Errorf("reset remote machine: %w. You can also manually run 'uncloud-uninstall' "+
"on the remote machine to fully uninstall Uncloud from it", err)
}
fmt.Println("Resetting the remote machine...")
if err := waitMachineReady(ctx, machineClient, 1*time.Minute); err != nil {
return fmt.Errorf("wait for machine to be ready after reset: %w", err)
}
return nil
}
// waitMachineReady waits for the machine to be ready to serve requests.
func waitMachineReady(ctx context.Context, machineClient pb.MachineClient, timeout time.Duration) error {
boff := backoff.WithContext(backoff.NewExponentialBackOff(
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(timeout),
), ctx)
inspect := func() error {
_, err := machineClient.Inspect(ctx, &emptypb.Empty{})
if err != nil {
return fmt.Errorf("inspect machine: %w", err)
}
return nil
}
return backoff.Retry(inspect, boff)
}
+36
View File
@@ -0,0 +1,36 @@
package cli
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestInstallCmd(t *testing.T) {
t.Run("root", func(t *testing.T) {
cmd := installCmd("root", "")
assert.NotContains(t, cmd, "sudo")
assert.NotContains(t, cmd, "UNCLOUD_GROUP_ADD_USER")
})
// Test with version
t.Run("root with version", func(t *testing.T) {
cmd := installCmd("root", "v1.2.3")
assert.NotContains(t, cmd, "sudo")
assert.NotContains(t, cmd, "UNCLOUD_GROUP_ADD_USER")
assert.Contains(t, cmd, "UNCLOUD_VERSION=v1.2.3")
})
t.Run("nonroot", func(t *testing.T) {
cmd := installCmd("nonroot", "")
assert.Contains(t, cmd, "sudo")
assert.Contains(t, cmd, "UNCLOUD_GROUP_ADD_USER=nonroot")
})
t.Run("nonroot with version", func(t *testing.T) {
cmd := installCmd("nonroot", "v1.2.3")
assert.Contains(t, cmd, "sudo")
assert.Contains(t, cmd, "UNCLOUD_GROUP_ADD_USER=nonroot")
assert.Contains(t, cmd, "UNCLOUD_VERSION=v1.2.3")
})
}
+38
View File
@@ -0,0 +1,38 @@
package cli
import (
"os"
"github.com/charmbracelet/huh"
"golang.org/x/term"
)
func Confirm() (bool, error) {
var confirmed bool
form := huh.NewForm(
huh.NewGroup(
huh.NewConfirm().
Title(
"Do you want to continue?",
).
Affirmative("Yes!").
Negative("No").
Value(&confirmed),
),
).WithAccessible(true)
if err := form.Run(); err != nil {
return false, err
}
return confirmed, nil
}
// IsStdinTerminal checks if the standard input is a terminal (TTY).
func IsStdinTerminal() bool {
return term.IsTerminal(int(os.Stdin.Fd()))
}
// IsStdoutTerminal checks if the standard output is a terminal (TTY).
func IsStdoutTerminal() bool {
return term.IsTerminal(int(os.Stdout.Fd()))
}
+258
View File
@@ -0,0 +1,258 @@
package corrosion
import (
"encoding/binary"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"net/netip"
"time"
)
// AdminClient is a client for the Corrosion admin API.
type AdminClient struct {
sockPath string
}
func NewAdminClient(sockPath string) (*AdminClient, error) {
return &AdminClient{sockPath: sockPath}, nil
}
type Response struct {
JSON map[string]any
// Err is set if the response is an error or if an error occurred while processing the response.
Err error
}
// SendCommand sends a command to the Corrosion admin API and returns a channel that will receive responses.
// The channel will be closed after sending the last or error response. The caller must read from the channel until
// it is closed.
func (c *AdminClient) SendCommand(cmd []byte) (<-chan Response, error) {
conn, err := net.Dial("unix", c.sockPath)
if err != nil {
return nil, fmt.Errorf("connect to admin socket: %w", err)
}
if _, err = conn.Write(encodeFrame(cmd)); err != nil {
conn.Close()
return nil, fmt.Errorf("send command: %w", err)
}
ch := make(chan Response)
go func() {
defer close(ch)
defer conn.Close()
for {
r := Response{}
data, err := readFrame(conn)
if err != nil {
r.Err = err
ch <- r
return
}
var decoded any
if err = json.Unmarshal(data, &decoded); err != nil {
r.Err = fmt.Errorf("unmarshal response: %w", err)
ch <- r
return
}
switch v := decoded.(type) {
case string:
if v == "Success" {
return
}
// Ignore other strings.
case map[string]any:
if errData, ok := v["Error"].(map[string]any); ok {
if errMsg, ok := errData["msg"].(string); ok {
r.Err = errors.New(errMsg)
} else {
r.Err = fmt.Errorf("invalid error response: %v", errData)
}
ch <- r
return
} else if jsonData, ok := v["Json"].(map[string]any); ok {
r.JSON = jsonData
ch <- r
}
// Ignore other maps.
default:
// Ignore other types.
}
}
}()
return ch, nil
}
// encodeFrame encodes a length_delimited Tokio frame by prefacing frame data with a frame head that specifies
// the length of the frame.
func encodeFrame(data []byte) []byte {
encoded := make([]byte, 4+len(data))
binary.BigEndian.PutUint32(encoded, uint32(len(data)))
copy(encoded[4:], data)
return encoded
}
// readFrame reads a length_delimited Tokio frame from the connection by extracting the frame data that follows
// the frame head.
func readFrame(conn net.Conn) ([]byte, error) {
// Read the frame head (4 bytes).
head := make([]byte, 4)
if _, err := io.ReadFull(conn, head); err != nil {
return nil, fmt.Errorf("read frame head: %w", err)
}
// Read the frame data (length specified in the frame head).
length := binary.BigEndian.Uint32(head)
data := make([]byte, length)
if _, err := io.ReadFull(conn, data); err != nil {
return nil, fmt.Errorf("read frame data: %w", err)
}
return data, nil
}
type ClusterMembershipState struct {
ID string
Addr netip.AddrPort
State string
Timestamp time.Time
}
var (
// MembershipStateAlive indicates that the member is active.
MembershipStateAlive string = "Alive"
// MembershipStateSuspect indicates that the member is active, but at least one cluster member suspects its down.
// For all purposes, a Suspect member is treated as if it were Alive until either it refutes the suspicion
// (becoming Alive) or fails to do so (being declared Down).
MembershipStateSuspect string = "Suspect"
// MembershipStateDown indicates that the member is confirmed Down. A member that reaches this state can't join
// the cluster with the same identity until the cluster forgets this knowledge.
MembershipStateDown string = "Down"
)
func parseClusterMembershipState(json map[string]any) (ClusterMembershipState, error) {
// Example JSON:
// {
// "id": {
// "addr": "[fdcc:1d51:6bae:6bb2:53c0:8796:1be0:b783]:51001",
// "cluster_id": 0,
// "id": "10d69d6f-6578-4dcf-a285-e860e40c5f06",
// "ts": 7435936225798880256
// },
// "incarnation": 0,
// "state": "Down"
// }
var state ClusterMembershipState
var err error
idObj, ok := json["id"].(map[string]any)
if !ok {
return state, fmt.Errorf("missing or invalid 'id' field")
}
// ID
if id, ok := idObj["id"].(string); ok {
state.ID = id
} else {
return state, fmt.Errorf("missing or invalid 'id' field")
}
// Addr
if addr, ok := idObj["addr"].(string); ok {
state.Addr, err = netip.ParseAddrPort(addr)
if err != nil {
return state, fmt.Errorf("parse 'addr' field: %w", err)
}
} else {
return state, fmt.Errorf("missing or invalid 'addr' field")
}
// State
if stateStr, ok := json["state"].(string); ok {
switch stateStr {
case MembershipStateAlive, MembershipStateSuspect, MembershipStateDown:
state.State = stateStr
default:
return state, fmt.Errorf("invalid 'state' field: %s", stateStr)
}
} else {
return state, fmt.Errorf("missing or invalid 'state' field")
}
// Timestamp
if ts, ok := idObj["ts"].(float64); ok {
state.Timestamp = ntp64ToTime(uint64(ts))
} else {
return state, fmt.Errorf("missing or invalid 'ts' field")
}
return state, nil
}
// nt64ToTime converts a 64-bit NTP timestamp relative to the Unix epoch (1st Jan 1970) to time.Time.
// See for more details: https://datatracker.ietf.org/doc/html/rfc5905#section-6
func ntp64ToTime(ntp uint64) time.Time {
// The NTP timestamp returned from Corrosion is relative to the Unix epoch (1st Jan 1970)
// so no need to subtract the 70 years offset.
secs := uint32(ntp >> 32)
frac := uint32(ntp)
// Convert the fraction to nanoseconds: frac * 1e9 / 2^32
nsecs := (uint64(frac) * 1000_000_000) >> 32
return time.Unix(int64(secs), int64(nsecs))
}
// ClusterMembershipStates returns the current membership SWIM states of all cluster members.
// If latest is true, only the latest state of each member is returned.
func (c *AdminClient) ClusterMembershipStates(latest bool) ([]ClusterMembershipState, error) {
respCh, err := c.SendCommand([]byte("{\"Cluster\":\"MembershipStates\"}"))
if err != nil {
return nil, err
}
var (
states []ClusterMembershipState
latestStates map[string]ClusterMembershipState
parseErr error
)
if latest {
latestStates = make(map[string]ClusterMembershipState)
}
for r := range respCh {
if r.Err != nil {
// It's safe to return here because the channel is closed after the first error response.
return nil, r.Err
}
s, err := parseClusterMembershipState(r.JSON)
if err != nil {
// Do not return early to drain the channel.
parseErr = errors.Join(parseErr, err)
} else {
if latest {
if existing, ok := latestStates[s.ID]; !ok || existing.Timestamp.Before(s.Timestamp) {
latestStates[s.ID] = s
}
} else {
states = append(states, s)
}
}
}
if latest {
states = make([]ClusterMembershipState, 0, len(latestStates))
for _, s := range latestStates {
states = append(states, s)
}
}
return states, parseErr
}
+234
View File
@@ -0,0 +1,234 @@
package corrosion
import (
"context"
"crypto/tls"
"errors"
"fmt"
"log/slog"
"net"
"net/http"
"net/netip"
"net/url"
"time"
"github.com/cenkalti/backoff/v4"
"golang.org/x/net/http2"
)
const (
// http2ConnectTimeout is the maximum amount of time an HTTP2 client will wait for a connection to be established.
http2ConnectTimeout = 3 * time.Second
// http2MaxRetryTime is the maximum amount of time an HTTP2 client will retry a request.
http2MaxRetryTime = 10 * time.Second
// resubscribeMaxRetryTime is the maximum amount of time an API client will retry resubscribing to a query after
// an error occurs.
resubscribeMaxRetryTime = 60 * time.Second
)
// APIClient is a client for the Corrosion API.
type APIClient struct {
baseURL *url.URL
client *http.Client
newResubBackoff func() backoff.BackOff
}
// NewAPIClient creates a new Corrosion API client. The client retries on network errors using an exponential backoff
// policy with a maximum interval of 1 second and a maximum elapsed time of 10 seconds.
// It automatically resubscribes to active subscriptions if an error occurs using an exponential backoff policy with a
// maximum interval of 1 second and a maximum elapsed time of 60 seconds.
// Use the WithHTTP2Client option to provide a custom HTTP client and the WithResubscribeBackoff option to change the
// backoff policy for resubscribing to a query.
func NewAPIClient(addr netip.AddrPort, opts ...APIClientOption) (*APIClient, error) {
baseURL, err := url.Parse(fmt.Sprintf("http://%s", addr))
if err != nil {
return nil, fmt.Errorf("invalid URL: %w", err)
}
c := &APIClient{
baseURL: baseURL,
client: &http.Client{
Transport: &RetryRoundTripper{
Base: &http2.Transport{
AllowHTTP: true,
DialTLSContext: func(ctx context.Context, network, addr string, _ *tls.Config) (net.Conn, error) {
dialer := &net.Dialer{
Timeout: http2ConnectTimeout,
}
return dialer.DialContext(ctx, network, addr)
},
},
NewBackoff: func() backoff.BackOff {
return backoff.NewExponentialBackOff(
backoff.WithInitialInterval(100*time.Millisecond),
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(http2MaxRetryTime),
)
},
},
},
newResubBackoff: func() backoff.BackOff {
return backoff.NewExponentialBackOff(
backoff.WithInitialInterval(100*time.Millisecond),
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(resubscribeMaxRetryTime),
)
},
}
for _, opt := range opts {
opt(c)
}
return c, nil
}
type APIClientOption func(*APIClient)
func WithHTTP2Client(client *http.Client) APIClientOption {
return func(c *APIClient) {
c.client = client
}
}
// WithResubscribeBackoff sets the backoff policy for resubscribing to a query if an error occurs.
// Pass nil to disable resubscribing.
func WithResubscribeBackoff(newBackoff func() backoff.BackOff) APIClientOption {
return func(c *APIClient) {
c.newResubBackoff = newBackoff
}
}
type RetryRoundTripper struct {
Base http.RoundTripper
// NewBackoff creates a new backoff policy for each request.
NewBackoff func() backoff.BackOff
}
func (rt *RetryRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
roundTrip := func() (*http.Response, error) {
resp, err := rt.Base.RoundTrip(req)
if err != nil {
var opErr *net.OpError
if errors.As(err, &opErr) {
// Not certain, but I expect operational errors should generally be retryable.
slog.Debug("Retrying corrosion API request due to network error.", "error", err)
return nil, err
}
// Don't retry on other errors.
return nil, backoff.Permanent(err)
}
// Success, don't retry.
return resp, err
}
boff := backoff.WithContext(rt.NewBackoff(), req.Context())
return backoff.RetryWithData(roundTrip, boff)
}
type RetrySubscription struct {
ctx context.Context
cancel context.CancelFunc
client *APIClient
sub *Subscription
changes chan *ChangeEvent
lastChangeID uint64
err error
backoff backoff.BackOff
}
func NewRetrySubscription(ctx context.Context, client *APIClient, sub *Subscription, boff backoff.BackOff) *RetrySubscription {
ctx, cancel := context.WithCancel(ctx)
if boff == nil {
boff = backoff.NewExponentialBackOff(
backoff.WithInitialInterval(100*time.Millisecond),
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(60*time.Second),
)
}
return &RetrySubscription{
ctx: ctx,
cancel: cancel,
client: client,
sub: sub,
backoff: backoff.WithContext(boff, ctx),
}
}
func (rs *RetrySubscription) ID() string {
return rs.sub.ID()
}
func (rs *RetrySubscription) Changes() (<-chan *ChangeEvent, error) {
if rs.changes != nil {
return rs.changes, nil
}
// Ensure the rows are consumed if they have been requested.
changes, err := rs.sub.Changes()
if err != nil {
return nil, err
}
rs.lastChangeID = rs.sub.lastChangeID
go func() {
defer close(rs.changes)
var change *ChangeEvent
for {
select {
case change = <-changes:
case <-rs.ctx.Done():
return
}
if change != nil {
select {
case rs.changes <- change:
case <-rs.ctx.Done():
return
}
rs.lastChangeID = change.ChangeID
continue
}
// The underlying subscription has been closed due to an error or context cancellation.
// Return if the context is done or try to resubscribe otherwise.
if rs.ctx.Err() != nil {
return
}
if err = rs.resubscribe(); err != nil {
// resubscribe returns a permanent error after unsuccessful retries.
rs.err = fmt.Errorf("resubscribe to query: %w", err)
return
}
changes, err = rs.sub.Changes()
if err != nil {
// Unexpected error but report it anyway.
rs.err = err
return
}
}
}()
return rs.changes, nil
}
func (rs *RetrySubscription) resubscribe() error {
return backoff.Retry(func() error {
slog.Debug("Resubscribing to Corrosion query.", "id", rs.ID(), "from_change", rs.lastChangeID)
sub, err := rs.client.ResubscribeContext(rs.ctx, rs.ID(), rs.lastChangeID)
if err != nil {
return err
}
rs.sub = sub
return nil
}, rs.backoff)
}
func (rs *RetrySubscription) Err() error {
return rs.err
}
func (rs *RetrySubscription) Close() {
rs.cancel()
}
+327
View File
@@ -0,0 +1,327 @@
package corrosion
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
)
type Statement struct {
Query string `json:"query"`
Params []any `json:"params"`
}
type ExecResponse struct {
Results []ExecResult `json:"results"`
Time float64 `json:"time"`
Version *uint `json:"version"`
}
type ExecResult struct {
RowsAffected uint `json:"rows_affected"`
Time float64 `json:"time"`
Error *string `json:"error"`
}
// ExecContext writes changes to the Corrosion database for propagation through the cluster. The args are for any
// placeholder parameters in the query. Corrosion does not sync schema changes made using this method. Use Corrosion's
// schema_files to create and update the cluster's database schema.
func (c *APIClient) ExecContext(ctx context.Context, query string, args ...any) (*ExecResult, error) {
statements := []Statement{
{
Query: query,
Params: args,
},
}
resp, err := c.ExecMultiContext(ctx, statements...)
if err != nil {
return nil, err
}
if len(resp.Results) == 0 {
return nil, fmt.Errorf("no results: %+v", resp)
}
return &resp.Results[0], nil
}
// ExecMultiContext writes changes to the Corrosion database for propagation through the cluster.
// Unlike ExecContext, this method allows multiple statements to be executed in a single transaction.
func (c *APIClient) ExecMultiContext(ctx context.Context, statements ...Statement) (*ExecResponse, error) {
body, err := json.Marshal(statements)
if err != nil {
return nil, fmt.Errorf("marshal queries: %w", err)
}
transactionsURL := c.baseURL.JoinPath("/v1/transactions").String()
req, err := http.NewRequestWithContext(ctx, "POST", transactionsURL, bytes.NewReader(body))
if err != nil {
return nil, fmt.Errorf("create request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
resp, err := c.client.Do(req)
if err != nil {
return nil, fmt.Errorf("send request: %w", err)
}
defer resp.Body.Close()
var execResp ExecResponse
if resp.StatusCode == http.StatusOK {
if err = json.NewDecoder(resp.Body).Decode(&execResp); err != nil {
return nil, fmt.Errorf("decode response: %w", err)
}
// The response may still contain DB errors even if the status code is OK. Return them along with the response.
var errs []error
for _, result := range execResp.Results {
if result.Error != nil {
errs = append(errs, errors.New(*result.Error))
}
}
return &execResp, errors.Join(errs...)
} else if resp.StatusCode == http.StatusInternalServerError {
// If the response is an Internal Server Error, the response body may contain the error encoded as ExecResponse.
respBody, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("read response body: %w", err)
}
if err = json.Unmarshal(respBody, &execResp); err != nil {
return nil, fmt.Errorf("internal server error: %s", respBody)
}
if len(execResp.Results) > 0 && execResp.Results[0].Error != nil {
return nil, errors.New(*execResp.Results[0].Error)
}
return nil, fmt.Errorf("internal server error: %s", respBody)
}
respBody, err := io.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("read response body: %w", err)
}
return nil, fmt.Errorf("unexpected status code %d: %s", resp.StatusCode, respBody)
}
type QueryEvent struct {
Columns []string `json:"columns"`
Row *RowEvent `json:"row"`
EOQ *EndOfQuery `json:"eoq"`
Change *ChangeEvent `json:"change"`
// Error is a server-side error that occurred during query execution. It's considered fatal for the client
// as it cannot be recovered from server-side.
Error *string `json:"error"`
}
type EndOfQuery struct {
Time float64 `json:"time"`
ChangeID *uint64 `json:"change_id"`
}
type RowEvent struct {
RowID uint64
Values []json.RawMessage
}
func (re *RowEvent) UnmarshalJSON(data []byte) error {
var raw []json.RawMessage
if err := json.Unmarshal(data, &raw); err != nil {
return fmt.Errorf("invalid row event: %w", err)
}
if len(raw) != 2 {
return fmt.Errorf("invalid row event: expected an array of 2 elements")
}
if err := json.Unmarshal(raw[0], &re.RowID); err != nil {
return fmt.Errorf("invalid row event: %w", err)
}
if err := json.Unmarshal(raw[1], &re.Values); err != nil {
return fmt.Errorf("invalid row event: %w", err)
}
return nil
}
func (re *RowEvent) MarshalJSON() ([]byte, error) {
return json.Marshal([]any{re.RowID, re.Values})
}
// QueryContext executes a query that returns rows, typically a SELECT.
// The args are for any placeholder parameters in the query.
func (c *APIClient) QueryContext(ctx context.Context, query string, args ...any) (*Rows, error) {
statement := Statement{
Query: query,
Params: args,
}
body, err := json.Marshal(statement)
if err != nil {
return nil, fmt.Errorf("marshal query: %w", err)
}
queriesURL := c.baseURL.JoinPath("/v1/queries").String()
req, err := http.NewRequestWithContext(ctx, "POST", queriesURL, bytes.NewReader(body))
if err != nil {
return nil, fmt.Errorf("create request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
resp, err := c.client.Do(req)
if err != nil {
return nil, fmt.Errorf("send request: %w", err)
}
if resp.StatusCode != http.StatusOK {
respBody, err := io.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
return nil, fmt.Errorf("read response body: %w", err)
}
return nil, fmt.Errorf("unexpected status code %d: %s", resp.StatusCode, respBody)
}
rows, err := newRows(ctx, resp.Body, true)
if err != nil {
resp.Body.Close()
return nil, fmt.Errorf("parse query response: %w", err)
}
return rows, nil
}
// Rows is the result of a query. Its cursor starts before the first row of the result set.
// Use [Rows.Next] to advance from row to row.
type Rows struct {
ctx context.Context
body io.ReadCloser
decoder *json.Decoder
eoq *EndOfQuery
closeOnEOQ bool
columns []string
row RowEvent
err error
}
func newRows(ctx context.Context, body io.ReadCloser, closeOnEOQ bool) (*Rows, error) {
select {
case <-ctx.Done():
return nil, ctx.Err()
default:
}
decoder := json.NewDecoder(body)
var e QueryEvent
if err := decoder.Decode(&e); err != nil {
return nil, fmt.Errorf("decode query event: %w", err)
}
if e.Columns == nil {
return nil, fmt.Errorf("expected columns event, got: %+v", e)
}
return &Rows{
ctx: ctx,
body: body,
decoder: decoder,
closeOnEOQ: closeOnEOQ,
columns: e.Columns,
}, nil
}
// Columns returns the column names.
func (rs *Rows) Columns() []string {
return rs.columns
}
// Next prepares the next result row for reading with the [Rows.Scan] method. It returns true on success, or false
// if there is no next result row or an error happened while preparing it. [Rows.Err] should be consulted to distinguish
// between the two cases.
//
// Every call to [Rows.Scan], even the first one, must be preceded by a call to [Rows.Next].
func (rs *Rows) Next() bool {
select {
case <-rs.ctx.Done():
rs.err = rs.ctx.Err()
_ = rs.Close()
return false
default:
}
var e QueryEvent
if err := rs.decoder.Decode(&e); err != nil {
rs.err = fmt.Errorf("decode query event: %w", err)
_ = rs.Close()
return false
}
// Server-side query error.
if e.Error != nil {
rs.err = fmt.Errorf("query error: %s", *e.Error)
_ = rs.Close()
return false
}
if e.Row != nil {
if len(e.Row.Values) != len(rs.columns) {
rs.err = fmt.Errorf("expected %d column values, got %d", len(rs.columns), len(e.Row.Values))
_ = rs.Close()
return false
}
rs.row = *e.Row
return true
}
if e.EOQ != nil {
rs.eoq = e.EOQ
// Rows could be used as part of a subscription, so don't close the body if so.
if rs.closeOnEOQ {
_ = rs.Close()
}
return false
}
rs.err = fmt.Errorf("expected row or eof event, got: %+v", e)
_ = rs.Close()
return false
}
// Err returns the error, if any, that was encountered during iteration.
// Err may be called after an explicit or implicit [Rows.Close].
func (rs *Rows) Err() error {
return rs.err
}
// Scan copies the columns in the current row into the values pointed at by dest.
// The number of values in dest must be the same as the number of columns in [Rows].
// Scan converts JSON-encoded column values to the provided Go types using [json.Unmarshal].
func (rs *Rows) Scan(dest ...any) error {
if rs.err != nil {
return rs.err
}
if len(dest) != len(rs.columns) {
return fmt.Errorf("expected %d values, got %d", len(rs.columns), len(dest))
}
for i, v := range rs.row.Values {
if err := json.Unmarshal(v, dest[i]); err != nil {
return fmt.Errorf("unmarshal column value #%d: %w", i, err)
}
}
return nil
}
// Time returns the time taken to execute the query in seconds. It's only available after all rows have been consumed.
// It doesn't include the time to send the query, receive the response, or iterate over the rows.
func (rs *Rows) Time() (float64, error) {
if rs.eoq == nil {
if rs.Err() != nil {
return 0, fmt.Errorf("time is not available: %w", rs.Err())
}
return 0, errors.New("time is not available until all rows are consumed")
}
return rs.eoq.Time, nil
}
// Close closes the [Rows], preventing further enumeration. If [Rows.Next] is called and returns false,
// the [Rows] are closed automatically and it will suffice to check the result of [Rows.Err].
// Close is idempotent and does not affect the result of [Rows.Err].
func (rs *Rows) Close() error {
return rs.body.Close()
}
+324
View File
@@ -0,0 +1,324 @@
package corrosion
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"log/slog"
"net/http"
"strconv"
"github.com/cenkalti/backoff/v4"
)
type ChangeType string
var (
ChangeTypeInsert ChangeType = "insert"
ChangeTypeUpdate ChangeType = "update"
ChangeTypeDelete ChangeType = "delete"
)
type ChangeEvent struct {
Type ChangeType
RowID uint64
Values []json.RawMessage
ChangeID uint64
}
func (ce *ChangeEvent) UnmarshalJSON(data []byte) error {
var raw []json.RawMessage
if err := json.Unmarshal(data, &raw); err != nil {
return fmt.Errorf("invalid change event: %w", err)
}
if len(raw) != 4 {
return fmt.Errorf("invalid change event: expected an array of 4 elements")
}
if err := json.Unmarshal(raw[0], &ce.Type); err != nil {
return fmt.Errorf("invalid change event type: %w", err)
}
if err := json.Unmarshal(raw[1], &ce.RowID); err != nil {
return fmt.Errorf("invalid change event row ID: %w", err)
}
if err := json.Unmarshal(raw[2], &ce.Values); err != nil {
return fmt.Errorf("invalid change event values: %w", err)
}
if err := json.Unmarshal(raw[3], &ce.ChangeID); err != nil {
return fmt.Errorf("invalid change event change ID: %w", err)
}
return nil
}
func (ce *ChangeEvent) MarshalJSON() ([]byte, error) {
return json.Marshal([]any{ce.Type, ce.RowID, ce.Values, ce.ChangeID})
}
// Scan copies the column values in the change event into the values pointed at by dest.
// The number of values in dest must be the same as the number of columns in the change.
// Scan converts JSON-encoded column values to the provided Go types using [json.Unmarshal].
func (ce *ChangeEvent) Scan(dest ...any) error {
if len(dest) != len(ce.Values) {
return fmt.Errorf("expected %d values, got %d", len(ce.Values), len(dest))
}
for i, v := range ce.Values {
if err := json.Unmarshal(v, dest[i]); err != nil {
return fmt.Errorf("unmarshal column value #%d: %w", i, err)
}
}
return nil
}
// Subscription receives updates from the Corrosion database for a desired SQL query.
type Subscription struct {
ctx context.Context
cancel context.CancelFunc
id string
rows *Rows
body io.ReadCloser
decoder *json.Decoder
resubscribe func(ctx context.Context, fromChange uint64) (*Subscription, error)
changes chan *ChangeEvent
lastChangeID uint64
err error
}
func newSubscription(
ctx context.Context,
id string,
rows *Rows,
body io.ReadCloser,
decoder *json.Decoder,
resubscribe func(ctx context.Context, fromChange uint64) (*Subscription, error),
) *Subscription {
ctx, cancel := context.WithCancel(ctx)
if decoder == nil {
decoder = json.NewDecoder(body)
}
return &Subscription{
ctx: ctx,
cancel: cancel,
id: id,
rows: rows,
body: body,
decoder: decoder,
resubscribe: resubscribe,
}
}
// ID returns the subscription ID.
func (s *Subscription) ID() string {
return s.id
}
// Rows returns the rows of the query or nil if skipRows was true when creating the subscription or if the subscription
// was created with [APIClient.ResubscribeContext].
func (s *Subscription) Rows() *Rows {
return s.rows
}
// Changes returns a channel that receives change events for the query. Changes are not available until all rows
// are consumed. The channel is closed when the context is done, or an error occurs while reading the changes,
// or when the subscription is closed explicitly. If it's closed due to an error, [Subscription.Err] will return
// the error.
func (s *Subscription) Changes() (<-chan *ChangeEvent, error) {
if s.changes != nil {
return s.changes, nil
}
if s.rows != nil {
if s.rows.eoq == nil {
return nil, errors.New("changes are not available until all rows are consumed")
}
s.lastChangeID = *s.rows.eoq.ChangeID
}
s.changes = make(chan *ChangeEvent)
go func() {
// Close the body when the context is done to unblock the decoder in the following goroutine.
<-s.ctx.Done()
s.body.Close()
}()
go s.handleChangeEvents()
return s.changes, nil
}
func (s *Subscription) handleChangeEvents() {
defer s.cancel()
defer close(s.changes)
for {
select {
case <-s.ctx.Done():
return
default:
}
var e QueryEvent
var err error
if err = s.decoder.Decode(&e); err != nil {
// Do not report an error that occurred due to context cancellation, just return.
if s.ctx.Err() != nil {
return
}
err = fmt.Errorf("decode query event: %w", err)
} else if e.Error != nil {
err = fmt.Errorf("query error: %s", *e.Error)
} else if e.Change == nil {
err = fmt.Errorf("expected change event, got: %+v", e)
} else if s.lastChangeID != 0 && e.Change.ChangeID != s.lastChangeID+1 {
// If skipRows is true, the last change ID is unknown.
err = fmt.Errorf("missed a change: expected change ID %d, got %d",
s.lastChangeID+1, e.Change.ChangeID)
}
if err == nil {
s.lastChangeID = e.Change.ChangeID
select {
case s.changes <- e.Change:
case <-s.ctx.Done():
return
}
} else {
// Report the error if resubscribing is disabled.
if s.resubscribe == nil {
s.err = err
return
}
slog.Info("Resubscribing to Corrosion query due to an error.",
"err", err, "id", s.id, "from_change", s.lastChangeID)
sub, sErr := s.resubscribe(s.ctx, s.lastChangeID)
if sErr != nil {
// resubscribe returns a permanent error after unsuccessful retries.
s.err = fmt.Errorf("resubscribe to query with backoff: %w", sErr)
return
}
// Reset the subscription to the new one.
s.rows = nil
s.body = sub.body
s.decoder = sub.decoder
// Do not close the sub to not close the body.
sub.cancel()
}
}
}
// Err returns the error, if any, that was encountered during fetching changes.
// Err may be called after an explicit or implicit [Subscription.Close].
func (s *Subscription) Err() error {
return s.err
}
func (s *Subscription) Close() error {
s.cancel()
return s.body.Close()
}
// SubscribeContext creates a subscription to receive updates for a desired SQL query. If skipRows is false,
// Subscription.Rows must be consumed before Subscription.Changes can be called. If skipRows is true, Subscription.Rows
// will return nil.
func (c *APIClient) SubscribeContext(
ctx context.Context, query string, args []any, skipRows bool,
) (*Subscription, error) {
statement := Statement{
Query: query,
Params: args,
}
body, err := json.Marshal(statement)
if err != nil {
return nil, fmt.Errorf("marshal query: %w", err)
}
subURL := c.baseURL.JoinPath("/v1/subscriptions")
if skipRows {
q := subURL.Query()
q.Set("skip_rows", "true")
subURL.RawQuery = q.Encode()
}
req, err := http.NewRequestWithContext(ctx, "POST", subURL.String(), bytes.NewReader(body))
if err != nil {
return nil, fmt.Errorf("create request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
resp, err := c.client.Do(req)
if err != nil {
return nil, fmt.Errorf("send request: %w", err)
}
if resp.StatusCode != http.StatusOK {
respBody, err := io.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
return nil, fmt.Errorf("read response body: %w", err)
}
return nil, fmt.Errorf("unexpected status code %d: %s", resp.StatusCode, respBody)
}
id := resp.Header.Get("corro-query-id")
if id == "" {
resp.Body.Close()
return nil, errors.New("missing corro-query-id header in response")
}
if skipRows {
return newSubscription(ctx, id, nil, resp.Body, nil, c.resubscribeWithBackoffFn(id)), nil
}
rows, err := newRows(ctx, resp.Body, false)
if err != nil {
resp.Body.Close()
return nil, fmt.Errorf("parse query response: %w", err)
}
return newSubscription(ctx, id, rows, rows.body, rows.decoder, c.resubscribeWithBackoffFn(id)), nil
}
func (c *APIClient) resubscribeWithBackoffFn(id string) func(context.Context, uint64) (*Subscription, error) {
if c.newResubBackoff == nil {
return nil
}
return func(ctx context.Context, fromChange uint64) (*Subscription, error) {
return backoff.RetryWithData(func() (*Subscription, error) {
slog.Debug("Retrying to resubscribe to Corrosion query.", "id", id, "from_change", fromChange)
return c.ResubscribeContext(ctx, id, fromChange)
}, c.newResubBackoff())
}
}
func (c *APIClient) ResubscribeContext(ctx context.Context, id string, fromChange uint64) (*Subscription, error) {
subURL := c.baseURL.JoinPath("/v1/subscriptions", id)
q := subURL.Query()
q.Set("from", strconv.FormatUint(fromChange, 10))
subURL.RawQuery = q.Encode()
req, err := http.NewRequestWithContext(ctx, "GET", subURL.String(), nil)
if err != nil {
return nil, fmt.Errorf("create request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
resp, err := c.client.Do(req)
if err != nil {
return nil, fmt.Errorf("send request: %w", err)
}
if resp.StatusCode != http.StatusOK {
respBody, err := io.ReadAll(resp.Body)
resp.Body.Close()
if err != nil {
return nil, fmt.Errorf("read response body: %w", err)
}
return nil, fmt.Errorf("unexpected status code %d: %s", resp.StatusCode, respBody)
}
return newSubscription(ctx, id, nil, resp.Body, nil, c.resubscribeWithBackoffFn(id)), nil
}
+4 -3
View File
@@ -3,9 +3,10 @@ package daemon
import (
"context"
"fmt"
systemd "github.com/coreos/go-systemd/daemon"
"log/slog"
"uncloud/internal/machine"
systemd "github.com/coreos/go-systemd/daemon"
"github.com/psviderski/uncloud/internal/machine"
)
type Daemon struct {
@@ -35,7 +36,7 @@ func (d *Daemon) Run(ctx context.Context) error {
case <-d.machine.Started():
_, err := systemd.SdNotify(false, systemd.SdNotifyReady)
if err != nil {
slog.Error("Failed to notify systemd that the daemon is ready.", "error", err)
slog.Error("Failed to notify systemd that the daemon is ready.", "err", err)
}
case <-ctx.Done():
}
+4 -3
View File
@@ -5,8 +5,9 @@ import (
"fmt"
"net/netip"
"os"
"uncloud/internal/machine"
"uncloud/internal/machine/network"
"github.com/psviderski/uncloud/internal/machine"
"github.com/psviderski/uncloud/internal/machine/network"
)
// MachineToken returns the local machine's token that can be used for adding the machine to a cluster.
@@ -37,5 +38,5 @@ func MachineToken(dataDir string) (machine.Token, error) {
for i, ip := range ips {
endpoints[i] = netip.AddrPortFrom(ip, network.WireGuardPort)
}
return machine.NewToken(state.Network.PublicKey, endpoints), nil
return machine.NewToken(state.Network.PublicKey, publicIP, endpoints), nil
}
+34
View File
@@ -0,0 +1,34 @@
package dns
const (
RecordTypeA RecordType = "A"
RecordTypeAAAA RecordType = "AAAA"
)
type RecordType string
type DomainResponse struct {
Name string `json:"name,omitempty"`
Token string `json:"token,omitempty"`
}
type RecordRequest struct {
Name string `json:"name,omitempty"`
Type RecordType `json:"type,omitempty"`
Values []string `json:"values,omitempty"`
}
type RecordResponse struct {
RecordRequest
FQDN string `json:"fqdn,omitempty"`
}
type AuthErrorResponse struct {
Status int `json:"status,omitempty"`
Message string `json:"msg,omitempty"`
Data authErrorData `json:"data,omitempty"`
}
type authErrorData struct {
NoDomain bool `json:"noDomain,omitempty"`
}
+152
View File
@@ -0,0 +1,152 @@
package dns
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io"
"log/slog"
"net/http"
)
// The dns package code is based on https://github.com/acorn-io/runtime/blob/main/pkg/dns.
// Client handles interactions with the Uncloud DNS API service.
type Client interface {
// ReserveDomain calls Uncloud DNS to reserve a new domain. It returns the domain, a token for authentication,
// and an error.
ReserveDomain(endpoint string) (string, string, error)
// CreateRecords calls Uncloud DNS to create or update DNS records based on the supplied RecordRequests
// for the specified domain.
CreateRecords(endpoint, domain, token string, records []RecordRequest) ([]RecordResponse, error)
}
// ErrAuthNoDomain indicates that a request failed authentication because the domain was not found.
// If encountered, a new domain needs to be reserved.
var ErrAuthNoDomain = errors.New("the supplied domain failed authentication")
// NewClient creates a new AcornDNS client
func NewClient() Client {
return &client{
c: http.DefaultClient,
}
}
type client struct {
c *http.Client
}
func (c *client) ReserveDomain(endpoint string) (string, string, error) {
url := fmt.Sprintf("%s/%s", endpoint, "domains")
req, err := c.request(http.MethodPost, url, nil, "")
if err != nil {
return "", "", err
}
resp := &DomainResponse{}
err = c.do(req, resp)
if err != nil {
return "", "", err
}
return resp.Name, resp.Token, nil
}
func (c *client) CreateRecords(endpoint, domain, token string, records []RecordRequest) ([]RecordResponse, error) {
url := fmt.Sprintf("%s/domains/%s/records", endpoint, domain)
var resp []RecordResponse
for _, recordRequest := range records {
body, err := jsonBody(recordRequest)
if err != nil {
return resp, err
}
req, err := c.request(http.MethodPost, url, body, token)
if err != nil {
return resp, err
}
var recordResp RecordResponse
if err = c.do(req, &recordResp); err != nil {
return resp, err
}
resp = append(resp, recordResp)
}
return resp, nil
}
func (c *client) request(method string, url string, body io.Reader, token string) (*http.Request, error) {
req, err := http.NewRequest(method, url, body)
if err != nil {
return nil, err
}
req.Header.Add("Content-Type", "application/json")
if token != "" {
bearer := "Bearer " + token
req.Header.Add("Authorization", bearer)
}
return req, nil
}
func (c *client) do(req *http.Request, responseBody any) error {
slog.Debug("Making request to DNS service.", "method", req.Method, "url", req.URL)
resp, err := c.c.Do(req)
if err != nil {
return err
}
slog.Debug("Response code for request to DNS service.",
"method", req.Method, "url", req.URL, "code", resp.StatusCode)
// When err is nil, resp contains a non-nil resp.Body which must be closed.
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return fmt.Errorf("read response body: %w", err)
}
if resp.StatusCode == http.StatusUnauthorized {
var authError AuthErrorResponse
err = json.Unmarshal(body, &authError)
if err != nil {
return fmt.Errorf("unmarshal auth error response: %w", err)
}
if authError.Data.NoDomain {
return ErrAuthNoDomain
}
return errors.New("authentication failed")
}
if code := resp.StatusCode; code < 200 || code > 300 {
return fmt.Errorf("unexpected response status code: %d", code)
}
if responseBody != nil {
err = json.Unmarshal(body, responseBody)
if err != nil {
return fmt.Errorf("unmarshal response body (%s): %w", string(body), err)
}
}
return nil
}
func jsonBody(payload any) (io.Reader, error) {
buf := &bytes.Buffer{}
err := json.NewEncoder(buf).Encode(payload)
if err != nil {
return nil, err
}
return buf, nil
}
+54
View File
@@ -0,0 +1,54 @@
package docker
import (
"context"
"errors"
"fmt"
"log/slog"
"time"
"github.com/cenkalti/backoff/v4"
"github.com/docker/docker/client"
)
type Client struct {
*client.Client
}
// WaitDaemonReady waits for the Docker daemon to start and be ready to serve requests.
func WaitDaemonReady(ctx context.Context, cli *client.Client) error {
// Retry to ping the Docker daemon until it's ready or the context is canceled.
boff := backoff.WithContext(backoff.NewExponentialBackOff(
backoff.WithInitialInterval(100*time.Millisecond),
backoff.WithMaxInterval(1*time.Second),
backoff.WithMaxElapsedTime(0),
), ctx)
waitingLogged := false
ping := func() error {
_, err := cli.Ping(ctx)
if err == nil {
if waitingLogged {
slog.Info("Docker daemon is ready.")
}
return nil
}
if !client.IsErrConnectionFailed(err) {
return backoff.Permanent(fmt.Errorf("connect to Docker daemon: %w", err))
}
if !waitingLogged {
slog.Info("Waiting for Docker daemon to start and be ready.")
waitingLogged = true
}
return err
}
if err := backoff.Retry(ping, boff); err != nil {
if errors.Is(err, context.Canceled) {
return nil
}
return fmt.Errorf("ping Docker: %w", err)
}
return nil
}
+70
View File
@@ -0,0 +1,70 @@
package docker
import (
"context"
"errors"
"fmt"
"time"
"github.com/containerd/errdefs"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/image"
"github.com/docker/go-connections/nat"
)
// CreateContainerWithImagePull creates a container. If the image is missing, it pulls the image first.
func (cli *Client) CreateContainerWithImagePull(
ctx context.Context, name string, config *container.Config, hostConfig *container.HostConfig,
) (container.CreateResponse, error) {
resp, err := cli.ContainerCreate(ctx, config, hostConfig, nil, nil, name)
if err == nil || !errdefs.IsNotFound(err) {
return resp, err
}
pullCh, err := cli.PullImage(ctx, config.Image, image.PullOptions{})
if err != nil {
return resp, fmt.Errorf("pull image: %w", err)
}
// Drain the pull channel until it's closed (image is fully pulled) or an error occurs.
// If the context is canceled during the pull, the channel will receive a context cancellation error.
for msg := range pullCh {
if msg.Err != nil {
return resp, fmt.Errorf("pull image: %w", msg.Err)
}
}
// Create container again after image pull.
if resp, err = cli.ContainerCreate(ctx, config, hostConfig, nil, nil, name); err != nil {
return resp, err
}
return resp, nil
}
// WaitPortPublished waits for a container port to be published on the host which happens asynchronously.
func (cli *Client) WaitPortPublished(ctx context.Context, containerID string, port nat.Port) ([]nat.PortBinding, error) {
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
for {
c, err := cli.ContainerInspect(ctx, containerID)
if err != nil {
return nil, fmt.Errorf("inspect container: %w", err)
}
binding, ok := c.NetworkSettings.Ports[port]
if ok && len(binding) > 0 {
return binding, nil
}
select {
case <-time.After(10 * time.Millisecond):
case <-ctx.Done():
if errors.Is(ctx.Err(), context.DeadlineExceeded) {
return nil, errors.New("timeout")
}
return nil, ctx.Err()
}
}
}
+132
View File
@@ -0,0 +1,132 @@
package docker
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"os"
dockercommand "github.com/docker/cli/cli/command"
dockerconfig "github.com/docker/cli/cli/config"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/pkg/jsonmessage"
)
type PullPushImageMessage struct {
Message jsonmessage.JSONMessage
Err error
}
// PullImage pulls an image and returns a channel to receive progress messages.
func (cli *Client) PullImage(
ctx context.Context, image string, opts image.PullOptions,
) (<-chan PullPushImageMessage, error) {
if opts.RegistryAuth == "" {
// Try to retrieve the authentication token for the image from the default local Docker config file.
if encodedAuth, err := RetrieveLocalDockerRegistryAuth(image); err == nil {
opts.RegistryAuth = encodedAuth
}
}
respBody, err := cli.ImagePull(ctx, image, opts)
if err != nil {
return nil, err
}
ch := make(chan PullPushImageMessage)
go processPullPushImageResp(ctx, respBody, ch)
return ch, nil
}
// PushImage pushes an image and returns a channel to receive progress messages.
func (cli *Client) PushImage(
ctx context.Context, image string, opts image.PushOptions,
) (<-chan PullPushImageMessage, error) {
if opts.RegistryAuth == "" {
// Try to retrieve the authentication token for the image from the default local Docker config file.
encodedAuth, _ := RetrieveLocalDockerRegistryAuth(image)
if encodedAuth == "" {
// If no credentials are found, provide an encoded empty auth config to work around the bug in Docker:
// https://github.com/moby/moby/issues/50729
var err error
if encodedAuth, err = registry.EncodeAuthConfig(registry.AuthConfig{}); err != nil {
return nil, fmt.Errorf("encode empty auth config: %w", err)
}
}
opts.RegistryAuth = encodedAuth
}
respBody, err := cli.ImagePush(ctx, image, opts)
if err != nil {
return nil, err
}
ch := make(chan PullPushImageMessage)
go processPullPushImageResp(ctx, respBody, ch)
return ch, nil
}
// processPullPushImageResp decodes JSON messages from the image pull/push response body and
// sends them to the provided channel.
func processPullPushImageResp(ctx context.Context, respBody io.ReadCloser, ch chan<- PullPushImageMessage) {
defer respBody.Close()
defer close(ch)
decoder := json.NewDecoder(respBody)
for {
var jm jsonmessage.JSONMessage
if err := decoder.Decode(&jm); err != nil {
if errors.Is(err, io.EOF) {
return
}
ch <- PullPushImageMessage{Err: fmt.Errorf("decode image pull/push message: %w", err)}
return
}
msg := PullPushImageMessage{Message: jm}
if jm.Error != nil {
msg.Err = errors.New(jm.Error.Message)
}
select {
case <-ctx.Done():
ch <- PullPushImageMessage{Err: ctx.Err()}
return
default:
ch <- msg
}
}
}
// RetrieveLocalDockerRegistryAuth retrieves the authentication token for the specified image from the local Docker
// config file. It returns the encoded authentication token if it contains any credentials, or an empty string if
// no credentials are found.
func RetrieveLocalDockerRegistryAuth(image string) (string, error) {
// Try to retrieve the authentication token for the image from the default local Docker config file.
dockerConfig := dockerconfig.LoadDefaultConfigFile(os.Stderr)
encodedAuth, err := dockercommand.RetrieveAuthTokenFromImage(dockerConfig, image)
if err != nil {
return "", err
}
// The encodedAuth can be a base64-encoded "{}" (empty JSON object) or include a server address but no credentials.
// Return encodedAuth only if it contains any credentials.
auth, err := registry.DecodeAuthConfig(encodedAuth)
if err != nil {
return "", fmt.Errorf("decode auth config: %w", err)
}
if auth.Username == "" &&
auth.Password == "" &&
auth.Auth == "" &&
auth.IdentityToken == "" &&
auth.RegistryToken == "" {
return "", nil
}
return encodedAuth, nil
}
+73
View File
@@ -0,0 +1,73 @@
package fs
import (
"fmt"
"os"
"os/user"
"strconv"
"strings"
)
func ExpandHomeDir(path string) string {
if len(path) == 0 {
return path
}
if path[0] == '~' {
home, err := os.UserHomeDir()
if err != nil {
return path
}
return strings.Replace(path, "~", home, 1)
}
return path
}
// LookupUIDGID returns the user and group IDs for the given username.
func LookupUIDGID(username string) (uid, gid int, err error) {
usr, err := user.Lookup(username)
if err != nil {
err = fmt.Errorf("lookup user %q: %w", username, err)
return
}
uid, err = strconv.Atoi(usr.Uid)
if err != nil {
err = fmt.Errorf("parse %q user ID (UID) %q: %w", username, usr.Uid, err)
return
}
gid, err = strconv.Atoi(usr.Gid)
if err != nil {
err = fmt.Errorf("parse %q user group ID (GID) %q: %w", username, usr.Gid, err)
return
}
return
}
func Chown(path, username, group string) error {
uid, gid := -1, -1
if username != "" {
usr, err := user.Lookup(username)
if err != nil {
return fmt.Errorf("lookup user %q: %w", username, err)
}
uid, err = strconv.Atoi(usr.Uid)
if err != nil {
return fmt.Errorf("parse %q user ID (UID) %q: %w", username, usr.Uid, err)
}
}
if group != "" {
grp, err := user.LookupGroup(group)
if err != nil {
return fmt.Errorf("lookup group %q: %w", group, err)
}
gid, err = strconv.Atoi(grp.Gid)
if err != nil {
return fmt.Errorf("parse %q group ID (GID) %q: %w", group, grp.Gid, err)
}
}
if err := os.Chown(path, uid, gid); err != nil {
return fmt.Errorf("chown %q: %w", path, err)
}
return nil
}
+22
View File
@@ -0,0 +1,22 @@
package fs
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestExpandHomeDir(t *testing.T) {
t.Run("empty", func(t *testing.T) {
assert.Equal(t, "", ExpandHomeDir(""))
})
t.Run("no home", func(t *testing.T) {
assert.Equal(t, "/path", ExpandHomeDir("/path"))
})
t.Run("home", func(t *testing.T) {
t.Setenv("HOME", "/home/user")
assert.Equal(t, "/home/user/path", ExpandHomeDir("~/path"))
})
}
+82
View File
@@ -0,0 +1,82 @@
package log
import (
"context"
"fmt"
"io"
"log/slog"
"sync"
)
// SlogTextHandler extends the standard [slog.TextHandler] to provide custom formatting that is very similar
// to the default slog handler. It prefixes each log entry with a right-padded level indicator and message,
// while excluding the default time, level, and message attributes from the structured output.
//
// The output format is:
// LEVEL MESSAGE key1=value1 key2=value2
type SlogTextHandler struct {
*slog.TextHandler
opts slog.HandlerOptions
mu sync.Mutex
w io.Writer
}
func NewSlogTextHandler(w io.Writer, opts *slog.HandlerOptions) *SlogTextHandler {
if opts == nil {
opts = &slog.HandlerOptions{}
}
// Remove time, level, and message from the default attributes.
opts.ReplaceAttr = func(groups []string, a slog.Attr) slog.Attr {
if a.Key == slog.TimeKey || a.Key == slog.LevelKey || a.Key == slog.MessageKey {
return slog.Attr{}
}
return a
}
return &SlogTextHandler{
TextHandler: slog.NewTextHandler(w, opts),
opts: *opts,
w: w,
}
}
func (h *SlogTextHandler) Enabled(ctx context.Context, level slog.Level) bool {
return h.TextHandler.Enabled(ctx, level)
}
func (h *SlogTextHandler) WithAttrs(attrs []slog.Attr) slog.Handler {
return &SlogTextHandler{
TextHandler: h.TextHandler.WithAttrs(attrs).(*slog.TextHandler),
opts: h.opts,
w: h.w,
}
}
func (h *SlogTextHandler) WithGroup(name string) slog.Handler {
return &SlogTextHandler{
TextHandler: h.TextHandler.WithGroup(name).(*slog.TextHandler),
opts: h.opts,
w: h.w,
}
}
func (h *SlogTextHandler) Handle(ctx context.Context, r slog.Record) error {
// Only log entries that are at or above the minimum level (INFO by default).
minLevel := slog.LevelInfo
if h.opts.Level != nil {
minLevel = h.opts.Level.Level()
}
if r.Level < minLevel {
return nil
}
h.mu.Lock()
defer h.mu.Unlock()
levelMsg := fmt.Sprintf("%-5s %s ", r.Level.String(), r.Message)
if _, err := h.w.Write([]byte(levelMsg)); err != nil {
return err
}
return h.TextHandler.Handle(ctx, r)
}
+176
View File
@@ -0,0 +1,176 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.34.2
// protoc v5.27.3
// source: internal/machine/api/pb/caddy.proto
package pb
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
emptypb "google.golang.org/protobuf/types/known/emptypb"
timestamppb "google.golang.org/protobuf/types/known/timestamppb"
reflect "reflect"
sync "sync"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type GetCaddyConfigResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
// The generated Caddyfile content.
Caddyfile string `protobuf:"bytes,1,opt,name=caddyfile,proto3" json:"caddyfile,omitempty"`
// Timestamp when the config was last modified.
ModifiedAt *timestamppb.Timestamp `protobuf:"bytes,2,opt,name=modified_at,json=modifiedAt,proto3" json:"modified_at,omitempty"`
}
func (x *GetCaddyConfigResponse) Reset() {
*x = GetCaddyConfigResponse{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_caddy_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *GetCaddyConfigResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*GetCaddyConfigResponse) ProtoMessage() {}
func (x *GetCaddyConfigResponse) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_caddy_proto_msgTypes[0]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use GetCaddyConfigResponse.ProtoReflect.Descriptor instead.
func (*GetCaddyConfigResponse) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_caddy_proto_rawDescGZIP(), []int{0}
}
func (x *GetCaddyConfigResponse) GetCaddyfile() string {
if x != nil {
return x.Caddyfile
}
return ""
}
func (x *GetCaddyConfigResponse) GetModifiedAt() *timestamppb.Timestamp {
if x != nil {
return x.ModifiedAt
}
return nil
}
var File_internal_machine_api_pb_caddy_proto protoreflect.FileDescriptor
var file_internal_machine_api_pb_caddy_proto_rawDesc = []byte{
0x0a, 0x23, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69,
0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x70, 0x62, 0x2f, 0x63, 0x61, 0x64, 0x64, 0x79, 0x2e,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x03, 0x61, 0x70, 0x69, 0x1a, 0x1b, 0x67, 0x6f, 0x6f, 0x67,
0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x65, 0x6d, 0x70, 0x74,
0x79, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x1f, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61,
0x6d, 0x70, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x73, 0x0a, 0x16, 0x47, 0x65, 0x74, 0x43,
0x61, 0x64, 0x64, 0x79, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e,
0x73, 0x65, 0x12, 0x1c, 0x0a, 0x09, 0x63, 0x61, 0x64, 0x64, 0x79, 0x66, 0x69, 0x6c, 0x65, 0x18,
0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x63, 0x61, 0x64, 0x64, 0x79, 0x66, 0x69, 0x6c, 0x65,
0x12, 0x3b, 0x0a, 0x0b, 0x6d, 0x6f, 0x64, 0x69, 0x66, 0x69, 0x65, 0x64, 0x5f, 0x61, 0x74, 0x18,
0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d,
0x70, 0x52, 0x0a, 0x6d, 0x6f, 0x64, 0x69, 0x66, 0x69, 0x65, 0x64, 0x41, 0x74, 0x32, 0x49, 0x0a,
0x05, 0x43, 0x61, 0x64, 0x64, 0x79, 0x12, 0x40, 0x0a, 0x09, 0x47, 0x65, 0x74, 0x43, 0x6f, 0x6e,
0x66, 0x69, 0x67, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f,
0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x1b, 0x2e, 0x61, 0x70,
0x69, 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x64, 0x64, 0x79, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67,
0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x37, 0x5a, 0x35, 0x67, 0x69, 0x74, 0x68,
0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x70, 0x73, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73, 0x6b,
0x69, 0x2f, 0x75, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e,
0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x70,
0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
}
var (
file_internal_machine_api_pb_caddy_proto_rawDescOnce sync.Once
file_internal_machine_api_pb_caddy_proto_rawDescData = file_internal_machine_api_pb_caddy_proto_rawDesc
)
func file_internal_machine_api_pb_caddy_proto_rawDescGZIP() []byte {
file_internal_machine_api_pb_caddy_proto_rawDescOnce.Do(func() {
file_internal_machine_api_pb_caddy_proto_rawDescData = protoimpl.X.CompressGZIP(file_internal_machine_api_pb_caddy_proto_rawDescData)
})
return file_internal_machine_api_pb_caddy_proto_rawDescData
}
var file_internal_machine_api_pb_caddy_proto_msgTypes = make([]protoimpl.MessageInfo, 1)
var file_internal_machine_api_pb_caddy_proto_goTypes = []any{
(*GetCaddyConfigResponse)(nil), // 0: api.GetCaddyConfigResponse
(*timestamppb.Timestamp)(nil), // 1: google.protobuf.Timestamp
(*emptypb.Empty)(nil), // 2: google.protobuf.Empty
}
var file_internal_machine_api_pb_caddy_proto_depIdxs = []int32{
1, // 0: api.GetCaddyConfigResponse.modified_at:type_name -> google.protobuf.Timestamp
2, // 1: api.Caddy.GetConfig:input_type -> google.protobuf.Empty
0, // 2: api.Caddy.GetConfig:output_type -> api.GetCaddyConfigResponse
2, // [2:3] is the sub-list for method output_type
1, // [1:2] is the sub-list for method input_type
1, // [1:1] is the sub-list for extension type_name
1, // [1:1] is the sub-list for extension extendee
0, // [0:1] is the sub-list for field type_name
}
func init() { file_internal_machine_api_pb_caddy_proto_init() }
func file_internal_machine_api_pb_caddy_proto_init() {
if File_internal_machine_api_pb_caddy_proto != nil {
return
}
if !protoimpl.UnsafeEnabled {
file_internal_machine_api_pb_caddy_proto_msgTypes[0].Exporter = func(v any, i int) any {
switch v := v.(*GetCaddyConfigResponse); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_internal_machine_api_pb_caddy_proto_rawDesc,
NumEnums: 0,
NumMessages: 1,
NumExtensions: 0,
NumServices: 1,
},
GoTypes: file_internal_machine_api_pb_caddy_proto_goTypes,
DependencyIndexes: file_internal_machine_api_pb_caddy_proto_depIdxs,
MessageInfos: file_internal_machine_api_pb_caddy_proto_msgTypes,
}.Build()
File_internal_machine_api_pb_caddy_proto = out.File
file_internal_machine_api_pb_caddy_proto_rawDesc = nil
file_internal_machine_api_pb_caddy_proto_goTypes = nil
file_internal_machine_api_pb_caddy_proto_depIdxs = nil
}

Some files were not shown because too many files have changed in this diff Show More