Compare commits

...
210 Commits
Author SHA1 Message Date
Pasha Sviderski 0b32cd2095 chore: upgrade unregistry 0.3.1 -> 0.4.1 with a fix for pushing big images 2025-12-16 20:08:40 +10:00
Pasha Sviderski 425b974331 fix: panic when initialising new cluster with --connect 2025-12-16 11:28:11 +10:00
Pasha Sviderski db09dec855 chore: add more tests for MatchesDockerVolume and update error message when volume spec doesn't match 2025-12-15 18:12:12 +10:00
Justin BradfordandGitHub 20cdcdf917 fix: Set volume driver name to "local" if not specified (#219) 2025-12-15 17:50:36 +10:00
Pasha Sviderski 796363f23c fix: TestCollectContainers unit tests 2025-12-12 21:02:22 +10:00
Pasha Sviderski cb0df2169f feat: add IP ADDRESS column to 'uc ps' and 'uc inspect' 2025-12-12 20:23:43 +10:00
Pasha Sviderski 95afc7f289 chore: sort 'uc inspect' containers by CREATED 2025-12-12 20:13:02 +10:00
Pasha Sviderski bc65c73c51 chore: add CREATED column to 'uc ps', fallback sorting by CREATED 2025-12-12 20:01:49 +10:00
Pasha Sviderski 596bc561b8 chore: 'uc stop' - make --timeout default explicit, update flag and long descriptions 2025-12-12 19:16:17 +10:00
Pasha Sviderski f215fae6c7 chore: group deploy and service management commands in uc --help 2025-12-12 18:52:08 +10:00
Pasha Sviderski 5707c37b63 docs: replace managed DNS domain: cluster.uncloud.run -> uncld.dev 2025-12-12 18:21:11 +10:00
Pasha Sviderski 5780d3a8a4 docs: regenereate CLI reference 2025-12-10 20:48:28 +10:00
Pasha Sviderski ac18ff3e8b docs: add examples section to uc logs command 2025-12-10 20:47:42 +10:00
a9a720f3c3 feat: Add uc service start/stop commands (#195)
* feat: Add `uc service start/stop` commands

* Add nolint:dupl for service start/stop commands

Co-authored-by: Pasha Sviderski <me@psviderski.name>

* Add --signal and --timeout options to uc service stop

* Check whether service stop --timeout flag was used to set options properly

---------

Co-authored-by: Pasha Sviderski <me@psviderski.name>
2025-12-09 15:58:09 +10:00
4b4b721d49 feat(uc/caddy): expose 443/udp port for HTTP3 (#204)
* feat(caddy): expose 443/udp port for HTTP3

* expose udp port 443 for caddy to support new HTTP3

* chore: Add comment for HTTP/3 (QUIC) port configuration

Added a comment to clarify the purpose of the UDP port configuration.

---------

Co-authored-by: Anton Ovchinnikov <anton@tonyo.info>
2025-12-08 19:21:30 +10:00
Justin BradfordandGitHub 6373d4097c fix: volume driver options are not used if driver if not specified [bug #210] (#211) 2025-12-08 15:58:43 +10:00
Pasha Sviderski dfe12ca644 docs: add #1 repo of the day badge to README 2025-12-08 12:23:50 +10:00
Pasha Sviderski 2ef8a6fd20 docs: update landing value section 2025-12-05 00:28:37 +10:00
Pasha Sviderski 27cfe1c4f3 feat: 'uc logs' without argements streams logs for services in the local Compose file 2025-12-04 15:40:24 +10:00
Pasha Sviderski ebd4622592 lint 2025-12-04 14:26:25 +10:00
Pasha Sviderski 4fa30ab8cb test: e2e test for client.ServiceLogs 2025-12-04 14:24:10 +10:00
Pasha Sviderski 1e6aaf451e feat: support filtering service logs by machine (-m/--machine) 2025-12-03 15:01:23 +10:00
Pasha Sviderski bf65fbaca5 feat --utc flag for 'uc logs' command to print timestamps in UTC 2025-12-03 12:52:20 +10:00
79dc05cb66 feat: service logs command to stream service logs (#196)
* chore: fix landing logo shadow

* feat: implement service logs command with colored output and strict ordering

* revert Makefile

* fix after rebase

* move logs command under services with root shortcut

* simplify server options for streaming logs and CLI flags

* refactor ContainerLogs grpc server

* update --tail flag

* minor docker.proto

* refactor client ServiceLogs and ContainerLogs

* move ProxyMachinesContext from api to client pkg

* minor refactor proto Stream

* add api/logs

* implement LogMerger

* fix LogMerger to correctly use semaphore

* increate inflish entries to 100 per stream

* send heartbeats

* refactor ContainerLogs to synchronise Send of entries and heartbeats to the steram

* minor logmerege

* remove ContainerName form ServiceLogEntryMetadata

* refactor ContainerLogs into docker.Service

* detect stalled container logs streams

* update logmerger tests

* fix comment in test

* refactor LogMerger with options

* uc logs: format one or multiple services

* make LogMerger emit heartbeats, emit entries <= watermark, rewrite tests

* update uc logs with new LogMerger

* go mod tidy

* fix after merge

---------

Co-authored-by: Evgenii Orlov <evgenii.orlov@semrush.com>
2025-12-02 19:50:48 +10:00
Anton OvchinnikovandGitHub 234985b57d feat(install): Add install-only mode to the install script (#194) 2025-11-30 20:00:33 +01:00
Justin BradfordandGitHub d89bfb8e03 feat: add uc ps command to list all containers in the cluster (#165)
* feat: Add `uc ps` command to list all containers in the cluster

* Fix indentation

* Improvements to the container state highlights code (use enum, better var naming)

* Add group/sort option by "health"

* Rename `uc ps --group-by` flag to `--sort`. Also, remove `--context` flag.

* Use huh.spinner with standard style/type

* Remove pointless TrimPrefix on container name

* Move machine lookup out of services loop

* Use more efficient `ListServiceContainers`

* Use consistent header other

* Use api.ProxyMachinesContext helper for context to query all machines for service containers

* Make errored MachineServiceContainer entries a warning, not return an error immediately, in `uc ps`

* Use lipgloss table instead of tabwriter for "uc ps" command

* Handle nil metadata in `uc ps` on container when cluster only has one machine
2025-11-27 20:05:58 +10:00
91a09d5d37 feat: add CLI connector for unix domain socket (#186)
* feat: Add cli connector for unix domain socket

* Update cmd/uncloud/main.go

---------

Co-authored-by: Pasha Sviderski <me@psviderski.name>
2025-11-25 21:45:07 +10:00
Pasha Sviderski e87207eae3 chore: fix landing logo shadow 2025-11-24 09:40:36 +10:00
Anton Ovchinnikov 3c33973269 chore(exec): Update TODO regarding signal forwarding 2025-11-23 23:58:53 +01:00
Pasha Sviderski 3019377310 chore: update logo for docs 2025-11-21 22:41:17 +10:00
Pasha Sviderski 53def3a624 chore: update white color in logo-title-dark.svg 2025-11-21 22:38:13 +10:00
Pasha Sviderski eb23e956cb docs: README subtitle minor 2025-11-21 22:33:09 +10:00
Pasha Sviderski d029d4d0bf docs: update uncloud logo with better spacing 2025-11-21 22:26:29 +10:00
Pasha Sviderski 4433095045 chore: show links to docs and Discord for the root 'uc --help' command 2025-11-21 22:08:32 +10:00
Pasha Sviderski 05c8117f7b chore: add redirect from uncloud.run/discord to discord invite 2025-11-21 21:03:48 +10:00
Pasha Sviderski 3fbba6d540 chore: sort services in 'uc ls' output by name 2025-11-21 18:11:20 +10:00
Pasha Sviderski 439d956ac2 docs: Deploy an app: build secrets are not supported (resolves #181) 2025-11-20 17:32:37 +10:00
Pasha Sviderski 055cfae272 docs: Compose compatibility matrix: deploy.rollback_config and deploy.update_config are not supported (resolves #179) 2025-11-20 17:15:40 +10:00
Justin BradfordandGitHub 76b4369aa6 feat: store machine id on connection entries in Uncloud config so it can be removed with machine (#182)
* fix: Store machine id on connection entries so it can be removed when machine is

* Add "GetContextOverrideOrCurrent" helper method to cli and use in `uc machine rm` to clean up connections
2025-11-20 16:49:37 +10:00
Pasha Sviderski 74c7738bc3 docs: update 'uc service rm' description that volumes are preserved 2025-11-20 16:36:36 +10:00
Pasha Sviderski f6d75d507c lint 2025-11-20 16:04:06 +10:00
Pasha Sviderski 6a1cd7bf87 refactor: replace loadProjectFromContent with compose.LoadProjectFromContent 2025-11-20 16:03:47 +10:00
Pasha Sviderski 57205e1149 fix: new Compose deployment with volumes and --recreate flag (fixes #176) 2025-11-20 15:19:44 +10:00
Pasha Sviderski ce4a8cce72 docs: regenereate CLI reference 2025-11-20 10:53:37 +10:00
Justin BradfordandGitHub 75085b3929 refactor: Simplify dns e2e test using new ExecContainer functionality (#178)
* refactor: Simplify dns e2e test using new ExecContainer functionality

* Fix linter errors
2025-11-20 08:43:58 +10:00
Pasha Sviderski c489fcc63c docs: typo 2025-11-20 08:06:26 +10:00
Pasha Sviderski e30ea89990 chore: enable CGO to be able to build fsevent required by updated compose on macOS 2025-11-20 08:06:26 +10:00
Pasha Sviderski 9ddcbd6de0 docs: render CLI reference 2025-11-20 08:06:26 +10:00
Justin BradfordandGitHub 939645e144 refactor: Make --context cli flag a global option (#174)
* Make `--context` flag a global option and remove duplication

* Remove unused dns showOptions struct

* First pass of refactoring "active context" determination and validation

* Undo some refactoring in cli.go to keep diff more focused
2025-11-19 20:30:32 +10:00
Justin BradfordandGitHub 1d413734d8 feat: Interactive 'uc ctx connection` command to select preferred cluster connection (#170)
* feat: Interactive uc command to select preferred cluster connection

https://github.com/psviderski/uncloud/issues/125

* Simplify connection selection and reordering by using the index instead of a pointer

* Move connection reordering logic to a new SetDefaultConnection on context
2025-11-19 13:53:29 +10:00
JiraliteandGitHub 2e655d870c chore: extract version from latest redirect in install_cli.sh (#180) 2025-11-18 13:31:35 +10:00
Luis LavenaandGitHub eea81723b8 feat(ssh+cli): Support SSH CLI on machine init and add commands (#173)
* Add SSHCLIRemote type with buildSSHArgs

Implement's Executor interface (Run, Stream, Close)

* Add UseSSHCLI field to RemoteMachine struct

Allow us to differentiate regular ssh from ssh+cli.

* Parse ssh+cli:// scheme on machine init/add

* Support SSH CLI when provisioning

* Update docs to reflect ssh+cli:// support

* Revert removal of `init` examples

Fixes copy & pasta mistake when documenting the new ssh+cli connection
methods.
2025-11-17 20:11:31 +10:00
Pasha Sviderski bbdeedb44e docs: add Image tag format reference doc 2025-11-12 23:12:59 +10:00
Pasha Sviderski 3e64af910a docs: split 'Deploy to specific machines' and 'Deploy a global service' into separate guides 2025-11-12 18:35:00 +10:00
Pasha Sviderski fcdbf785bb docs: init Guides section 2025-11-12 15:54:29 +10:00
Pasha Sviderski 7ebcba7c36 docs: add 'Deploy an app' how-to guide, init 'Image tag format' reference 2025-11-12 15:54:05 +10:00
Pasha Sviderski 90d244fea8 chore: 'uc image push' fix long description 2025-11-12 09:28:54 +10:00
Pasha Sviderski 497b9483f6 chore: fix the examples for 'uc image push' command after changing default 2025-11-12 09:25:19 +10:00
Pasha Sviderski e33fd40033 docs: split Compose docs into reference and concepts 2025-11-11 23:06:21 +10:00
Pasha Sviderski 2b93a1c089 chore: update --container flag help that accepts ID prefix 2025-11-10 08:41:25 +10:00
Anton OvchinnikovandGitHub 15d4ef5518 feat: Allow container matching by ID prefix (#175) 2025-11-10 08:38:20 +10:00
Anton Ovchinnikov 2f5f515696 build: Bump go tests timeout to 15 min 2025-11-09 18:26:28 +01:00
Anton Ovchinnikov 30bb33f1a0 fix(exec): Handle context cancellation server-side 2025-11-09 18:19:41 +01:00
Anton Ovchinnikov 7bff706aa0 ref: Rewrite config tests with Exec 2025-11-09 17:37:07 +01:00
Pasha SviderskiandGitHub b0cd752619 docs: add line break before uncloud-recipes block 2025-11-07 21:53:46 +10:00
Pasha Sviderski 52dd0c71a5 docs: link uncloud-recipes repo in README 2025-11-07 21:47:41 +10:00
Pasha Sviderski 2d292ed053 chore: remove x-machines from website compose 2025-11-07 18:31:05 +10:00
Pasha Sviderski 9770151583 fix: 'uc deploy' push image to all machines if x-machines is not specified 2025-11-07 15:38:52 +10:00
Pasha Sviderski 89db50e8df chore: consistent casing 2025-11-07 14:29:27 +10:00
Pasha Sviderski 3ac5992e1d chore: use short container ID for stop operation in deploy plan 2025-11-07 13:53:30 +10:00
Pasha Sviderski 8c4f21ff81 docs: regenerate CLI reference docs 2025-11-07 13:29:30 +10:00
Pasha Sviderski 4de068c46e chore: pin corrosion version to the latest working version v0.2.2 2025-11-07 12:51:40 +10:00
Pasha Sviderski b6dfc2175f fix: reconfigure WireGuard peers when listed >=1 machines in cluster store (fixes #155) 2025-11-07 12:51:40 +10:00
Luis LavenaandGitHub 48d2239f5d Connect to remote SSH nodes using SSH CLI (#152)
* Connect to remote SSH nodes using SSH CLI

Replace Go-native SSH implementation with SSH CLI execution
to support diverse SSH configurations and agents. Implements
'uncloudd dial-stdio' subcommand that proxies gRPC connections
over stdin/stdout, similar to Docker's approach.

This change addresses compatibility issues with:
- SSH agents exposing many keys (1Password, causing "too many
  authentication failures")
- Tailscale SSH (which doesn't support advanced SSH channel
  types like direct-streamlocal)
- Custom SSH configurations in ~/.ssh/config

The dial-stdio approach reduces SSH feature requirements by
streaming the unix socket connection over stdin/stdout instead
of using SSH channel forwarding.

Changes:
- Add 'uncloudd dial-stdio' hidden subcommand for socket proxy
- Add SSHCLIConnector using ssh command + dial-stdio
- Update connection logic to use SSH CLI connector
- Maintain backward compatibility with SSHKeyFile config

Resolves #131

* Fix sshcli tests missing ConnectionTimeout

Introduced short connection timeout on the first change but forgot to
update tests to match.

* Add SSHCLI field and update MachineConnection String() format

- Add SSHCLI field to support ssh_cli YAML configuration
- Update String() to use URI-like format (ssh://, ssh+cli://, tcp://)

* Add Validate() method and tests for MachineConnection

- Add Validate() to ensure connection methods are mutually exclusive
- Add tests for validation and String() method

* Unify SSH connector configs to use SSHConnectorConfig

* Update connectCluster to support both SSH connector types

* Restore Go SSH connector as default for machine init/add

Revert provisionOrConnectRemoteMachine to use Go SSH connector:
- Root users: reuse SSH connection from provisioning
- Non-root users: establish new connection for group membership
- Remove SSH CLI as default connector

SSH CLI connector remains available via ssh_cli config field.

* Add sshCLIDialer with DialContext method

Implement proxy.ContextDialer for SSHCLIConnector using SSH -W flag.
Each dial spawns a new SSH process for TCP forwarding, enabling
independent connections separate from the gRPC dial-stdio connection.

* Implement SSHCLIConnector.Dialer() method

Return sshCLIDialer instead of error, enabling uc image push
functionality with SSHCLIConnector. Validates connector is
configured before returning dialer.

* Fix half-closing implementation matchin Docker's approach

* Use testify assertions for connection tests

* Allow ssh+cli:// to be used with --connect

This way I can skip the configuration file while testing things out,
and confirm it works correctly:

  $ unset SSH_AUTH_SOCK

  $ ./uncloud --connect ssh://provision@blatta11 machine ls
  Error: connect to cluster: connect to machine: SSH login to
    provision@blatta11:22: connect using SSH agent: connect to SSH
    agent: dial unix: missing address

  $ ./uncloud --connect ssh+cli://provision@blatta11 machine ls
  NAME       STATE   ADDRESS         PUBLIC IP   WIREGUARD ENDPOINTS                                                                                                                  MACHINE ID
  blatta11   Up      10.210.0.1/24   -           100.64.0.22:51820, ...

* Validates configuration before connecting to cluster

* Do not tie client constructor with real validation

No longer attempt to validate the connection when instantiating a new
client.

Later on we could validate it in different places.

* Cleanup test and remove AI-slop

There were some serious slop in those tests, so took the time to clean
them up and kept only the relevant ones.

There is some repetition between buildSSHArgs and buildDialArgs but can
be tackled at a later stage.

* Fix connection representation tests

Prefix connection with ssh ssh+cli respectively.

* Wait for stdout before returning

Missed copy & pasta from Docker dial-stdio implementation (this happens
when you stare at the code for too long that it burns your eyes).
2025-11-06 09:44:29 +10:00
Anton Ovchinnikov 06d988583d doc: Generate docs 2025-11-05 23:48:30 +01:00
Anton OvchinnikovandGitHub abea3e229e feat: "exec" command to start processes inside remote containers (#139) 2025-11-05 23:47:24 +01:00
Justin BradfordandGitHub 1c3f210848 bug: Use GITHUB_TOKEN if present in scripts/install_cli.sh (#169)
https://github.com/psviderski/uncloud/issues/163
2025-11-05 15:52:28 +10:00
Pasha Sviderski 722615cd9b chore: go mod tidy 2025-11-04 18:22:29 +10:00
Pasha Sviderski 809d1eca77 lint 2025-11-04 18:20:10 +10:00
Pasha Sviderski 6e4439bf49 chore: show container image for the remove container plan operation 2025-11-04 18:15:16 +10:00
Pasha Sviderski 882f2f5d03 feat(build,deploy): build service images using compose (bake/buildkit), push to cluster, and deploy 2025-11-04 18:14:42 +10:00
Pasha Sviderski dde23b5549 chore: increase timeout 5->30s for waiting for machines to be ready in ucind cluster 2025-11-04 12:40:15 +10:00
Pasha Sviderski c3ecd378f8 feat(build): --push and --push-registry flags to push build images to cluster or registries 2025-11-03 15:38:59 +10:00
Pasha Sviderski 6da8e98c62 chore: add TODO to refactor docker client method 2025-11-03 13:45:45 +10:00
Pasha Sviderski 4d591f9efc lint 2025-11-01 16:07:36 +10:00
Pasha Sviderski 26c3699a9e feat: process image templates in compose services, set default git-based tags if not specified 2025-11-01 16:06:52 +10:00
Pasha Sviderski 328ace1fd1 chore: implement image template processing using git repo state (not enabled) 2025-11-01 15:50:36 +10:00
Pasha Sviderski 91bf135a4f docs: minor README update 2025-11-01 14:01:23 +10:00
Pasha Sviderski e2b3260926 docs: update README unregistry is already implemented, not coming soon 2025-11-01 14:00:38 +10:00
Pasha Sviderski 179289b504 test: InspectGitState when git not available 2025-10-31 17:34:55 +10:00
Pasha Sviderski bd8a843972 chore: inspecrt git state without an error if git utility not available 2025-10-31 17:17:29 +10:00
Pasha Sviderski 7e366ff980 chore: add gitutils package to inspect local git repo 2025-10-31 15:57:43 +10:00
Pasha Sviderski d0c23fa840 lint: fix ineffassign 2025-10-30 21:01:18 +10:00
Pasha Sviderski 4bc47af05b refactor: strip project name from volume names without mangling project name 2025-10-30 20:54:15 +10:00
Pasha Sviderski 65c3e0a68d feat(build): print if no services to build, build dependencies --dep, check build configuration --check 2025-10-30 20:54:15 +10:00
Pasha Sviderski 5c4f49b685 feat: new cbuild (compose build) command using Compose implementation 2025-10-30 20:54:15 +10:00
Justin BradfordandGitHub ba6290d616 feat: add "nearest.<service>.internal" mode to internal DNS (#159)
* feat: Adding a "nearest.service.local" mode to internal DNS

* Pass listenAddr to internal DNS server (rather than calling into network.MachineIP)

Also adds an exact round-robin via "rr.{service}" mode. Not sure we want that over random or not, though.

* Remove the resolved count and rotating round-robin. "rr.name" mode is now the same as default with random shuffle.

* Add test on nearest internal DNS lookup

* Add docs on internal DNS order modes
2025-10-29 14:25:59 +10:00
Anton OvchinnikovandGitHub 6af7a9861d feat: Allow to match containers against truncated ID (#160) 2025-10-28 18:44:04 +01:00
Justin BradfordandGitHub 3b4bcb7932 feat: support service.gpus and reservations.devices (#156)
* feat: support compose.yaml `service.gpus` and `service.deploy.resources.reservations.devices` DeviceRequests to Docker container
* Fix lint error
* Adjust test comments to work with linter
* Rename DeviceRequests to DeviceReservations to be more consistent with compose nomenclature
* Device reservation changes are immutable and should trigger a container recreate rather than update
2025-10-27 16:49:38 +10:00
Luis LavenaandGitHub 0153e9f9e2 fix: progress percent overflow in image push (#153) 2025-10-22 14:52:57 +10:00
Pasha Sviderski 9a0b1a4ce3 chore: print (custom Caddy config) for service ENDPOINTS if x-caddy is used 2025-10-21 11:42:18 +10:00
Anton Ovchinnikov e251dd950b fix: Improve wording for containerd warning 2025-10-16 23:33:25 +02:00
Pasha Sviderski 40862df038 fix: compose variables are now interpolated from .env file (fixes #144) 2025-10-15 17:45:53 +10:00
Pasha Sviderski dc569175ef docs: add info that caddy could be deployed on specific machines 2025-10-13 20:40:11 +10:00
Pasha Sviderski 6ba98e4576 fix: 'uc image push' closed network connection error if takes >10s (fixes #141) 2025-10-13 20:19:46 +10:00
Pasha Sviderski 75b932b66e docs: regenerate CLI reference docs 2025-10-10 18:57:15 +10:00
Pasha Sviderski 5aa85ab501 fix(push): always provide encoded empty auth config to work around panic in Docker 2025-10-09 15:14:10 +10:00
Pasha Sviderski 521ccf2026 chore(push): run unregistry only if containerd image store enabled for docker 2025-10-09 12:48:51 +10:00
Pasha Sviderski e8b4dd0bd1 chore(push): fail 'image push' when containerd image store not used on target machine 2025-10-09 12:11:18 +10:00
Pasha Sviderski 265042e2c8 lint 2025-10-08 16:38:40 +10:00
Justin BradfordandGitHub 1c85112d4f docs: Rough notes for "services" docs with examples of internal DNS lookups and container ENV (#136) 2025-10-08 16:35:52 +10:00
Pasha Sviderski 0fc5032b27 feat(images): add IN USE column to highlight which images are used by any container 2025-10-08 15:43:54 +10:00
Pasha Sviderski 893ab1f9f5 fix: negotiate docker client API version in e2e test 2025-10-08 14:18:11 +10:00
Pasha Sviderski a30ed600b6 chore: update docker and compose Go dependencies to the latest versions 2025-10-08 13:56:58 +10:00
Pasha Sviderski a2c6cbe633 chore(images): filter images by name with wildcard pattern using Docker filter 2025-10-08 12:33:13 +10:00
Pasha Sviderski 1f9c2c873d feat(images): list images filtered by name 2025-10-07 20:39:31 +10:00
Pasha Sviderski 509c9eb7d3 chore(images): format platforms as pills, sort images by name 2025-10-07 19:14:40 +10:00
Pasha Sviderski fba03b3df7 feat(images): list image platforms and do not group images by machine 2025-10-07 17:06:52 +10:00
Pasha Sviderski c6d18f5d49 chore: NAME -> IMAGE column name 2025-10-07 15:12:05 +10:00
3e804d9ecc feat: set UNCLOUD_MACHINE_ID in service container ENV (#135)
* Set machine id in service container ENV

* Add machine id to clone of the container's env map to avoid exposing serialization format

* fix: panic on setting UNCLOUD_MACHINE_ID env var when Env is nil in the spec

---------

Co-authored-by: Pasha Sviderski <me@psviderski.name>
2025-10-07 14:12:00 +10:00
Pasha Sviderski 784d86d8e8 fix: test build for docker server 2025-10-03 21:10:02 +10:00
Pasha Sviderski 3bac1e61b9 chore: go mod tidy 2025-10-03 20:54:59 +10:00
Pasha Sviderski 6d2e100d63 feat(images): 'uc image ls' and 'uc images' (alias) commands to list images on machines 2025-10-03 20:53:30 +10:00
Pasha Sviderski cce81ecaad chore(images): simplifies ListImages service, server, and client to only list Docker images and not directly access containerd 2025-10-03 18:25:39 +10:00
Pasha Sviderski f8bf6bccd3 test: basic e2e test for ListImages 2025-10-03 16:55:20 +10:00
Pasha Sviderski 82fd9744bf chore: remove unused imports 2025-10-03 15:03:19 +10:00
Pasha Sviderski cdff036935 feat(images): gRPC client and server for ListImages 2025-10-03 14:23:18 +10:00
Pasha Sviderski 458d282357 chore: correct comment and add TODO to move machine resolution to grpc-proxy router 2025-10-02 20:12:45 +10:00
Pasha Sviderski 59074a275f chore(images): initialise Docker service with containerd client 2025-10-02 15:51:54 +10:00
Pasha Sviderski 92975eec51 feat(images): add gRPC proto ListImages 2025-10-02 12:50:53 +10:00
Pasha Sviderski e0d11a8abc chore: configure Docker to use containerd image store by default (closes #129) 2025-10-01 15:00:19 +10:00
Pasha Sviderski fa004d63bb chore: add firewall rule to allow cluster machines to push to other machines unregistry 2025-10-01 12:34:03 +10:00
Pasha Sviderski a217643ac9 chore: shorten socat proxy container progress message 2025-09-30 19:44:54 +10:00
Pasha Sviderski fc3297c7b7 chore: go mod tidy 2025-09-30 19:31:14 +10:00
Pasha Sviderski 684f0d38ee feat(push): '--platform all' to push image to all machines in cluster 2025-09-30 19:23:01 +10:00
Pasha Sviderski 332e7bca90 feat(push): --platform flag to push a specific platform of a multi-platform image 2025-09-30 18:59:03 +10:00
Pasha Sviderski ae458d260f chore: refactor duplication 2025-09-30 15:42:01 +10:00
Pasha Sviderski 6b758c810e docs: image push example 2025-09-30 15:24:40 +10:00
Pasha Sviderski ffed906cef feat(push): 'image push' to single or multiple machines via unregistry proxy 2025-09-30 15:18:26 +10:00
Pasha Sviderski bcc504b8cc chore: internal Docker client PushImage 2025-09-30 13:58:34 +10:00
Anton Ovchinnikov 782f4cb20b chore: Update AI.md with instructions around tests and docs 2025-09-29 00:02:58 +02:00
Pasha Sviderski aebd2ac232 docs: review and update Compose support matrix 2025-09-26 22:42:07 +10:00
Pasha Sviderski 2969b40ad4 chore: return a proxy dialer for SSH connections 2025-09-26 21:24:56 +10:00
Pasha Sviderski e0045c7570 chore: refactor gRPC docker client 2025-09-26 21:24:56 +10:00
Pasha Sviderski bc6f2fd49d chore: fix docker client CreateContainerWithImagePull, add WaitPortPublished 2025-09-26 21:24:56 +10:00
Pasha Sviderski cd25b973ea fix: docker client PullImage 2025-09-26 21:24:56 +10:00
Pasha Sviderski 27bbbe7186 chore: internal docker package with handy PullImage and CreateContainerWithImagePull methods 2025-09-26 21:24:56 +10:00
Pasha Sviderski 20cf98d42e fix: return an error if 'uc rm' tries to remove a service that doesn't exist 2025-09-26 21:24:56 +10:00
Anton OvchinnikovandGitHub 63c4de512e feat: Initial support for Compose configs (#116) 2025-09-26 21:24:10 +10:00
Anton Ovchinnikov 337c15de35 chore: Add node to Mise 2025-09-24 23:16:36 +02:00
Justin BradfordandGitHub 2d361d3ee6 Adding machine.service.internal dns entries (#123) 2025-09-24 20:06:54 +10:00
Pasha Sviderski 977fdf3cc3 chore: add Proxy for proxing local connections to a remote TCP address using the dialer 2025-09-24 19:52:37 +10:00
Pasha Sviderski 9f6880b701 feat(push): stub for 'image push' command and Dialer interface for cluster connectors 2025-09-23 16:54:47 +10:00
Pasha Sviderski 1a1afece7d fix: e2e test rebind the registry port to unoccupied 5001 2025-09-23 15:26:36 +10:00
Pasha Sviderski 08f233c5b1 fix: auto-detect containerd.sock path and conditionally start unregistry 2025-09-23 15:10:34 +10:00
Pasha Sviderski 9438654d99 docs: fix comment in schema.sql 2025-09-23 12:21:45 +10:00
Pasha Sviderski 60926f64cc feat: show IMAGE column when listing services (uc ls) #13 2025-09-22 14:03:45 +10:00
Pasha Sviderski b6b8286118 feat: run embedded unregistry on machine IP in uncloudd daemon 2025-09-19 20:43:20 +10:00
Pasha Sviderski 8f906cfd95 fix: do not auto-confirm deploy plan on CI (no TTY), require explicit --yes or UNCLOUD_AUTO_CONFIRM=true 2025-09-18 12:12:06 +10:00
Pasha Sviderski da03ed8ece docs: generate CLI reference docs 2025-09-17 15:11:36 +10:00
Pasha Sviderski 7239cd3a6b docs: clarify 'machine init' help, --context flag description, add examples 2025-09-17 15:10:59 +10:00
Pasha Sviderski 3db07ca2d4 feat: allow setting config and connection with UNCLOUD_CONFIG and UNCLOUD_CONNECT env vars 2025-09-16 10:17:44 +10:00
Pasha Sviderski f8fe3c7af3 feat: --no-install flag for 'machine init/add' to skip installing Uncloud daemon and dependencies #122 2025-09-15 20:44:35 +10:00
Pasha Sviderski b6e9766f40 lint: gRPC status.Error for static error messages 2025-09-15 17:11:12 +10:00
Pasha Sviderski 1f62e30a08 chore: update mise checksums for linux-x64 2025-09-13 22:02:11 +10:00
Pasha Sviderski 88f05ffd75 chore: update mise checksums for macos-arm64 2025-09-13 21:50:40 +10:00
Pasha Sviderski 375bb385a2 chore: bump golangci-lint to 2.4.0 on CI 2025-09-13 21:44:45 +10:00
Pasha Sviderski 51840e0366 chore: bump golangci-lint to 2.4.0 2025-09-13 21:20:48 +10:00
Pasha Sviderski 831c581e0f feat: try each cluster connection in order until one succeeds, add progress spinner 2025-09-13 21:12:20 +10:00
Pasha Sviderski e1734a1aa6 chore: bump Go to 1.25.1 2025-09-13 21:06:17 +10:00
Pasha Sviderski 84376bb34c chore: mise.lock include checksums for linux-x64 2025-09-12 23:18:45 +10:00
Pasha Sviderski 6d0b1dc1ee chore bump mise version on CI to 2025.9.6 2025-09-12 22:58:26 +10:00
Pasha Sviderski bc078472d4 feat: add -y|--yes flag for 'us deploy' to auto-confirm deployment plan 2025-09-12 22:52:06 +10:00
Pasha Sviderski 8a273dbf63 chore: rename cluster -> context option in commands for consistency 2025-09-11 20:21:07 +10:00
Pasha Sviderski 001acfc6a5 docs: remove --watch flag from compose file for deploying caddy 2025-09-10 17:32:15 +10:00
Pasha Sviderski 7b88b8b810 fix: write generated Caddyfile to disk only if succesfully loaded into local Caddy (means valid) 2025-09-10 16:59:31 +10:00
Pasha Sviderski 97bdb8eae1 fix: skip validation for user-defined Caddy configs if caddy not running locally 2025-09-10 15:04:45 +10:00
Pasha Sviderski 5baa8087e5 chore: load Caddy config via admin API instead of watching Caddyfile change on fs 2025-09-09 19:10:32 +10:00
Pasha Sviderski d25864e52f feat: automatically generate a unique 'default-N' context name for new cluster (fixes #113) 2025-09-08 20:35:17 +10:00
Anton Ovchinnikov ec2787c99c fix: Handle implicit relative path for config
Fixes #117
2025-09-07 15:58:09 +02:00
Anton Ovchinnikov 9963f9df2d chore: Update bug template 2025-09-07 15:13:15 +02:00
Anton Ovchinnikov 835834322b chore: Add GH bug report template 2025-09-07 15:02:59 +02:00
Pasha Sviderski 51ba3c7df8 fix: landing navbar on mobile 2025-09-04 18:15:52 +10:00
Pasha Sviderski 047f661462 docs: finish the Deploy demo app page 2025-09-03 20:42:26 +10:00
Pasha Sviderski 5a1e61ccff fix: broken links to completions docs in CLI reference 2025-08-28 20:15:54 +10:00
Pasha Sviderski db60a81b2d chore: lint 2025-08-28 20:09:48 +10:00
Pasha Sviderski 7c19323ea1 docs: 'uc docs' command to generate CLI reference docs 2025-08-28 20:09:18 +10:00
Pasha Sviderski 290e6db98e docs(ingress): Publishing service and Managing Caddy 2025-08-28 17:40:39 +10:00
Pasha Sviderski df29d9ba43 docs: Ingress & HTTP overview 2025-08-27 22:26:50 +10:00
Pasha Sviderski 64769081d9 fix: format for --publish flag 2025-08-27 22:25:32 +10:00
Pasha Sviderski 0107363d41 fix: allow host mode x-ports with x-caddy in compose 2025-08-26 19:15:09 +10:00
Pasha Sviderski 48dc1dd624 chore: delete unused image digest resolver 2025-08-25 20:36:56 +10:00
Pasha Sviderski 3a6eef410a feat: add --caddyfile flag to 'uc caddy deploy' and 'uc run' commands 2025-08-22 20:52:01 +10:00
Pasha Sviderski 9f5ca9a33d feat: add 'uc caddy config' command to show the current Caddyfile 2025-08-22 20:20:02 +10:00
Pasha Sviderski b046b78398 feat: migrate Caddy to generated Caddyfile, mount persistent data volume 2025-08-22 19:51:15 +10:00
Pasha Sviderski ff213e71d3 fix: unmarshaling of ServiceSpec in ServiceContainer struct 2025-08-22 17:25:28 +10:00
Pasha Sviderski 3cda5cc564 chore: caddy client to get caddy config 2025-08-22 15:58:21 +10:00
Pasha Sviderski 75fdbaf2f4 chore: relax ports+Caddy spec validation to allow host mode ports 2025-08-22 15:34:59 +10:00
Pasha Sviderski 81f4e3a67a chore: fix mockery for linux in .mise.lock 2025-08-21 17:31:43 +10:00
Pasha Sviderski e99e769455 chore: add Caddy gRPC service to retrieve Caddyfile config from machines 2025-08-21 17:04:42 +10:00
Pasha Sviderski 11949eeb3b chore: include validation errors for user-defined Caddy configs as a comment in Caddyfile 2025-08-21 14:01:24 +10:00
Pasha Sviderski b437659678 chore: update comments in generated Caddyfile 2025-08-21 13:14:54 +10:00
Pasha Sviderski c01365b416 chore: add header to generated Caddyfile that it's autogenerated 2025-08-21 08:54:06 +10:00
Pasha Sviderski 813c397644 chore: change reverse_proxy upstreams from 'to' to the directive arguments 2025-08-21 08:32:45 +10:00
Pasha Sviderski 5cc005a423 feat: validate and append custom per-service Caddy configs to generated Caddyfile 2025-08-21 08:18:59 +10:00
Pasha Sviderski 066d411367 chore: parse Created time on container with CreatedTime 2025-08-20 22:54:51 +10:00
Pasha Sviderski 03970862ab feat: concatenate custom Caddy configs for services into final Caddyfile (no upstream interpolation) 2025-08-19 22:01:48 +10:00
Pasha Sviderski 8bf9fc0c9c chore: introduce mockery for generating mocks, generate for CaddyfileValidator 2025-08-18 18:50:47 +10:00
Pasha Sviderski 93fef88fac chore: refactor Caddyfile generator to accept a validator 2025-08-18 18:48:32 +10:00
Pasha Sviderski 455174ccb0 chore: generate sites in Caddyfile from x-ports alongside caddy.json 2025-08-15 18:29:23 +10:00
249 changed files with 20018 additions and 2213 deletions
+36
View File
@@ -0,0 +1,36 @@
---
name: Bug Report
about: Create a report to help us improve
title: "[BUG] "
labels: bug
assignees: ""
---
**Describe the bug**
<!-- A clear and concise description of what the bug is. -->
**How to reproduce**
<!-- Steps to reproduce the behavior:
1. Run ...
2. Do ...
-->
**Expected behavior**
<!-- A clear and concise description of what you expected to happen. -->
**Environment:**
- Uncloud versions:
- Control (client) node (`uc --version`):
- Uncloud daemon (from the server) (`uncloudd --version`):
- OS version (`uname -a`):
- Client (control node):
- Server:
**Additional context**
<!-- Add any other context about the problem here. -->
+3 -3
View File
@@ -28,7 +28,7 @@ jobs:
- name: Set up Go - name: Set up Go
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0 uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with: with:
go-version: "1.23.2" go-version: "1.25.1"
- name: Install dependencies - name: Install dependencies
run: | run: |
@@ -40,7 +40,7 @@ jobs:
run: | run: |
make ucind-image make ucind-image
make test make test
timeout-minutes: 10 timeout-minutes: 15
check-protobuf: check-protobuf:
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
@@ -57,7 +57,7 @@ jobs:
- name: Install Mise - name: Install Mise
uses: jdx/mise-action@13abe502c30c1559a5c37dff303831bab82c9402 # v2.2.3 uses: jdx/mise-action@13abe502c30c1559a5c37dff303831bab82c9402 # v2.2.3
with: with:
version: "2025.6.5" version: "2025.9.6"
env: env:
GITHUB_TOKEN: ${{ github.token }} GITHUB_TOKEN: ${{ github.token }}
+2 -2
View File
@@ -23,12 +23,12 @@ jobs:
- name: Set up Go - name: Set up Go
uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0 uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with: with:
go-version: "1.23.2" go-version: "1.25.1"
- name: golangci-lint - name: golangci-lint
uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0 uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0
with: with:
version: v2.2.2 version: v2.4.0
- name: Format code - name: Format code
run: | run: |
+2 -1
View File
@@ -16,7 +16,8 @@ builds:
main: ./cmd/uncloud main: ./cmd/uncloud
binary: uncloud binary: uncloud
env: env:
- CGO_ENABLED=0 # Use CGO to be able to build fsevent required by compose on macOS.
- CGO_ENABLED={{ if eq .Os "darwin" }}1{{ else }}0{{ end }}
goos: goos:
- darwin - darwin
- linux - linux
+80 -22
View File
@@ -1,39 +1,97 @@
[tools.go] [[tools."aqua:vektra/mockery"]]
version = "1.23.10" version = "3.5.3"
backend = "aqua:vektra/mockery"
[tools."aqua:vektra/mockery".platforms.linux-x64]
checksum = "sha256:ebce416b0175338525246c376885a1579ca6cd4d4015140ba0c70e6b5339a39c"
size = 5498859
url = "https://github.com/vektra/mockery/releases/download/v3.5.3/mockery_3.5.3_Linux_x86_64.tar.gz"
[tools."aqua:vektra/mockery".platforms.macos-arm64]
checksum = "sha256:a3a94b14c7414e148f2252199ffc4a0108d311358f3d336cbe05bb73cb203704"
size = 5238877
url = "https://github.com/vektra/mockery/releases/download/v3.5.3/mockery_3.5.3_Darwin_arm64.tar.gz"
[[tools.go]]
version = "1.25.1"
backend = "core:go" backend = "core:go"
[tools.go.checksums] [tools.go.platforms.linux-x64]
"go1.23.10.darwin-arm64.tar.gz" = "sha256:25c64bfa8a8fd8e7f62fb54afa4354af8409a4bb2358c2699a1003b733e6fce5" checksum = "sha256:7716a0d940a0f6ae8e1f3b3f4f36299dc53e31b16840dbd171254312c41ca12e"
"go1.23.10.linux-amd64.tar.gz" = "sha256:535f9f81802499f2a7dbfa70abb8fda3793725fcc29460f719815f6e10b5fd60" size = 59660846
url = "https://dl.google.com/go/go1.25.1.linux-amd64.tar.gz"
[tools.golangci-lint] [tools.go.platforms.macos-arm64]
version = "2.2.2" checksum = "sha256:68deebb214f39d542e518ebb0598a406ab1b5a22bba8ec9ade9f55fb4dd94a6c"
size = 57906702
url = "https://dl.google.com/go/go1.25.1.darwin-arm64.tar.gz"
[[tools.golangci-lint]]
version = "2.4.0"
backend = "aqua:golangci/golangci-lint" backend = "aqua:golangci/golangci-lint"
[tools.golangci-lint.checksums] [tools.golangci-lint.platforms.linux-x64]
"golangci-lint-2.2.2-darwin-arm64.tar.gz" = "sha256:d84d94d042c0d495fd1746f3d18948a75de163b17a14e8de3ef840928dd2df74" checksum = "sha256:fae792524c04424c0ac369f5b8076f04b45cf29fc945a370e55d369a8dc11840"
"golangci-lint-2.2.2-linux-amd64.tar.gz" = "sha256:c27fbde948a87d326feacd21df2f61a9c54dbd2e3bfa185c0a1cd6917a6f964f" size = 14126779
url = "https://github.com/golangci/golangci-lint/releases/download/v2.4.0/golangci-lint-2.4.0-linux-amd64.tar.gz"
[tools.protoc] [tools.golangci-lint.platforms.macos-arm64]
checksum = "sha256:cd4dd53fa09b6646baff5fd22b8c64d91db02c21c7496df27992d75d34feec59"
size = 13542526
url = "https://github.com/golangci/golangci-lint/releases/download/v2.4.0/golangci-lint-2.4.0-darwin-arm64.tar.gz"
[[tools.node]]
version = "22.20.0"
backend = "core:node"
[tools.node.platforms.linux-x64]
checksum = "sha256:eeaccb0378b79406f2208e8b37a62479c70595e20be6b659125eb77dd1ab2a29"
size = 56645685
url = "https://nodejs.org/dist/v22.20.0/node-v22.20.0-linux-x64.tar.gz"
[tools.node.platforms.macos-arm64]
checksum = "sha256:cc04a76a09f79290194c0646f48fec40354d88969bec467789a5d55dd097f949"
size = 49838299
url = "https://nodejs.org/dist/v22.20.0/node-v22.20.0-darwin-arm64.tar.gz"
[[tools.protoc]]
version = "27.3" version = "27.3"
backend = "aqua:protocolbuffers/protobuf/protoc" backend = "aqua:protocolbuffers/protobuf/protoc"
[tools.protoc.checksums] [tools.protoc.platforms.linux-x64]
"protoc-27.3-linux-x86_64.zip" = "sha256:6dab2adab83f915126cab53540d48957c40e9e9023969c3e84d44bfb936c7741" checksum = "blake3:8c2938accf7061164ea3968529a47ced1db4824ca553857f6903c68ae4347739"
"protoc-27.3-osx-aarch_64.zip" = "sha256:b22116bd97cdbd7ea25346abe635a9df268515fe5ef5afa93cd9a68fc2513f84" size = 3235873
url = "https://github.com/protocolbuffers/protobuf/releases/download/v27.3/protoc-27.3-linux-x86_64.zip"
[tools.protoc-gen-go] [tools.protoc.platforms.macos-arm64]
checksum = "blake3:650c2494c003e9fddd6f4164a0a36e31da51ae7e1e5208da8cb560a4fe886fcd"
size = 2372763
url = "https://github.com/protocolbuffers/protobuf/releases/download/v27.3/protoc-27.3-osx-aarch_64.zip"
[[tools.protoc-gen-go]]
version = "1.34.2" version = "1.34.2"
backend = "aqua:protocolbuffers/protobuf-go/protoc-gen-go" backend = "aqua:protocolbuffers/protobuf-go/protoc-gen-go"
[tools.protoc-gen-go.checksums] [tools.protoc-gen-go.platforms.linux-x64]
"protoc-gen-go.v1.34.2.darwin.arm64.tar.gz" = "sha256:17aca7f948dbb624049030cf841e35895cf34183ba006e721247fdeb95ff2780" checksum = "blake3:f765e393e0f0b25070c13848d88f5e6303e66090132afa1b753e981507a4174e"
"protoc-gen-go.v1.34.2.linux.amd64.tar.gz" = "sha256:b87bc134dee55576a842141bf0ed27761c635d746780fce5dee038c6dd16554f" size = 1848331
url = "https://github.com/protocolbuffers/protobuf-go/releases/download/v1.34.2/protoc-gen-go.v1.34.2.linux.amd64.tar.gz"
[tools.protoc-gen-go-grpc] [tools.protoc-gen-go.platforms.macos-arm64]
checksum = "blake3:c237256a7f718ac4ccdc037ee30025a0b7c3f79300d7831ec59d76d782443fd5"
size = 1784028
url = "https://github.com/protocolbuffers/protobuf-go/releases/download/v1.34.2/protoc-gen-go.v1.34.2.darwin.arm64.tar.gz"
[[tools.protoc-gen-go-grpc]]
version = "1.5.1" version = "1.5.1"
backend = "aqua:grpc/grpc-go/protoc-gen-go-grpc" backend = "aqua:grpc/grpc-go/protoc-gen-go-grpc"
[tools.protoc-gen-go-grpc.checksums] [tools.protoc-gen-go-grpc.platforms.linux-x64]
"protoc-gen-go-grpc.v1.5.1.darwin.arm64.tar.gz" = "sha256:d6083feb51dcfe59f26793e99ef01ef5eac68b64294ec2546711f614ac5878f3" checksum = "blake3:197739132f0408f017d0c83773ac1ace85c0572473fdd658ec6e4ccbf20e0552"
"protoc-gen-go-grpc.v1.5.1.linux.amd64.tar.gz" = "sha256:a6cac4ea731e54aea304ad44d704a69d1cdc82997084b33637e21a89dc9229d6" size = 4859466
url = "https://github.com/grpc/grpc-go/releases/download/cmd/protoc-gen-go-grpc/v1.5.1/protoc-gen-go-grpc.v1.5.1.linux.amd64.tar.gz"
[tools.protoc-gen-go-grpc.platforms.macos-arm64]
checksum = "blake3:3c668207565d23ea2cf006d699207e3a804ebb929a534bd38a3b4ff4e941e00c"
size = 4565944
url = "https://github.com/grpc/grpc-go/releases/download/cmd/protoc-gen-go-grpc/v1.5.1/protoc-gen-go-grpc.v1.5.1.darwin.arm64.tar.gz"
+4 -2
View File
@@ -2,8 +2,10 @@
experimental = true experimental = true
[tools] [tools]
go = "1.23" "aqua:vektra/mockery" = "3.5.3"
golangci-lint = "2.2.2" go = "1.25.1"
golangci-lint = "2.4.0"
node = "22.20.0"
protoc = "27.3" protoc = "27.3"
protoc-gen-go = "1.34.2" protoc-gen-go = "1.34.2"
protoc-gen-go-grpc = "1.5.1" protoc-gen-go-grpc = "1.5.1"
+4
View File
@@ -0,0 +1,4 @@
packages:
github.com/psviderski/uncloud/internal/machine/caddyconfig:
interfaces:
CaddyfileValidator:
+19 -5
View File
@@ -28,7 +28,7 @@ This document provides comprehensive information about the Uncloud project for A
- **Docker Compose compatibility**: Uses familiar Docker Compose format - **Docker Compose compatibility**: Uses familiar Docker Compose format
- **Zero-downtime deployments**: Rolling updates without service interruption - **Zero-downtime deployments**: Rolling updates without service interruption
- **Automatic HTTPS**: Caddy reverse proxy with Let's Encrypt integration - **Automatic HTTPS**: Caddy reverse proxy with Let's Encrypt integration
- **Managed DNS**: Free `*.cluster.uncloud.run` subdomains via Uncloud DNS service - **Managed DNS**: Free `*.xxxxxx.uncld.dev` subdomains via Uncloud DNS service
- **Cross-machine scaling**: Run containers across multiple machines - **Cross-machine scaling**: Run containers across multiple machines
### 🔧 Developer Experience ### 🔧 Developer Experience
@@ -88,6 +88,10 @@ This document provides comprehensive information about the Uncloud project for A
- **`scripts/`**: Installation and utility scripts - **`scripts/`**: Installation and utility scripts
- **`test/`**: Test suites and test infrastructure - **`test/`**: Test suites and test infrastructure
- **`website/`**: Documentation website (Docusaurus) - **`website/`**: Documentation website (Docusaurus)
- `landing/`: Landing page
- `docs/`: User documentation
- **`misc/`**: Design documents and guides - **`misc/`**: Design documents and guides
## Key Technologies ## Key Technologies
@@ -184,9 +188,11 @@ uc context use <name> # Switch context
### Testing ### Testing
- Unit tests alongside source files (`*_test.go`) - Test files and locations
- Integration tests in `test/e2e/` - Unit tests alongside source files (`*_test.go`)
- Test fixtures in `test/fixtures/` - Integration tests in `test/e2e/`
- Test fixtures in `test/fixtures/`
- Use table driven tests whenever possible
### Dependencies ### Dependencies
@@ -233,7 +239,7 @@ uc context use <name> # Switch context
- `internal/machine/machine.go`: Core machine management - `internal/machine/machine.go`: Core machine management
- `pkg/api/`: Public API definitions - `pkg/api/`: Public API definitions
- `misc/design.md`: Architecture and design philosophy - `misc/design.md`: Architecture and design philosophy
- `README.md`: User-facing documentation - `README.md`: Repository README
### Configuration Files ### Configuration Files
@@ -242,3 +248,11 @@ uc context use <name> # Switch context
- `Dockerfile`: Container build instructions forUncloud-in-Docker (used for testing) - `Dockerfile`: Container build instructions forUncloud-in-Docker (used for testing)
This document should help AI assistants understand the project structure, make informed suggestions, and contribute effectively to the Uncloud codebase. This document should help AI assistants understand the project structure, make informed suggestions, and contribute effectively to the Uncloud codebase.
## Documentation
Instructions when generating documentation:
- Use conversational language write as if you were speaking to a friend.
- Keep sentences simple, optimize for clarity and understanding.
- Place the subject before the action whenever possible. Example: prefer “The function loads data” over “Data is loaded by the function.”
+2 -2
View File
@@ -1,6 +1,6 @@
ARG ALPINE_VERSION=3.20.3 ARG ALPINE_VERSION=3.20.3
FROM golang:1.23.2-alpine AS uncloudd FROM golang:1.25.1-alpine AS uncloudd
ARG TARGETOS ARG TARGETOS
ARG TARGETARCH ARG TARGETARCH
@@ -13,7 +13,7 @@ COPY go.mod go.sum ./
RUN go mod download && go mod verify RUN go mod download && go mod verify
COPY . . COPY . .
RUN GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build -o uncloudd cmd/uncloudd/main.go RUN GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build -o uncloudd ./cmd/uncloudd
FROM alpine:${ALPINE_VERSION} AS corrosion-download FROM alpine:${ALPINE_VERSION} AS corrosion-download
+8 -1
View File
@@ -38,7 +38,7 @@ demo-reset:
.PHONY: ucind-cluster .PHONY: ucind-cluster
ucind-cluster: ucind-cluster:
go run ./cmd/ucind cluster rm && go run ./cmd/ucind cluster create -m 3 go run ./cmd/ucind cluster rm && go run ./cmd/ucind cluster create -m $(if $(MACHINES_COUNT),$(MACHINES_COUNT),3)
.PHONY: proto .PHONY: proto
proto: proto:
@@ -66,6 +66,10 @@ ucind-image:
ucind-multiarch-image-push: ucind-multiarch-image-push:
docker buildx build --push --platform linux/amd64,linux/arm64 -t "$(UCIND_IMAGE)" --target ucind . docker buildx build --push --platform linux/amd64,linux/arm64 -t "$(UCIND_IMAGE)" --target ucind .
.PHONY: mocks
mocks:
@mockery
.PHONY: test .PHONY: test
test: test:
ifeq ($(TEST_NAME),) ifeq ($(TEST_NAME),)
@@ -111,3 +115,6 @@ _lint:
# Uncloud daemon won't likely support OS other than Linux anytime soon, so for now we can rely on that. # Uncloud daemon won't likely support OS other than Linux anytime soon, so for now we can rely on that.
GOOS=linux golangci-lint run $(ARGS) GOOS=linux golangci-lint run $(ARGS)
.PHONY: cli-docs
cli-docs:
go run ./cmd/uncloud docs
+17 -10
View File
@@ -1,7 +1,7 @@
<div align="center"> <div align="center">
<img src="./website/landing/images/logo.svg" height="100" width="100" alt="Uncloud logo"/> <img src="./website/landing/images/logo-title.svg#gh-light-mode-only" alt="Uncloud logo"/>
<h1>Uncloud</h1> <img src="./website/landing/images/logo-title-dark.svg#gh-dark-mode-only" alt="Uncloud logo"/>
<p><strong>Docker simplicity. Multi-machine power.</strong></p> <p><strong>Docker simplicity. Multi-machine power</strong></p>
<p> <p>
<a href="https://uncloud.run/docs"><img src="https://img.shields.io/badge/Docs-blue.svg?style=for-the-badge&logo=gitbook&logoColor=white" alt="Documentation"></a> <a href="https://uncloud.run/docs"><img src="https://img.shields.io/badge/Docs-blue.svg?style=for-the-badge&logo=gitbook&logoColor=white" alt="Documentation"></a>
@@ -25,12 +25,14 @@ complexity of Kubernetes.
## ✨ Features ## ✨ Features
* **Deploy anywhere**: Combine cloud VMs, dedicated servers, and bare metal into a unified computing environment * **Deploy anywhere**: Combine cloud VMs, dedicated servers, and bare metal into a unified computing environment,
regardless of location or provider. regardless of location or provider.
* **Docker Compose**: Familiar [Docker Compose](https://compose-spec.io/) format for defining services and volumes. No * **Docker Compose**: Familiar [Docker Compose](https://compose-spec.io/) format for defining services and volumes. No
need to learn a new bespoke DSL. need to learn a new bespoke DSL.
* **Zero-downtime deployments**: Rolling updates without service interruption. Automatic rollback on failure is coming * **Zero-downtime deployments**: Rolling updates without service interruption. Automatic rollback on failure is coming
soon. soon.
* **[Unregistry](https://github.com/psviderski/unregistry) integration**: Build and push your Docker images directly to
your machines without an external registry. It will transfer only the missing layers, making it fast and efficient.
* **Service discovery**: Built-in DNS server resolves service names to container IPs. * **Service discovery**: Built-in DNS server resolves service names to container IPs.
* **Persistent storage**: Run stateful services with Docker volumes managed across machines. * **Persistent storage**: Run stateful services with Docker volumes managed across machines.
* **Zero-config private network**: Automatic WireGuard mesh with peer discovery and NAT traversal. Containers get unique * **Zero-config private network**: Automatic WireGuard mesh with peer discovery and NAT traversal. Containers get unique
@@ -38,18 +40,13 @@ complexity of Kubernetes.
* **No control plane**: Fully decentralised design eliminates single points of failure and reduces operational overhead. * **No control plane**: Fully decentralised design eliminates single points of failure and reduces operational overhead.
* **Imperative over declarative**: Favoring imperative operations over state reconciliation simplifies both the mental * **Imperative over declarative**: Favoring imperative operations over state reconciliation simplifies both the mental
model and troubleshooting. model and troubleshooting.
* **Managed DNS**: Automatic DNS records `*.<id>.cluster.uncloud.run` for services with public access via managed * **Managed DNS**: Automatic DNS records `*.xxxxxx.uncld.dev` for services with public access via managed
[Uncloud DNS](https://github.com/psviderski/uncloud-dns) service. [Uncloud DNS](https://github.com/psviderski/uncloud-dns) service.
* **Automatic HTTPS**: Built-in Caddy reverse proxy handles TLS certificate provisioning and renewal using Let's * **Automatic HTTPS**: Built-in Caddy reverse proxy handles TLS certificate provisioning and renewal using Let's
Encrypt. Encrypt.
* **Docker-like CLI**: Familiar commands for managing both infrastructure and applications. * **Docker-like CLI**: Familiar commands for managing both infrastructure and applications.
* **Remote management**: Control your entire infrastructure through SSH access to any single machine in the cluster. * **Remote management**: Control your entire infrastructure through SSH access to any single machine in the cluster.
### 🚀 Coming soon
* **[Unregistry](https://github.com/psviderski/unregistry) integration**: Push your Docker images directly to your
machines without an external registry. It will transfer only the missing layers, making it fast and efficient.
## 🎬 Quick demo ## 🎬 Quick demo
The screenshot below demonstrates how I use Uncloud to deploy https://uncloud.run website to 2 remote machines from The screenshot below demonstrates how I use Uncloud to deploy https://uncloud.run website to 2 remote machines from
@@ -67,6 +64,12 @@ on-premises in just a couple minutes.
<img src="https://embed-ssl.wistia.com/deliveries/3cf7014a48b93afc556444bed3e39a8c.jpg?image_crop_resized=900x526&image_play_button_rounded=true&image_play_button_size=2x&image_play_button_color=18181Be0" alt="Uncloud demo" width="450" height="263" /> <img src="https://embed-ssl.wistia.com/deliveries/3cf7014a48b93afc556444bed3e39a8c.jpg?image_crop_resized=900x526&image_play_button_rounded=true&image_play_button_size=2x&image_play_button_color=18181Be0" alt="Uncloud demo" width="450" height="263" />
</a> </a>
<br>
<br>
> **📚 Want more examples?** Check out the [**uncloud-recipes**](https://github.com/psviderski/uncloud-recipes)
> repository for community recipes and templates for deploying popular services on Uncloud.
## 💫 Why Uncloud? ## 💫 Why Uncloud?
Modern cloud platforms like Heroku and Render offer amazing developer experiences but at a premium price. Traditional Modern cloud platforms like Heroku and Render offer amazing developer experiences but at a premium price. Traditional
@@ -334,6 +337,10 @@ These companies and projects are helping Uncloud with their generous sponsorship
## ❤️ Contributors ## ❤️ Contributors
<a href="https://trendshift.io/repositories/14069">
<img alt="Trendshift" src="https://trendshift.io/api/badge/repositories/14069" />
</a>
Thank you [@cedws](https://github.com/cedws) for being the first contributor to Uncloud! 🎉 Thank you [@cedws](https://github.com/cedws) for being the first contributor to Uncloud! 🎉
<a href="https://github.com/psviderski/uncloud/graphs/contributors"> <a href="https://github.com/psviderski/uncloud/graphs/contributors">
+67 -33
View File
@@ -10,68 +10,102 @@ import (
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
// NewBuildCommand creates a new command to build services from a Compose file. type buildOptions struct {
cli.BuildServicesOptions
files []string
profiles []string
}
// NewBuildCommand creates a new command to build images for services from a Compose file.
func NewBuildCommand() *cobra.Command { func NewBuildCommand() *cobra.Command {
opts := cli.BuildOptions{} opts := buildOptions{}
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "build [FLAGS] [SERVICE...]", Use: "build [FLAGS] [SERVICE...]",
Short: "Build services from a Compose file.", Short: "Build services from a Compose file.",
Long: `Build images for services from a Compose file using local Docker.
By default, built images remain on the local Docker host. Use --push to upload them
to cluster machines or --push-registry to upload them to external registries.`,
Example: ` # Build all services that have a build section in compose.yaml.
uc build
# Build specific services that have a build section.
uc build web api
# Build services and push images to all cluster machines or service x-machines if specified.
uc build --push
# Build services and push images to specific machines.
uc build --push -m machine1,machine2
# Build services and push images to external registries (e.g., Docker Hub).
uc build --push-registry
# Build services with build arguments, pull newer base images before building, and don't use cache.
uc build --build-arg NODE_VERSION=24 --build-arg ENV=production --no-cache --pull`,
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
if len(args) > 0 {
opts.Services = args opts.Services = args
}
return runBuild(cmd.Context(), uncli, opts) return runBuild(cmd.Context(), uncli, opts)
}, },
GroupID: "service",
} }
cmd.Flags().StringSliceVarP(&opts.Files, "file", "f", nil, cmd.Flags().StringArrayVar(&opts.BuildArgs, "build-arg", nil,
"One or more Compose files to build (default compose.yaml)") "Set a build-time variable for services. Used in Dockerfiles that declare the variable with ARG.\n"+
cmd.Flags().StringSliceVarP(&opts.Profiles, "profile", "p", nil, "Can be specified multiple times. Format: --build-arg VAR=VALUE")
cmd.Flags().BoolVar(&opts.Check, "check", false,
"Check the build configuration for services without building them.")
cmd.Flags().BoolVar(&opts.Deps, "deps", false,
"Also build services declared as dependencies of the selected services.")
cmd.Flags().StringSliceVarP(&opts.files, "file", "f", nil,
"One or more Compose files to build. (default compose.yaml)")
cmd.Flags().StringSliceVarP(&opts.Machines, "machine", "m", nil,
"Machine names or IDs to push the built images to (requires --push).\n"+
"Can be specified multiple times or as a comma-separated list. (default is all machines or x-machines)")
cmd.Flags().BoolVar(&opts.NoCache, "no-cache", false,
"Do not use cache when building images.")
cmd.Flags().StringSliceVarP(&opts.profiles, "profile", "p", nil,
"One or more Compose profiles to enable.") "One or more Compose profiles to enable.")
cmd.Flags().BoolVarP(&opts.Push, "push", "P", false, cmd.Flags().BoolVar(&opts.Pull, "pull", false,
"Push built images to the registry after building. (default false)") "Always attempt to pull newer versions of base images before building.")
cmd.Flags().BoolVarP(&opts.NoCache, "no-cache", "n", false, cmd.Flags().BoolVar(&opts.PushCluster, "push", false,
"Do not use cache when building images. (default false)") "Upload the built images to cluster machines after building.\n"+
"Use --machine to specify which machines. (default is all machines)")
cmd.Flags().BoolVar(&opts.PushRegistry, "push-registry", false,
"Upload the built images to external registries (e.g., Docker Hub) after building.")
return cmd return cmd
} }
// TODO: deduplicate with a similar functino for deploy options // runBuild parses the Compose file(s) and builds the images for selected services.
// projectOpts returns the project options for the Compose file(s). func runBuild(ctx context.Context, uncli *cli.CLI, opts buildOptions) error {
func projectOptsFromBuildOpts(opts cli.BuildOptions) []composecli.ProjectOptionsFn { // Validate push flags.
projectOpts := []composecli.ProjectOptionsFn{} if opts.PushCluster && opts.PushRegistry {
return fmt.Errorf("cannot specify both --push and --push-registry: choose one push target")
if len(opts.Profiles) > 0 {
projectOpts = append(projectOpts, composecli.WithDefaultProfiles(opts.Profiles...))
} }
return projectOpts machines := cli.ExpandCommaSeparatedValues(opts.Machines)
} // Special handling for an explicit "all" keyword to push to all machines.
if len(machines) == 1 && machines[0] == "all" {
machines = nil
}
opts.Machines = machines
// runBuild parses the Compose file(s), builds the services, and pushes them if requested. project, err := compose.LoadProject(ctx, opts.files, composecli.WithDefaultProfiles(opts.profiles...))
func runBuild(ctx context.Context, uncli *cli.CLI, opts cli.BuildOptions) error {
projectOpts := projectOptsFromBuildOpts(opts)
project, err := compose.LoadProject(ctx, opts.Files, projectOpts...)
if err != nil { if err != nil {
return fmt.Errorf("load compose file(s): %w", err) return fmt.Errorf("load compose file(s): %w", err)
} }
if len(opts.Services) > 0 { servicesToBuild, err := cli.ServicesThatNeedBuild(project, opts.Services, opts.Deps)
project, err = project.WithSelectedServices(opts.Services)
if err != nil { if err != nil {
return fmt.Errorf("select services: %w", err) return fmt.Errorf("determine services to build: %w", err)
} }
}
servicesToBuild := cli.GetServicesThatNeedBuild(project)
if len(servicesToBuild) == 0 { if len(servicesToBuild) == 0 {
fmt.Println("No services to build.") fmt.Println("No services to build.")
return nil return nil
} }
return cli.BuildServices(ctx, servicesToBuild, opts) return uncli.BuildServices(ctx, project, opts.BuildServicesOptions)
} }
+70
View File
@@ -0,0 +1,70 @@
package caddy
import (
"context"
"fmt"
"os"
"github.com/alecthomas/chroma/v2/quick"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
type configOptions struct {
machine string
noColor bool
}
func NewConfigCommand() *cobra.Command {
opts := configOptions{}
cmd := &cobra.Command{
Use: "config",
Short: "Show the current Caddy configuration (Caddyfile).",
Long: "Display the current Caddy configuration (Caddyfile) from the connected machine or a specified one.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return runConfig(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringVarP(&opts.machine, "machine", "m", "",
"Name or ID of the machine to get the configuration from. (default is connected machine)")
cmd.Flags().BoolVar(&opts.noColor, "no-color", false,
"Disable syntax highlighting for the output.")
return cmd
}
func runConfig(ctx context.Context, uncli *cli.CLI, opts configOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
if opts.machine != "" {
// If a specific machine is requested, use it to get the Caddy configuration.
ctx, _, err = clusterClient.ProxyMachinesContext(ctx, []string{opts.machine})
if err != nil {
return err
}
}
config, err := clusterClient.Caddy.GetConfig(ctx, nil)
if err != nil {
return fmt.Errorf("get Caddy config: %w", err)
}
// Print the Caddyfile with syntax highlighting.
if opts.noColor {
fmt.Print(config.Caddyfile)
} else {
if err = quick.Highlight(os.Stdout, config.Caddyfile, "caddy", "terminal256", "monokai"); err != nil {
// If highlighting fails, fall back to plain output.
fmt.Print(config.Caddyfile)
}
}
return nil
}
+17 -9
View File
@@ -5,6 +5,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"maps" "maps"
"os"
"slices" "slices"
"strings" "strings"
@@ -18,9 +19,9 @@ import (
) )
type deployOptions struct { type deployOptions struct {
caddyfile string
image string image string
machines []string machines []string
context string
} }
func NewDeployCommand() *cobra.Command { func NewDeployCommand() *cobra.Command {
@@ -37,21 +38,28 @@ func NewDeployCommand() *cobra.Command {
}, },
} }
cmd.Flags().StringVar(&opts.caddyfile, "caddyfile", "",
"Path to a custom global Caddy config (Caddyfile) that will be prepended to the auto-generated Caddy config.")
cmd.Flags().StringVar(&opts.image, "image", "", cmd.Flags().StringVar(&opts.image, "image", "",
"Caddy Docker image to deploy. (default caddy:LATEST_VERSION)") "Caddy Docker image to deploy. (default caddy:LATEST_VERSION)")
cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil, cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil,
"Machine names to deploy to. Can be specified multiple times or as a comma-separated "+ "Machine names or IDs to deploy to. Can be specified multiple times or as a comma-separated "+
"list of machine names. (default is all machines)") "list. (default is all machines)")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context to deploy to. (default is the current context)",
)
return cmd return cmd
} }
func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error { func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context) caddyfile := ""
if opts.caddyfile != "" {
data, err := os.ReadFile(opts.caddyfile)
if err != nil {
return fmt.Errorf("read Caddyfile: %w", err)
}
caddyfile = strings.TrimSpace(string(data))
}
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
@@ -91,7 +99,7 @@ func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
placement := api.Placement{ placement := api.Placement{
Machines: cli.ExpandCommaSeparatedValues(opts.machines), Machines: cli.ExpandCommaSeparatedValues(opts.machines),
} }
d, err := clusterClient.NewCaddyDeployment(opts.image, placement) d, err := clusterClient.NewCaddyDeployment(opts.image, caddyfile, placement)
if err != nil { if err != nil {
return fmt.Errorf("create caddy deployment: %w", err) return fmt.Errorf("create caddy deployment: %w", err)
} }
+1
View File
@@ -10,6 +10,7 @@ func NewRootCommand() *cobra.Command {
Short: "Manage Caddy reverse proxy service.", Short: "Manage Caddy reverse proxy service.",
} }
cmd.AddCommand( cmd.AddCommand(
NewConfigCommand(),
NewDeployCommand(), NewDeployCommand(),
) )
return cmd return cmd
+78
View File
@@ -0,0 +1,78 @@
package context
import (
"fmt"
"github.com/charmbracelet/huh"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/cli/config"
"github.com/spf13/cobra"
)
func NewConnectionCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "connection",
Short: "Choose a new default connection for the current context.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return selectConnection(uncli)
},
}
return cmd
}
func selectConnection(uncli *cli.CLI) error {
if uncli.Config == nil {
return fmt.Errorf("connection management is not available: Uncloud configuration file is not being used")
}
if len(uncli.Config.Contexts) == 0 {
return fmt.Errorf("no contexts found in Uncloud config (%s)", uncli.Config.Path())
}
currentCtxName := uncli.Config.CurrentContext
currentCtx, ok := uncli.Config.Contexts[currentCtxName]
if !ok {
return fmt.Errorf("current context '%s' not found", currentCtxName)
}
if len(currentCtx.Connections) == 0 {
return fmt.Errorf("no connections found in context '%s'", currentCtxName)
}
var selectedConnection int
form := huh.NewForm(
huh.NewGroup(
huh.NewSelect[int]().
Title("Select a default connection").
Options(buildConnectionOptions(currentCtx.Connections)...).
Value(&selectedConnection),
),
)
if err := form.Run(); err != nil {
return fmt.Errorf("select connection: %w", err)
}
currentCtx.SetDefaultConnection(selectedConnection)
selected := currentCtx.Connections[0]
if err := uncli.Config.Save(); err != nil {
return fmt.Errorf("save config: %w", err)
}
fmt.Printf("Default connection for context '%s' is now '%s'.\n", currentCtxName, selected)
return nil
}
func buildConnectionOptions(connections []config.MachineConnection) []huh.Option[int] {
options := make([]huh.Option[int], len(connections))
for i, conn := range connections {
key := conn.String()
opt := huh.NewOption(key, i)
if i == 0 {
opt.Key += " (default)"
opt = opt.Selected(true)
}
options[i] = opt
}
return options
}
+1 -8
View File
@@ -12,8 +12,6 @@ import (
) )
func NewListCommand() *cobra.Command { func NewListCommand() *cobra.Command {
var clusterContext string
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "ls", Use: "ls",
Aliases: []string{"list"}, Aliases: []string{"list"},
@@ -24,11 +22,6 @@ func NewListCommand() *cobra.Command {
}, },
} }
cmd.Flags().StringVarP(
&clusterContext, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd return cmd
} }
@@ -51,7 +44,7 @@ func list(uncli *cli.CLI) error {
for _, name := range contextNames { for _, name := range contextNames {
current := "" current := ""
if name == currentContext { if name == currentContext {
current = "*" current = ""
} }
connCount := len(uncli.Config.Contexts[name].Connections) connCount := len(uncli.Config.Contexts[name].Connections)
fmt.Fprintf(tw, "%s\t%s\t%d\n", name, current, connCount) fmt.Fprintf(tw, "%s\t%s\t%d\n", name, current, connCount)
+1
View File
@@ -19,6 +19,7 @@ func NewRootCommand() *cobra.Command {
cmd.AddCommand( cmd.AddCommand(
NewListCommand(), NewListCommand(),
NewUseCommand(), NewUseCommand(),
NewConnectionCommand(),
) )
return cmd return cmd
+76 -31
View File
@@ -6,6 +6,7 @@ import (
"fmt" "fmt"
"strings" "strings"
"github.com/charmbracelet/lipgloss"
composecli "github.com/compose-spec/compose-go/v2/cli" composecli "github.com/compose-spec/compose-go/v2/cli"
"github.com/docker/compose/v2/pkg/progress" "github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/internal/cli" "github.com/psviderski/uncloud/internal/cli"
@@ -17,13 +18,14 @@ import (
) )
type deployOptions struct { type deployOptions struct {
cli.BuildServicesOptions
files []string files []string
profiles []string profiles []string
services []string services []string
noBuild bool noBuild bool
recreate bool recreate bool
yes bool
context string
} }
// NewDeployCommand creates a new command to deploy services from a Compose file. // NewDeployCommand creates a new command to deploy services from a Compose file.
@@ -33,26 +35,34 @@ func NewDeployCommand() *cobra.Command {
Use: "deploy [FLAGS] [SERVICE...]", Use: "deploy [FLAGS] [SERVICE...]",
Short: "Deploy services from a Compose file.", Short: "Deploy services from a Compose file.",
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI) cli.BindEnvToFlag(cmd, "yes", "UNCLOUD_AUTO_CONFIRM")
if len(args) > 0 { uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.services = args opts.services = args
}
return runDeploy(cmd.Context(), uncli, opts) return runDeploy(cmd.Context(), uncli, opts)
}, },
GroupID: "service",
} }
cmd.Flags().StringArrayVar(&opts.BuildServicesOptions.BuildArgs, "build-arg", nil,
"Set a build-time variable for services. Used in Dockerfiles that declare the variable with ARG.\n"+
"Can be specified multiple times. Format: --build-arg VAR=VALUE")
cmd.Flags().BoolVar(&opts.BuildServicesOptions.Pull, "build-pull", false,
"Always attempt to pull newer versions of base images before building service images.")
cmd.Flags().StringSliceVarP(&opts.files, "file", "f", nil, cmd.Flags().StringSliceVarP(&opts.files, "file", "f", nil,
"One or more Compose files to deploy services from. (default compose.yaml)") "One or more Compose files to deploy services from. (default compose.yaml)")
cmd.Flags().BoolVar(&opts.noBuild, "no-build", false,
"Do not build new images before deploying services.")
cmd.Flags().BoolVar(&opts.BuildServicesOptions.NoCache, "no-cache", false,
"Do not use cache when building images.")
cmd.Flags().StringSliceVarP(&opts.profiles, "profile", "p", nil, cmd.Flags().StringSliceVarP(&opts.profiles, "profile", "p", nil,
"One or more Compose profiles to enable.") "One or more Compose profiles to enable.")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context to deploy to (default is the current context)")
cmd.Flags().BoolVarP(&opts.noBuild, "no-build", "n", false,
"Do not build images before deploying services. (default false)")
cmd.Flags().BoolVar(&opts.recreate, "recreate", false, cmd.Flags().BoolVar(&opts.recreate, "recreate", false,
"Recreate containers even if their configuration and image haven't changed.") "Recreate containers even if their configuration and image haven't changed.")
cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
"Auto-confirm deployment plan. Should be explicitly set when running non-interactively,\n"+
"e.g., in CI/CD pipelines. [$UNCLOUD_AUTO_CONFIRM]")
// TODO: Consider adding a filter flag to specify which machines to deploy to but keep the rest running. // TODO: Consider adding a filter flag to specify which machines to deploy to but keep the rest running.
// Could be useful to test a new version on a subset of machines before rolling out to all. // Could be useful to test a new version on a subset of machines before rolling out to all.
@@ -60,22 +70,9 @@ func NewDeployCommand() *cobra.Command {
return cmd return cmd
} }
// projectOpts returns the project options for the Compose file(s).
func projectOpts(opts deployOptions) []composecli.ProjectOptionsFn {
projectOpts := []composecli.ProjectOptionsFn{}
if len(opts.profiles) > 0 {
projectOpts = append(projectOpts, composecli.WithDefaultProfiles(opts.profiles...))
}
return projectOpts
}
// runDeploy parses the Compose file(s) and deploys the services. // runDeploy parses the Compose file(s) and deploys the services.
func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error { func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
projectOpts := projectOpts(opts) project, err := compose.LoadProject(ctx, opts.files, composecli.WithDefaultProfiles(opts.profiles...))
project, err := compose.LoadProject(ctx, opts.files, projectOpts...)
if err != nil { if err != nil {
return fmt.Errorf("load compose file(s): %w", err) return fmt.Errorf("load compose file(s): %w", err)
} }
@@ -88,29 +85,69 @@ func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
} }
} }
servicesToBuild := cli.GetServicesThatNeedBuild(project) servicesToBuild, err := cli.ServicesThatNeedBuild(project, opts.services, false)
if err != nil {
return fmt.Errorf("determine services to build: %w", err)
}
if len(servicesToBuild) > 0 { if len(servicesToBuild) > 0 {
if opts.noBuild { if opts.noBuild {
fmt.Println("Not building services as requested.") fmt.Println("Not building services as requested.")
} else { } else {
buildOpts := cli.BuildOptions{ // Build service images without pushing them to cluster yet to not connect to the cluster twice.
Push: true, opts.BuildServicesOptions.Deps = true // build dependencies as deploy includes them by default
NoCache: false, opts.BuildServicesOptions.Services = opts.services
}
if err := cli.BuildServices(ctx, servicesToBuild, buildOpts); err != nil { if err = uncli.BuildServices(ctx, project, opts.BuildServicesOptions); err != nil {
return fmt.Errorf("build services: %w", err) return fmt.Errorf("build services: %w", err)
} }
} }
fmt.Println()
} }
clusterClient, err := uncli.ConnectCluster(ctx, opts.context) clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
defer clusterClient.Close() defer clusterClient.Close()
if len(servicesToBuild) > 0 && !opts.noBuild {
// Push built service images to cluster machines one at a time.
var errs []error
for _, s := range servicesToBuild {
if s.Image == "" {
// Skip services without an image name (shouldn't happen for services with build config).
continue
}
// Push to the specified x-machines or to *all* cluster machines if not specified.
var pushOpts client.PushImageOptions
if machines, ok := s.Extensions[compose.MachinesExtensionKey].(compose.MachinesSource); ok {
pushOpts.Machines = machines
}
if len(pushOpts.Machines) == 0 {
pushOpts.AllMachines = true
}
boldStyle := lipgloss.NewStyle().Bold(true)
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
if err = clusterClient.PushImage(ctx, s.Image, pushOpts); err != nil {
return fmt.Errorf("push image '%s' for service '%s': %w", s.Image, s.Name, err)
}
return nil
}, uncli.ProgressOut(), fmt.Sprintf("Pushing image %s to cluster", boldStyle.Render(s.Image)))
// Collect errors to try pushing all images.
if err != nil {
errs = append(errs, err)
}
}
if err = errors.Join(errs...); err != nil {
return err
}
fmt.Println()
}
var strategy deploy.Strategy var strategy deploy.Strategy
if opts.recreate { if opts.recreate {
strategy = &deploy.RollingStrategy{ForceRecreate: true} strategy = &deploy.RollingStrategy{ForceRecreate: true}
@@ -130,12 +167,19 @@ func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
return nil return nil
} }
fmt.Println("Deployment plan:") fmt.Println(lipgloss.NewStyle().Bold(true).Render("Deployment plan"))
if err = printPlan(ctx, clusterClient, plan); err != nil { if err = printPlan(ctx, clusterClient, plan); err != nil {
return fmt.Errorf("print deployment plan: %w", err) return fmt.Errorf("print deployment plan: %w", err)
} }
fmt.Println() fmt.Println()
// Ask for plan confirmation before proceeding with the deployment unless auto-confirmed with --yes.
if !opts.yes {
if !cli.IsStdinTerminal() {
return errors.New("cannot ask to confirm deployment plan in non-interactive mode, " +
"use --yes flag or set UNCLOUD_AUTO_CONFIRM=true to auto-confirm")
}
confirmed, err := cli.Confirm() confirmed, err := cli.Confirm()
if err != nil { if err != nil {
return fmt.Errorf("confirm deployment: %w", err) return fmt.Errorf("confirm deployment: %w", err)
@@ -144,6 +188,7 @@ func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
fmt.Println("Cancelled. No changes were made.") fmt.Println("Cancelled. No changes were made.")
return nil return nil
} }
}
return progress.RunWithTitle(ctx, func(ctx context.Context) error { return progress.RunWithTitle(ctx, func(ctx context.Context) error {
if err := plan.Execute(ctx, clusterClient); err != nil { if err := plan.Execute(ctx, clusterClient); err != nil {
+3 -14
View File
@@ -12,32 +12,21 @@ import (
"google.golang.org/protobuf/types/known/emptypb" "google.golang.org/protobuf/types/known/emptypb"
) )
type releaseOptions struct {
context string
}
func NewReleaseCommand() *cobra.Command { func NewReleaseCommand() *cobra.Command {
opts := releaseOptions{}
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "release", Use: "release",
Short: "Release the reserved cluster domain.", Short: "Release the reserved cluster domain.",
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
return release(cmd.Context(), uncli, opts) return release(cmd.Context(), uncli)
}, },
} }
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd return cmd
} }
func release(ctx context.Context, uncli *cli.CLI, opts releaseOptions) error { func release(ctx context.Context, uncli *cli.CLI) error {
client, err := uncli.ConnectCluster(ctx, opts.context) client, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
+1 -6
View File
@@ -19,7 +19,6 @@ const DefaultUncloudDNSAPIEndpoint = "https://dns.uncloud.run/v1"
type reserveOptions struct { type reserveOptions struct {
endpoint string endpoint string
context string
} }
func NewReserveCommand() *cobra.Command { func NewReserveCommand() *cobra.Command {
@@ -36,16 +35,12 @@ func NewReserveCommand() *cobra.Command {
cmd.Flags().StringVar(&opts.endpoint, "endpoint", DefaultUncloudDNSAPIEndpoint, cmd.Flags().StringVar(&opts.endpoint, "endpoint", DefaultUncloudDNSAPIEndpoint,
"API endpoint for the Uncloud DNS service.") "API endpoint for the Uncloud DNS service.")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd return cmd
} }
func reserve(ctx context.Context, uncli *cli.CLI, opts reserveOptions) error { func reserve(ctx context.Context, uncli *cli.CLI, opts reserveOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context) clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
+1 -1
View File
@@ -9,7 +9,7 @@ func NewRootCommand() *cobra.Command {
Use: "dns", Use: "dns",
Short: "Manage cluster domain in Uncloud DNS.", Short: "Manage cluster domain in Uncloud DNS.",
Long: "Manage cluster domain in Uncloud DNS.\n" + Long: "Manage cluster domain in Uncloud DNS.\n" +
"DNS commands allow you to reserve or release a unique '<id>.cluster.uncloud.run' domain for your " + "DNS commands allow you to reserve or release a unique 'xxxxxx.uncld.dev' domain for your " +
"cluster. When reserved, Caddy service deployments will automatically update DNS records to route " + "cluster. When reserved, Caddy service deployments will automatically update DNS records to route " +
"traffic to the services in the cluster.", "traffic to the services in the cluster.",
} }
+3 -14
View File
@@ -10,32 +10,21 @@ import (
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
type showOptions struct {
context string
}
func NewShowCommand() *cobra.Command { func NewShowCommand() *cobra.Command {
opts := showOptions{}
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "show", Use: "show",
Short: "Print the cluster domain name.", Short: "Print the cluster domain name.",
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
return show(cmd.Context(), uncli, opts) return show(cmd.Context(), uncli)
}, },
} }
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd return cmd
} }
func show(ctx context.Context, uncli *cli.CLI, opts showOptions) error { func show(ctx context.Context, uncli *cli.CLI) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context) clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
+122
View File
@@ -0,0 +1,122 @@
package main
import (
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
"github.com/spf13/cobra"
"github.com/spf13/cobra/doc"
)
const docsDir = "website/docs/9-cli-reference"
type cmdWrapper struct {
cmd *cobra.Command
}
// NewDocsCommand creates a new hidden command to generate CLI reference docs.
func NewDocsCommand() *cobra.Command {
wrapper := &cmdWrapper{}
cmd := &cobra.Command{
Use: "docs",
Short: "Generate Uncloud CLI reference docs",
SilenceUsage: true,
DisableFlagsInUseLine: true,
Hidden: true,
Args: cobra.NoArgs,
ValidArgsFunction: cobra.NoFileCompletions,
RunE: func(cmd *cobra.Command, _ []string) error {
// Remove existing markdown files.
mdFiles, err := filepath.Glob(filepath.Join(docsDir, "*.md"))
if err != nil {
return fmt.Errorf("list existing CLI docs: %w", err)
}
for _, f := range mdFiles {
if err = os.Remove(f); err != nil {
return fmt.Errorf("remove '%s': %w", f, err)
}
}
// Generate new CLI reference docs.
wrapper.cmd.Root().DisableAutoGenTag = true
if err := doc.GenMarkdownTree(cmd.Root(), docsDir); err != nil {
return fmt.Errorf("generate CLI docs: %w", err)
}
// Remove *completion*.md files that contain malformatted code blocks that break Docusaurus.
mdFiles, err = filepath.Glob(filepath.Join(docsDir, "*completion*.md"))
if err != nil {
return fmt.Errorf("list generated CLI docs: %w", err)
}
for _, f := range mdFiles {
if err = os.Remove(f); err != nil {
return fmt.Errorf("remove '%s': %w", f, err)
}
}
// Post-process generated markdown files.
mdFiles, err = filepath.Glob(filepath.Join(docsDir, "*.md"))
if err != nil {
return fmt.Errorf("list generated CLI docs: %w", err)
}
for _, f := range mdFiles {
if err = postProcessMarkdown(f); err != nil {
return fmt.Errorf("post-process '%s': %w", f, err)
}
}
return nil
},
}
wrapper.cmd = cmd
return cmd
}
// postProcessMarkdown applies transformations to generated markdown files.
func postProcessMarkdown(filename string) error {
data, err := os.ReadFile(filename)
if err != nil {
return err
}
content := string(data)
// Replace "SEE ALSO" with "See also".
content = strings.ReplaceAll(content, "SEE ALSO", "See also")
// Escape <id> to avoid Docusaurus treating it as an HTML tag.
content = strings.ReplaceAll(content, "<id>", "\\<id>")
// Remove broken links to completion docs.
if strings.Contains(content, "[uc completion") {
lines := strings.Split(content, "\n")
var filteredLines []string
for _, line := range lines {
if !strings.Contains(line, "[uc completion") {
filteredLines = append(filteredLines, line)
}
}
content = strings.Join(filteredLines, "\n")
}
// Adjust heading levels. Process from shortest to longest to avoid double replacements.
replacements := []struct {
old, new string
}{
{`(?m)^## `, `# `},
{`(?m)^### `, `## `},
{`(?m)^#### `, `### `},
{`(?m)^##### `, `#### `},
}
for _, r := range replacements {
re := regexp.MustCompile(r.old)
content = re.ReplaceAllString(content, r.new)
}
return os.WriteFile(filename, []byte(content), 0o644)
}
+307
View File
@@ -0,0 +1,307 @@
package image
import (
"context"
"fmt"
"slices"
"sort"
"strings"
"time"
"github.com/charmbracelet/lipgloss"
"github.com/charmbracelet/lipgloss/table"
"github.com/containerd/platforms"
"github.com/docker/docker/api/types/image"
"github.com/docker/go-units"
"github.com/muesli/termenv"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type listOptions struct {
machines []string
nameFilter string
}
func NewListCommand() *cobra.Command {
opts := listOptions{}
cmd := &cobra.Command{
Use: "ls [REPO:[TAG]]",
Aliases: []string{"list"},
Short: "List images on machines in the cluster.",
Long: "List images on machines in the cluster. By default, on all machines. Optionally filter by image name.",
Example: ` # List all images on all machines.
uc image ls
# List images on specific machine.
uc image ls -m machine1
# List images on multiple machines.
uc image ls -m machine1,machine2
# List images filtered by name (with any tag) on all machines.
uc image ls myapp
# List images filtered by name pattern on specific machine.
uc image ls "myapp:1.*" -m machine1`,
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) > 0 {
opts.nameFilter = args[0]
}
uncli := cmd.Context().Value("cli").(*cli.CLI)
return list(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil,
"Filter images by machine name or ID. Can be specified multiple times or as a comma-separated list. "+
"(default is include all machines)")
return cmd
}
// imageRow represents a single image with its metadata for display.
type imageRow struct {
id string
name string
platforms string
createdHuman string
createdUnix int64
size string
inUse string
store string
machine string
}
func list(ctx context.Context, uncli *cli.CLI, opts listOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
// Get all machines to create ID to name mapping.
allMachines, err := clusterClient.ListMachines(ctx, nil)
if err != nil {
return fmt.Errorf("list machines: %w", err)
}
machineIDToName := make(map[string]string)
for _, machineMember := range allMachines {
if machineMember.Machine != nil && machineMember.Machine.Id != "" && machineMember.Machine.Name != "" {
machineIDToName[machineMember.Machine.Id] = machineMember.Machine.Name
}
}
machines := cli.ExpandCommaSeparatedValues(opts.machines)
clusterImages, err := clusterClient.ListImages(ctx, api.ImageFilter{
Machines: machines,
Name: opts.nameFilter,
})
if err != nil {
return fmt.Errorf("list images: %w", err)
}
// Collect all images from all machines.
var rows []imageRow
for _, machineImages := range clusterImages {
// Get machine name for better readability.
machineName := machineImages.Metadata.Machine
if m := allMachines.FindByNameOrID(machineName); m != nil {
machineName = m.Machine.Name
}
store := "docker"
if machineImages.ContainerdStore {
store = "containerd"
}
// Process each image for this machine.
for _, img := range machineImages.Images {
// Show the first 12 chars without 'sha256:' as the image ID like Docker does.
id := strings.TrimPrefix(img.ID, "sha256:")[:12]
name := "<none>"
if len(img.RepoTags) > 0 && img.RepoTags[0] != "<none>:<none>" {
name = img.RepoTags[0]
}
imgPlatforms, _ := imagePlatforms(img)
formattedPlatforms := formatPlatforms(imgPlatforms)
created := ""
createdAt := time.Unix(img.Created, 0)
if !createdAt.IsZero() {
created = units.HumanDuration(time.Now().UTC().Sub(createdAt)) + " ago"
}
size := units.HumanSizeWithPrecision(float64(img.Size), 3)
// Check if the image is in use by any containers. Only supported by Docker API >=1.51
inUse := "-"
if img.Containers != -1 { // -1 means the info is not available.
if img.Containers > 0 {
inUse = lipgloss.NewStyle().Foreground(lipgloss.Color("10")).Render("●")
} else {
inUse = lipgloss.NewStyle().Foreground(lipgloss.Color("8")).Render("○")
}
}
rows = append(rows, imageRow{
id: id,
name: name,
platforms: formattedPlatforms,
createdHuman: created,
createdUnix: img.Created,
size: size,
inUse: inUse,
store: store,
machine: machineName,
})
}
}
if len(rows) == 0 {
if opts.nameFilter != "" {
fmt.Printf("No images matching '%s' found.\n", opts.nameFilter)
} else {
fmt.Println("No images found.")
}
return nil
}
// Sort images by name, then by machine name.
sort.Slice(rows, func(i, j int) bool {
if rows[i].name != rows[j].name {
return rows[i].name < rows[j].name
}
return rows[i].machine < rows[j].machine
})
// Print the images in a table format.
fmt.Println(formatImageTable(rows))
return nil
}
// imagePlatforms returns a list of platforms supported by the image and a boolean indicating if it's multi-platform.
func imagePlatforms(img image.Summary) ([]string, bool) {
var formattedPlatforms []string
multiPlatform := false
for _, m := range img.Manifests {
if m.Kind != image.ManifestKindImage || !m.Available {
continue
}
if m.ID != img.ID {
// There is an image manifest that has digest different from the main image digest.
// This means the image manifest is an index or a manifest list (multi-platform image).
multiPlatform = true
}
formattedPlatforms = append(formattedPlatforms, platforms.Format(m.ImageData.Platform))
}
slices.Sort(formattedPlatforms)
return formattedPlatforms, multiPlatform
}
func formatPlatforms(platforms []string) string {
if len(platforms) == 0 {
return "-"
}
platformStyle := lipgloss.NewStyle().
BorderForeground(lipgloss.Color("152")).
Foreground(lipgloss.Color("0")).
Background(lipgloss.Color("152"))
// Use fancy pill borders only if the output is a terminal with color support.
if lipgloss.ColorProfile() != termenv.Ascii {
platformStyle = platformStyle.Border(lipgloss.Border{Left: "", Right: ""}, false, true, false, true)
}
styledPlatforms := make([]string, len(platforms))
for i, p := range platforms {
styledPlatforms[i] = platformStyle.Render(p)
}
return strings.Join(styledPlatforms, " ")
}
func formatImageTable(rows []imageRow) string {
columns := []struct {
name string
hide bool
}{
{name: "IMAGE ID"},
{name: "NAME"},
{name: "PLATFORMS"},
{name: "CREATED"},
{name: "SIZE"},
{name: "IN USE"},
{name: "STORE"},
{name: "MACHINE"},
}
// Hide the "IN USE" column if none of the images have that info available.
inUseInfoAvailable := slices.ContainsFunc(rows, func(r imageRow) bool {
return r.inUse != "-"
})
if !inUseInfoAvailable {
// Hide "IN USE" column.
columns[5].hide = true
}
t := table.New().
// Remove the default border.
Border(lipgloss.Border{}).
BorderTop(false).
BorderBottom(false).
BorderLeft(false).
BorderRight(false).
BorderHeader(false).
BorderColumn(false).
StyleFunc(func(row, col int) lipgloss.Style {
if row == table.HeaderRow {
return lipgloss.NewStyle().Bold(true).PaddingRight(3)
}
// Regular style for data rows with padding.
return lipgloss.NewStyle().PaddingRight(3)
})
var headers []string
for _, col := range columns {
if !col.hide {
headers = append(headers, col.name)
}
}
t.Headers(headers...)
for _, row := range rows {
values := []string{
row.id,
row.name,
row.platforms,
row.createdHuman,
row.size,
row.inUse,
row.store,
row.machine,
}
var filteredValues []string
for i, v := range values {
if !columns[i].hide {
filteredValues = append(filteredValues, v)
}
}
t.Row(filteredValues...)
}
return t.String()
}
+94
View File
@@ -0,0 +1,94 @@
package image
import (
"context"
"fmt"
"github.com/containerd/platforms"
"github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/client"
"github.com/spf13/cobra"
)
type pushOptions struct {
image string
machines []string
platform string
}
func NewPushCommand() *cobra.Command {
opts := pushOptions{}
cmd := &cobra.Command{
Use: "push IMAGE",
Short: "Upload a local Docker image to the cluster.",
Long: `Upload a local Docker image to the cluster transferring only the missing layers.
The image is uploaded to all cluster machines (default) or the specified machine(s).`,
Example: ` # Push image to all machines in the cluster.
uc image push myapp:latest
# Push image to specific machine.
uc image push myapp:latest -m machine1
# Push image to multiple machines.
uc image push myapp:latest -m machine1,machine2,machine3
# Push a specific platform of a multi-platform image.
uc image push myapp:latest --platform linux/amd64`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.image = args[0]
return push(cmd.Context(), uncli, opts)
},
}
cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil,
"Machine names or IDs to push the image to. Can be specified multiple times or as a comma-separated list. "+
"(default is all machines)")
cmd.Flags().StringVar(
&opts.platform, "platform", "",
"Push a specific platform of a multi-platform image (e.g., linux/amd64, linux/arm64).\n"+
"Local Docker must be configured to use containerd image store to support multi-platform images.",
)
return cmd
}
func push(ctx context.Context, uncli *cli.CLI, opts pushOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
machines := cli.ExpandCommaSeparatedValues(opts.machines)
pushOpts := client.PushImageOptions{}
// Special handling for an explicit "all" keyword to push to all machines.
if len(machines) == 1 && machines[0] == "all" {
pushOpts.AllMachines = true
} else if len(machines) > 0 {
pushOpts.Machines = machines
} else {
// Default is to push to all machines in the cluster.
pushOpts.AllMachines = true
}
if opts.platform != "" {
p, err := platforms.Parse(opts.platform)
if err != nil {
return fmt.Errorf("invalid platform '%s': %w", opts.platform, err)
}
pushOpts.Platform = &p
}
return progress.RunWithTitle(ctx, func(ctx context.Context) error {
if err = clusterClient.PushImage(ctx, opts.image, pushOpts); err != nil {
return fmt.Errorf("push image to cluster: %w", err)
}
return nil
}, uncli.ProgressOut(), fmt.Sprintf("Pushing image %s to cluster", opts.image))
}
+19
View File
@@ -0,0 +1,19 @@
package image
import (
"github.com/spf13/cobra"
)
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "image",
Short: "Manage images on machines in the cluster.",
}
cmd.AddCommand(
NewListCommand(),
NewPushCommand(),
)
return cmd
}
+19
View File
@@ -0,0 +1,19 @@
package main
import (
"strings"
"github.com/psviderski/uncloud/cmd/uncloud/image"
"github.com/spf13/cobra"
)
// NewImagesCommand returns the 'image ls' command modified to work as 'images'.
func NewImagesCommand() *cobra.Command {
listCmd := image.NewListCommand()
listCmd.Use = "images [IMAGE]"
// Remove 'list' alias since this command is already an alias.
listCmd.Aliases = nil
listCmd.Example = strings.ReplaceAll(listCmd.Example, "uc image ls", "uc images")
return listCmd
}
+22 -8
View File
@@ -5,6 +5,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"net/netip" "net/netip"
"strings"
"time" "time"
"github.com/cenkalti/backoff/v4" "github.com/cenkalti/backoff/v4"
@@ -22,9 +23,9 @@ import (
type addOptions struct { type addOptions struct {
name string name string
noCaddy bool noCaddy bool
noInstall bool
publicIP string publicIP string
sshKey string sshKey string
context string
version string version string
} }
@@ -33,11 +34,22 @@ func NewAddCommand() *cobra.Command {
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "add [USER@]HOST[:PORT]", Use: "add [USER@]HOST[:PORT]",
Short: "Add a remote machine to a cluster.", Short: "Add a remote machine to a cluster.",
Long: `Add a new machine to an existing Uncloud cluster.
Connection methods:
ssh://user@host - Use built-in SSH library (default, no prefix required)
ssh+cli://user@host - Use system SSH command (supports ProxyJump, SSH config)`,
Args: cobra.ExactArgs(1), Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
user, host, port, err := config.SSHDestination(args[0]).Parse() // Determine if SSH CLI needs to be used and strip scheme
destination := args[0]
useSSHCLI := strings.HasPrefix(destination, "ssh+cli://")
destination = strings.TrimPrefix(destination, "ssh+cli://")
destination = strings.TrimPrefix(destination, "ssh://")
user, host, port, err := config.SSHDestination(destination).Parse()
if err != nil { if err != nil {
return fmt.Errorf("parse remote machine: %w", err) return fmt.Errorf("parse remote machine: %w", err)
} }
@@ -46,6 +58,7 @@ func NewAddCommand() *cobra.Command {
Host: host, Host: host,
Port: port, Port: port,
KeyPath: opts.sshKey, KeyPath: opts.sshKey,
UseSSHCLI: useSSHCLI,
} }
return add(cmd.Context(), uncli, remoteMachine, opts) return add(cmd.Context(), uncli, remoteMachine, opts)
@@ -56,6 +69,11 @@ func NewAddCommand() *cobra.Command {
&opts.noCaddy, "no-caddy", false, &opts.noCaddy, "no-caddy", false,
"Don't deploy Caddy reverse proxy service to the machine.", "Don't deploy Caddy reverse proxy service to the machine.",
) )
cmd.Flags().BoolVar(
&opts.noInstall, "no-install", false,
"Skip installation of Docker, Uncloud daemon, and dependencies on the machine. "+
"Assumes they're already installed and running.",
)
cmd.Flags().StringVar( cmd.Flags().StringVar(
&opts.publicIP, "public-ip", "auto", &opts.publicIP, "public-ip", "auto",
"Public IP address of the machine for ingress configuration. Use 'auto' for automatic detection, "+ "Public IP address of the machine for ingress configuration. Use 'auto' for automatic detection, "+
@@ -70,10 +88,6 @@ func NewAddCommand() *cobra.Command {
&opts.version, "version", "latest", &opts.version, "version", "latest",
"Version of the Uncloud daemon to install on the machine.", "Version of the Uncloud daemon to install on the machine.",
) )
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context to add the machine to. (default is the current context)",
)
return cmd return cmd
} }
@@ -94,10 +108,10 @@ func add(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine,
} }
clusterClient, machineClient, err := uncli.AddMachine(ctx, cli.AddMachineOptions{ clusterClient, machineClient, err := uncli.AddMachine(ctx, cli.AddMachineOptions{
Context: opts.context,
MachineName: opts.name, MachineName: opts.name,
PublicIP: publicIP, PublicIP: publicIP,
RemoteMachine: remoteMachine, RemoteMachine: remoteMachine,
SkipInstall: opts.noInstall,
Version: opts.version, Version: opts.version,
}) })
if err != nil { if err != nil {
@@ -146,7 +160,7 @@ func add(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine,
// TODO: scale the existing Caddy service to the new machine instead of running a new deployment // TODO: scale the existing Caddy service to the new machine instead of running a new deployment
// that may cause a small downtime. // that may cause a small downtime.
d, err := clusterClient.NewCaddyDeployment(caddyImage, api.Placement{}) d, err := clusterClient.NewCaddyDeployment(caddyImage, "", api.Placement{})
if err != nil { if err != nil {
return fmt.Errorf("create caddy deployment: %w", err) return fmt.Errorf("create caddy deployment: %w", err)
} }
+45 -8
View File
@@ -4,6 +4,7 @@ import (
"context" "context"
"fmt" "fmt"
"net/netip" "net/netip"
"strings"
"github.com/docker/compose/v2/pkg/progress" "github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/cmd/uncloud/caddy" "github.com/psviderski/uncloud/cmd/uncloud/caddy"
@@ -22,6 +23,7 @@ type initOptions struct {
network string network string
noCaddy bool noCaddy bool
noDNS bool noDNS bool
noInstall bool
publicIP string publicIP string
sshKey string sshKey string
version string version string
@@ -31,9 +33,26 @@ type initOptions struct {
func NewInitCommand() *cobra.Command { func NewInitCommand() *cobra.Command {
opts := initOptions{} opts := initOptions{}
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "init [USER@HOST:PORT]", Use: "init [schema://]USER@HOST[:PORT]",
Short: "Initialise a new cluster with a remote machine as the first member.", Short: "Initialise a new cluster with a remote machine as the first member.",
// TODO: include usage examples of initialising a remote machine. Long: `Initialise a new cluster by setting up a remote machine as the first member.
This command creates a new context in your Uncloud config to manage the cluster.
Connection methods:
ssh://user@host - Use built-in SSH library (default, no prefix required)
ssh+cli://user@host - Use system SSH command (supports ProxyJump, SSH config)`,
Example: ` # Initialise a new cluster with default settings.
uc machine init root@<your-server-ip>
# Initialise with a context name 'prod' in the Uncloud config (~/.config/uncloud/config.yaml) and machine name 'vps1'.
uc machine init root@<your-server-ip> -c prod -n vps1
# Initialise with a non-root user and custom SSH port and key.
uc machine init ubuntu@<your-server-ip>:2222 -i ~/.ssh/mykey
# Initialise without Caddy (no reverse proxy) and without an automatically managed domain name (xxxxxx.uncld.dev).
# You can deploy Caddy with 'uc caddy deploy' and reserve a domain with 'uc dns reserve' later.
uc machine init root@<your-server-ip> --no-caddy --no-dns`,
// TODO: support initialising a cluster on the local machine. // TODO: support initialising a cluster on the local machine.
Args: cobra.MaximumNArgs(1), Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
@@ -41,7 +60,13 @@ func NewInitCommand() *cobra.Command {
var remoteMachine *cli.RemoteMachine var remoteMachine *cli.RemoteMachine
if len(args) > 0 { if len(args) > 0 {
user, host, port, err := config.SSHDestination(args[0]).Parse() // Determine if SSH CLI is requested and strip scheme
destination := args[0]
useSSHCLI := strings.HasPrefix(destination, "ssh+cli://")
destination = strings.TrimPrefix(destination, "ssh+cli://")
destination = strings.TrimPrefix(destination, "ssh://")
user, host, port, err := config.SSHDestination(destination).Parse()
if err != nil { if err != nil {
return fmt.Errorf("parse remote machine: %w", err) return fmt.Errorf("parse remote machine: %w", err)
} }
@@ -50,6 +75,7 @@ func NewInitCommand() *cobra.Command {
Host: host, Host: host,
Port: port, Port: port,
KeyPath: opts.sshKey, KeyPath: opts.sshKey,
UseSSHCLI: useSSHCLI,
} }
} }
@@ -68,11 +94,16 @@ func NewInitCommand() *cobra.Command {
) )
cmd.Flags().BoolVar( cmd.Flags().BoolVar(
&opts.noCaddy, "no-caddy", false, &opts.noCaddy, "no-caddy", false,
"Don't deploy Caddy reverse proxy service to the machine.", "Don't deploy Caddy reverse proxy service to the machine. You can deploy it later with 'uc caddy deploy'.",
) )
cmd.Flags().BoolVar( cmd.Flags().BoolVar(
&opts.noDNS, "no-dns", false, &opts.noDNS, "no-dns", false,
"Don't reserve a cluster domain in Uncloud DNS.", "Don't reserve a cluster domain in Uncloud DNS. You can reserve it later with 'uc dns reserve'.",
)
cmd.Flags().BoolVar(
&opts.noInstall, "no-install", false,
"Skip installation of Docker, Uncloud daemon, and dependencies on the machine. "+
"Assumes they're already installed and running.",
) )
cmd.Flags().StringVar( cmd.Flags().StringVar(
&opts.publicIP, "public-ip", "auto", &opts.publicIP, "public-ip", "auto",
@@ -89,14 +120,19 @@ func NewInitCommand() *cobra.Command {
"Version of the Uncloud daemon to install on the machine.", "Version of the Uncloud daemon to install on the machine.",
) )
cmd.Flags().StringVarP( cmd.Flags().StringVarP(
&opts.context, "context", "c", "default", &opts.context, "context", "c", cli.DefaultContextName,
"Name of the created context for the initialised cluster in the Uncloud config.", "Name of the new context to be created in the Uncloud config to manage the cluster.",
) )
return cmd return cmd
} }
func initCluster(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine, opts initOptions) error { func initCluster(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine, opts initOptions) error {
if uncli.Config == nil {
// Config is nil when connecting directly to a remote machine (--connect) without using Uncloud config.
return fmt.Errorf("do not specify --connect when initialising a new cluster")
}
netPrefix, err := netip.ParsePrefix(opts.network) netPrefix, err := netip.ParsePrefix(opts.network)
if err != nil { if err != nil {
return fmt.Errorf("parse network CIDR: %w", err) return fmt.Errorf("parse network CIDR: %w", err)
@@ -121,6 +157,7 @@ func initCluster(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteM
Network: netPrefix, Network: netPrefix,
PublicIP: publicIP, PublicIP: publicIP,
RemoteMachine: remoteMachine, RemoteMachine: remoteMachine,
SkipInstall: opts.noInstall,
Version: opts.version, Version: opts.version,
}) })
if err != nil { if err != nil {
@@ -147,7 +184,7 @@ func initCluster(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteM
} }
if !opts.noCaddy { if !opts.noCaddy {
d, err := client.NewCaddyDeployment("", api.Placement{}) d, err := client.NewCaddyDeployment("", "", api.Placement{})
if err != nil { if err != nil {
return fmt.Errorf("create caddy deployment: %w", err) return fmt.Errorf("create caddy deployment: %w", err)
} }
+6 -11
View File
@@ -14,25 +14,20 @@ import (
) )
func NewListCommand() *cobra.Command { func NewListCommand() *cobra.Command {
var contextName string
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "ls", Use: "ls",
Aliases: []string{"list"}, Aliases: []string{"list"},
Short: "List machines in a cluster.", Short: "List machines in a cluster.",
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
return list(cmd.Context(), uncli, contextName) return list(cmd.Context(), uncli)
}, },
} }
cmd.Flags().StringVarP(
&contextName, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd return cmd
} }
func list(ctx context.Context, uncli *cli.CLI, clusterName string) error { func list(ctx context.Context, uncli *cli.CLI) error {
client, err := uncli.ConnectCluster(ctx, clusterName) client, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
@@ -46,7 +41,7 @@ func list(ctx context.Context, uncli *cli.CLI, clusterName string) error {
// Print the list of machines in a table format. // Print the list of machines in a table format.
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0) tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
// Print header. // Print header.
if _, err = fmt.Fprintln(tw, "NAME\tSTATE\tADDRESS\tPUBLIC IP\tWIREGUARD ENDPOINTS"); err != nil { if _, err = fmt.Fprintln(tw, "NAME\tSTATE\tADDRESS\tPUBLIC IP\tWIREGUARD ENDPOINTS\tMACHINE ID"); err != nil {
return fmt.Errorf("write header: %w", err) return fmt.Errorf("write header: %w", err)
} }
// Print rows. // Print rows.
@@ -68,8 +63,8 @@ func list(ctx context.Context, uncli *cli.CLI, clusterName string) error {
} }
if _, err = fmt.Fprintf( if _, err = fmt.Fprintf(
tw, "%s\t%s\t%s\t%s\t%s\n", m.Name, capitalise(member.State.String()), subnet, publicIP, tw, "%s\t%s\t%s\t%s\t%s\t%s\n", m.Name, capitalise(member.State.String()), subnet, publicIP,
strings.Join(endpoints, ", "), strings.Join(endpoints, ", "), member.Machine.Id,
); err != nil { ); err != nil {
return fmt.Errorf("write row: %w", err) return fmt.Errorf("write row: %w", err)
} }
+3 -8
View File
@@ -9,7 +9,6 @@ import (
) )
func NewRenameCommand() *cobra.Command { func NewRenameCommand() *cobra.Command {
var contextName string
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "rename OLD_NAME NEW_NAME", Use: "rename OLD_NAME NEW_NAME",
Short: "Rename a machine in the cluster.", Short: "Rename a machine in the cluster.",
@@ -20,18 +19,14 @@ configuration including network settings, public IP, and cluster membership.`,
Args: cobra.ExactArgs(2), Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
return rename(cmd.Context(), uncli, contextName, args[0], args[1]) return rename(cmd.Context(), uncli, args[0], args[1])
}, },
} }
cmd.Flags().StringVarP(
&contextName, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd return cmd
} }
func rename(ctx context.Context, uncli *cli.CLI, contextName, oldName, newName string) error { func rename(ctx context.Context, uncli *cli.CLI, oldName, newName string) error {
client, err := uncli.ConnectCluster(ctx, contextName) client, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return err return err
} }
+17 -7
View File
@@ -22,7 +22,6 @@ import (
type removeOptions struct { type removeOptions struct {
noReset bool noReset bool
yes bool yes bool
context string
} }
func NewRmCommand() *cobra.Command { func NewRmCommand() *cobra.Command {
@@ -39,8 +38,6 @@ func NewRmCommand() *cobra.Command {
}, },
} }
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false, cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
"Do not prompt for confirmation before removing the machine.") "Do not prompt for confirmation before removing the machine.")
cmd.Flags().BoolVar(&opts.noReset, "no-reset", false, cmd.Flags().BoolVar(&opts.noReset, "no-reset", false,
@@ -51,14 +48,14 @@ func NewRmCommand() *cobra.Command {
func remove(ctx context.Context, uncli *cli.CLI, nameOrID string, opts removeOptions) error { func remove(ctx context.Context, uncli *cli.CLI, nameOrID string, opts removeOptions) error {
// TODO: automatically choose a connection to the machine that is not being removed. // TODO: automatically choose a connection to the machine that is not being removed.
client, err := uncli.ConnectCluster(ctx, opts.context) client, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
defer client.Close() defer client.Close()
// Verify the machine exists and list all service containers on it including stopped ones. // Verify the machine exists and list all service containers on it including stopped ones.
mctx, machines, err := api.ProxyMachinesContext(ctx, client, []string{nameOrID}) mctx, machines, err := client.ProxyMachinesContext(ctx, []string{nameOrID})
if err != nil { if err != nil {
return err return err
} }
@@ -155,8 +152,21 @@ func remove(ctx context.Context, uncli *cli.CLI, nameOrID string, opts removeOpt
} }
} }
// TODO: remove the connection to the machine from the uncloud config if it exists. We need a way to associate // Remove the connection to the machine from the uncloud config if it exists.
// the machine with its connection in the config, e.g. by storing the machine name in the connection metadata. if uncli.Config != nil {
contextName := uncli.GetContextOverrideOrCurrent()
if context, ok := uncli.Config.Contexts[contextName]; ok {
for i, c := range context.Connections {
if c.MachineID == m.Id {
context.Connections = slices.Delete(context.Connections, i, i+1)
break
}
}
if err := uncli.Config.Save(); err != nil {
return fmt.Errorf("save config: %w", err)
}
}
}
// TODO: If Caddy was running on this machine and a cluster domain is reserved, // TODO: If Caddy was running on this machine and a cluster domain is reserved,
// let the user know that the DNS records should be updated. // let the user know that the DNS records should be updated.
+1 -1
View File
@@ -8,7 +8,7 @@ func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "machine", Use: "machine",
Aliases: []string{"m"}, Aliases: []string{"m"},
Short: "Manage machines in an Uncloud cluster.", Short: "Manage machines in the cluster.",
} }
cmd.AddCommand( cmd.AddCommand(
NewAddCommand(), NewAddCommand(),
+1 -6
View File
@@ -13,7 +13,6 @@ import (
type updateOptions struct { type updateOptions struct {
name string name string
publicIP string publicIP string
context string
} }
func NewUpdateCommand() *cobra.Command { func NewUpdateCommand() *cobra.Command {
@@ -43,10 +42,6 @@ At least one flag must be specified to perform an update operation.`,
&opts.publicIP, "public-ip", "", &opts.publicIP, "public-ip", "",
fmt.Sprintf("Public IP address of the machine for ingress configuration. Use '%s' or '' to remove the public IP.", PublicIPNone), fmt.Sprintf("Public IP address of the machine for ingress configuration. Use '%s' or '' to remove the public IP.", PublicIPNone),
) )
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd return cmd
} }
@@ -57,7 +52,7 @@ func update(ctx context.Context, uncli *cli.CLI, cmd *cobra.Command, opts update
return fmt.Errorf("at least one update flag must be specified (--name, --public-ip)") return fmt.Errorf("at least one update flag must be specified (--name, --public-ip)")
} }
client, err := uncli.ConnectCluster(ctx, opts.context) client, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return err return err
} }
+65 -18
View File
@@ -6,15 +6,18 @@ import (
"net/netip" "net/netip"
"strings" "strings"
"github.com/charmbracelet/lipgloss"
"github.com/psviderski/uncloud/cmd/uncloud/caddy" "github.com/psviderski/uncloud/cmd/uncloud/caddy"
cmdcontext "github.com/psviderski/uncloud/cmd/uncloud/context" cmdcontext "github.com/psviderski/uncloud/cmd/uncloud/context"
"github.com/psviderski/uncloud/cmd/uncloud/dns" "github.com/psviderski/uncloud/cmd/uncloud/dns"
"github.com/psviderski/uncloud/cmd/uncloud/image"
"github.com/psviderski/uncloud/cmd/uncloud/machine" "github.com/psviderski/uncloud/cmd/uncloud/machine"
"github.com/psviderski/uncloud/cmd/uncloud/service" "github.com/psviderski/uncloud/cmd/uncloud/service"
"github.com/psviderski/uncloud/cmd/uncloud/volume" "github.com/psviderski/uncloud/cmd/uncloud/volume"
"github.com/psviderski/uncloud/internal/cli" "github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/cli/config" "github.com/psviderski/uncloud/internal/cli/config"
"github.com/psviderski/uncloud/internal/fs" "github.com/psviderski/uncloud/internal/fs"
"github.com/psviderski/uncloud/internal/log"
"github.com/psviderski/uncloud/internal/version" "github.com/psviderski/uncloud/internal/version"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
@@ -22,17 +25,24 @@ import (
type globalOptions struct { type globalOptions struct {
configPath string configPath string
connect string connect string
context string
} }
func main() { func main() {
log.InitLoggerFromEnv()
opts := globalOptions{} opts := globalOptions{}
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "uncloud", Use: "uc",
Short: "A CLI tool for managing Uncloud resources such as clusters, machines, and services.", Short: "A CLI tool for managing Uncloud resources such as machines, services, and volumes.",
Version: version.String(), Version: version.String(),
SilenceUsage: true, SilenceUsage: true,
SilenceErrors: true, SilenceErrors: true,
PersistentPreRunE: func(cmd *cobra.Command, args []string) error { PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
cli.BindEnvToFlag(cmd, "connect", "UNCLOUD_CONNECT")
cli.BindEnvToFlag(cmd, "context", "UNCLOUD_CONTEXT")
cli.BindEnvToFlag(cmd, "uncloud-config", "UNCLOUD_CONFIG")
var conn *config.MachineConnection var conn *config.MachineConnection
if opts.connect != "" { if opts.connect != "" {
if strings.HasPrefix(opts.connect, "tcp://") { if strings.HasPrefix(opts.connect, "tcp://") {
@@ -43,11 +53,17 @@ func main() {
conn = &config.MachineConnection{ conn = &config.MachineConnection{
TCP: &addrPort, TCP: &addrPort,
} }
} else { } else if strings.HasPrefix(opts.connect, "ssh+cli://") {
dest := opts.connect dest := opts.connect[len("ssh+cli://"):]
if strings.HasPrefix(dest, "ssh://") { conn = &config.MachineConnection{
dest = dest[len("ssh://"):] SSHCLI: config.SSHDestination(dest),
} }
} else if strings.HasPrefix(opts.connect, "unix://") {
conn = &config.MachineConnection{
Unix: opts.connect[len("unix://"):],
}
} else {
dest := strings.TrimPrefix(opts.connect, "ssh://")
conn = &config.MachineConnection{ conn = &config.MachineConnection{
SSH: config.SSHDestination(dest), SSH: config.SSHDestination(dest),
} }
@@ -55,7 +71,7 @@ func main() {
} }
configPath := fs.ExpandHomeDir(opts.configPath) configPath := fs.ExpandHomeDir(opts.configPath)
uncli, err := cli.New(configPath, conn) uncli, err := cli.New(configPath, conn, opts.context)
if err != nil { if err != nil {
return fmt.Errorf("initialise CLI: %w", err) return fmt.Errorf("initialise CLI: %w", err)
} }
@@ -65,27 +81,58 @@ func main() {
} }
cmd.PersistentFlags().StringVar(&opts.connect, "connect", "", cmd.PersistentFlags().StringVar(&opts.connect, "connect", "",
"Connect to a remote cluster machine without using the Uncloud configuration file.\n"+ "Connect to a remote cluster machine without using the Uncloud configuration file. [$UNCLOUD_CONNECT]\n"+
"Format: [ssh://]user@host[:port] or tcp://host:port") "Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], tcp://host:port, or unix:///path/to/uncloud.sock")
// TODO: allow to override using UNCLOUD_CONFIG env var.
cmd.PersistentFlags().StringVar(&opts.configPath, "uncloud-config", "~/.config/uncloud/config.yaml", cmd.PersistentFlags().StringVar(&opts.configPath, "uncloud-config", "~/.config/uncloud/config.yaml",
"Path to the Uncloud configuration file.") "Path to the Uncloud configuration file. [$UNCLOUD_CONFIG]")
_ = cmd.MarkPersistentFlagFilename("uncloud-config", "yaml", "yml") _ = cmd.MarkPersistentFlagFilename("uncloud-config", "yaml", "yml")
// TODO: make --context a global flag and pass it as a value of the command context. cmd.PersistentFlags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context to use (default is the current context). [$UNCLOUD_CONTEXT]")
// Set custom help function to show links to docs and Discord only for the root 'uc' command.
defaultHelpFunc := cmd.HelpFunc()
cmd.SetHelpFunc(func(c *cobra.Command, args []string) {
defaultHelpFunc(c, args)
// Only show links for the root 'uc' command.
if c.Name() == "uc" {
urlStyle := lipgloss.NewStyle().
Underline(true).
Foreground(lipgloss.Color("12")) // light blue
fmt.Fprintln(c.OutOrStdout())
fmt.Fprintf(c.OutOrStdout(), "Learn more about Uncloud: %s\n",
urlStyle.Render("https://uncloud.run/docs"))
fmt.Fprintf(c.OutOrStdout(), "Join our Discord community: %s\n",
urlStyle.Render("https://uncloud.run/discord"))
}
})
cmd.AddGroup(&cobra.Group{
ID: "service",
Title: "Deploy and manage services:",
})
cmd.AddCommand( cmd.AddCommand(
NewDeployCommand(),
NewBuildCommand(), NewBuildCommand(),
NewDeployCommand(),
NewDocsCommand(),
NewImagesCommand(),
NewPsCommand(),
caddy.NewRootCommand(), caddy.NewRootCommand(),
cmdcontext.NewRootCommand(), cmdcontext.NewRootCommand(),
dns.NewRootCommand(), dns.NewRootCommand(),
image.NewRootCommand(),
machine.NewRootCommand(), machine.NewRootCommand(),
service.NewRootCommand(), service.NewRootCommand(),
service.NewInspectCommand(), service.NewExecCommand("service"),
service.NewListCommand(), service.NewInspectCommand("service"),
service.NewRmCommand(), service.NewListCommand("service"),
service.NewRunCommand(), service.NewLogsCommand("service"),
service.NewScaleCommand(), service.NewRmCommand("service"),
service.NewRunCommand("service"),
service.NewScaleCommand("service"),
service.NewStartCommand("service"),
service.NewStopCommand("service"),
volume.NewRootCommand(), volume.NewRootCommand(),
) )
cobra.CheckErr(cmd.Execute()) cobra.CheckErr(cmd.Execute())
+284
View File
@@ -0,0 +1,284 @@
package main
import (
"context"
"fmt"
"sort"
"time"
"github.com/charmbracelet/huh/spinner"
"github.com/charmbracelet/lipgloss"
"github.com/charmbracelet/lipgloss/table"
"github.com/docker/docker/api/types/container"
"github.com/docker/go-units"
"github.com/spf13/cobra"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/client"
)
const (
sortByService = "service"
sortByMachine = "machine"
sortByHealth = "health"
)
type containerHighlight int
const (
highlightDanger containerHighlight = iota
highlightWarning
highlightSuccess
highlightNormal
)
type psOptions struct {
sortBy string
}
func NewPsCommand() *cobra.Command {
opts := psOptions{}
cmd := &cobra.Command{
Use: "ps",
Short: "List all service containers.",
Long: `List all service containers across all machines in the cluster.
This command provides a comprehensive overview of all running containers that are part of a service,
making it easy to see the distribution and status of containers across the cluster.`,
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
if opts.sortBy != sortByService && opts.sortBy != sortByMachine && opts.sortBy != sortByHealth {
return fmt.Errorf("invalid value for --sort: %q, must be one of '%s', '%s' or '%s'", opts.sortBy,
sortByService, sortByMachine, sortByHealth)
}
return runPs(cmd.Context(), uncli, opts)
},
GroupID: "service",
}
cmd.Flags().StringVarP(&opts.sortBy, "sort", "s", sortByService,
"Sort containers by 'service', 'machine', or 'health'.")
return cmd
}
type containerInfo struct {
serviceName string
machineName string
id string
name string
image string
status string
highlight containerHighlight
created time.Time
ip string
}
func runPs(ctx context.Context, uncli *cli.CLI, opts psOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
var containers []containerInfo
err = spinner.New().
Title(" Collecting container info...").
Type(spinner.MiniDot).
Style(lipgloss.NewStyle().Foreground(lipgloss.Color("3"))).
ActionWithErr(func(ctx context.Context) error {
containers, err = collectContainers(ctx, clusterClient)
return err
}).
Run()
if err != nil {
return fmt.Errorf("collect containers: %w", err)
}
// Sort the containers based on the sorting option.
sort.SliceStable(containers, func(i, j int) bool {
a, b := containers[i], containers[j]
switch opts.sortBy {
case sortByHealth:
if a.highlight != b.highlight {
return a.highlight < b.highlight
}
if a.serviceName != b.serviceName {
return a.serviceName < b.serviceName
}
case sortByMachine:
if a.machineName != b.machineName {
return a.machineName < b.machineName
}
if a.serviceName != b.serviceName {
return a.serviceName < b.serviceName
}
default: // sortByService
if a.serviceName != b.serviceName {
return a.serviceName < b.serviceName
}
}
// Fallback to creation time (newest first).
return a.created.After(b.created)
})
return printContainers(containers)
}
func printContainers(containers []containerInfo) error {
t := table.New().
// Remove the default border.
Border(lipgloss.Border{}).
BorderTop(false).
BorderBottom(false).
BorderLeft(false).
BorderRight(false).
BorderHeader(false).
BorderColumn(false).
StyleFunc(func(row, col int) lipgloss.Style {
if row == table.HeaderRow {
return lipgloss.NewStyle().Bold(true).PaddingRight(3)
}
// Regular style for data rows with padding.
return lipgloss.NewStyle().PaddingRight(3)
})
t.Headers("SERVICE", "CONTAINER ID", "CONTAINER NAME", "IMAGE", "CREATED", "STATUS", "IP ADDRESS", "MACHINE")
for _, ctr := range containers {
id := ctr.id
if len(id) > 12 {
id = id[:12]
}
created := units.HumanDuration(time.Now().UTC().Sub(ctr.created)) + " ago"
var statusStyle lipgloss.Style
switch ctr.highlight {
case highlightSuccess:
statusStyle = lipgloss.NewStyle().Foreground(lipgloss.Color("2")) // Green
case highlightDanger:
statusStyle = lipgloss.NewStyle().Foreground(lipgloss.Color("1")) // Red
case highlightWarning:
statusStyle = lipgloss.NewStyle().Foreground(lipgloss.Color("3")) // Yellow
default:
statusStyle = lipgloss.NewStyle() // Default
}
t.Row(
ctr.serviceName,
id,
ctr.name,
ctr.image,
created,
statusStyle.Render(ctr.status),
ctr.ip,
ctr.machineName,
)
}
fmt.Println(t)
return nil
}
func collectContainers(ctx context.Context, cli *client.Client) ([]containerInfo, error) {
listCtx, machines, err := cli.ProxyMachinesContext(ctx, nil)
if err != nil {
return nil, fmt.Errorf("proxy machines context: %w", err)
}
// Create a map of IP to machine name for resolving response metadata
machinesNamesByIP := make(map[string]string)
for _, m := range machines {
if addr, err := m.Machine.Network.ManagementIp.ToAddr(); err == nil {
machinesNamesByIP[addr.String()] = m.Machine.Name
}
}
// List all service containers across all machines in the cluster.
machineContainers, err := cli.Docker.ListServiceContainers(
listCtx, "", container.ListOptions{All: true},
)
if err != nil {
return nil, fmt.Errorf("list service containers: %w", err)
}
var containers []containerInfo
for _, msc := range machineContainers {
machineName := "unknown"
// Metadata can be nil if the request was broadcasted to only one machine.
if msc.Metadata == nil && len(machineContainers) > 1 {
return nil, fmt.Errorf("something went wrong with gRPC proxy: metadata is missing for a machine response")
}
if msc.Metadata != nil && msc.Metadata.Error != "" {
client.PrintWarning(fmt.Sprintf("failed to list containers on machine %s: %s", machineName,
msc.Metadata.Error))
continue
}
if msc.Metadata != nil {
var ok bool
machineName, ok = machinesNamesByIP[msc.Metadata.Machine]
if !ok {
// Fallback to machine's IP as name.
machineName = msc.Metadata.Machine
}
} else {
// Fallback to the first available machine name.
if len(machines) > 0 {
machineName = machines[0].Machine.Name
}
}
for _, ctr := range msc.Containers {
if ctr.Container.State == nil || ctr.Container.Config == nil {
continue
}
status, err := ctr.Container.HumanState()
if err != nil {
return nil, fmt.Errorf("get human state for container %s: %w", ctr.Container.ID, err)
}
var highlight containerHighlight
healthStatus := ""
if ctr.Container.State.Health != nil {
healthStatus = ctr.Container.State.Health.Status
}
if healthStatus == container.Unhealthy || ctr.Container.State.Status == "dead" || ctr.Container.State.OOMKilled || ctr.Container.State.Dead {
highlight = highlightDanger
} else if healthStatus == container.Healthy {
highlight = highlightSuccess
} else if ctr.Container.State.Status == "running" {
highlight = highlightNormal
} else { // Other non-critical but noteworthy states
highlight = highlightWarning
}
created, _ := time.Parse(time.RFC3339Nano, ctr.Container.Created)
ip := ctr.Container.UncloudNetworkIP()
ipStr := ""
// The container might not have an IP if it's not running or uses the host network.
if ip.IsValid() {
ipStr = ip.String()
}
info := containerInfo{
serviceName: ctr.ServiceName(),
machineName: machineName,
id: ctr.Container.ID,
name: ctr.Container.Name,
image: ctr.Container.Config.Image,
status: status,
highlight: highlight,
created: created,
ip: ipStr,
}
containers = append(containers, info)
}
}
return containers, nil
}
+442
View File
@@ -0,0 +1,442 @@
package main
import (
"context"
"encoding/json"
"net/netip"
"testing"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/machine/docker"
"github.com/psviderski/uncloud/pkg/client"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/grpc"
"google.golang.org/protobuf/types/known/emptypb"
)
// mockDockerClient implements pb.DockerClient
type mockDockerClient struct {
pb.DockerClient // Embed to avoid implementing all methods
listResp *pb.ListServiceContainersResponse
listErr error
}
func (m *mockDockerClient) ListServiceContainers(ctx context.Context, in *pb.ListServiceContainersRequest, opts ...grpc.CallOption) (*pb.ListServiceContainersResponse, error) {
return m.listResp, m.listErr
}
// mockClusterClient implements pb.ClusterClient
type mockClusterClient struct {
pb.ClusterClient // Embed to avoid implementing all methods
machinesResp *pb.ListMachinesResponse
machinesErr error
}
func (m *mockClusterClient) ListMachines(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*pb.ListMachinesResponse, error) {
return m.machinesResp, m.machinesErr
}
func TestCollectContainers_NilMetadata(t *testing.T) {
// Setup container data
containerData := map[string]interface{}{
"Id": "container1",
"Name": "test-container",
"Config": map[string]interface{}{
"Image": "test-image",
},
"State": map[string]interface{}{
"Status": "running",
"StartedAt": "2023-01-01T12:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
},
"NetworkSettings": map[string]interface{}{
"Networks": map[string]interface{}{},
},
}
containerJSON, err := json.Marshal(containerData)
require.NoError(t, err)
serviceSpecJSON, err := json.Marshal(map[string]interface{}{})
require.NoError(t, err)
// Setup mocks
mockDocker := &mockDockerClient{
listResp: &pb.ListServiceContainersResponse{
Messages: []*pb.MachineServiceContainers{
{
Metadata: nil, // Simulating the issue: nil metadata
Containers: []*pb.ServiceContainer{
{
Container: containerJSON,
ServiceSpec: serviceSpecJSON,
},
},
},
},
},
}
machineIP := "10.0.0.1"
mockCluster := &mockClusterClient{
machinesResp: &pb.ListMachinesResponse{
Machines: []*pb.MachineMember{
{
Machine: &pb.MachineInfo{
Name: "machine-1",
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(netip.MustParseAddr(machineIP)),
},
},
State: pb.MachineMember_UP,
},
},
},
}
// Construct client with mocks
cli := &client.Client{
Docker: &docker.Client{
GRPCClient: mockDocker,
},
ClusterClient: mockCluster,
}
// Execute
containers, err := collectContainers(context.Background(), cli)
require.NoError(t, err)
// Verify
assert.Len(t, containers, 1)
if len(containers) > 0 {
c := containers[0]
assert.Equal(t, "container1", c.id)
assert.Equal(t, "test-container", c.name)
assert.Equal(t, "machine-1", c.machineName, "Should fall back to the single machine name when metadata is nil")
}
}
func TestCollectContainers_NilMetadata_MultipleMachines_Error(t *testing.T) {
// If we have multiple machines but receive nil metadata, it should return an error as it is ambiguous
// Setup container data
containerData1 := map[string]interface{}{
"Id": "container1",
}
containerJSON1, _ := json.Marshal(containerData1)
containerData2 := map[string]interface{}{
"Id": "container2",
"Config": map[string]interface{}{
"Image": "test-image",
},
"State": map[string]interface{}{
"Status": "running",
"StartedAt": "2023-01-01T12:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
},
"NetworkSettings": map[string]interface{}{
"Networks": map[string]interface{}{},
},
}
containerJSON2, _ := json.Marshal(containerData2)
serviceSpecJSON, _ := json.Marshal(map[string]interface{}{})
// Setup mocks
mockDocker := &mockDockerClient{
listResp: &pb.ListServiceContainersResponse{
Messages: []*pb.MachineServiceContainers{
{
Metadata: nil, // Nil metadata
Containers: []*pb.ServiceContainer{
{
Container: containerJSON1,
ServiceSpec: serviceSpecJSON,
},
},
},
{
Metadata: &pb.Metadata{Machine: "10.0.0.2"},
Containers: []*pb.ServiceContainer{
{
Container: containerJSON2,
ServiceSpec: serviceSpecJSON,
},
},
},
},
},
}
mockCluster := &mockClusterClient{
machinesResp: &pb.ListMachinesResponse{
Machines: []*pb.MachineMember{
{
Machine: &pb.MachineInfo{
Name: "machine-1",
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(netip.MustParseAddr("10.0.0.1")),
},
},
},
{
Machine: &pb.MachineInfo{
Name: "machine-2",
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(netip.MustParseAddr("10.0.0.2")),
},
},
},
},
},
}
// Construct client with mocks
cli := &client.Client{
Docker: &docker.Client{
GRPCClient: mockDocker,
},
ClusterClient: mockCluster,
}
// Execute
_, err := collectContainers(context.Background(), cli)
require.Error(t, err)
assert.Contains(t, err.Error(), "metadata is missing for a machine response")
}
func TestCollectContainers_MetadataPresent_MultipleMachines(t *testing.T) {
// Verify correct mapping of containers to machines when metadata is present
// Setup container data
containerData1 := map[string]interface{}{
"Id": "container1",
"Name": "container-1",
"Config": map[string]interface{}{
"Image": "image-1",
},
"State": map[string]interface{}{
"Status": "running",
"StartedAt": "2023-01-01T12:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
},
"NetworkSettings": map[string]interface{}{
"Networks": map[string]interface{}{},
},
}
containerJSON1, _ := json.Marshal(containerData1)
containerData2 := map[string]interface{}{
"Id": "container2",
"Name": "container-2",
"Config": map[string]interface{}{
"Image": "image-2",
},
"State": map[string]interface{}{
"Status": "running",
"StartedAt": "2023-01-01T12:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
},
"NetworkSettings": map[string]interface{}{
"Networks": map[string]interface{}{},
},
}
containerJSON2, _ := json.Marshal(containerData2)
serviceSpecJSON, _ := json.Marshal(map[string]interface{}{})
// Setup mocks
mockDocker := &mockDockerClient{
listResp: &pb.ListServiceContainersResponse{
Messages: []*pb.MachineServiceContainers{
{
Metadata: &pb.Metadata{Machine: "10.0.0.1"},
Containers: []*pb.ServiceContainer{
{
Container: containerJSON1,
ServiceSpec: serviceSpecJSON,
},
},
},
{
Metadata: &pb.Metadata{Machine: "10.0.0.2"},
Containers: []*pb.ServiceContainer{
{
Container: containerJSON2,
ServiceSpec: serviceSpecJSON,
},
},
},
},
},
}
mockCluster := &mockClusterClient{
machinesResp: &pb.ListMachinesResponse{
Machines: []*pb.MachineMember{
{
Machine: &pb.MachineInfo{
Name: "machine-1",
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(netip.MustParseAddr("10.0.0.1")),
},
},
},
{
Machine: &pb.MachineInfo{
Name: "machine-2",
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(netip.MustParseAddr("10.0.0.2")),
},
},
},
},
},
}
cli := &client.Client{
Docker: &docker.Client{GRPCClient: mockDocker},
ClusterClient: mockCluster,
}
containers, err := collectContainers(context.Background(), cli)
require.NoError(t, err)
assert.Len(t, containers, 2)
// Order is not guaranteed by the map iteration in logic or parallel fetch (though here it's mocked sequential),
// but collectContainers just appends.
// We'll find them by ID.
for _, c := range containers {
if c.id == "container1" {
assert.Equal(t, "machine-1", c.machineName)
} else if c.id == "container2" {
assert.Equal(t, "machine-2", c.machineName)
} else {
t.Errorf("unexpected container id: %s", c.id)
}
}
}
func TestCollectContainers_NilMetadata_NoMachines(t *testing.T) {
// Case: 1 msc with nil metadata but no machines at all
containerData := map[string]interface{}{
"Id": "container1",
"Config": map[string]interface{}{
"Image": "test-image",
},
"State": map[string]interface{}{
"Status": "running",
"StartedAt": "2023-01-01T12:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
},
"NetworkSettings": map[string]interface{}{
"Networks": map[string]interface{}{},
},
}
containerJSON, _ := json.Marshal(containerData)
serviceSpecJSON, _ := json.Marshal(map[string]interface{}{})
mockDocker := &mockDockerClient{
listResp: &pb.ListServiceContainersResponse{
Messages: []*pb.MachineServiceContainers{
{
Metadata: nil,
Containers: []*pb.ServiceContainer{
{
Container: containerJSON,
ServiceSpec: serviceSpecJSON,
},
},
},
},
},
}
// No machines in cluster response
mockCluster := &mockClusterClient{
machinesResp: &pb.ListMachinesResponse{
Machines: []*pb.MachineMember{},
},
}
cli := &client.Client{
Docker: &docker.Client{GRPCClient: mockDocker},
ClusterClient: mockCluster,
}
containers, err := collectContainers(context.Background(), cli)
require.NoError(t, err)
assert.Len(t, containers, 1)
if len(containers) > 0 {
assert.Equal(t, "unknown", containers[0].machineName)
}
}
func TestCollectContainers_MetadataPresent_NotInMapping(t *testing.T) {
// Case: msc with metadata that is not in the IP-to-name mapping
containerData := map[string]interface{}{
"Id": "container1",
"Config": map[string]interface{}{
"Image": "test-image",
},
"State": map[string]interface{}{
"Status": "running",
"StartedAt": "2023-01-01T12:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
},
"NetworkSettings": map[string]interface{}{
"Networks": map[string]interface{}{},
},
}
containerJSON, _ := json.Marshal(containerData)
serviceSpecJSON, _ := json.Marshal(map[string]interface{}{})
mockDocker := &mockDockerClient{
listResp: &pb.ListServiceContainersResponse{
Messages: []*pb.MachineServiceContainers{
{
Metadata: &pb.Metadata{Machine: "10.0.0.99"}, // Unknown IP
Containers: []*pb.ServiceContainer{
{
Container: containerJSON,
ServiceSpec: serviceSpecJSON,
},
},
},
},
},
}
mockCluster := &mockClusterClient{
machinesResp: &pb.ListMachinesResponse{
Machines: []*pb.MachineMember{
{
Machine: &pb.MachineInfo{
Name: "machine-1",
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(netip.MustParseAddr("10.0.0.1")),
},
},
},
},
},
}
cli := &client.Client{
Docker: &docker.Client{GRPCClient: mockDocker},
ClusterClient: mockCluster,
}
containers, err := collectContainers(context.Background(), cli)
require.NoError(t, err)
assert.Len(t, containers, 1)
if len(containers) > 0 {
// Should fallback to the IP/string in metadata
assert.Equal(t, "10.0.0.99", containers[0].machineName)
}
}
+128
View File
@@ -0,0 +1,128 @@
package service
import (
"context"
"fmt"
"os"
"github.com/docker/cli/cli/streams"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type execCliOptions struct {
detach bool
interactive bool
noTty bool
containerId string
}
var DEFAULT_COMMAND = []string{"sh", "-c", "command -v bash >/dev/null 2>&1 && exec bash || exec sh"}
func NewExecCommand(groupID string) *cobra.Command {
opts := execCliOptions{}
execCmd := &cobra.Command{
Use: "exec [OPTIONS] SERVICE [COMMAND ARGS...]",
Short: "Execute a command in a running service container.",
Long: `Execute a command (interactive shell by default) in a running container within a service.
If the service has multiple replicas and no container ID is specified, the command will be executed in a random container.
`,
Example: `
# Start an interactive shell ("bash" or "sh" will be tried by default)
uc exec web-service
# Start an interactive shell with explicit command
uc exec web-service /bin/zsh
# List files in the specific container of the service; --container accepts full ID or a (unique) prefix
uc exec --container d792e web-service ls -la
# Pipe input to a command inside the service container
cat backup.sql | uc exec -T db-service psql -U postgres mydb
# Run a task in the background (detached mode)
uc exec -d web-service /scripts/cleanup.sh`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
serviceName := args[0]
command := args[1:]
if len(command) == 0 {
command = DEFAULT_COMMAND
}
return runExec(cmd.Context(), uncli, serviceName, command, opts)
},
GroupID: groupID,
}
execCmd.Flags().BoolVarP(&opts.detach, "detach", "d", false, "Detached mode: run command in the background")
execCmd.Flags().BoolVarP(&opts.noTty, "no-tty", "T", !cli.IsStdoutTerminal(),
"Disable pseudo-TTY allocation. By default 'uc exec' allocates a TTY when connected to a terminal.")
// Keep "-i" and "-t" flags hidden for compatibility with docker exec
execCmd.Flags().BoolVarP(&opts.interactive, "interactive", "i", true, "Keep STDIN open even if not attached")
execCmd.Flags().MarkHidden("interactive")
execCmd.Flags().BoolP("tty", "t", false, "Allocate a pseudo-TTY")
execCmd.Flags().MarkHidden("tty")
// Common flags
execCmd.Flags().StringVar(&opts.containerId, "container", "",
"ID of the container to exec into. Accepts full ID or a unique prefix "+
"(default is the random container of the service)")
// This tells Cobra that all flags must come before positional arguments, so that
// commands with their own flags can be handled correctly.
execCmd.Flags().SetInterspersed(false)
return execCmd
}
func runExec(ctx context.Context, uncli *cli.CLI, serviceName string, command []string, opts execCliOptions) error {
if !opts.detach {
// Check if we're trying to attach to a TTY from a non-TTY client, e.g.
// when doing an 'cmd | uc exec ...'
stdin := streams.NewIn(os.Stdin)
// TODO: this logic/behavior mirrors docker-compose, but we can be smarter about it and detect TTY dynamically
if err := stdin.CheckTty(opts.interactive, !opts.noTty); err != nil {
return fmt.Errorf("check TTY: %w; use -T option to disable TTY allocation", err)
}
}
client, err := uncli.ConnectClusterWithOptions(ctx, cli.ConnectOptions{
ShowProgress: false,
})
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
execConfig := api.ExecOptions{
Command: command,
AttachStdin: opts.interactive,
Tty: !opts.noTty,
Detach: opts.detach,
}
if !opts.detach {
execConfig.AttachStdout = true
execConfig.AttachStderr = true
}
exitCode, err := client.ExecContainer(ctx, serviceName, opts.containerId, execConfig)
if err != nil {
return fmt.Errorf("exec container: %w", err)
}
// For non-detached mode, exit with the same code as the executed command
if !opts.detach {
if exitCode != 0 {
os.Exit(exitCode)
}
}
return nil
}
+29 -15
View File
@@ -4,21 +4,22 @@ import (
"context" "context"
"fmt" "fmt"
"os" "os"
"slices"
"text/tabwriter" "text/tabwriter"
"time" "time"
"github.com/docker/docker/pkg/stringid" "github.com/docker/docker/pkg/stringid"
"github.com/docker/go-units" "github.com/docker/go-units"
"github.com/psviderski/uncloud/internal/cli" "github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
type inspectOptions struct { type inspectOptions struct {
service string service string
cluster string
} }
func NewInspectCommand() *cobra.Command { func NewInspectCommand(groupID string) *cobra.Command {
opts := inspectOptions{} opts := inspectOptions{}
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "inspect SERVICE", Use: "inspect SERVICE",
@@ -29,16 +30,13 @@ func NewInspectCommand() *cobra.Command {
opts.service = args[0] opts.service = args[0]
return inspect(cmd.Context(), uncli, opts) return inspect(cmd.Context(), uncli, opts)
}, },
GroupID: groupID,
} }
cmd.Flags().StringVarP(
&opts.cluster, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd return cmd
} }
func inspect(ctx context.Context, uncli *cli.CLI, opts inspectOptions) error { func inspect(ctx context.Context, uncli *cli.CLI, opts inspectOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.cluster) client, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
@@ -58,23 +56,31 @@ func inspect(ctx context.Context, uncli *cli.CLI, opts inspectOptions) error {
machinesNamesByID[m.Machine.Id] = m.Machine.Name machinesNamesByID[m.Machine.Id] = m.Machine.Name
} }
fmt.Printf("ID: %s\n", svc.ID) fmt.Printf("Service ID: %s\n", svc.ID)
fmt.Printf("Name: %s\n", svc.Name) fmt.Printf("Name: %s\n", svc.Name)
fmt.Printf("Mode: %s\n", svc.Mode) fmt.Printf("Mode: %s\n", svc.Mode)
fmt.Println() fmt.Println()
// Parse created times for sorting and display.
createdTimes := make(map[string]time.Time, len(svc.Containers))
for _, ctr := range svc.Containers {
createdTimes[ctr.Container.ID], _ = time.Parse(time.RFC3339Nano, ctr.Container.Created)
}
// Sort containers by created time (newest first).
slices.SortFunc(svc.Containers, func(a, b api.MachineServiceContainer) int {
return createdTimes[b.Container.ID].Compare(createdTimes[a.Container.ID])
})
// Print the list of containers in a table format. // Print the list of containers in a table format.
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0) tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
if _, err = fmt.Fprintln(tw, "CONTAINER ID\tIMAGE\tCREATED\tSTATUS\tMACHINE"); err != nil { if _, err = fmt.Fprintln(tw, "CONTAINER ID\tIMAGE\tCREATED\tSTATUS\tIP ADDRESS\tMACHINE"); err != nil {
return fmt.Errorf("write header: %w", err) return fmt.Errorf("write header: %w", err)
} }
now := time.Now().UTC()
for _, ctr := range svc.Containers { for _, ctr := range svc.Containers {
createdAt, err := time.Parse(time.RFC3339Nano, ctr.Container.Created) created := units.HumanDuration(now.Sub(createdTimes[ctr.Container.ID])) + " ago"
if err != nil {
return fmt.Errorf("parse created time: %w", err)
}
created := units.HumanDuration(time.Now().UTC().Sub(createdAt)) + " ago"
machine := machinesNamesByID[ctr.MachineID] machine := machinesNamesByID[ctr.MachineID]
if machine == "" { if machine == "" {
@@ -85,13 +91,21 @@ func inspect(ctx context.Context, uncli *cli.CLI, opts inspectOptions) error {
return fmt.Errorf("get human state: %w", err) return fmt.Errorf("get human state: %w", err)
} }
ip := ctr.Container.UncloudNetworkIP()
ipStr := ""
// The container might not have an IP if it's not running or uses the host network.
if ip.IsValid() {
ipStr = ip.String()
}
_, err = fmt.Fprintf( _, err = fmt.Fprintf(
tw, tw,
"%s\t%s\t%s\t%s\t%s\n", "%s\t%s\t%s\t%s\t%s\t%s\n",
stringid.TruncateID(ctr.Container.ID), stringid.TruncateID(ctr.Container.ID),
ctr.Container.Config.Image, ctr.Container.Config.Image,
created, created,
state, state,
ipStr,
machine, machine,
) )
if err != nil { if err != nil {
+336
View File
@@ -0,0 +1,336 @@
package service
import (
"context"
"errors"
"fmt"
"os"
"slices"
"strconv"
"strings"
"time"
"github.com/charmbracelet/lipgloss"
mapset "github.com/deckarep/golang-set/v2"
"github.com/docker/docker/pkg/stringid"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client"
"github.com/psviderski/uncloud/pkg/client/compose"
"github.com/spf13/cobra"
)
type logsOptions struct {
files []string
follow bool
tail string
since string
until string
utc bool
machines []string
}
func NewLogsCommand(groupID string) *cobra.Command {
var options logsOptions
cmd := &cobra.Command{
Use: "logs [SERVICE...]",
Aliases: []string{"log"},
Short: "View service logs.",
Long: `View logs from all replicas of the specified service(s) across all machines in the cluster.
If no services are specified, streams logs from all services defined in the Compose file
(compose.yaml by default or the file(s) specified with --file).`,
Example: ` # View recent logs for a service.
uc logs web
# Stream logs in real-time (follow mode).
uc logs -f web
# View logs from multiple services.
uc logs web api db
# View logs from all services in compose.yaml.
uc logs
# Show last 20 lines per replica (default is 100).
uc logs -n 20 web
# Show all logs without line limit.
uc logs -n all web
# View logs from a specific time range.
uc logs --since 3h --until 1h30m web
# View logs only from replicas running on specific machines.
uc logs -m machine1,machine2 web api`,
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return runLogs(cmd.Context(), uncli, args, options)
},
GroupID: groupID,
}
cmd.Flags().StringSliceVar(&options.files, "file", nil,
"One or more Compose files to load service names from when no services are specified. (default compose.yaml)")
cmd.Flags().BoolVarP(&options.follow, "follow", "f", false,
"Continually stream new logs.")
cmd.Flags().StringSliceVarP(&options.machines, "machine", "m", nil,
"Filter logs by machine name or ID. Can be specified multiple times or as a comma-separated list.")
cmd.Flags().StringVar(&options.since, "since", "",
"Show logs generated on or after the given timestamp. Accepts relative duration, RFC 3339 date, or Unix timestamp.\n"+
"Examples:\n"+
" --since 2m30s Relative duration (2 minutes 30 seconds ago)\n"+
" --since 1h Relative duration (1 hour ago)\n"+
" --since 2025-11-24 RFC 3339 date only (midnight using local timezone)\n"+
" --since 2024-05-14T22:50:00 RFC 3339 date/time using local timezone\n"+
" --since 2024-01-31T10:30:00Z RFC 3339 date/time in UTC\n"+
" --since 1763953966 Unix timestamp (seconds since January 1, 1970)")
cmd.Flags().StringVarP(&options.tail, "tail", "n", "100",
"Show the most recent logs and limit the number of lines shown per replica. Use 'all' to show all logs.")
cmd.Flags().StringVar(&options.until, "until", "",
"Show logs generated before the given timestamp. Accepts relative duration, RFC 3339 date, or Unix timestamp.\n"+
"See --since for examples.")
cmd.Flags().BoolVar(&options.utc, "utc", false,
"Print timestamps in UTC instead of local timezone.")
return cmd
}
func runLogs(ctx context.Context, uncli *cli.CLI, serviceNames []string, opts logsOptions) error {
// If no services specified, try to load them from the Compose file(s).
if len(serviceNames) == 0 {
project, err := compose.LoadProject(ctx, opts.files)
if err != nil {
return fmt.Errorf("load compose file(s): %w", err)
}
// View logs for all services, including disabled by inactive profiles.
serviceNames = append(project.ServiceNames(), project.DisabledServiceNames()...)
if len(serviceNames) == 0 {
return errors.New("no services found in compose file(s)")
}
}
// Parse tail option.
tail := -1
if opts.tail != "all" {
tailInt, err := strconv.Atoi(opts.tail)
if err != nil {
return fmt.Errorf("invalid --tail value '%s': %w", opts.tail, err)
}
tail = tailInt
}
c, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer c.Close()
logsOpts := api.ServiceLogsOptions{
Follow: opts.follow,
Tail: tail,
Since: opts.since,
Until: opts.until,
Machines: cli.ExpandCommaSeparatedValues(opts.machines),
}
// Collect log streams from all services.
machineIDsSet := mapset.NewSet[string]()
svcStreams := make([]<-chan api.ServiceLogEntry, 0, len(serviceNames))
for _, serviceName := range serviceNames {
svc, ch, err := c.ServiceLogs(ctx, serviceName, logsOpts)
if err != nil {
return fmt.Errorf("stream logs for service '%s': %w", serviceName, err)
}
svcStreams = append(svcStreams, ch)
machineIDs := svc.MachineIDs()
machineIDsSet.Append(machineIDs...)
}
var stream <-chan api.ServiceLogEntry
if len(serviceNames) == 1 {
stream = svcStreams[0]
} else {
// Merge all service streams into a single sorted stream without stall detection as its handled per-service.
merger := client.NewLogMerger(svcStreams, client.LogMergerOptions{})
stream = merger.Stream()
}
// Fetch machine names for all machines (machineIDsSet) service containers are running on.
machines, err := c.ListMachines(ctx, &api.MachineFilter{NamesOrIDs: machineIDsSet.ToSlice()})
if err != nil {
return fmt.Errorf("list machines: %w", err)
}
machineNames := make([]string, 0, len(machines))
for _, m := range machines {
machineNames = append(machineNames, m.Machine.Name)
}
formatter := newLogFormatter(machineNames, serviceNames, opts.utc)
// Print merged logs.
for entry := range stream {
if entry.Err != nil {
formatter.printError(entry)
continue
}
formatter.printEntry(entry)
}
return nil
}
// Available colors for machine/service differentiation.
var colorPalette = []lipgloss.Color{
lipgloss.Color("10"), // Bright green
lipgloss.Color("11"), // Bright yellow
lipgloss.Color("12"), // Bright blue
lipgloss.Color("13"), // Bright magenta
lipgloss.Color("14"), // Bright cyan
lipgloss.Color("2"), // Green
lipgloss.Color("3"), // Yellow
lipgloss.Color("4"), // Blue
lipgloss.Color("5"), // Magenta
lipgloss.Color("6"), // Cyan
}
// logFormatter handles formatting and printing of log entries with dynamic column alignment.
type logFormatter struct {
machineNames []string
serviceNames []string
maxMachineWidth int
maxServiceWidth int
utc bool
}
func newLogFormatter(machineNames, serviceNames []string, utc bool) *logFormatter {
slices.Sort(machineNames)
slices.Sort(serviceNames)
maxMachineWidth := 0
for _, name := range machineNames {
if len(name) > maxMachineWidth {
maxMachineWidth = len(name)
}
}
maxServiceWidth := 0
for _, name := range serviceNames {
if len(name) > maxServiceWidth {
maxServiceWidth = len(name)
}
}
return &logFormatter{
machineNames: machineNames,
serviceNames: serviceNames,
maxMachineWidth: maxMachineWidth,
maxServiceWidth: maxServiceWidth,
utc: utc,
}
}
// formatTimestamp formats timestamp using local timezone or UTC if configured.
func (f *logFormatter) formatTimestamp(t time.Time) string {
if f.utc {
t = t.UTC()
} else {
t = t.In(time.Local)
}
dimStyle := lipgloss.NewStyle().Faint(true)
return dimStyle.Render(t.Format(time.StampMilli))
}
func (f *logFormatter) formatMachine(name string) string {
style := lipgloss.NewStyle().Bold(true).PaddingRight(f.maxMachineWidth - len(name))
if len(f.serviceNames) == 1 {
// Machine name is coloured for single-service logs.
i := slices.Index(f.machineNames, name)
if i == -1 {
f.machineNames = append(f.machineNames, name)
i = len(f.machineNames) - 1
}
style = style.Foreground(colorPalette[i%len(colorPalette)])
}
return style.Render(name)
}
func (f *logFormatter) formatServiceContainer(serviceName, containerID string) string {
styleService := lipgloss.NewStyle().Bold(true).PaddingRight(f.maxServiceWidth - len(serviceName))
styleContainer := lipgloss.NewStyle().Faint(true)
if len(f.serviceNames) > 1 {
// Service name is coloured for multi-service logs.
i := slices.Index(f.serviceNames, serviceName)
if i == -1 {
f.serviceNames = append(f.serviceNames, serviceName)
i = len(f.serviceNames) - 1
}
styleService = styleService.Foreground(colorPalette[i%len(colorPalette)])
}
return styleService.Render(serviceName) + styleContainer.Render("["+containerID[:5]+"]")
}
// printEntry prints a single log entry with proper formatting.
func (f *logFormatter) printEntry(entry api.ServiceLogEntry) {
if entry.Stream != api.LogStreamStdout && entry.Stream != api.LogStreamStderr {
return
}
var output strings.Builder
// Timestamp
output.WriteString(f.formatTimestamp(entry.Timestamp))
output.WriteString(" ")
// Machine name
output.WriteString(f.formatMachine(entry.Metadata.MachineName))
output.WriteString(" ")
// Service[container_id]
output.WriteString(f.formatServiceContainer(entry.Metadata.ServiceName, entry.Metadata.ContainerID))
output.WriteString(" ")
// Message
output.Write(entry.Message)
// Print to appropriate stream.
if entry.Stream == api.LogStreamStderr {
fmt.Fprint(os.Stderr, output.String())
} else {
fmt.Print(output.String())
}
}
// printError prints an error entry (e.g., stalled stream warning).
func (f *logFormatter) printError(entry api.ServiceLogEntry) {
if entry.Metadata.ContainerID != "" {
msg := fmt.Sprintf("WARNING: log stream from %s[%s] on machine '%s'",
entry.Metadata.ServiceName,
stringid.TruncateID(entry.Metadata.ContainerID),
entry.Metadata.MachineName)
if errors.Is(entry.Err, api.ErrLogStreamStalled) {
msg += " stopped responding"
} else {
msg += fmt.Sprintf(": %v", entry.Err)
}
style := lipgloss.NewStyle().Bold(true).Foreground(lipgloss.Color("11")) // Bold bright yellow
fmt.Fprintln(os.Stderr, style.Render(msg))
} else {
msg := fmt.Sprintf("ERROR: %v", entry.Err)
style := lipgloss.NewStyle().Bold(true).Foreground(lipgloss.Color("9")) // Bold bright red
fmt.Fprintln(os.Stderr, style.Render(msg))
}
}
+27 -20
View File
@@ -4,34 +4,31 @@ import (
"context" "context"
"fmt" "fmt"
"os" "os"
"slices"
"strings" "strings"
"text/tabwriter" "text/tabwriter"
"github.com/psviderski/uncloud/internal/cli" "github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
func NewListCommand() *cobra.Command { func NewListCommand(groupID string) *cobra.Command {
// TODO(lhf): rename to context
var cluster string
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "ls", Use: "ls",
Aliases: []string{"list"}, Aliases: []string{"list"},
Short: "List services.", Short: "List services.",
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
return list(cmd.Context(), uncli, cluster) return list(cmd.Context(), uncli)
}, },
GroupID: groupID,
} }
cmd.Flags().StringVarP(
&cluster, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd return cmd
} }
func list(ctx context.Context, uncli *cli.CLI, clusterName string) error { func list(ctx context.Context, uncli *cli.CLI) error {
client, err := uncli.ConnectCluster(ctx, clusterName) client, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
@@ -42,15 +39,15 @@ func list(ctx context.Context, uncli *cli.CLI, clusterName string) error {
return fmt.Errorf("list services: %w", err) return fmt.Errorf("list services: %w", err)
} }
serviceNames := make(map[string]struct{}, len(services)) // Sort services by name.
haveDuplicateNames := false haveDuplicateNames := false
for _, svc := range services { slices.SortFunc(services, func(a, b api.Service) int {
if _, exists := serviceNames[svc.Name]; exists { if a.Name == b.Name {
haveDuplicateNames = true haveDuplicateNames = true
break return strings.Compare(a.ID, b.ID)
}
serviceNames[svc.Name] = struct{}{}
} }
return strings.Compare(a.Name, b.Name)
})
// Print the list of services in a table format. // Print the list of services in a table format.
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0) tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
@@ -61,19 +58,29 @@ func list(ctx context.Context, uncli *cli.CLI, clusterName string) error {
return fmt.Errorf("write header: %w", err) return fmt.Errorf("write header: %w", err)
} }
} }
if _, err = fmt.Fprintln(tw, "NAME\tMODE\tREPLICAS\tENDPOINTS"); err != nil { if _, err = fmt.Fprintln(tw, "NAME\tMODE\tREPLICAS\tIMAGE\tENDPOINTS"); err != nil {
return fmt.Errorf("write header: %w", err) return fmt.Errorf("write header: %w", err)
} }
for _, s := range services { for _, s := range services {
endpointsSlice := s.Endpoints() images := strings.Join(s.Images(), ", ")
endpoints := strings.Join(endpointsSlice, ", ") endpoints := strings.Join(s.Endpoints(), ", ")
// If no endpoints from ports, check if the service uses custom Caddy config.
if endpoints == "" {
for _, ctr := range s.Containers {
if ctr.Container.ServiceSpec.CaddyConfig() != "" {
endpoints = "(custom Caddy config)"
}
}
}
if haveDuplicateNames { if haveDuplicateNames {
if _, err = fmt.Fprintf(tw, "%s\t", s.ID); err != nil { if _, err = fmt.Fprintf(tw, "%s\t", s.ID); err != nil {
return fmt.Errorf("write row: %w", err) return fmt.Errorf("write row: %w", err)
} }
} }
if _, err = fmt.Fprintf(tw, "%s\t%s\t%d\t%s\n", s.Name, s.Mode, len(s.Containers), endpoints); err != nil { if _, err = fmt.Fprintf(tw, "%s\t%s\t%d\t%s\t%s\n",
s.Name, s.Mode, len(s.Containers), images, endpoints); err != nil {
return fmt.Errorf("write row: %w", err) return fmt.Errorf("write row: %w", err)
} }
} }
+9 -8
View File
@@ -11,31 +11,32 @@ import (
type rmOptions struct { type rmOptions struct {
services []string services []string
cluster string
} }
func NewRmCommand() *cobra.Command { func NewRmCommand(groupID string) *cobra.Command {
opts := rmOptions{} opts := rmOptions{}
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "rm SERVICE [SERVICE...]", Use: "rm SERVICE [SERVICE...]",
Aliases: []string{"remove", "delete"}, Aliases: []string{"remove", "delete"},
Short: "Remove one or more services.", Short: "Remove one or more services.",
Long: `Remove one or more services.
The volumes used by the services are preserved and should be removed separately
with 'uc volume rm'. Anonymous Docker volumes (automatically created from VOLUME
directives in image Dockerfiles) are automatically removed with their containers.`,
Args: cobra.MinimumNArgs(1), Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error { RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI) uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.services = args opts.services = args
return rm(cmd.Context(), uncli, opts) return rm(cmd.Context(), uncli, opts)
}, },
GroupID: groupID,
} }
cmd.Flags().StringVarP(
&opts.cluster, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd return cmd
} }
func rm(ctx context.Context, uncli *cli.CLI, opts rmOptions) error { func rm(ctx context.Context, uncli *cli.CLI, opts rmOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.cluster) client, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
@@ -50,5 +51,5 @@ func rm(ctx context.Context, uncli *cli.CLI, opts rmOptions) error {
}, uncli.ProgressOut(), "Removing service "+s) }, uncli.ProgressOut(), "Removing service "+s)
} }
return nil return err
} }
+10 -6
View File
@@ -8,14 +8,18 @@ func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "service", Use: "service",
Aliases: []string{"svc"}, Aliases: []string{"svc"},
Short: "Manage services in an Uncloud cluster.", Short: "Manage services in the cluster.",
} }
cmd.AddCommand( cmd.AddCommand(
NewInspectCommand(), NewExecCommand(""),
NewListCommand(), NewInspectCommand(""),
NewRmCommand(), NewListCommand(""),
NewRunCommand(), NewLogsCommand(""),
NewScaleCommand(), NewRmCommand(""),
NewRunCommand(""),
NewScaleCommand(""),
NewStartCommand(""),
NewStopCommand(""),
) )
return cmd return cmd
} }
+23 -10
View File
@@ -17,6 +17,7 @@ import (
) )
type runOptions struct { type runOptions struct {
caddyfile string
command []string command []string
cpu dockeropts.NanoCPUs cpu dockeropts.NanoCPUs
entrypoint string entrypoint string
@@ -33,11 +34,9 @@ type runOptions struct {
replicas uint replicas uint
user string user string
volumes []string volumes []string
cluster string
} }
func NewRunCommand() *cobra.Command { func NewRunCommand(groupID string) *cobra.Command {
opts := runOptions{} opts := runOptions{}
cmd := &cobra.Command{ cmd := &cobra.Command{
@@ -55,8 +54,12 @@ func NewRunCommand() *cobra.Command {
return run(cmd.Context(), uncli, opts) return run(cmd.Context(), uncli, opts)
}, },
GroupID: groupID,
} }
cmd.Flags().StringVar(&opts.caddyfile, "caddyfile", "",
"Path to a custom Caddy config (Caddyfile) for the service. "+
"Cannot be used together with non-@host published ports.")
cmd.Flags().VarP(&opts.cpu, "cpu", "", cmd.Flags().VarP(&opts.cpu, "cpu", "",
"Maximum number of CPU cores a service container can use. Fractional values are allowed: "+ "Maximum number of CPU cores a service container can use. Fractional values are allowed: "+
"0.5 for half a core or 2.25 for two and a quarter cores.") "0.5 for half a core or 2.25 for two and a quarter cores.")
@@ -82,7 +85,7 @@ func NewRunCommand() *cobra.Command {
"Give extended privileges to service containers. This is a security risk and should be used with caution.") "Give extended privileges to service containers. This is a security risk and should be used with caution.")
cmd.Flags().StringSliceVarP(&opts.publish, "publish", "p", nil, cmd.Flags().StringSliceVarP(&opts.publish, "publish", "p", nil,
"Publish a service port to make it accessible outside the cluster. Can be specified multiple times.\n"+ "Publish a service port to make it accessible outside the cluster. Can be specified multiple times.\n"+
"Format: [hostname:][load_balancer_port:]container_port[/protocol] or [host_ip:]:host_port:container_port[/protocol]@host\n"+ "Format: [hostname:]container_port[/protocol] or [host_ip:]host_port:container_port[/protocol]@host\n"+
"Supported protocols: tcp, udp, http, https (default is tcp). If a hostname for http(s) port is not specified\n"+ "Supported protocols: tcp, udp, http, https (default is tcp). If a hostname for http(s) port is not specified\n"+
"and a cluster domain is reserved, service-name.cluster-domain will be used as the hostname.\n"+ "and a cluster domain is reserved, service-name.cluster-domain will be used as the hostname.\n"+
"Examples:\n"+ "Examples:\n"+
@@ -108,11 +111,6 @@ func NewRunCommand() *cobra.Command {
" -v /data/uploads:/app/uploads Bind mount /data/uploads host directory to /app/uploads in container\n"+ " -v /data/uploads:/app/uploads Bind mount /data/uploads host directory to /app/uploads in container\n"+
" -v /host/path:/container/path:ro Bind mount a host directory or file as read-only") " -v /host/path:/container/path:ro Bind mount a host directory or file as read-only")
cmd.Flags().StringVarP(
&opts.cluster, "context", "c", "",
"Name of the cluster context to run the service in. (default is the current context)",
)
return cmd return cmd
} }
@@ -122,7 +120,7 @@ func run(ctx context.Context, uncli *cli.CLI, opts runOptions) error {
return err return err
} }
clusterClient, err := uncli.ConnectCluster(ctx, opts.cluster) clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
@@ -161,6 +159,15 @@ func run(ctx context.Context, uncli *cli.CLI, opts runOptions) error {
func prepareServiceSpec(opts runOptions) (api.ServiceSpec, error) { func prepareServiceSpec(opts runOptions) (api.ServiceSpec, error) {
var spec api.ServiceSpec var spec api.ServiceSpec
caddyfile := ""
if opts.caddyfile != "" {
data, err := os.ReadFile(opts.caddyfile)
if err != nil {
return spec, fmt.Errorf("read Caddyfile: %w", err)
}
caddyfile = strings.TrimSpace(string(data))
}
env, err := parseEnv(opts.env) env, err := parseEnv(opts.env)
if err != nil { if err != nil {
return spec, err return spec, err
@@ -218,6 +225,12 @@ func prepareServiceSpec(opts runOptions) (api.ServiceSpec, error) {
Volumes: volumes, Volumes: volumes,
} }
if caddyfile != "" {
spec.Caddy = &api.CaddySpec{
Config: caddyfile,
}
}
// Overwrite the default ENTRYPOINT of the image or reset it if an empty string is passed. // Overwrite the default ENTRYPOINT of the image or reset it if an empty string is passed.
if opts.entrypoint != "" { if opts.entrypoint != "" {
spec.Container.Entrypoint = []string{opts.entrypoint} spec.Container.Entrypoint = []string{opts.entrypoint}
+3 -8
View File
@@ -15,10 +15,9 @@ import (
type scaleOptions struct { type scaleOptions struct {
service string service string
replicas uint replicas uint
cluster string
} }
func NewScaleCommand() *cobra.Command { func NewScaleCommand(groupID string) *cobra.Command {
opts := scaleOptions{} opts := scaleOptions{}
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "scale SERVICE REPLICAS", Use: "scale SERVICE REPLICAS",
@@ -37,13 +36,9 @@ func NewScaleCommand() *cobra.Command {
return scale(cmd.Context(), uncli, opts) return scale(cmd.Context(), uncli, opts)
}, },
GroupID: groupID,
} }
cmd.Flags().StringVarP(
&opts.cluster, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd return cmd
} }
@@ -57,7 +52,7 @@ func scale(ctx context.Context, uncli *cli.CLI, opts scaleOptions) error {
) )
} }
clusterClient, err := uncli.ConnectCluster(ctx, opts.cluster) clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
+54
View File
@@ -0,0 +1,54 @@
//nolint:dupl
package service
import (
"context"
"fmt"
"github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
type startOptions struct {
services []string
}
func NewStartCommand(groupID string) *cobra.Command {
opts := startOptions{}
cmd := &cobra.Command{
Use: "start SERVICE [SERVICE...]",
Short: "Start one or more services.",
Long: `Start one or more previously stopped services.
Starts all containers of the specified service(s) across all machines in the cluster.
Services can be specified by name or ID.`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.services = args
return start(cmd.Context(), uncli, opts)
},
GroupID: groupID,
}
return cmd
}
func start(ctx context.Context, uncli *cli.CLI, opts startOptions) error {
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
for _, s := range opts.services {
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
if err = client.StartService(ctx, s); err != nil {
return fmt.Errorf("start service '%s': %w", s, err)
}
return nil
}, uncli.ProgressOut(), "Starting service "+s)
}
return err
}
+68
View File
@@ -0,0 +1,68 @@
//nolint:dupl
package service
import (
"context"
"fmt"
"github.com/docker/compose/v2/pkg/progress"
"github.com/docker/docker/api/types/container"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
type stopOptions struct {
services []string
signal string
timeout int
}
func NewStopCommand(groupID string) *cobra.Command {
opts := stopOptions{}
cmd := &cobra.Command{
Use: "stop SERVICE [SERVICE...]",
Short: "Stop one or more services.",
Long: `Stop one or more running services.
Gracefully stops all containers of the specified service(s) across all machines in the cluster.
Services can be specified by name or ID. Stopped services can be restarted with 'uc start'.`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.services = args
return stop(cmd.Context(), uncli, opts)
},
GroupID: groupID,
}
cmd.Flags().StringVarP(&opts.signal, "signal", "s", "",
"Signal to send to each container's main process.\n"+
"Can be a signal name (SIGTERM, SIGINT, SIGHUP, etc.) or a number. (default SIGTERM)")
cmd.Flags().IntVarP(&opts.timeout, "timeout", "t", 10,
"Seconds to wait for each container to stop gracefully before forcibly killing it with SIGKILL.\n"+
"Use -1 to wait indefinitely.")
return cmd
}
func stop(ctx context.Context, uncli *cli.CLI, opts stopOptions) error {
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
stopOpts := container.StopOptions{
Signal: opts.signal,
Timeout: &opts.timeout,
}
for _, s := range opts.services {
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
if err = client.StopService(ctx, s, stopOpts); err != nil {
return fmt.Errorf("stop service '%s': %w", s, err)
}
return nil
}, uncli.ProgressOut(), "Stopping service "+s)
}
return err
}
+1 -4
View File
@@ -19,7 +19,6 @@ type createOptions struct {
driverOpts []string driverOpts []string
labels []string labels []string
machine string machine string
context string
} }
func NewCreateCommand() *cobra.Command { func NewCreateCommand() *cobra.Command {
@@ -50,14 +49,12 @@ func NewCreateCommand() *cobra.Command {
"Labels to assign to the volume in the form of 'key=value' pairs. Can be specified multiple times.") "Labels to assign to the volume in the form of 'key=value' pairs. Can be specified multiple times.")
cmd.Flags().StringVarP(&opts.machine, "machine", "m", "", cmd.Flags().StringVarP(&opts.machine, "machine", "m", "",
"Name or ID of the machine to create the volume on.") "Name or ID of the machine to create the volume on.")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd return cmd
} }
func create(ctx context.Context, uncli *cli.CLI, name string, opts createOptions) error { func create(ctx context.Context, uncli *cli.CLI, name string, opts createOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context) client, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
+1 -4
View File
@@ -13,7 +13,6 @@ import (
type inspectOptions struct { type inspectOptions struct {
machine string machine string
context string
} }
func NewInspectCommand() *cobra.Command { func NewInspectCommand() *cobra.Command {
@@ -32,14 +31,12 @@ func NewInspectCommand() *cobra.Command {
cmd.Flags().StringVarP(&opts.machine, "machine", "m", "", cmd.Flags().StringVarP(&opts.machine, "machine", "m", "",
"Name or ID of the machine where the volume is located. "+ "Name or ID of the machine where the volume is located. "+
"If not specified, the volume will be searched across all machines.") "If not specified, the volume will be searched across all machines.")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd return cmd
} }
func inspect(ctx context.Context, uncli *cli.CLI, name string, opts inspectOptions) error { func inspect(ctx context.Context, uncli *cli.CLI, name string, opts inspectOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context) client, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
+1 -5
View File
@@ -16,7 +16,6 @@ import (
type listOptions struct { type listOptions struct {
machines []string machines []string
quiet bool quiet bool
context string
} }
func NewListCommand() *cobra.Command { func NewListCommand() *cobra.Command {
@@ -38,14 +37,11 @@ func NewListCommand() *cobra.Command {
cmd.Flags().BoolVarP(&opts.quiet, "quiet", "q", false, cmd.Flags().BoolVarP(&opts.quiet, "quiet", "q", false,
"Only display volume names.") "Only display volume names.")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd return cmd
} }
func list(ctx context.Context, uncli *cli.CLI, opts listOptions) error { func list(ctx context.Context, uncli *cli.CLI, opts listOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context) client, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
+1 -5
View File
@@ -14,7 +14,6 @@ type removeOptions struct {
force bool force bool
machines []string machines []string
yes bool yes bool
context string
} }
func NewRemoveCommand() *cobra.Command { func NewRemoveCommand() *cobra.Command {
@@ -41,14 +40,11 @@ func NewRemoveCommand() *cobra.Command {
cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false, cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
"Do not prompt for confirmation before removing the volume(s).") "Do not prompt for confirmation before removing the volume(s).")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd return cmd
} }
func remove(ctx context.Context, uncli *cli.CLI, names []string, opts removeOptions) error { func remove(ctx context.Context, uncli *cli.CLI, names []string, opts removeOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context) client, err := uncli.ConnectCluster(ctx)
if err != nil { if err != nil {
return fmt.Errorf("connect to cluster: %w", err) return fmt.Errorf("connect to cluster: %w", err)
} }
+1 -1
View File
@@ -7,7 +7,7 @@ import (
func NewRootCommand() *cobra.Command { func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "volume", Use: "volume",
Short: "Manage volumes in an Uncloud cluster.", Short: "Manage volumes in the cluster.",
} }
cmd.AddCommand( cmd.AddCommand(
NewCreateCommand(), NewCreateCommand(),
+128
View File
@@ -0,0 +1,128 @@
package main
import (
"context"
"fmt"
"io"
"net"
"os"
"github.com/psviderski/uncloud/internal/machine"
"github.com/spf13/cobra"
)
func newDialStdioCommand() *cobra.Command {
var socketPath string
cmd := &cobra.Command{
Use: "dial-stdio",
Short: "Proxy stdin/stdout to the Uncloud API socket",
Hidden: true,
RunE: func(cmd *cobra.Command, args []string) error {
return runDialStdio(cmd.Context(), socketPath, os.Stdin, os.Stdout)
},
}
cmd.Flags().StringVar(&socketPath, "socket", machine.DefaultUncloudSockPath,
"Path to the Uncloud API socket")
return cmd
}
// halfReadCloser is the read side of a half-duplex connection.
type halfReadCloser interface {
io.Reader
CloseRead() error
}
// halfWriteCloser is the write side of a half-duplex connection.
type halfWriteCloser interface {
io.Writer
CloseWrite() error
}
// halfReadCloserWrapper wraps an io.ReadCloser to implement halfReadCloser.
type halfReadCloserWrapper struct {
io.ReadCloser
}
func (x *halfReadCloserWrapper) CloseRead() error {
return x.Close()
}
// halfWriteCloserWrapper wraps an io.WriteCloser to implement halfWriteCloser.
type halfWriteCloserWrapper struct {
io.WriteCloser
}
func (x *halfWriteCloserWrapper) CloseWrite() error {
return x.Close()
}
func runDialStdio(ctx context.Context, socketPath string, stdin io.Reader, stdout io.Writer) error {
// Connect to the unix socket.
var dialer net.Dialer
conn, err := dialer.DialContext(ctx, "unix", socketPath)
if err != nil {
return fmt.Errorf("connect to socket %q: %w", socketPath, err)
}
defer conn.Close()
// Wrap stdin/stdout to support half-closing.
var stdinCloser halfReadCloser
if c, ok := stdin.(halfReadCloser); ok {
stdinCloser = c
} else if c, ok := stdin.(io.ReadCloser); ok {
stdinCloser = &halfReadCloserWrapper{c}
}
var stdoutCloser halfWriteCloser
if c, ok := stdout.(halfWriteCloser); ok {
stdoutCloser = c
} else if c, ok := stdout.(io.WriteCloser); ok {
stdoutCloser = &halfWriteCloserWrapper{c}
}
// Copy data bidirectionally between stdin/stdout and the socket.
stdin2socket := make(chan error, 1)
socket2stdout := make(chan error, 1)
// Copy from stdin to socket.
go func() {
_, err := io.Copy(conn, stdin)
stdin2socket <- err
// Close write side of connection after stdin is done.
if unixConn, ok := conn.(*net.UnixConn); ok {
unixConn.CloseWrite()
}
if stdinCloser != nil {
stdinCloser.CloseRead()
}
}()
// Copy from socket to stdout.
go func() {
_, err := io.Copy(stdout, conn)
socket2stdout <- err
// Close read side of connection after socket is done sending.
if unixConn, ok := conn.(*net.UnixConn); ok {
unixConn.CloseRead()
}
if stdoutCloser != nil {
stdoutCloser.CloseWrite()
}
}()
select {
case err = <-stdin2socket:
if err != nil {
return err
}
// wait for stdout
err = <-socket2stdout
case err = <-socket2stdout:
// return immediately, matching Docker's approach
// (stdin is never closed when TTY)
}
return err
}
+3
View File
@@ -42,6 +42,9 @@ func main() {
"Directory for storing persistent machine state") "Directory for storing persistent machine state")
_ = cmd.MarkFlagDirname("data-dir") _ = cmd.MarkFlagDirname("data-dir")
// Add dial-stdio subcommand.
cmd.AddCommand(newDialStdioCommand())
// ctx is canceled when the daemon command is interrupted. // ctx is canceled when the daemon command is interrupted.
ctx, cancel := context.WithCancel(context.Background()) ctx, cancel := context.WithCancel(context.Background())
+187 -89
View File
@@ -1,24 +1,30 @@
module github.com/psviderski/uncloud module github.com/psviderski/uncloud
go 1.23.0 go 1.25
require ( require (
github.com/BurntSushi/toml v1.4.0 github.com/BurntSushi/toml v1.5.0
github.com/Masterminds/semver v1.5.0 github.com/Masterminds/semver v1.5.0
github.com/Masterminds/squirrel v1.5.4 github.com/Masterminds/squirrel v1.5.4
github.com/alecthomas/chroma/v2 v2.20.0
github.com/caddyserver/caddy/v2 v2.8.4 github.com/caddyserver/caddy/v2 v2.8.4
github.com/cenkalti/backoff/v4 v4.3.0 github.com/cenkalti/backoff/v4 v4.3.0
github.com/charmbracelet/bubbles v0.21.0
github.com/charmbracelet/bubbletea v1.3.10
github.com/charmbracelet/huh v0.6.0 github.com/charmbracelet/huh v0.6.0
github.com/charmbracelet/lipgloss v0.13.0 github.com/charmbracelet/huh/spinner v0.0.0-20251110114415-25888d17260b
github.com/compose-spec/compose-go/v2 v2.4.5 github.com/charmbracelet/lipgloss v1.1.0
github.com/compose-spec/compose-go/v2 v2.9.0
github.com/containerd/errdefs v1.0.0
github.com/containerd/platforms v1.0.0-rc.1
github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf github.com/coreos/go-systemd v0.0.0-20191104093116-d3cd4ed1dbcf
github.com/deckarep/golang-set/v2 v2.8.0 github.com/deckarep/golang-set/v2 v2.8.0
github.com/dgraph-io/badger/v3 v3.2103.5 github.com/dgraph-io/badger/v3 v3.2103.5
github.com/distribution/reference v0.6.0 github.com/distribution/reference v0.6.0
github.com/docker/cli v27.5.0+incompatible github.com/docker/cli v28.5.0+incompatible
github.com/docker/compose/v2 v2.31.0 github.com/docker/compose/v2 v2.40.0
github.com/docker/docker v27.4.0-rc.2+incompatible github.com/docker/docker v28.5.0+incompatible
github.com/docker/go-connections v0.5.0 github.com/docker/go-connections v0.6.0
github.com/docker/go-units v0.5.0 github.com/docker/go-units v0.5.0
github.com/goccy/go-yaml v1.17.1 github.com/goccy/go-yaml v1.17.1
github.com/google/go-cmp v0.7.0 github.com/google/go-cmp v0.7.0
@@ -36,45 +42,66 @@ require (
github.com/lmittmann/tint v1.0.5 github.com/lmittmann/tint v1.0.5
github.com/miekg/dns v1.1.65 github.com/miekg/dns v1.1.65
github.com/mitchellh/mapstructure v1.5.0 github.com/mitchellh/mapstructure v1.5.0
github.com/moby/term v0.5.0 github.com/moby/term v0.5.2
github.com/muesli/termenv v0.16.0
github.com/opencontainers/go-digest v1.0.0 github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.1.0 github.com/opencontainers/image-spec v1.1.1
github.com/psviderski/unregistry v0.4.1
github.com/siderolabs/discovery-api v0.1.4 github.com/siderolabs/discovery-api v0.1.4
github.com/siderolabs/discovery-client v0.1.9 github.com/siderolabs/discovery-client v0.1.9
github.com/siderolabs/grpc-proxy v0.5.1 github.com/siderolabs/grpc-proxy v0.5.1
github.com/spf13/cobra v1.8.1 github.com/spf13/cobra v1.10.1
github.com/stretchr/testify v1.10.0 github.com/stretchr/testify v1.11.1
github.com/vishvananda/netlink v1.3.0 github.com/vishvananda/netlink v1.3.1
go.uber.org/zap v1.27.0 go.uber.org/zap v1.27.0
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba go4.org/netipx v0.0.0-20231129151722-fdeea329fbba
golang.org/x/crypto v0.33.0 golang.org/x/crypto v0.41.0
golang.org/x/net v0.35.0 golang.org/x/net v0.43.0
golang.org/x/sync v0.11.0 golang.org/x/sync v0.17.0
golang.org/x/sys v0.31.0 golang.org/x/sys v0.36.0
golang.org/x/term v0.34.0
golang.zx2c4.com/wireguard v0.0.0-20231211153847-12269c276173 golang.zx2c4.com/wireguard v0.0.0-20231211153847-12269c276173
golang.zx2c4.com/wireguard/wgctrl v0.0.0-20230429144221-925a1e7659e6 golang.zx2c4.com/wireguard/wgctrl v0.0.0-20230429144221-925a1e7659e6
google.golang.org/genproto/googleapis/rpc v0.0.0-20241209162323-e6fa225c2576 google.golang.org/genproto/googleapis/rpc v0.0.0-20250528174236-200df99c418a
google.golang.org/grpc v1.68.1 google.golang.org/grpc v1.74.2
google.golang.org/protobuf v1.36.3 google.golang.org/protobuf v1.36.9
modernc.org/sqlite v1.36.3 modernc.org/sqlite v1.36.3
) )
require ( require (
cloud.google.com/go/kms v1.19.0 // indirect
cloud.google.com/go/longrunning v0.6.0 // indirect
filippo.io/edwards25519 v1.1.0 // indirect filippo.io/edwards25519 v1.1.0 // indirect
github.com/AdaLogics/go-fuzz-headers v0.0.0-20230811130428-ced1acdcaa24 // indirect github.com/AlecAivazis/survey/v2 v2.3.7 // indirect
github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 // indirect github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96 // indirect
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
github.com/DefangLabs/secret-detector v0.0.0-20250403165618-22662109213e // indirect
github.com/Masterminds/goutils v1.1.1 // indirect github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.2.1 // indirect github.com/Masterminds/semver/v3 v3.4.0 // indirect
github.com/Masterminds/sprig/v3 v3.2.3 // indirect github.com/Masterminds/sprig/v3 v3.2.3 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/OneOfOne/xxhash v1.2.8 // indirect github.com/Microsoft/hcsshim v0.13.0 // indirect
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d // indirect
github.com/antlr4-go/antlr/v4 v4.13.0 // indirect github.com/antlr4-go/antlr/v4 v4.13.0 // indirect
github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect
github.com/armon/circbuf v0.0.0-20190214190532-5111143e8da2 // indirect github.com/armon/circbuf v0.0.0-20190214190532-5111143e8da2 // indirect
github.com/armon/go-metrics v0.4.1 // indirect github.com/armon/go-metrics v0.4.1 // indirect
github.com/armon/go-radix v1.0.0 // indirect github.com/armon/go-radix v1.0.0 // indirect
github.com/aryann/difflib v0.0.0-20210328193216-ff5ff6dc229b // indirect github.com/aryann/difflib v0.0.0-20210328193216-ff5ff6dc229b // indirect
github.com/atotto/clipboard v0.1.4 // indirect github.com/atotto/clipboard v0.1.4 // indirect
github.com/aws/aws-sdk-go-v2 v1.30.3 // indirect
github.com/aws/aws-sdk-go-v2/config v1.27.27 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.17.27 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.11 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.15 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.15 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.11.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.11.17 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.22.4 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.26.4 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.30.3 // indirect
github.com/aws/smithy-go v1.20.3 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/beorn7/perks v1.0.1 // indirect github.com/beorn7/perks v1.0.1 // indirect
github.com/bgentry/speakeasy v0.2.0 // indirect github.com/bgentry/speakeasy v0.2.0 // indirect
@@ -84,67 +111,86 @@ require (
github.com/catppuccin/go v0.2.0 // indirect github.com/catppuccin/go v0.2.0 // indirect
github.com/cespare/xxhash v1.1.0 // indirect github.com/cespare/xxhash v1.1.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/charmbracelet/bubbles v0.20.0 // indirect github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc // indirect
github.com/charmbracelet/bubbletea v1.1.1 // indirect github.com/charmbracelet/x/ansi v0.10.1 // indirect
github.com/charmbracelet/x/ansi v0.3.2 // indirect github.com/charmbracelet/x/cellbuf v0.0.13-0.20250311204145-2c3ea96c31dd // indirect
github.com/charmbracelet/x/exp/strings v0.0.0-20240919170804-a4978c8e603a // indirect github.com/charmbracelet/x/exp/strings v0.0.0-20240919170804-a4978c8e603a // indirect
github.com/charmbracelet/x/term v0.2.0 // indirect github.com/charmbracelet/x/term v0.2.1 // indirect
github.com/chzyer/readline v1.5.1 // indirect github.com/chzyer/readline v1.5.1 // indirect
github.com/cloudflare/cfssl v1.6.4 // indirect github.com/containerd/cgroups/v3 v3.0.5 // indirect
github.com/containerd/console v1.0.4 // indirect github.com/containerd/console v1.0.5 // indirect
github.com/containerd/containerd v1.7.24 // indirect github.com/containerd/containerd/api v1.9.0 // indirect
github.com/containerd/containerd/api v1.7.19 // indirect github.com/containerd/containerd/v2 v2.1.4 // indirect
github.com/containerd/continuity v0.4.4 // indirect github.com/containerd/continuity v0.4.5 // indirect
github.com/containerd/errdefs v0.3.0 // indirect github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/containerd/fifo v1.1.0 // indirect
github.com/containerd/log v0.1.0 // indirect github.com/containerd/log v0.1.0 // indirect
github.com/containerd/platforms v0.2.1 // indirect github.com/containerd/plugin v1.0.0 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.16.3 // indirect github.com/containerd/stargz-snapshotter/estargz v0.16.3 // indirect
github.com/containerd/ttrpc v1.2.5 // indirect github.com/containerd/ttrpc v1.2.7 // indirect
github.com/containerd/typeurl/v2 v2.2.0 // indirect github.com/containerd/typeurl/v2 v2.2.3 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.6 // indirect github.com/cpuguy83/go-md2man/v2 v2.0.7 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/decred/dcrd/dcrec/secp256k1/v4 v4.3.0 // indirect github.com/decred/dcrd/dcrec/secp256k1/v4 v4.3.0 // indirect
github.com/dgraph-io/badger v1.6.2 // indirect github.com/dgraph-io/badger v1.6.2 // indirect
github.com/dgraph-io/badger/v2 v2.2007.4 // indirect github.com/dgraph-io/badger/v2 v2.2007.4 // indirect
github.com/dgraph-io/ristretto v0.1.1 // indirect github.com/dgraph-io/ristretto v0.1.1 // indirect
github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect
github.com/docker/buildx v0.18.0 // indirect github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/distribution/distribution/v3 v3.0.0 // indirect
github.com/dlclark/regexp2 v1.11.5 // indirect
github.com/docker/buildx v0.29.1 // indirect
github.com/docker/cli-docs-tool v0.10.0 // indirect
github.com/docker/distribution v2.8.3+incompatible // indirect github.com/docker/distribution v2.8.3+incompatible // indirect
github.com/docker/docker-credential-helpers v0.8.2 // indirect github.com/docker/docker-credential-helpers v0.9.3 // indirect
github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c // indirect github.com/docker/go v1.5.1-1.0.20160303222718-d30aec9fd63c // indirect
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect
github.com/docker/go-metrics v0.0.1 // indirect github.com/docker/go-metrics v0.0.1 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect github.com/dustin/go-humanize v1.0.1 // indirect
github.com/eiannone/keyboard v0.0.0-20220611211555-0d226195f203 // indirect
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
github.com/fatih/color v1.17.0 // indirect github.com/fatih/color v1.18.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fsnotify/fsevents v0.2.0 // indirect
github.com/fvbommel/sortorder v1.1.0 // indirect github.com/fvbommel/sortorder v1.1.0 // indirect
github.com/fxamacker/cbor/v2 v2.7.0 // indirect
github.com/go-jose/go-jose/v3 v3.0.3 // indirect github.com/go-jose/go-jose/v3 v3.0.3 // indirect
github.com/go-kit/kit v0.13.0 // indirect github.com/go-kit/kit v0.13.0 // indirect
github.com/go-kit/log v0.2.1 // indirect github.com/go-kit/log v0.2.1 // indirect
github.com/go-logfmt/logfmt v0.6.0 // indirect github.com/go-logfmt/logfmt v0.6.0 // indirect
github.com/go-logr/logr v1.4.2 // indirect github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-openapi/jsonpointer v0.21.0 // indirect
github.com/go-openapi/jsonreference v0.20.2 // indirect
github.com/go-openapi/swag v0.23.0 // indirect
github.com/go-sql-driver/mysql v1.8.1 // indirect github.com/go-sql-driver/mysql v1.8.1 // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/go-viper/mapstructure/v2 v2.0.0 // indirect github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/godbus/dbus/v5 v5.1.0 // indirect github.com/godbus/dbus/v5 v5.1.0 // indirect
github.com/gofrs/flock v0.12.1 // indirect github.com/gofrs/flock v0.12.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/glog v1.2.2 // indirect github.com/golang-jwt/jwt/v5 v5.2.2 // indirect
github.com/golang/glog v1.2.5 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.4 // indirect github.com/golang/protobuf v1.5.4 // indirect
github.com/golang/snappy v0.0.4 // indirect github.com/golang/snappy v0.0.4 // indirect
github.com/google/btree v1.1.2 // indirect github.com/google/btree v1.1.2 // indirect
github.com/google/cel-go v0.20.1 // indirect github.com/google/cel-go v0.20.1 // indirect
github.com/google/flatbuffers v24.3.25+incompatible // indirect github.com/google/flatbuffers v24.3.25+incompatible // indirect
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e // indirect github.com/google/gnostic-models v0.6.8 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/pprof v0.0.0-20250403155104-27863c87afa6 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/google/uuid v1.6.0 // indirect github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/handlers v1.5.2 // indirect
github.com/gorilla/mux v1.8.1 // indirect github.com/gorilla/mux v1.8.1 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.24.0 // indirect github.com/gorilla/websocket v1.5.3 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-immutable-radix v1.3.1 // indirect github.com/hashicorp/go-immutable-radix v1.3.1 // indirect
github.com/hashicorp/go-immutable-radix/v2 v2.1.0 // indirect
github.com/hashicorp/go-msgpack v0.5.5 // indirect github.com/hashicorp/go-msgpack v0.5.5 // indirect
github.com/hashicorp/go-msgpack/v2 v2.1.2 // indirect github.com/hashicorp/go-msgpack/v2 v2.1.2 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect github.com/hashicorp/go-multierror v1.1.1 // indirect
@@ -152,13 +198,15 @@ require (
github.com/hashicorp/go-syslog v1.0.0 // indirect github.com/hashicorp/go-syslog v1.0.0 // indirect
github.com/hashicorp/go-version v1.7.0 // indirect github.com/hashicorp/go-version v1.7.0 // indirect
github.com/hashicorp/golang-lru v1.0.2 // indirect github.com/hashicorp/golang-lru v1.0.2 // indirect
github.com/hashicorp/golang-lru/arc/v2 v2.0.7 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/hashicorp/logutils v1.0.0 // indirect github.com/hashicorp/logutils v1.0.0 // indirect
github.com/hashicorp/mdns v1.0.5 // indirect github.com/hashicorp/mdns v1.0.5 // indirect
github.com/huandu/xstrings v1.5.0 // indirect github.com/huandu/xstrings v1.5.0 // indirect
github.com/imdario/mergo v0.3.16 // indirect github.com/imdario/mergo v0.3.16 // indirect
github.com/in-toto/in-toto-golang v0.5.0 // indirect github.com/in-toto/in-toto-golang v0.9.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/inhies/go-bytesize v0.0.0-20220417184213-4913239db9cf // indirect
github.com/ipfs/bbloom v0.0.4 // indirect github.com/ipfs/bbloom v0.0.4 // indirect
github.com/ipfs/go-block-format v0.2.0 // indirect github.com/ipfs/go-block-format v0.2.0 // indirect
github.com/ipfs/go-ipfs-util v0.0.3 // indirect github.com/ipfs/go-ipfs-util v0.0.3 // indirect
@@ -176,8 +224,12 @@ require (
github.com/jackc/pgtype v1.14.0 // indirect github.com/jackc/pgtype v1.14.0 // indirect
github.com/jackc/pgx/v4 v4.18.3 // indirect github.com/jackc/pgx/v4 v4.18.3 // indirect
github.com/jbenet/goprocess v0.1.4 // indirect github.com/jbenet/goprocess v0.1.4 // indirect
github.com/jonboulle/clockwork v0.5.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/josharian/native v1.1.0 // indirect github.com/josharian/native v1.1.0 // indirect
github.com/klauspost/compress v1.17.11 // indirect github.com/json-iterator/go v1.1.12 // indirect
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
github.com/klauspost/compress v1.18.0 // indirect
github.com/klauspost/cpuid/v2 v2.2.9 // indirect github.com/klauspost/cpuid/v2 v2.2.9 // indirect
github.com/lann/builder v0.0.0-20180802200727-47ae307949d0 // indirect github.com/lann/builder v0.0.0-20180802200727-47ae307949d0 // indirect
github.com/lann/ps v0.0.0-20150810152359-62de8c46ede0 // indirect github.com/lann/ps v0.0.0-20150810152359-62de8c46ede0 // indirect
@@ -187,6 +239,7 @@ require (
github.com/libp2p/go-libp2p-pubsub v0.11.0 // indirect github.com/libp2p/go-libp2p-pubsub v0.11.0 // indirect
github.com/libp2p/go-msgio v0.3.0 // indirect github.com/libp2p/go-msgio v0.3.0 // indirect
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/manifoldco/promptui v0.9.0 // indirect github.com/manifoldco/promptui v0.9.0 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect github.com/mattn/go-isatty v0.0.20 // indirect
@@ -206,19 +259,26 @@ require (
github.com/mitchellh/go-ps v1.0.0 // indirect github.com/mitchellh/go-ps v1.0.0 // indirect
github.com/mitchellh/hashstructure/v2 v2.0.2 // indirect github.com/mitchellh/hashstructure/v2 v2.0.2 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/moby/buildkit v0.17.2 // indirect github.com/moby/buildkit v0.25.0 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/go-archive v0.1.0 // indirect
github.com/moby/locker v1.0.1 // indirect github.com/moby/locker v1.0.1 // indirect
github.com/moby/patternmatcher v0.6.0 // indirect github.com/moby/patternmatcher v0.6.0 // indirect
github.com/moby/spdystream v0.5.0 // indirect
github.com/moby/sys/atomicwriter v0.1.0 // indirect
github.com/moby/sys/capability v0.4.0 // indirect
github.com/moby/sys/mountinfo v0.7.2 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect github.com/moby/sys/sequential v0.6.0 // indirect
github.com/moby/sys/signal v0.7.1 // indirect github.com/moby/sys/signal v0.7.1 // indirect
github.com/moby/sys/user v0.3.0 // indirect github.com/moby/sys/symlink v0.3.0 // indirect
github.com/moby/sys/user v0.4.0 // indirect
github.com/moby/sys/userns v0.1.0 // indirect github.com/moby/sys/userns v0.1.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/morikuni/aec v1.0.0 // indirect github.com/morikuni/aec v1.0.0 // indirect
github.com/mr-tron/base58 v1.2.0 // indirect github.com/mr-tron/base58 v1.2.0 // indirect
github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect
github.com/muesli/cancelreader v0.2.2 // indirect github.com/muesli/cancelreader v0.2.2 // indirect
github.com/muesli/termenv v0.15.3-0.20240618155329-98d742f6907a // indirect
github.com/multiformats/go-base32 v0.1.0 // indirect github.com/multiformats/go-base32 v0.1.0 // indirect
github.com/multiformats/go-base36 v0.2.0 // indirect github.com/multiformats/go-base36 v0.2.0 // indirect
github.com/multiformats/go-multiaddr v0.13.0 // indirect github.com/multiformats/go-multiaddr v0.13.0 // indirect
@@ -229,93 +289,131 @@ require (
github.com/multiformats/go-multistream v0.5.0 // indirect github.com/multiformats/go-multistream v0.5.0 // indirect
github.com/multiformats/go-varint v0.0.7 // indirect github.com/multiformats/go-varint v0.0.7 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/onsi/ginkgo/v2 v2.22.0 // indirect github.com/onsi/ginkgo/v2 v2.22.0 // indirect
github.com/opencontainers/runtime-spec v1.2.1 // indirect
github.com/opencontainers/selinux v1.12.0 // indirect
github.com/pelletier/go-toml v1.9.5 // indirect github.com/pelletier/go-toml v1.9.5 // indirect
github.com/pires/go-proxyproto v0.7.0 // indirect github.com/pires/go-proxyproto v0.7.0 // indirect
github.com/pkg/errors v0.9.1 // indirect github.com/pkg/errors v0.9.1 // indirect
github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/polydawn/refmt v0.89.0 // indirect github.com/polydawn/refmt v0.89.0 // indirect
github.com/posener/complete v1.2.3 // indirect github.com/posener/complete v1.2.3 // indirect
github.com/prometheus/client_golang v1.20.5 // indirect github.com/prometheus/client_golang v1.22.0 // indirect
github.com/prometheus/client_model v0.6.1 // indirect github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.61.0 // indirect github.com/prometheus/common v0.62.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect github.com/prometheus/procfs v0.15.1 // indirect
github.com/quic-go/qpack v0.5.1 // indirect github.com/quic-go/qpack v0.5.1 // indirect
github.com/quic-go/quic-go v0.48.2 // indirect github.com/quic-go/quic-go v0.48.2 // indirect
github.com/redis/go-redis/extra/rediscmd/v9 v9.0.5 // indirect
github.com/redis/go-redis/extra/redisotel/v9 v9.0.5 // indirect
github.com/redis/go-redis/v9 v9.7.3 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/rivo/uniseg v0.4.7 // indirect github.com/rivo/uniseg v0.4.7 // indirect
github.com/rs/xid v1.5.0 // indirect github.com/rs/xid v1.5.0 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/santhosh-tekuri/jsonschema/v6 v6.0.1 // indirect
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 // indirect github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 // indirect
github.com/secure-systems-lab/go-securesystemslib v0.4.0 // indirect github.com/secure-systems-lab/go-securesystemslib v0.6.0 // indirect
github.com/serialx/hashring v0.0.0-20200727003509-22c0c7ab6b1b // indirect
github.com/shibumi/go-pathspec v1.3.0 // indirect github.com/shibumi/go-pathspec v1.3.0 // indirect
github.com/shopspring/decimal v1.4.0 // indirect github.com/shopspring/decimal v1.4.0 // indirect
github.com/shurcooL/sanitized_anchor_name v1.0.0 // indirect github.com/shurcooL/sanitized_anchor_name v1.0.0 // indirect
github.com/siderolabs/gen v0.4.8 // indirect github.com/siderolabs/gen v0.4.8 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect github.com/sirupsen/logrus v1.9.3 // indirect
github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 // indirect
github.com/slackhq/nebula v1.6.1 // indirect github.com/slackhq/nebula v1.6.1 // indirect
github.com/smallstep/certificates v0.26.1 // indirect github.com/smallstep/certificates v0.26.1 // indirect
github.com/smallstep/nosql v0.6.1 // indirect github.com/smallstep/nosql v0.6.1 // indirect
github.com/smallstep/pkcs7 v0.0.0-20231024181729-3b98ecc1ca81 // indirect github.com/smallstep/pkcs7 v0.1.1 // indirect
github.com/smallstep/scep v0.0.0-20231024192529-aee96d7ad34d // indirect github.com/smallstep/scep v0.0.0-20231024192529-aee96d7ad34d // indirect
github.com/smallstep/truststore v0.13.0 // indirect github.com/smallstep/truststore v0.13.0 // indirect
github.com/spaolacci/murmur3 v1.1.0 // indirect github.com/spaolacci/murmur3 v1.1.0 // indirect
github.com/spf13/cast v1.7.0 // indirect github.com/spf13/cast v1.7.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect github.com/spf13/pflag v1.0.10 // indirect
github.com/stoewer/go-strcase v1.2.0 // indirect github.com/stoewer/go-strcase v1.2.0 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/tailscale/tscert v0.0.0-20240517230440-bbccfbf48933 // indirect github.com/tailscale/tscert v0.0.0-20240517230440-bbccfbf48933 // indirect
github.com/theupdateframework/notary v0.7.0 // indirect github.com/theupdateframework/notary v0.7.0 // indirect
github.com/tonistiigi/dchapes-mode v0.0.0-20241001053921-ca0759fec205 // indirect github.com/tilt-dev/fsnotify v1.4.8-0.20220602155310-fff9c274a375 // indirect
github.com/tonistiigi/fsutil v0.0.0-20241028165955-397af5306b5c // indirect github.com/tonistiigi/dchapes-mode v0.0.0-20250318174251-73d941a28323 // indirect
github.com/tonistiigi/go-csvvalue v0.0.0-20240710180619-ddb21b71c0b4 // indirect github.com/tonistiigi/fsutil v0.0.0-20250605211040-586307ad452f // indirect
github.com/tonistiigi/go-csvvalue v0.0.0-20240814133006-030d3b2625d0 // indirect
github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea // indirect github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea // indirect
github.com/tonistiigi/vt100 v0.0.0-20240514184818-90bafcd6abab // indirect github.com/tonistiigi/vt100 v0.0.0-20240514184818-90bafcd6abab // indirect
github.com/urfave/cli v1.22.16 // indirect github.com/urfave/cli v1.22.17 // indirect
github.com/vbatts/tar-split v0.11.6 // indirect github.com/vbatts/tar-split v0.12.1 // indirect
github.com/vishvananda/netns v0.0.4 // indirect github.com/vishvananda/netns v0.0.5 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect github.com/x448/float16 v0.8.4 // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect github.com/xhit/go-str2duration/v2 v2.1.0 // indirect
github.com/xeipuuv/gojsonschema v1.2.0 // indirect github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
github.com/zclconf/go-cty v1.17.0 // indirect
github.com/zeebo/blake3 v0.2.4 // indirect github.com/zeebo/blake3 v0.2.4 // indirect
go.etcd.io/bbolt v1.3.11 // indirect go.etcd.io/bbolt v1.4.3 // indirect
go.opencensus.io v0.24.0 // indirect go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect go.opentelemetry.io/auto/sdk v1.1.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0 // indirect go.opentelemetry.io/contrib/bridges/prometheus v0.57.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.46.1 // indirect go.opentelemetry.io/contrib/exporters/autoexport v0.57.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0 // indirect go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0 // indirect
go.opentelemetry.io/otel v1.33.0 // indirect go.opentelemetry.io/contrib/instrumentation/net/http/httptrace/otelhttptrace v0.60.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v0.44.0 // indirect go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.33.0 // indirect go.opentelemetry.io/otel v1.36.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.27.0 // indirect go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc v0.8.0 // indirect
go.opentelemetry.io/otel/metric v1.33.0 // indirect go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp v0.8.0 // indirect
go.opentelemetry.io/otel/sdk v1.33.0 // indirect go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.35.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.32.0 // indirect go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.35.0 // indirect
go.opentelemetry.io/otel/trace v1.33.0 // indirect go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.35.0 // indirect
go.opentelemetry.io/proto/otlp v1.4.0 // indirect go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.35.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.35.0 // indirect
go.opentelemetry.io/otel/exporters/prometheus v0.54.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdoutlog v0.8.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.32.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.32.0 // indirect
go.opentelemetry.io/otel/log v0.8.0 // indirect
go.opentelemetry.io/otel/metric v1.36.0 // indirect
go.opentelemetry.io/otel/sdk v1.36.0 // indirect
go.opentelemetry.io/otel/sdk/log v0.8.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.36.0 // indirect
go.opentelemetry.io/otel/trace v1.36.0 // indirect
go.opentelemetry.io/proto/otlp v1.5.0 // indirect
go.step.sm/cli-utils v0.9.0 // indirect go.step.sm/cli-utils v0.9.0 // indirect
go.step.sm/crypto v0.45.0 // indirect go.step.sm/crypto v0.45.0 // indirect
go.step.sm/linkedca v0.20.1 // indirect go.step.sm/linkedca v0.20.1 // indirect
go.uber.org/automaxprocs v1.5.3 // indirect go.uber.org/automaxprocs v1.5.3 // indirect
go.uber.org/mock v0.5.0 // indirect go.uber.org/mock v0.6.0 // indirect
go.uber.org/multierr v1.11.0 // indirect go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap/exp v0.2.0 // indirect go.uber.org/zap/exp v0.2.0 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/crypto/x509roots/fallback v0.0.0-20240507223354-67b13616a595 // indirect golang.org/x/crypto/x509roots/fallback v0.0.0-20240507223354-67b13616a595 // indirect
golang.org/x/exp v0.0.0-20241215155358-4a5509556b9e // indirect golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0 // indirect
golang.org/x/mod v0.23.0 // indirect golang.org/x/mod v0.27.0 // indirect
golang.org/x/term v0.29.0 // indirect golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/text v0.22.0 // indirect golang.org/x/text v0.28.0 // indirect
golang.org/x/time v0.8.0 // indirect golang.org/x/time v0.11.0 // indirect
golang.org/x/tools v0.30.0 // indirect golang.org/x/tools v0.36.0 // indirect
golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 // indirect golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 // indirect
google.golang.org/genproto v0.0.0-20240401170217-c3f982113cda // indirect google.golang.org/genproto/googleapis/api v0.0.0-20250528174236-200df99c418a // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20241209162323-e6fa225c2576 // indirect gopkg.in/evanphx/json-patch.v4 v4.12.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect
gvisor.dev/gvisor v0.0.0-20230927004350-cbd86285d259 // indirect gvisor.dev/gvisor v0.0.0-20230927004350-cbd86285d259 // indirect
howett.net/plist v1.0.0 // indirect howett.net/plist v1.0.0 // indirect
k8s.io/api v0.32.3 // indirect
k8s.io/apimachinery v0.32.3 // indirect
k8s.io/client-go v0.32.3 // indirect
k8s.io/klog/v2 v2.130.1 // indirect
k8s.io/kube-openapi v0.0.0-20241105132330-32ad38e42d3f // indirect
k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738 // indirect
lukechampine.com/blake3 v1.3.0 // indirect lukechampine.com/blake3 v1.3.0 // indirect
modernc.org/libc v1.61.13 // indirect modernc.org/libc v1.61.13 // indirect
modernc.org/mathutil v1.7.1 // indirect modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.8.2 // indirect modernc.org/memory v1.8.2 // indirect
sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 // indirect
sigs.k8s.io/structured-merge-diff/v4 v4.4.2 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
tags.cncf.io/container-device-interface v1.0.1 // indirect
) )
+458 -259
View File
File diff suppressed because it is too large Load Diff
+176 -153
View File
@@ -2,175 +2,198 @@ package cli
import ( import (
"context" "context"
"encoding/base64" "errors"
"encoding/json"
"fmt" "fmt"
"os" "strings"
"github.com/charmbracelet/lipgloss"
composetypes "github.com/compose-spec/compose-go/v2/types" composetypes "github.com/compose-spec/compose-go/v2/types"
"github.com/distribution/reference" mapset "github.com/deckarep/golang-set/v2"
"github.com/docker/cli/cli/config" "github.com/docker/cli/cli/command"
"github.com/docker/docker/api/types" "github.com/docker/cli/cli/flags"
"github.com/docker/docker/api/types/image" composeapi "github.com/docker/compose/v2/pkg/api"
dockerclient "github.com/docker/docker/client" composev2 "github.com/docker/compose/v2/pkg/compose"
"github.com/docker/docker/pkg/archive" "github.com/docker/compose/v2/pkg/progress"
"github.com/docker/docker/pkg/jsonmessage" "github.com/psviderski/uncloud/pkg/client"
"github.com/docker/docker/registry" "github.com/psviderski/uncloud/pkg/client/compose"
"github.com/moby/term"
) )
type BuildOptions struct { // BuildServicesOptions contains options for building services in a Compose project.
Files []string type BuildServicesOptions struct {
Profiles []string // BuildArgs sets build-time variables for services. Used in Dockerfiles that declare variables with ARG.
Services []string BuildArgs []string
Push bool // Check the build configuration for services without building them.
Check bool
// Deps enables to also build services declared as dependencies of the selected Services.
Deps bool
// NoCache disables the use of cache when building images.
NoCache bool NoCache bool
// Pull attempts to pull newer versions of the base images before building.
Pull bool
// Services specifies which services to build. If empty, all services with a build config are built.
Services []string
// Push targets are mutually exclusive.
// PushCluster uploads the built images to cluster machines after building.
PushCluster bool
// PushRegistry uploads the built images to external registries after building.
PushRegistry bool
// Cluster-specific options (only used if PushCluster is true).
// Machines is a list of machine names or IDs to push the image to. If empty, images are pushed to all machines.
Machines []string
} }
// GetServicesThatNeedBuild returns a map of services that require building // BuildServices builds images for services in the Compose project.
func GetServicesThatNeedBuild(project *composetypes.Project) map[string]composetypes.ServiceConfig { func (cli *CLI) BuildServices(ctx context.Context, project *composetypes.Project, opts BuildServicesOptions) error {
servicesToBuild := make(map[string]composetypes.ServiceConfig, len(project.Services)) // Validate push targets.
for serviceName, service := range project.Services { if opts.PushCluster && opts.PushRegistry {
if service.Build == nil { return fmt.Errorf("cannot specify both PushCluster and PushRegistry: choose one push target")
}
// Build service images using Compose implementation.
dockerCli, err := command.NewDockerCli()
if err != nil {
return fmt.Errorf("create docker client: %w", err)
}
// Initialise the Docker CLI with default options.
if err = dockerCli.Initialize(flags.NewClientOptions()); err != nil {
return fmt.Errorf("initialise docker client: %w", err)
}
composeService := composev2.NewComposeService(dockerCli)
buildOpts := composeapi.BuildOptions{
Args: composetypes.NewMappingWithEquals(opts.BuildArgs),
Check: opts.Check,
Deps: opts.Deps,
NoCache: opts.NoCache,
Pull: opts.Pull,
Push: opts.PushRegistry,
Services: opts.Services,
}
if err = composeService.Build(ctx, project, buildOpts); err != nil {
return err
}
if !opts.PushCluster {
return nil
}
// Push built service images to cluster machines.
builtServices, err := ServicesThatNeedBuild(project, opts.Services, opts.Deps)
if err != nil {
return fmt.Errorf("determine built services: %w", err)
}
if len(builtServices) == 0 {
// No services were built, nothing to push.
return nil
}
// Add a line break after the build output.
fmt.Fprintln(cli.ProgressOut())
clusterClient, err := cli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
// Push one service image at a time.
var errs []error
for _, s := range builtServices {
if s.Image == "" {
// Skip services without an image name (shouldn't happen for services with build config).
continue continue
} }
servicesToBuild[serviceName] = service
}
return servicesToBuild
}
// BuildServices builds the services defined in the provided map. // Push to the specified machines falling back to service x-machines.
func BuildServices(ctx context.Context, servicesToBuild map[string]composetypes.ServiceConfig, opts BuildOptions) error { // If none specified, push to *all* cluster machines.
fmt.Println("Building services...") var pushOpts client.PushImageOptions
// Init docker client (can be local or remote, depending on DOCKER_HOST environment variable) if len(opts.Machines) > 0 {
dockerCli, err := dockerclient.NewClientWithOpts(dockerclient.FromEnv, dockerclient.WithAPIVersionNegotiation()) pushOpts.Machines = opts.Machines
if err != nil { } else if machines, ok := s.Extensions[compose.MachinesExtensionKey].(compose.MachinesSource); ok {
return err pushOpts.Machines = machines
}
defer dockerCli.Close()
serviceImages := make(map[string]string, len(servicesToBuild))
// Build the services using the local docker client and compose libraries
for _, service := range servicesToBuild {
fmt.Printf("Building service: %s\n", service.Name)
imageName, err := buildSingleService(ctx, dockerCli, service, opts)
if err != nil {
return fmt.Errorf("build service %s: %w", service.Name, err)
}
serviceImages[service.Name] = imageName
}
fmt.Printf("Service images are built.\n")
if opts.Push {
err = pushServiceImages(ctx, dockerCli, serviceImages)
} }
return err if len(pushOpts.Machines) == 0 {
} pushOpts.AllMachines = true
// buildSingleService builds a single service using the Docker client and Compose libraries.
func buildSingleService(ctx context.Context, dockerCli *dockerclient.Client, service composetypes.ServiceConfig, opts BuildOptions) (string, error) {
if service.Build == nil {
return "", fmt.Errorf("service %s has no build configuration", service.Name)
}
if service.Image == "" {
return "", fmt.Errorf("service %s has no image specified; building services without image is not supported yet", service.Name)
} }
buildContextPath := service.Build.Context boldStyle := lipgloss.NewStyle().Bold(true)
imageName := service.Image err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
if err = clusterClient.PushImage(ctx, s.Image, pushOpts); err != nil {
// Create a tar archive of the build context return fmt.Errorf("push image '%s' for service '%s': %w", s.Image, s.Name, err)
buildContext, err := archive.TarWithOptions(buildContextPath, &archive.TarOptions{})
if err != nil {
return "", fmt.Errorf("failed to create build context for service %s: %w", service.Name, err)
}
buildOptions := types.ImageBuildOptions{
// TODO: Support Dockerfiles outside the build context
// See https://github.com/docker/compose/blob/cf89fd1aa1328d5af77658ccc5a1e1b29981ae80/pkg/compose/build_classic.go#L92
Dockerfile: service.Build.Dockerfile,
Tags: []string{imageName},
Remove: true, // Remove intermediate containers
NoCache: opts.NoCache,
}
buildResponse, err := dockerCli.ImageBuild(ctx, buildContext, buildOptions)
if err != nil {
return "", fmt.Errorf("failed to build image for service %s: %w", service.Name, err)
}
defer buildResponse.Body.Close()
// Display the build response
fd, isTerminal := term.GetFdInfo(os.Stdout)
if err := jsonmessage.DisplayJSONMessagesStream(buildResponse.Body, os.Stdout, fd, isTerminal, nil); err != nil {
return "", fmt.Errorf("failed to display build response for service %s: %w", service.Name, err)
}
return imageName, nil
}
// pushSingleServiceImage pushes a single service image.
func pushSingleServiceImage(ctx context.Context, dockerCli *dockerclient.Client, serviceName string, imageName string) error {
ref, err := reference.ParseNormalizedNamed(imageName)
if err != nil {
return err
}
repoInfo, err := registry.ParseRepositoryInfo(ref)
if err != nil {
return err
}
registryKey := repoInfo.Index.Name
if repoInfo.Index.Official {
registryKey = registry.IndexServer
}
// Load the Docker config file with auth details, if available
configFile := config.LoadDefaultConfigFile(os.Stderr)
authConfig, err := configFile.GetAuthConfig(registryKey)
if err != nil {
return err
}
authJSON, err := json.Marshal(authConfig)
if err != nil {
return fmt.Errorf("failed to marshal auth config for registry %s: %w", registryKey, err)
}
authStr := base64.URLEncoding.EncodeToString(authJSON)
pushOptions := image.PushOptions{
RegistryAuth: authStr,
}
pushResponse, err := dockerCli.ImagePush(ctx, imageName, pushOptions)
if err != nil {
return fmt.Errorf("failed to push image %s: %w", imageName, err)
}
defer pushResponse.Close()
fmt.Printf("Pushing image %s for service %s...\n", imageName, serviceName)
fd, isTerminal := term.GetFdInfo(os.Stdout)
if err := jsonmessage.DisplayJSONMessagesStream(pushResponse, os.Stdout, fd, isTerminal, nil); err != nil {
return fmt.Errorf("failed to display push response for image %s: %w", imageName, err)
}
fmt.Printf("Image %s pushed successfully.\n", imageName)
return nil
}
// pushServiceImages pushes all built service images to the registry.
func pushServiceImages(ctx context.Context, dockerCli *dockerclient.Client, serviceImages map[string]string) error {
fmt.Printf("Pushing images...\n")
for serviceName, imageName := range serviceImages {
if err := pushSingleServiceImage(ctx, dockerCli, serviceName, imageName); err != nil {
return fmt.Errorf("push image for service %s: %w", serviceName, err)
}
} }
return nil return nil
}, cli.ProgressOut(), fmt.Sprintf("Pushing image %s to cluster", boldStyle.Render(s.Image)))
// Collect errors to try pushing all images.
if err != nil {
errs = append(errs, err)
}
}
return errors.Join(errs...)
}
// ServicesThatNeedBuild returns a list of services that require building.
// deps indicates whether to include services that are dependencies of the selected services.
// Implementation is based on the logic from docker/compose/v2/pkg/compose/build.go.
func ServicesThatNeedBuild(
project *composetypes.Project, selectedServices []string, deps bool,
) ([]composetypes.ServiceConfig, error) {
servicesToBuild := make([]composetypes.ServiceConfig, 0, len(project.Services))
var policy composetypes.DependencyOption = composetypes.IgnoreDependencies
if deps {
policy = composetypes.IncludeDependencies
}
// Also include services used as build.additional_contexts with service: prefix.
selectedServices = includeAdditionalContextsServices(project, selectedServices)
// Some build dependencies we just introduced may not be enabled, enable them.
var err error
if project, err = project.WithServicesEnabled(selectedServices...); err != nil {
return nil, err
}
if project, err = project.WithSelectedServices(selectedServices); err != nil {
return nil, err
}
err = project.ForEachService(selectedServices, func(serviceName string, service *composetypes.ServiceConfig) error {
if service.Build != nil {
servicesToBuild = append(servicesToBuild, *service)
}
return nil
}, policy)
if err != nil {
return nil, err
}
return servicesToBuild, nil
}
// includeAdditionalContextsServices adds services referenced in build.additional_contexts to the list
// of selected services.
func includeAdditionalContextsServices(project *composetypes.Project, selectedServices []string) []string {
servicesWithDependencies := mapset.NewSet(selectedServices...)
for _, service := range selectedServices {
s, ok := project.Services[service]
if !ok {
s = project.DisabledServices[service]
}
if s.Build != nil {
for _, target := range s.Build.AdditionalContexts {
if name, found := strings.CutPrefix(target, composetypes.ServicePrefix); found {
servicesWithDependencies.Add(name)
}
}
}
}
if servicesWithDependencies.Cardinality() > len(selectedServices) {
return includeAdditionalContextsServices(project, servicesWithDependencies.ToSlice())
}
return servicesWithDependencies.ToSlice()
} }
+141 -60
View File
@@ -2,7 +2,6 @@ package cli
import ( import (
"context" "context"
"errors"
"fmt" "fmt"
"net/netip" "net/netip"
"os" "os"
@@ -10,7 +9,6 @@ import (
"github.com/docker/cli/cli/streams" "github.com/docker/cli/cli/streams"
"github.com/psviderski/uncloud/internal/cli/config" "github.com/psviderski/uncloud/internal/cli/config"
"github.com/psviderski/uncloud/internal/fs"
"github.com/psviderski/uncloud/internal/machine" "github.com/psviderski/uncloud/internal/machine"
"github.com/psviderski/uncloud/internal/machine/api/pb" "github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/sshexec" "github.com/psviderski/uncloud/internal/sshexec"
@@ -26,18 +24,20 @@ const (
// DefaultSSHKeyPath is the fallback location for the SSH private key when provisioning remote machines. // DefaultSSHKeyPath is the fallback location for the SSH private key when provisioning remote machines.
// Used when no key is explicitly provided and SSH agent authentication fails. // Used when no key is explicitly provided and SSH agent authentication fails.
DefaultSSHKeyPath = "~/.ssh/id_ed25519" DefaultSSHKeyPath = "~/.ssh/id_ed25519"
defaultContextName = "default" DefaultContextName = "default"
) )
type CLI struct { type CLI struct {
Config *config.Config Config *config.Config
conn *config.MachineConnection conn *config.MachineConnection
contextOverride string
} }
// New creates a new CLI instance with the given config path or remote machine connection. // New creates a new CLI instance with the given config path or remote machine connection.
// If the connection is provided, the config is ignored for all operations which is useful for interacting with // If the connection is provided, the config is ignored for all operations which is useful for interacting with
// a cluster without creating a config. // a cluster without creating a config.
func New(configPath string, conn *config.MachineConnection) (*CLI, error) { // If a non-empty context name is given, it will override the current default.
func New(configPath string, conn *config.MachineConnection, contextName string) (*CLI, error) {
if conn != nil { if conn != nil {
return &CLI{conn: conn}, nil return &CLI{conn: conn}, nil
} }
@@ -49,6 +49,7 @@ func New(configPath string, conn *config.MachineConnection) (*CLI, error) {
return &CLI{ return &CLI{
Config: cfg, Config: cfg,
contextOverride: contextName,
}, nil }, nil
} }
@@ -70,11 +71,30 @@ func (cli *CLI) SetCurrentContext(name string) error {
return cli.Config.Save() return cli.Config.Save()
} }
// ConnectCluster connects to a cluster using the given context name or the current context if not specified. func (cli *CLI) GetContextOverrideOrCurrent() string {
contextName := cli.contextOverride
if contextName == "" {
contextName = cli.Config.CurrentContext
}
return contextName
}
// ConnectCluster connects to a cluster using the context override or the current context if not specified.
// If the CLI was initialised with a machine connection, the config is ignored and the connection is used instead. // If the CLI was initialised with a machine connection, the config is ignored and the connection is used instead.
func (cli *CLI) ConnectCluster(ctx context.Context, contextName string) (*client.Client, error) { func (cli *CLI) ConnectCluster(ctx context.Context) (*client.Client, error) {
return cli.ConnectClusterWithOptions(ctx, ConnectOptions{
// Default to showing progress for CLI usage.
ShowProgress: true,
})
}
// ConnectClusterWithOptions connects to a cluster with the given options.
// If the CLI was initialised with a machine connection, the config is ignored and the connection is used instead.
// If the CLI has an override context, it is used instead of the current default.
// Options are useful when using the CLI as a library where you may want to disable visual feedback.
func (cli *CLI) ConnectClusterWithOptions(ctx context.Context, opts ConnectOptions) (*client.Client, error) {
if cli.conn != nil { if cli.conn != nil {
return connectCluster(ctx, *cli.conn) return ConnectCluster(ctx, *cli.conn, opts)
} }
if len(cli.Config.Contexts) == 0 { if len(cli.Config.Contexts) == 0 {
@@ -84,6 +104,8 @@ func (cli *CLI) ConnectCluster(ctx context.Context, contextName string) (*client
cli.Config.Path(), cli.Config.Path(),
) )
} }
contextName := cli.contextOverride
if contextName == "" { if contextName == "" {
// If the cluster is not specified, use the current cluster if set. // If the cluster is not specified, use the current cluster if set.
if cli.Config.CurrentContext == "" { if cli.Config.CurrentContext == "" {
@@ -116,38 +138,20 @@ func (cli *CLI) ConnectCluster(ctx context.Context, contextName string) (*client
) )
} }
// TODO: iterate over all connections and try to connect to the cluster using the first successful connection. // Try each connection in order until one succeeds.
conn := cfg.Connections[0] var lastErr error
for _, conn := range cfg.Connections {
c, err := connectCluster(ctx, conn) c, err := ConnectCluster(ctx, conn, opts)
if err != nil { if err == nil {
return nil, fmt.Errorf("connect to cluster (context '%s'): %w", contextName, err)
}
return c, nil return c, nil
}
func connectCluster(ctx context.Context, conn config.MachineConnection) (*client.Client, error) {
if conn.SSH != "" {
user, host, port, err := conn.SSH.Parse()
if err != nil {
return nil, fmt.Errorf("parse SSH connection %q: %w", conn.SSH, err)
} }
keyPath := fs.ExpandHomeDir(conn.SSHKeyFile) lastErr = err
sshConfig := &connector.SSHConnectorConfig{
User: user,
Host: host,
Port: port,
KeyPath: keyPath,
}
return client.New(ctx, connector.NewSSHConnector(sshConfig))
} else if conn.TCP != nil && conn.TCP.IsValid() {
return client.New(ctx, connector.NewTCPConnector(*conn.TCP))
} }
return nil, errors.New("connection configuration is invalid") return nil, fmt.Errorf("failed to connect to cluster context '%s': "+
"all connections (%d) in the Uncloud config (%s) failed; last error: %w",
contextName, len(cfg.Connections), cli.Config.Path(), lastErr)
} }
type InitClusterOptions struct { type InitClusterOptions struct {
@@ -156,6 +160,7 @@ type InitClusterOptions struct {
Network netip.Prefix Network netip.Prefix
PublicIP *netip.Addr PublicIP *netip.Addr
RemoteMachine *RemoteMachine RemoteMachine *RemoteMachine
SkipInstall bool
Version string Version string
} }
@@ -166,19 +171,16 @@ func (cli *CLI) InitCluster(ctx context.Context, opts InitClusterOptions) (*clie
return cli.initRemoteMachine(ctx, opts) return cli.initRemoteMachine(ctx, opts)
} }
// TODO: implement local machine initialisation // TODO: implement local machine initialisation
return nil, fmt.Errorf("local machine initialisation is not implemented yet") return nil, fmt.Errorf("local machine initialisation is not implemented yet. Please specify a remote machine")
} }
func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions) (*client.Client, error) { func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions) (*client.Client, error) {
contextName := opts.Context contextName, err := cli.newContextName(opts.Context)
if contextName == "" { if err != nil {
contextName = defaultContextName return nil, err
}
if _, ok := cli.Config.Contexts[contextName]; ok {
return nil, fmt.Errorf("cluster context '%s' already exists", contextName)
} }
machineClient, err := provisionRemoteMachine(ctx, opts.RemoteMachine, opts.Version) machineClient, err := provisionOrConnectRemoteMachine(ctx, opts.RemoteMachine, opts.SkipInstall, opts.Version)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -240,8 +242,21 @@ func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions)
// Save the machine's SSH connection details in the context config. // Save the machine's SSH connection details in the context config.
connCfg := config.MachineConnection{ connCfg := config.MachineConnection{
SSH: config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port),
SSHKeyFile: opts.RemoteMachine.KeyPath, SSHKeyFile: opts.RemoteMachine.KeyPath,
MachineID: resp.Machine.Id,
}
if opts.RemoteMachine.UseSSHCLI {
connCfg.SSHCLI = config.NewSSHDestination(
opts.RemoteMachine.User,
opts.RemoteMachine.Host,
opts.RemoteMachine.Port,
)
} else {
connCfg.SSH = config.NewSSHDestination(
opts.RemoteMachine.User,
opts.RemoteMachine.Host,
opts.RemoteMachine.Port,
)
} }
cli.Config.Contexts[contextName].Connections = append(cli.Config.Contexts[contextName].Connections, connCfg) cli.Config.Contexts[contextName].Connections = append(cli.Config.Contexts[contextName].Connections, connCfg)
if err = cli.Config.Save(); err != nil { if err = cli.Config.Save(); err != nil {
@@ -250,11 +265,37 @@ func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions)
return machineClient, nil return machineClient, nil
} }
// newContextName returns a unique name for a new cluster context. If the provided name is not DefaultContextName,
// and it's already taken, an error is returned. If the name is not provided or is DefaultContextName, the first
// available name "default[-N]" is returned.
func (cli *CLI) newContextName(name string) (string, error) {
if name == "" {
name = DefaultContextName
}
if _, exists := cli.Config.Contexts[name]; !exists {
return name, nil
}
// If non-default context already exists, error out.
if name != DefaultContextName {
return "", fmt.Errorf("cluster context '%s' already exists", name)
}
// The default context already exists, generate a numbered suffix to make it unique.
for i := 1; ; i++ {
name = fmt.Sprintf("%s-%d", DefaultContextName, i)
if _, exists := cli.Config.Contexts[name]; !exists {
return name, nil
}
}
}
type AddMachineOptions struct { type AddMachineOptions struct {
Context string
MachineName string MachineName string
PublicIP *netip.Addr PublicIP *netip.Addr
RemoteMachine *RemoteMachine RemoteMachine *RemoteMachine
SkipInstall bool
Version string Version string
} }
@@ -263,11 +304,8 @@ type AddMachineOptions struct {
// cluster. The machine client is connected to the new machine and can be used to interact with it. // cluster. The machine client is connected to the new machine and can be used to interact with it.
// Both client should be closed after use by the caller. // Both client should be closed after use by the caller.
func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client.Client, *client.Client, error) { func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client.Client, *client.Client, error) {
contextName := opts.Context contextName := cli.GetContextOverrideOrCurrent()
if contextName == "" { c, err := cli.ConnectCluster(ctx)
contextName = cli.Config.CurrentContext
}
c, err := cli.ConnectCluster(ctx, contextName)
if err != nil { if err != nil {
return nil, nil, fmt.Errorf("connect to cluster (context '%s'): %w", contextName, err) return nil, nil, fmt.Errorf("connect to cluster (context '%s'): %w", contextName, err)
} }
@@ -277,7 +315,7 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
} }
}() }()
machineClient, err := provisionRemoteMachine(ctx, opts.RemoteMachine, opts.Version) machineClient, err := provisionOrConnectRemoteMachine(ctx, opts.RemoteMachine, opts.SkipInstall, opts.Version)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
@@ -381,11 +419,21 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
// Save the machine's SSH connection details in the context config. // Save the machine's SSH connection details in the context config.
connCfg := config.MachineConnection{ connCfg := config.MachineConnection{
SSH: config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port),
SSHKeyFile: opts.RemoteMachine.KeyPath, SSHKeyFile: opts.RemoteMachine.KeyPath,
MachineID: addResp.Machine.Id,
} }
if contextName == "" { if opts.RemoteMachine.UseSSHCLI {
contextName = cli.Config.CurrentContext connCfg.SSHCLI = config.NewSSHDestination(
opts.RemoteMachine.User,
opts.RemoteMachine.Host,
opts.RemoteMachine.Port,
)
} else {
connCfg.SSH = config.NewSSHDestination(
opts.RemoteMachine.User,
opts.RemoteMachine.Host,
opts.RemoteMachine.Port,
)
} }
cli.Config.Contexts[contextName].Connections = append(cli.Config.Contexts[contextName].Connections, connCfg) cli.Config.Contexts[contextName].Connections = append(cli.Config.Contexts[contextName].Connections, connCfg)
if err = cli.Config.Save(); err != nil { if err = cli.Config.Save(); err != nil {
@@ -395,15 +443,45 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
return c, machineClient, nil return c, machineClient, nil
} }
// provisionRemoteMachine installs the Uncloud daemon and dependencies on the remote machine over SSH and returns // provisionOrConnectRemoteMachine installs the Uncloud daemon and dependencies on the remote machine over SSH and
// a machine API client to interact with the machine. The client should be closed after use by the caller. // returns a machine API client to interact with the machine. The client should be closed after use by the caller.
// The version parameter specifies the version of the Uncloud daemon to install. If empty, the latest version is used. // The version parameter specifies the version of the Uncloud daemon to install. If empty, the latest version is used.
// If skipInstall is true, the installation step is skipped, and it is assumed that the Uncloud daemon and dependencies
// are already installed and running.
// The remoteMachine.SSHKeyPath could be updated to the default SSH key path if it is not set and the SSH agent // The remoteMachine.SSHKeyPath could be updated to the default SSH key path if it is not set and the SSH agent
// authentication fails. // authentication fails.
func provisionRemoteMachine( func provisionOrConnectRemoteMachine(
ctx context.Context, remoteMachine *RemoteMachine, version string, ctx context.Context, remoteMachine *RemoteMachine, skipInstall bool, version string,
) (*client.Client, error) { ) (*client.Client, error) {
// Provision the remote machine by installing the Uncloud daemon and dependencies over SSH. // Use SSH CLI
if remoteMachine.UseSSHCLI {
exec := sshexec.NewSSHCLIRemote(
remoteMachine.User,
remoteMachine.Host,
remoteMachine.Port,
remoteMachine.KeyPath,
)
if !skipInstall {
if err := provisionMachine(ctx, exec, version); err != nil {
return nil, fmt.Errorf("provision machine: %w", err)
}
}
sshConfig := &connector.SSHConnectorConfig{
User: remoteMachine.User,
Host: remoteMachine.Host,
Port: remoteMachine.Port,
KeyPath: remoteMachine.KeyPath,
}
machineClient, err := client.New(ctx, connector.NewSSHCLIConnector(sshConfig))
if err != nil {
return nil, fmt.Errorf("connect to remote machine: %w", err)
}
return machineClient, nil
}
// Use Go SSH
sshClient, err := sshexec.Connect(remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath) sshClient, err := sshexec.Connect(remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath)
// If the SSH connection using SSH agent fails and no key path is provided, try to use the default SSH key. // If the SSH connection using SSH agent fails and no key path is provided, try to use the default SSH key.
if err != nil && remoteMachine.KeyPath == "" { if err != nil && remoteMachine.KeyPath == "" {
@@ -418,14 +496,17 @@ func provisionRemoteMachine(
config.NewSSHDestination(remoteMachine.User, remoteMachine.Host, remoteMachine.Port), err, config.NewSSHDestination(remoteMachine.User, remoteMachine.Host, remoteMachine.Port), err,
) )
} }
if !skipInstall {
// Provision the remote machine by installing the Uncloud daemon and dependencies over SSH.
exec := sshexec.NewRemote(sshClient) exec := sshexec.NewRemote(sshClient)
// Install and run the Uncloud daemon and dependencies on the remote machine.
if err = provisionMachine(ctx, exec, version); err != nil { if err = provisionMachine(ctx, exec, version); err != nil {
return nil, fmt.Errorf("provision machine: %w", err) return nil, fmt.Errorf("provision machine: %w", err)
} }
}
var machineClient *client.Client var machineClient *client.Client
if remoteMachine.User == "root" { if remoteMachine.User == "root" || skipInstall {
// Create a machine API client over the established SSH connection to the remote machine. // Create a machine API client over the established SSH connection to the remote machine.
machineClient, err = client.New(ctx, connector.NewSSHConnectorFromClient(sshClient)) machineClient, err = client.New(ctx, connector.NewSSHConnectorFromClient(sshClient))
} else { } else {
+4
View File
@@ -53,6 +53,10 @@ func (c *Config) Read() error {
func (c *Config) Save() error { func (c *Config) Save() error {
dir, _ := filepath.Split(c.path) dir, _ := filepath.Split(c.path)
// If dir is empty (e.g., when path is just a filename), use current directory
if dir == "" {
dir = "."
}
if err := os.MkdirAll(dir, 0o700); err != nil { if err := os.MkdirAll(dir, 0o700); err != nil {
return fmt.Errorf("create config directory '%s': %w", dir, err) return fmt.Errorf("create config directory '%s': %w", dir, err)
} }
+80
View File
@@ -0,0 +1,80 @@
package config
import (
"os"
"path/filepath"
"testing"
)
func TestConfig_Save(t *testing.T) {
t.Parallel()
// Create a temporary directory for the test
tmpDir := t.TempDir()
// Change to temp directory so relative paths resolve correctly
originalDir, err := os.Getwd()
if err != nil {
t.Fatalf("Failed to get current directory: %v", err)
}
defer func() {
if err := os.Chdir(originalDir); err != nil {
t.Logf("Failed to restore original directory: %v", err)
}
}()
tests := []struct {
name string
configPath string
contextName string
expectFileAt string // Expected file location for verification
useAbsolutePath bool // Whether to use absolute path for expectFileAt
}{
{
name: "relative path without prefix",
configPath: "test-config.yaml",
contextName: "test",
},
{
name: "relative path with prefix",
configPath: "./test-config-2.yaml",
contextName: "test2",
},
{
name: "absolute path",
configPath: filepath.Join(tmpDir, "absolute-config.yaml"),
contextName: "test3",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
if err := os.Chdir(tmpDir); err != nil {
t.Fatalf("Failed to change to temp directory: %v", err)
}
cfg := &Config{
CurrentContext: tt.contextName,
Contexts: map[string]*Context{
tt.contextName: {
Name: tt.contextName,
},
},
path: tt.configPath,
}
// This should not fail when saving the config
err := cfg.Save()
if err != nil {
t.Errorf("Expected no error when saving config, got: %v", err)
}
// Verify the file was created
if _, err := os.Stat(tt.configPath); os.IsNotExist(err) {
t.Errorf("Config file was not created at expected path: %s", tt.configPath)
}
})
}
}
+44
View File
@@ -1,6 +1,8 @@
package config package config
import ( import (
"errors"
"fmt"
"net" "net"
"net/netip" "net/netip"
"strconv" "strconv"
@@ -16,12 +18,54 @@ const (
type MachineConnection struct { type MachineConnection struct {
SSH SSHDestination `yaml:"ssh,omitempty"` SSH SSHDestination `yaml:"ssh,omitempty"`
SSHCLI SSHDestination `yaml:"ssh_cli,omitempty"`
SSHKeyFile string `yaml:"ssh_key_file,omitempty"` SSHKeyFile string `yaml:"ssh_key_file,omitempty"`
// TCP is the address and port of the machine's API server. // TCP is the address and port of the machine's API server.
// The pointer is used to omit the field when not set. Otherwise, yaml marshalling includes an empty object. // The pointer is used to omit the field when not set. Otherwise, yaml marshalling includes an empty object.
TCP *netip.AddrPort `yaml:"tcp,omitempty"` TCP *netip.AddrPort `yaml:"tcp,omitempty"`
// Unix is the path to the machine's API unix socket.
Unix string `yaml:"unix,omitempty"`
Host string `yaml:"host,omitempty"` Host string `yaml:"host,omitempty"`
PublicKey secret.Secret `yaml:"public_key,omitempty"` PublicKey secret.Secret `yaml:"public_key,omitempty"`
MachineID string `yaml:"machine_id,omitempty"`
}
func (c MachineConnection) String() string {
if c.SSH != "" {
return "ssh://" + string(c.SSH)
} else if c.SSHCLI != "" {
return "ssh+cli://" + string(c.SSHCLI)
} else if c.TCP != nil && c.TCP.IsValid() {
return fmt.Sprintf("tcp://%s", c.TCP)
} else if c.Unix != "" {
return fmt.Sprintf("unix://%s", c.Unix)
}
return "unknown connection"
}
func (c *MachineConnection) Validate() error {
setCount := 0
if c.SSH != "" {
setCount++
}
if c.SSHCLI != "" {
setCount++
}
if c.TCP != nil && c.TCP.IsValid() {
setCount++
}
if c.Unix != "" {
setCount++
}
if setCount == 0 {
return errors.New("no connection method specified (ssh, ssh_cli, tcp, or unix required)")
}
if setCount > 1 {
return errors.New("only one connection method allowed per connection (ssh, ssh_cli, tcp, or unix)")
}
return nil
} }
// SSHDestination represents an SSH destination string in the canonical form of "user@host:port". // SSHDestination represents an SSH destination string in the canonical form of "user@host:port".
+205
View File
@@ -0,0 +1,205 @@
package config
import (
"net/netip"
"testing"
"github.com/stretchr/testify/assert"
)
func TestMachineConnection_String(t *testing.T) {
t.Parallel()
tests := []struct {
name string
conn MachineConnection
want string
}{
{
name: "ssh connection",
conn: MachineConnection{
SSH: "user@host.com",
},
want: "ssh://user@host.com",
},
{
name: "ssh connection with port",
conn: MachineConnection{
SSH: "user@host.com:2222",
},
want: "ssh://user@host.com:2222",
},
{
name: "ssh_cli connection",
conn: MachineConnection{
SSHCLI: "user@host.com",
},
want: "ssh+cli://user@host.com",
},
{
name: "ssh_cli connection with port",
conn: MachineConnection{
SSHCLI: "user@host.com:2222",
},
want: "ssh+cli://user@host.com:2222",
},
{
name: "tcp connection",
conn: MachineConnection{
TCP: func() *netip.AddrPort {
addr := netip.MustParseAddrPort("10.0.0.1:8080")
return &addr
}(),
},
want: "tcp://10.0.0.1:8080",
},
{
name: "unix connection",
conn: MachineConnection{
Unix: "/run/uncloud/uncloud.sock",
},
want: "unix:///run/uncloud/uncloud.sock",
},
{
name: "no connection",
conn: MachineConnection{},
want: "unknown connection",
},
{
name: "ssh connection with machine id",
conn: MachineConnection{
SSH: "user@host.com",
MachineID: "ed98b9f7575308c340263cd279e3b568",
},
// String() does not use MachineID; just verifying entry with it is valid
want: "ssh://user@host.com",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := tt.conn.String()
assert.Equal(t, tt.want, got)
})
}
}
func TestMachineConnection_Validate(t *testing.T) {
t.Parallel()
tests := []struct {
name string
conn MachineConnection
wantErr bool
errMsg string
}{
{
name: "ssh only - valid",
conn: MachineConnection{
SSH: "user@host",
},
wantErr: false,
},
{
name: "ssh_cli only - valid",
conn: MachineConnection{
SSHCLI: "user@host",
},
wantErr: false,
},
{
name: "tcp only - valid",
conn: MachineConnection{
TCP: func() *netip.AddrPort {
addr := netip.MustParseAddrPort("10.0.0.1:8080")
return &addr
}(),
},
wantErr: false,
},
{
name: "unix only - valid",
conn: MachineConnection{
Unix: "/path/to/socket",
},
wantErr: false,
},
{
name: "no connection method - error",
conn: MachineConnection{},
wantErr: true,
errMsg: "no connection method specified",
},
{
name: "ssh and ssh_cli - error",
conn: MachineConnection{
SSH: "user@host",
SSHCLI: "user@host",
},
wantErr: true,
errMsg: "only one connection method allowed",
},
{
name: "ssh and unix - error",
conn: MachineConnection{
SSH: "user@host",
Unix: "/path/to/socket",
},
wantErr: true,
errMsg: "only one connection method allowed",
},
{
name: "ssh and tcp - error",
conn: MachineConnection{
SSH: "user@host",
TCP: func() *netip.AddrPort {
addr := netip.MustParseAddrPort("10.0.0.1:8080")
return &addr
}(),
},
wantErr: true,
errMsg: "only one connection method allowed",
},
{
name: "ssh_cli and tcp - error",
conn: MachineConnection{
SSHCLI: "user@host",
TCP: func() *netip.AddrPort {
addr := netip.MustParseAddrPort("10.0.0.1:8080")
return &addr
}(),
},
wantErr: true,
errMsg: "only one connection method allowed",
},
{
name: "all three - error",
conn: MachineConnection{
SSH: "user@host",
SSHCLI: "user@host2",
TCP: func() *netip.AddrPort {
addr := netip.MustParseAddrPort("10.0.0.1:8080")
return &addr
}(),
},
wantErr: true,
errMsg: "only one connection method allowed",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
err := tt.conn.Validate()
if tt.wantErr {
assert.Error(t, err)
assert.Contains(t, err.Error(), tt.errMsg)
} else {
assert.NoError(t, err)
}
})
}
}
+18
View File
@@ -4,3 +4,21 @@ type Context struct {
Name string `yaml:"-"` Name string `yaml:"-"`
Connections []MachineConnection `yaml:"connections"` Connections []MachineConnection `yaml:"connections"`
} }
func (c *Context) SetDefaultConnection(index int) {
// Do nothing if the index is out of range.
if index < 0 || index >= len(c.Connections) {
return
}
selected := c.Connections[index]
newConnections := make([]MachineConnection, 0, len(c.Connections))
newConnections = append(newConnections, selected)
for i, conn := range c.Connections {
if i != index {
newConnections = append(newConnections, conn)
}
}
c.Connections = newConnections
}
+212
View File
@@ -0,0 +1,212 @@
package cli
import (
"context"
"errors"
"fmt"
"os"
"time"
"github.com/charmbracelet/bubbles/spinner"
tea "github.com/charmbracelet/bubbletea"
"github.com/charmbracelet/lipgloss"
"github.com/psviderski/uncloud/internal/cli/config"
"github.com/psviderski/uncloud/internal/fs"
"github.com/psviderski/uncloud/pkg/client"
"github.com/psviderski/uncloud/pkg/client/connector"
)
// ConnectOptions configures the behavior of cluster connection attempts.
type ConnectOptions struct {
// Whether to show connection progress spinner if stdout is a terminal or progress logs if not.
ShowProgress bool
}
func ConnectCluster(ctx context.Context, conn config.MachineConnection, opts ConnectOptions) (*client.Client, error) {
if opts.ShowProgress {
return connectClusterWithProgress(ctx, conn)
}
return connectCluster(ctx, conn)
}
// connectClusterWithProgress connects to the cluster while displaying a progress spinner.
// If the stdout is not a terminal, it falls back to simple progress logs to stderr.
func connectClusterWithProgress(ctx context.Context, conn config.MachineConnection) (*client.Client, error) {
// If stdout is not a terminal, fall back to simple progress logs.
if !IsStdoutTerminal() {
fmt.Fprintln(os.Stderr, "Connecting to", conn.String())
cli, err := connectCluster(ctx, conn)
if err != nil {
fmt.Fprintln(os.Stderr, "Connection failed:", err)
} else {
fmt.Fprintln(os.Stderr, "Connected to cluster.")
}
return cli, err
}
// Run the connection TUI model.
p := tea.NewProgram(newConnectModel(ctx, conn))
model, err := p.Run()
if err != nil {
return nil, fmt.Errorf("run connection TUI: %w", err)
}
m := model.(connectModel)
return m.result.client, m.result.err
}
func connectCluster(ctx context.Context, conn config.MachineConnection) (*client.Client, error) {
// Determine which SSH type is configured
var sshDest config.SSHDestination
var useSSHCLI bool
// Validate connection configuration early to provide clear error messages.
if err := conn.Validate(); err != nil {
return nil, fmt.Errorf("invalid connection configuration: %w", err)
}
if conn.SSH != "" {
sshDest = conn.SSH
useSSHCLI = false
} else if conn.SSHCLI != "" {
sshDest = conn.SSHCLI
useSSHCLI = true
} else if conn.TCP != nil && conn.TCP.IsValid() {
return client.New(ctx, connector.NewTCPConnector(*conn.TCP))
} else if conn.Unix != "" {
return client.New(ctx, connector.NewUnixConnector(conn.Unix))
} else {
return nil, errors.New("connection configuration is invalid")
}
// Parse SSH destination and create config (shared for both types)
user, host, port, err := sshDest.Parse()
if err != nil {
return nil, fmt.Errorf("parse SSH connection %q: %w", sshDest, err)
}
keyPath := fs.ExpandHomeDir(conn.SSHKeyFile)
sshConfig := &connector.SSHConnectorConfig{
User: user,
Host: host,
Port: port,
KeyPath: keyPath,
}
// Create appropriate connector based on type
if useSSHCLI {
return client.New(ctx, connector.NewSSHCLIConnector(sshConfig))
}
return client.New(ctx, connector.NewSSHConnector(sshConfig))
}
// connectModel is a TUI model for connecting to a cluster with a progress spinner.
type connectModel struct {
ctx context.Context
conn config.MachineConnection
spinner spinner.Model
// showSpinner controls whether the spinner is visible (delayed to avoid flashing).
showSpinner bool
// done indicates whether the connection attempt has completed (successfully or with error).
done bool
// result holds the result of the connection attempt.
result connectResultMsg
}
type connectResultMsg struct {
client *client.Client
err error
}
// showSpinnerMsg is sent after a delay to show the spinner.
type showSpinnerMsg struct{}
func newConnectModel(ctx context.Context, conn config.MachineConnection) connectModel {
s := spinner.New()
s.Spinner = spinner.MiniDot
s.Style = lipgloss.NewStyle().Foreground(lipgloss.Color("3")) // the same yellow as in compose progress
return connectModel{
ctx: ctx,
conn: conn,
spinner: s,
}
}
func (m connectModel) Init() tea.Cmd {
return tea.Batch(
m.spinner.Tick,
m.connect(),
m.delayShowSpinner(),
)
}
func (m connectModel) connect() tea.Cmd {
return func() tea.Msg {
cli, err := connectCluster(m.ctx, m.conn)
return connectResultMsg{
client: cli,
err: err,
}
}
}
// delayShowSpinner returns a command that sends a message to show the spinner after a delay.
// This avoids flashing the spinner if the connection is fast.
func (m connectModel) delayShowSpinner() tea.Cmd {
return tea.Tick(500*time.Millisecond, func(t time.Time) tea.Msg {
return showSpinnerMsg{}
})
}
func (m connectModel) Update(msg tea.Msg) (tea.Model, tea.Cmd) {
select {
case <-m.ctx.Done():
m.result.err = m.ctx.Err()
m.done = true
return m, tea.Quit
default:
}
switch msg := msg.(type) {
case connectResultMsg:
m.result = msg
m.done = true
return m, tea.Quit
case showSpinnerMsg:
// Only show spinner if connection hasn't completed yet.
if !m.done {
m.showSpinner = true
}
return m, nil
case spinner.TickMsg:
var cmd tea.Cmd
m.spinner, cmd = m.spinner.Update(msg)
return m, cmd
case tea.KeyMsg:
if msg.Type == tea.KeyCtrlC {
m.result.err = fmt.Errorf("connection cancelled")
m.done = true
return m, tea.Quit
}
}
return m, nil
}
func (m connectModel) View() string {
// Don't show anything if done or spinner not yet visible.
if m.done || !m.showSpinner {
return ""
}
style := lipgloss.NewStyle().Foreground(lipgloss.Color("153"))
return fmt.Sprintf("%s %s\n",
m.spinner.View(),
fmt.Sprintf("Connecting to %s", style.Render(m.conn.String())),
)
}
+13
View File
@@ -1,7 +1,11 @@
package cli package cli
import ( import (
"log"
"os"
"strings" "strings"
"github.com/spf13/cobra"
) )
// ExpandCommaSeparatedValues takes a slice of strings and expands any comma-separated values into individual elements. // ExpandCommaSeparatedValues takes a slice of strings and expands any comma-separated values into individual elements.
@@ -21,3 +25,12 @@ func ExpandCommaSeparatedValues(values []string) []string {
return expanded return expanded
} }
// BindEnvToFlag assigns the value of an environment variable to the given command flag if the flag has not been set.
func BindEnvToFlag(cmd *cobra.Command, flagName, envVar string) {
if value := os.Getenv(envVar); value != "" && !cmd.Flags().Changed(flagName) {
if err := cmd.Flags().Set(flagName, value); err != nil {
log.Fatalf("Failed to bind environment variable '%s' to flag '%s': %v", envVar, flagName, err)
}
}
}
+1
View File
@@ -25,6 +25,7 @@ type RemoteMachine struct {
Host string Host string
Port int Port int
KeyPath string KeyPath string
UseSSHCLI bool // indicates ssh+cli:// should be used
} }
func installCmd(user string, version string) string { func installCmd(user string, version string) string {
+16 -1
View File
@@ -1,6 +1,11 @@
package cli package cli
import "github.com/charmbracelet/huh" import (
"os"
"github.com/charmbracelet/huh"
"golang.org/x/term"
)
func Confirm() (bool, error) { func Confirm() (bool, error) {
var confirmed bool var confirmed bool
@@ -21,3 +26,13 @@ func Confirm() (bool, error) {
return confirmed, nil return confirmed, nil
} }
// IsStdinTerminal checks if the standard input is a terminal (TTY).
func IsStdinTerminal() bool {
return term.IsTerminal(int(os.Stdin.Fd()))
}
// IsStdoutTerminal checks if the standard output is a terminal (TTY).
func IsStdoutTerminal() bool {
return term.IsTerminal(int(os.Stdout.Fd()))
}
+5
View File
@@ -11,7 +11,12 @@ import (
"github.com/docker/docker/client" "github.com/docker/docker/client"
) )
type Client struct {
*client.Client
}
// WaitDaemonReady waits for the Docker daemon to start and be ready to serve requests. // WaitDaemonReady waits for the Docker daemon to start and be ready to serve requests.
// TODO(lhf): convert into a Client method.
func WaitDaemonReady(ctx context.Context, cli *client.Client) error { func WaitDaemonReady(ctx context.Context, cli *client.Client) error {
// Retry to ping the Docker daemon until it's ready or the context is canceled. // Retry to ping the Docker daemon until it's ready or the context is canceled.
boff := backoff.WithContext(backoff.NewExponentialBackOff( boff := backoff.WithContext(backoff.NewExponentialBackOff(
+70
View File
@@ -0,0 +1,70 @@
package docker
import (
"context"
"errors"
"fmt"
"time"
"github.com/containerd/errdefs"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/image"
"github.com/docker/go-connections/nat"
)
// CreateContainerWithImagePull creates a container. If the image is missing, it pulls the image first.
func (cli *Client) CreateContainerWithImagePull(
ctx context.Context, name string, config *container.Config, hostConfig *container.HostConfig,
) (container.CreateResponse, error) {
resp, err := cli.ContainerCreate(ctx, config, hostConfig, nil, nil, name)
if err == nil || !errdefs.IsNotFound(err) {
return resp, err
}
pullCh, err := cli.PullImage(ctx, config.Image, image.PullOptions{})
if err != nil {
return resp, fmt.Errorf("pull image: %w", err)
}
// Drain the pull channel until it's closed (image is fully pulled) or an error occurs.
// If the context is canceled during the pull, the channel will receive a context cancellation error.
for msg := range pullCh {
if msg.Err != nil {
return resp, fmt.Errorf("pull image: %w", msg.Err)
}
}
// Create container again after image pull.
if resp, err = cli.ContainerCreate(ctx, config, hostConfig, nil, nil, name); err != nil {
return resp, err
}
return resp, nil
}
// WaitPortPublished waits for a container port to be published on the host which happens asynchronously.
func (cli *Client) WaitPortPublished(ctx context.Context, containerID string, port nat.Port) ([]nat.PortBinding, error) {
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
for {
c, err := cli.ContainerInspect(ctx, containerID)
if err != nil {
return nil, fmt.Errorf("inspect container: %w", err)
}
binding, ok := c.NetworkSettings.Ports[port]
if ok && len(binding) > 0 {
return binding, nil
}
select {
case <-time.After(10 * time.Millisecond):
case <-ctx.Done():
if errors.Is(ctx.Err(), context.DeadlineExceeded) {
return nil, errors.New("timeout")
}
return nil, ctx.Err()
}
}
}
+132
View File
@@ -0,0 +1,132 @@
package docker
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"os"
dockercommand "github.com/docker/cli/cli/command"
dockerconfig "github.com/docker/cli/cli/config"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/registry"
"github.com/docker/docker/pkg/jsonmessage"
)
type PullPushImageMessage struct {
Message jsonmessage.JSONMessage
Err error
}
// PullImage pulls an image and returns a channel to receive progress messages.
func (cli *Client) PullImage(
ctx context.Context, image string, opts image.PullOptions,
) (<-chan PullPushImageMessage, error) {
if opts.RegistryAuth == "" {
// Try to retrieve the authentication token for the image from the default local Docker config file.
if encodedAuth, err := RetrieveLocalDockerRegistryAuth(image); err == nil {
opts.RegistryAuth = encodedAuth
}
}
respBody, err := cli.ImagePull(ctx, image, opts)
if err != nil {
return nil, err
}
ch := make(chan PullPushImageMessage)
go processPullPushImageResp(ctx, respBody, ch)
return ch, nil
}
// PushImage pushes an image and returns a channel to receive progress messages.
func (cli *Client) PushImage(
ctx context.Context, image string, opts image.PushOptions,
) (<-chan PullPushImageMessage, error) {
if opts.RegistryAuth == "" {
// Try to retrieve the authentication token for the image from the default local Docker config file.
encodedAuth, _ := RetrieveLocalDockerRegistryAuth(image)
if encodedAuth == "" {
// If no credentials are found, provide an encoded empty auth config to work around the bug in Docker:
// https://github.com/moby/moby/issues/50729
var err error
if encodedAuth, err = registry.EncodeAuthConfig(registry.AuthConfig{}); err != nil {
return nil, fmt.Errorf("encode empty auth config: %w", err)
}
}
opts.RegistryAuth = encodedAuth
}
respBody, err := cli.ImagePush(ctx, image, opts)
if err != nil {
return nil, err
}
ch := make(chan PullPushImageMessage)
go processPullPushImageResp(ctx, respBody, ch)
return ch, nil
}
// processPullPushImageResp decodes JSON messages from the image pull/push response body and
// sends them to the provided channel.
func processPullPushImageResp(ctx context.Context, respBody io.ReadCloser, ch chan<- PullPushImageMessage) {
defer respBody.Close()
defer close(ch)
decoder := json.NewDecoder(respBody)
for {
var jm jsonmessage.JSONMessage
if err := decoder.Decode(&jm); err != nil {
if errors.Is(err, io.EOF) {
return
}
ch <- PullPushImageMessage{Err: fmt.Errorf("decode image pull/push message: %w", err)}
return
}
msg := PullPushImageMessage{Message: jm}
if jm.Error != nil {
msg.Err = errors.New(jm.Error.Message)
}
select {
case <-ctx.Done():
ch <- PullPushImageMessage{Err: ctx.Err()}
return
default:
ch <- msg
}
}
}
// RetrieveLocalDockerRegistryAuth retrieves the authentication token for the specified image from the local Docker
// config file. It returns the encoded authentication token if it contains any credentials, or an empty string if
// no credentials are found.
func RetrieveLocalDockerRegistryAuth(image string) (string, error) {
// Try to retrieve the authentication token for the image from the default local Docker config file.
dockerConfig := dockerconfig.LoadDefaultConfigFile(os.Stderr)
encodedAuth, err := dockercommand.RetrieveAuthTokenFromImage(dockerConfig, image)
if err != nil {
return "", err
}
// The encodedAuth can be a base64-encoded "{}" (empty JSON object) or include a server address but no credentials.
// Return encodedAuth only if it contains any credentials.
auth, err := registry.DecodeAuthConfig(encodedAuth)
if err != nil {
return "", fmt.Errorf("decode auth config: %w", err)
}
if auth.Username == "" &&
auth.Password == "" &&
auth.Auth == "" &&
auth.IdentityToken == "" &&
auth.RegistryToken == "" {
return "", nil
}
return encodedAuth, nil
}
+96
View File
@@ -0,0 +1,96 @@
package gitutil
import (
"fmt"
"os/exec"
"strconv"
"strings"
"time"
)
// GitState contains information about the Git repository state.
type GitState struct {
// Date is the current commit datetime.
Date time.Time
// IsDirty indicates whether there are uncommitted changes.
IsDirty bool
// IsRepo indicates whether the directory is a Git repository.
IsRepo bool
// SHA is the full SHA-1 (40 characters) of the current commit.
SHA string
}
// InspectGitState inspects the Git repo state from the specified directory.
// If the directory is not a Git repository, or if git utility is not available,
// returns GitState with IsRepo=false and no error.
func InspectGitState(dir string) (GitState, error) {
// Check if git command is available.
if _, err := exec.LookPath("git"); err != nil {
return GitState{}, nil
}
state := GitState{
IsRepo: isGitRepo(dir),
}
if !state.IsRepo {
return state, nil
}
// Get the current commit SHA.
sha, err := gitCommand(dir, "rev-parse", "HEAD")
if err != nil {
return state, fmt.Errorf("get current commit SHA: %w", err)
}
state.SHA = strings.TrimSpace(sha)
// Get the current commit timestamp.
timestamp, err := gitCommand(dir, "log", "-1", "--format=%ct")
if err != nil {
return state, fmt.Errorf("get current commit timestamp: %w", err)
}
seconds, err := strconv.ParseInt(strings.TrimSpace(timestamp), 10, 64)
if err != nil {
return state, fmt.Errorf("parse current commit timestamp: %w", err)
}
state.Date = time.Unix(seconds, 0).UTC()
// Check for uncommitted changes.
status, err := gitCommand(dir, "status", "--porcelain")
if err != nil {
return state, fmt.Errorf("check git status: %w", err)
}
state.IsDirty = strings.TrimSpace(status) != ""
return state, nil
}
// isGitRepo checks if the directory is a Git repository.
func isGitRepo(dir string) bool {
_, err := gitCommand(dir, "rev-parse", "--git-dir")
return err == nil
}
// gitCommand runs a git command in the specified directory.
func gitCommand(dir string, args ...string) (string, error) {
cmd := exec.Command("git", args...)
cmd.Dir = dir
output, err := cmd.Output()
if err != nil {
if exitErr, ok := err.(*exec.ExitError); ok {
return "", fmt.Errorf("git command failed: %s", exitErr.Stderr)
}
return "", err
}
return string(output), nil
}
// ShortSHA returns a truncated SHA with the specified length.
// If length is negative, zero, or greater than the SHA length, it returns the full SHA.
// Returns empty string if SHA is empty.
func (s *GitState) ShortSHA(length int) string {
if length <= 0 || length > len(s.SHA) {
length = len(s.SHA)
}
return s.SHA[:length]
}
+168
View File
@@ -0,0 +1,168 @@
package gitutil
import (
"os"
"os/exec"
"path/filepath"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestInspectGitState_GitNotAvailable(t *testing.T) {
// Reset the PATH env var to simulate git not being available.
t.Setenv("PATH", "")
tmpDir := t.TempDir()
state, err := InspectGitState(tmpDir)
require.NoError(t, err)
assert.False(t, state.IsRepo)
assert.Empty(t, state.SHA)
assert.Empty(t, state.ShortSHA(7))
assert.True(t, state.Date.IsZero())
assert.False(t, state.IsDirty)
}
func TestInspectGitState_NotARepo(t *testing.T) {
// Create a temporary directory that's not a git repo.
tmpDir := t.TempDir()
state, err := InspectGitState(tmpDir)
require.NoError(t, err)
// If tmpDir happens to be in a git repo (e.g., parent directory), skip this test.
// This is common when running tests from within the project itself.
if state.IsRepo {
t.Skip("tmpDir is within a git repository, skipping test")
}
assert.False(t, state.IsRepo)
assert.Empty(t, state.SHA)
assert.Empty(t, state.ShortSHA(7))
assert.True(t, state.Date.IsZero()) // Should have zero time if not a repo.
assert.False(t, state.IsDirty)
}
func TestInspectGitState_CleanRepo(t *testing.T) {
// Create a temporary git repo.
tmpDir := t.TempDir()
initGitRepo(t, tmpDir)
createAndCommitFile(t, tmpDir, "test.txt", "test content")
state, err := InspectGitState(tmpDir)
require.NoError(t, err)
assert.True(t, state.IsRepo)
assert.Len(t, state.SHA, 40)
assert.Equal(t, state.ShortSHA(7), state.SHA[:7])
assert.False(t, state.Date.IsZero())
assert.False(t, state.IsDirty)
}
func TestInspectGitState_DirtyRepo(t *testing.T) {
// Create a temporary git repo.
tmpDir := t.TempDir()
initGitRepo(t, tmpDir)
createAndCommitFile(t, tmpDir, "test.txt", "test content")
// Modify the file to make the repo dirty.
err := os.WriteFile(filepath.Join(tmpDir, "test.txt"), []byte("modified content"), 0o644)
require.NoError(t, err)
state, err := InspectGitState(tmpDir)
require.NoError(t, err)
assert.True(t, state.IsRepo)
assert.True(t, state.IsDirty)
}
func TestInspectGitState_UntrackedFiles(t *testing.T) {
// Create a temporary git repo.
tmpDir := t.TempDir()
initGitRepo(t, tmpDir)
createAndCommitFile(t, tmpDir, "test.txt", "test content")
// Add an untracked file.
err := os.WriteFile(filepath.Join(tmpDir, "untracked.txt"), []byte("untracked"), 0o644)
require.NoError(t, err)
state, err := InspectGitState(tmpDir)
require.NoError(t, err)
assert.True(t, state.IsRepo)
assert.True(t, state.IsDirty) // Untracked files make the repo dirty.
}
func TestGitState_ShortSHA(t *testing.T) {
state := &GitState{
SHA: "1234567890abcdef1234567890abcdef12345678",
}
tests := []struct {
name string
length int
expected string
}{
{"short 7", 7, "1234567"},
{"short 10", 10, "1234567890"},
{"full SHA", 40, "1234567890abcdef1234567890abcdef12345678"},
{"longer than SHA", 50, "1234567890abcdef1234567890abcdef12345678"},
{"negative", -42, "1234567890abcdef1234567890abcdef12345678"},
{"zero", 0, "1234567890abcdef1234567890abcdef12345678"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result := state.ShortSHA(tt.length)
assert.Equal(t, tt.expected, result)
})
}
}
func TestGitState_ShortSHA_Empty(t *testing.T) {
state := &GitState{
SHA: "",
}
result := state.ShortSHA(7)
assert.Empty(t, result)
}
// Helper functions.
func initGitRepo(t *testing.T, dir string) {
t.Helper()
// Initialize git repo.
runGitCommand(t, dir, "init")
runGitCommand(t, dir, "config", "user.email", "test@example.com")
runGitCommand(t, dir, "config", "user.name", "Test User")
}
func createAndCommitFile(t *testing.T, dir, filename, content string) {
t.Helper()
// Create file.
path := filepath.Join(dir, filename)
err := os.WriteFile(path, []byte(content), 0o644)
require.NoError(t, err)
// Commit file.
runGitCommand(t, dir, "add", filename)
runGitCommand(t, dir, "commit", "-m", "Add "+filename)
}
func runGitCommand(t *testing.T, dir string, args ...string) {
t.Helper()
cmd := exec.Command("git", args...)
cmd.Dir = dir
output, err := cmd.CombinedOutput()
require.NoError(t, err, "git command failed: %s", output)
}
+19
View File
@@ -0,0 +1,19 @@
package log
import (
"log/slog"
"os"
"slices"
"strings"
)
func InitLoggerFromEnv() {
debugValues := []string{"1", "true", "yes"}
if slices.Contains(debugValues, strings.ToLower(os.Getenv("DEBUG"))) {
logger := slog.New(NewSlogTextHandler(os.Stderr, &slog.HandlerOptions{
Level: slog.LevelDebug,
}))
slog.SetDefault(logger)
}
slog.Debug("logger initialized")
}
+176
View File
@@ -0,0 +1,176 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.34.2
// protoc v5.27.3
// source: internal/machine/api/pb/caddy.proto
package pb
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
emptypb "google.golang.org/protobuf/types/known/emptypb"
timestamppb "google.golang.org/protobuf/types/known/timestamppb"
reflect "reflect"
sync "sync"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
type GetCaddyConfigResponse struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
// The generated Caddyfile content.
Caddyfile string `protobuf:"bytes,1,opt,name=caddyfile,proto3" json:"caddyfile,omitempty"`
// Timestamp when the config was last modified.
ModifiedAt *timestamppb.Timestamp `protobuf:"bytes,2,opt,name=modified_at,json=modifiedAt,proto3" json:"modified_at,omitempty"`
}
func (x *GetCaddyConfigResponse) Reset() {
*x = GetCaddyConfigResponse{}
if protoimpl.UnsafeEnabled {
mi := &file_internal_machine_api_pb_caddy_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *GetCaddyConfigResponse) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*GetCaddyConfigResponse) ProtoMessage() {}
func (x *GetCaddyConfigResponse) ProtoReflect() protoreflect.Message {
mi := &file_internal_machine_api_pb_caddy_proto_msgTypes[0]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use GetCaddyConfigResponse.ProtoReflect.Descriptor instead.
func (*GetCaddyConfigResponse) Descriptor() ([]byte, []int) {
return file_internal_machine_api_pb_caddy_proto_rawDescGZIP(), []int{0}
}
func (x *GetCaddyConfigResponse) GetCaddyfile() string {
if x != nil {
return x.Caddyfile
}
return ""
}
func (x *GetCaddyConfigResponse) GetModifiedAt() *timestamppb.Timestamp {
if x != nil {
return x.ModifiedAt
}
return nil
}
var File_internal_machine_api_pb_caddy_proto protoreflect.FileDescriptor
var file_internal_machine_api_pb_caddy_proto_rawDesc = []byte{
0x0a, 0x23, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e, 0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69,
0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x70, 0x62, 0x2f, 0x63, 0x61, 0x64, 0x64, 0x79, 0x2e,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x03, 0x61, 0x70, 0x69, 0x1a, 0x1b, 0x67, 0x6f, 0x6f, 0x67,
0x6c, 0x65, 0x2f, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x65, 0x6d, 0x70, 0x74,
0x79, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x1a, 0x1f, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2f,
0x70, 0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2f, 0x74, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61,
0x6d, 0x70, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x22, 0x73, 0x0a, 0x16, 0x47, 0x65, 0x74, 0x43,
0x61, 0x64, 0x64, 0x79, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e,
0x73, 0x65, 0x12, 0x1c, 0x0a, 0x09, 0x63, 0x61, 0x64, 0x64, 0x79, 0x66, 0x69, 0x6c, 0x65, 0x18,
0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0x63, 0x61, 0x64, 0x64, 0x79, 0x66, 0x69, 0x6c, 0x65,
0x12, 0x3b, 0x0a, 0x0b, 0x6d, 0x6f, 0x64, 0x69, 0x66, 0x69, 0x65, 0x64, 0x5f, 0x61, 0x74, 0x18,
0x02, 0x20, 0x01, 0x28, 0x0b, 0x32, 0x1a, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70,
0x72, 0x6f, 0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x54, 0x69, 0x6d, 0x65, 0x73, 0x74, 0x61, 0x6d,
0x70, 0x52, 0x0a, 0x6d, 0x6f, 0x64, 0x69, 0x66, 0x69, 0x65, 0x64, 0x41, 0x74, 0x32, 0x49, 0x0a,
0x05, 0x43, 0x61, 0x64, 0x64, 0x79, 0x12, 0x40, 0x0a, 0x09, 0x47, 0x65, 0x74, 0x43, 0x6f, 0x6e,
0x66, 0x69, 0x67, 0x12, 0x16, 0x2e, 0x67, 0x6f, 0x6f, 0x67, 0x6c, 0x65, 0x2e, 0x70, 0x72, 0x6f,
0x74, 0x6f, 0x62, 0x75, 0x66, 0x2e, 0x45, 0x6d, 0x70, 0x74, 0x79, 0x1a, 0x1b, 0x2e, 0x61, 0x70,
0x69, 0x2e, 0x47, 0x65, 0x74, 0x43, 0x61, 0x64, 0x64, 0x79, 0x43, 0x6f, 0x6e, 0x66, 0x69, 0x67,
0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x37, 0x5a, 0x35, 0x67, 0x69, 0x74, 0x68,
0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x70, 0x73, 0x76, 0x69, 0x64, 0x65, 0x72, 0x73, 0x6b,
0x69, 0x2f, 0x75, 0x6e, 0x63, 0x6c, 0x6f, 0x75, 0x64, 0x2f, 0x69, 0x6e, 0x74, 0x65, 0x72, 0x6e,
0x61, 0x6c, 0x2f, 0x6d, 0x61, 0x63, 0x68, 0x69, 0x6e, 0x65, 0x2f, 0x61, 0x70, 0x69, 0x2f, 0x70,
0x62, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33,
}
var (
file_internal_machine_api_pb_caddy_proto_rawDescOnce sync.Once
file_internal_machine_api_pb_caddy_proto_rawDescData = file_internal_machine_api_pb_caddy_proto_rawDesc
)
func file_internal_machine_api_pb_caddy_proto_rawDescGZIP() []byte {
file_internal_machine_api_pb_caddy_proto_rawDescOnce.Do(func() {
file_internal_machine_api_pb_caddy_proto_rawDescData = protoimpl.X.CompressGZIP(file_internal_machine_api_pb_caddy_proto_rawDescData)
})
return file_internal_machine_api_pb_caddy_proto_rawDescData
}
var file_internal_machine_api_pb_caddy_proto_msgTypes = make([]protoimpl.MessageInfo, 1)
var file_internal_machine_api_pb_caddy_proto_goTypes = []any{
(*GetCaddyConfigResponse)(nil), // 0: api.GetCaddyConfigResponse
(*timestamppb.Timestamp)(nil), // 1: google.protobuf.Timestamp
(*emptypb.Empty)(nil), // 2: google.protobuf.Empty
}
var file_internal_machine_api_pb_caddy_proto_depIdxs = []int32{
1, // 0: api.GetCaddyConfigResponse.modified_at:type_name -> google.protobuf.Timestamp
2, // 1: api.Caddy.GetConfig:input_type -> google.protobuf.Empty
0, // 2: api.Caddy.GetConfig:output_type -> api.GetCaddyConfigResponse
2, // [2:3] is the sub-list for method output_type
1, // [1:2] is the sub-list for method input_type
1, // [1:1] is the sub-list for extension type_name
1, // [1:1] is the sub-list for extension extendee
0, // [0:1] is the sub-list for field type_name
}
func init() { file_internal_machine_api_pb_caddy_proto_init() }
func file_internal_machine_api_pb_caddy_proto_init() {
if File_internal_machine_api_pb_caddy_proto != nil {
return
}
if !protoimpl.UnsafeEnabled {
file_internal_machine_api_pb_caddy_proto_msgTypes[0].Exporter = func(v any, i int) any {
switch v := v.(*GetCaddyConfigResponse); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_internal_machine_api_pb_caddy_proto_rawDesc,
NumEnums: 0,
NumMessages: 1,
NumExtensions: 0,
NumServices: 1,
},
GoTypes: file_internal_machine_api_pb_caddy_proto_goTypes,
DependencyIndexes: file_internal_machine_api_pb_caddy_proto_depIdxs,
MessageInfos: file_internal_machine_api_pb_caddy_proto_msgTypes,
}.Build()
File_internal_machine_api_pb_caddy_proto = out.File
file_internal_machine_api_pb_caddy_proto_rawDesc = nil
file_internal_machine_api_pb_caddy_proto_goTypes = nil
file_internal_machine_api_pb_caddy_proto_depIdxs = nil
}
+20
View File
@@ -0,0 +1,20 @@
syntax = "proto3";
package api;
option go_package = "github.com/psviderski/uncloud/internal/machine/api/pb";
import "google/protobuf/empty.proto";
import "google/protobuf/timestamp.proto";
service Caddy {
// GetConfig retrieves the current Caddy configuration from the machine.
rpc GetConfig(google.protobuf.Empty) returns (GetCaddyConfigResponse);
}
message GetCaddyConfigResponse {
// The generated Caddyfile content.
string caddyfile = 1;
// Timestamp when the config was last modified.
google.protobuf.Timestamp modified_at = 2;
}
+124
View File
@@ -0,0 +1,124 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.5.1
// - protoc v5.27.3
// source: internal/machine/api/pb/caddy.proto
package pb
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
emptypb "google.golang.org/protobuf/types/known/emptypb"
)
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
// Requires gRPC-Go v1.64.0 or later.
const _ = grpc.SupportPackageIsVersion9
const (
Caddy_GetConfig_FullMethodName = "/api.Caddy/GetConfig"
)
// CaddyClient is the client API for Caddy service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type CaddyClient interface {
// GetConfig retrieves the current Caddy configuration from the machine.
GetConfig(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*GetCaddyConfigResponse, error)
}
type caddyClient struct {
cc grpc.ClientConnInterface
}
func NewCaddyClient(cc grpc.ClientConnInterface) CaddyClient {
return &caddyClient{cc}
}
func (c *caddyClient) GetConfig(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*GetCaddyConfigResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(GetCaddyConfigResponse)
err := c.cc.Invoke(ctx, Caddy_GetConfig_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
// CaddyServer is the server API for Caddy service.
// All implementations must embed UnimplementedCaddyServer
// for forward compatibility.
type CaddyServer interface {
// GetConfig retrieves the current Caddy configuration from the machine.
GetConfig(context.Context, *emptypb.Empty) (*GetCaddyConfigResponse, error)
mustEmbedUnimplementedCaddyServer()
}
// UnimplementedCaddyServer must be embedded to have
// forward compatible implementations.
//
// NOTE: this should be embedded by value instead of pointer to avoid a nil
// pointer dereference when methods are called.
type UnimplementedCaddyServer struct{}
func (UnimplementedCaddyServer) GetConfig(context.Context, *emptypb.Empty) (*GetCaddyConfigResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method GetConfig not implemented")
}
func (UnimplementedCaddyServer) mustEmbedUnimplementedCaddyServer() {}
func (UnimplementedCaddyServer) testEmbeddedByValue() {}
// UnsafeCaddyServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to CaddyServer will
// result in compilation errors.
type UnsafeCaddyServer interface {
mustEmbedUnimplementedCaddyServer()
}
func RegisterCaddyServer(s grpc.ServiceRegistrar, srv CaddyServer) {
// If the following call pancis, it indicates UnimplementedCaddyServer was
// embedded by pointer and is nil. This will cause panics if an
// unimplemented method is ever invoked, so we test this at initialization
// time to prevent it from happening at runtime later due to I/O.
if t, ok := srv.(interface{ testEmbeddedByValue() }); ok {
t.testEmbeddedByValue()
}
s.RegisterService(&Caddy_ServiceDesc, srv)
}
func _Caddy_GetConfig_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(emptypb.Empty)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(CaddyServer).GetConfig(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Caddy_GetConfig_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(CaddyServer).GetConfig(ctx, req.(*emptypb.Empty))
}
return interceptor(ctx, in, info, handler)
}
// Caddy_ServiceDesc is the grpc.ServiceDesc for Caddy service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
var Caddy_ServiceDesc = grpc.ServiceDesc{
ServiceName: "api.Caddy",
HandlerType: (*CaddyServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "GetConfig",
Handler: _Caddy_GetConfig_Handler,
},
},
Streams: []grpc.StreamDesc{},
Metadata: "internal/machine/api/pb/caddy.proto",
}
File diff suppressed because it is too large Load Diff
+82 -1
View File
@@ -5,6 +5,7 @@ package api;
option go_package = "github.com/psviderski/uncloud/internal/machine/api/pb"; option go_package = "github.com/psviderski/uncloud/internal/machine/api/pb";
import "google/protobuf/empty.proto"; import "google/protobuf/empty.proto";
import "google/protobuf/timestamp.proto";
import "internal/machine/api/pb/common.proto"; import "internal/machine/api/pb/common.proto";
service Docker { service Docker {
@@ -15,11 +16,15 @@ service Docker {
rpc ListContainers(ListContainersRequest) returns (ListContainersResponse); rpc ListContainers(ListContainersRequest) returns (ListContainersResponse);
rpc RemoveContainer(RemoveContainerRequest) returns (google.protobuf.Empty); rpc RemoveContainer(RemoveContainerRequest) returns (google.protobuf.Empty);
rpc ExecContainer(stream ExecContainerRequest) returns (stream ExecContainerResponse);
rpc ContainerLogs(ContainerLogsRequest) returns (stream ContainerLogEntry);
rpc PullImage(PullImageRequest) returns (stream JSONMessage); rpc PullImage(PullImageRequest) returns (stream JSONMessage);
rpc InspectImage(InspectImageRequest) returns (InspectImageResponse); rpc InspectImage(InspectImageRequest) returns (InspectImageResponse);
// InspectRemoteImage returns the image metadata for an image in a remote registry using the machine's // InspectRemoteImage returns the image metadata for an image in a remote registry using the machine's
// Docker auth credentials if necessary. // Docker auth credentials if necessary.
rpc InspectRemoteImage(InspectRemoteImageRequest) returns (InspectRemoteImageResponse); rpc InspectRemoteImage(InspectRemoteImageRequest) returns (InspectRemoteImageResponse);
rpc ListImages(ListImagesRequest) returns (ListImagesResponse);
rpc CreateVolume(CreateVolumeRequest) returns (CreateVolumeResponse); rpc CreateVolume(CreateVolumeRequest) returns (CreateVolumeResponse);
rpc ListVolumes(ListVolumesRequest) returns (ListVolumesResponse); rpc ListVolumes(ListVolumesRequest) returns (ListVolumesResponse);
@@ -91,6 +96,64 @@ message RemoveContainerRequest {
bytes options = 2; bytes options = 2;
} }
message ExecContainerRequest {
oneof payload {
// Initial configuration for the exec session. Must be sent as the first message.
ExecConfig config = 1;
// Raw stdin data to be written to the exec process.
bytes stdin = 2;
// TTY resize event (only used when TTY is enabled).
ResizeEvent resize = 3;
}
}
message ExecConfig {
// Container ID to execute the command in.
string container_id = 1;
// JSON serialised ExecOptions
bytes options = 2;
}
message ResizeEvent {
uint32 height = 1;
uint32 width = 2;
}
message ExecContainerResponse {
oneof payload {
// Exec instance ID returned after creating the exec.
string exec_id = 1;
// Raw stdout data from the exec process.
bytes stdout = 2;
// Raw stderr data from the exec process (only when TTY is disabled).
bytes stderr = 3;
// Exit code of the exec process. Sent as the final message.
int32 exit_code = 4;
}
}
message ContainerLogsRequest {
string container_id = 1;
// Options for logs retrieval.
bool follow = 2;
int32 tail = 3; // -1 means all
string since = 4; // https://www.rfc-editor.org/rfc/rfc3339.html timestamp or Go duration string
string until = 5; // https://www.rfc-editor.org/rfc/rfc3339.html timestamp or Go duration string
}
message ContainerLogEntry {
enum StreamType {
UNKNOWN = 0;
STDOUT = 1;
STDERR = 2;
HEARTBEAT = 3;
}
StreamType stream = 1;
google.protobuf.Timestamp timestamp = 2;
// Log line content. Empty for heartbeat entries.
bytes message = 3;
}
message PullImageRequest { message PullImageRequest {
string image = 1; string image = 1;
// JSON serialised image.PullOptions. // JSON serialised image.PullOptions.
@@ -113,7 +176,7 @@ message InspectImageResponse {
message Image { message Image {
Metadata metadata = 1; Metadata metadata = 1;
// JSON serialised types.ImageInspect. // JSON serialised image.InspectResponse.
bytes image = 2; bytes image = 2;
} }
@@ -134,6 +197,24 @@ message RemoteImage {
bytes manifest = 3; bytes manifest = 3;
} }
message ListImagesRequest {
// JSON serialised image.ListOptions.
bytes options = 1;
}
message ListImagesResponse {
// Must contain only one repeated messages field to allow broadcasting ListImages requests to multiple machines.
repeated MachineImages messages = 1;
}
message MachineImages {
Metadata metadata = 1;
// JSON serialised []image.Summary.
bytes images = 2;
// True if Docker uses the containerd image store, false if it uses its internal image store.
bool containerd_store = 3;
}
message CreateVolumeRequest { message CreateVolumeRequest {
// JSON serialised volume.CreateOptions. // JSON serialised volume.CreateOptions.
bytes options = 1; bytes options = 1;
+112 -1
View File
@@ -26,9 +26,12 @@ const (
Docker_StopContainer_FullMethodName = "/api.Docker/StopContainer" Docker_StopContainer_FullMethodName = "/api.Docker/StopContainer"
Docker_ListContainers_FullMethodName = "/api.Docker/ListContainers" Docker_ListContainers_FullMethodName = "/api.Docker/ListContainers"
Docker_RemoveContainer_FullMethodName = "/api.Docker/RemoveContainer" Docker_RemoveContainer_FullMethodName = "/api.Docker/RemoveContainer"
Docker_ExecContainer_FullMethodName = "/api.Docker/ExecContainer"
Docker_ContainerLogs_FullMethodName = "/api.Docker/ContainerLogs"
Docker_PullImage_FullMethodName = "/api.Docker/PullImage" Docker_PullImage_FullMethodName = "/api.Docker/PullImage"
Docker_InspectImage_FullMethodName = "/api.Docker/InspectImage" Docker_InspectImage_FullMethodName = "/api.Docker/InspectImage"
Docker_InspectRemoteImage_FullMethodName = "/api.Docker/InspectRemoteImage" Docker_InspectRemoteImage_FullMethodName = "/api.Docker/InspectRemoteImage"
Docker_ListImages_FullMethodName = "/api.Docker/ListImages"
Docker_CreateVolume_FullMethodName = "/api.Docker/CreateVolume" Docker_CreateVolume_FullMethodName = "/api.Docker/CreateVolume"
Docker_ListVolumes_FullMethodName = "/api.Docker/ListVolumes" Docker_ListVolumes_FullMethodName = "/api.Docker/ListVolumes"
Docker_RemoveVolume_FullMethodName = "/api.Docker/RemoveVolume" Docker_RemoveVolume_FullMethodName = "/api.Docker/RemoveVolume"
@@ -48,11 +51,14 @@ type DockerClient interface {
StopContainer(ctx context.Context, in *StopContainerRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) StopContainer(ctx context.Context, in *StopContainerRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
ListContainers(ctx context.Context, in *ListContainersRequest, opts ...grpc.CallOption) (*ListContainersResponse, error) ListContainers(ctx context.Context, in *ListContainersRequest, opts ...grpc.CallOption) (*ListContainersResponse, error)
RemoveContainer(ctx context.Context, in *RemoveContainerRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) RemoveContainer(ctx context.Context, in *RemoveContainerRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
ExecContainer(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[ExecContainerRequest, ExecContainerResponse], error)
ContainerLogs(ctx context.Context, in *ContainerLogsRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[ContainerLogEntry], error)
PullImage(ctx context.Context, in *PullImageRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[JSONMessage], error) PullImage(ctx context.Context, in *PullImageRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[JSONMessage], error)
InspectImage(ctx context.Context, in *InspectImageRequest, opts ...grpc.CallOption) (*InspectImageResponse, error) InspectImage(ctx context.Context, in *InspectImageRequest, opts ...grpc.CallOption) (*InspectImageResponse, error)
// InspectRemoteImage returns the image metadata for an image in a remote registry using the machine's // InspectRemoteImage returns the image metadata for an image in a remote registry using the machine's
// Docker auth credentials if necessary. // Docker auth credentials if necessary.
InspectRemoteImage(ctx context.Context, in *InspectRemoteImageRequest, opts ...grpc.CallOption) (*InspectRemoteImageResponse, error) InspectRemoteImage(ctx context.Context, in *InspectRemoteImageRequest, opts ...grpc.CallOption) (*InspectRemoteImageResponse, error)
ListImages(ctx context.Context, in *ListImagesRequest, opts ...grpc.CallOption) (*ListImagesResponse, error)
CreateVolume(ctx context.Context, in *CreateVolumeRequest, opts ...grpc.CallOption) (*CreateVolumeResponse, error) CreateVolume(ctx context.Context, in *CreateVolumeRequest, opts ...grpc.CallOption) (*CreateVolumeResponse, error)
ListVolumes(ctx context.Context, in *ListVolumesRequest, opts ...grpc.CallOption) (*ListVolumesResponse, error) ListVolumes(ctx context.Context, in *ListVolumesRequest, opts ...grpc.CallOption) (*ListVolumesResponse, error)
RemoveVolume(ctx context.Context, in *RemoveVolumeRequest, opts ...grpc.CallOption) (*emptypb.Empty, error) RemoveVolume(ctx context.Context, in *RemoveVolumeRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
@@ -130,9 +136,41 @@ func (c *dockerClient) RemoveContainer(ctx context.Context, in *RemoveContainerR
return out, nil return out, nil
} }
func (c *dockerClient) ExecContainer(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[ExecContainerRequest, ExecContainerResponse], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Docker_ServiceDesc.Streams[0], Docker_ExecContainer_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[ExecContainerRequest, ExecContainerResponse]{ClientStream: stream}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Docker_ExecContainerClient = grpc.BidiStreamingClient[ExecContainerRequest, ExecContainerResponse]
func (c *dockerClient) ContainerLogs(ctx context.Context, in *ContainerLogsRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[ContainerLogEntry], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Docker_ServiceDesc.Streams[1], Docker_ContainerLogs_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[ContainerLogsRequest, ContainerLogEntry]{ClientStream: stream}
if err := x.ClientStream.SendMsg(in); err != nil {
return nil, err
}
if err := x.ClientStream.CloseSend(); err != nil {
return nil, err
}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Docker_ContainerLogsClient = grpc.ServerStreamingClient[ContainerLogEntry]
func (c *dockerClient) PullImage(ctx context.Context, in *PullImageRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[JSONMessage], error) { func (c *dockerClient) PullImage(ctx context.Context, in *PullImageRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[JSONMessage], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...) cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Docker_ServiceDesc.Streams[0], Docker_PullImage_FullMethodName, cOpts...) stream, err := c.cc.NewStream(ctx, &Docker_ServiceDesc.Streams[2], Docker_PullImage_FullMethodName, cOpts...)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -169,6 +207,16 @@ func (c *dockerClient) InspectRemoteImage(ctx context.Context, in *InspectRemote
return out, nil return out, nil
} }
func (c *dockerClient) ListImages(ctx context.Context, in *ListImagesRequest, opts ...grpc.CallOption) (*ListImagesResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(ListImagesResponse)
err := c.cc.Invoke(ctx, Docker_ListImages_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *dockerClient) CreateVolume(ctx context.Context, in *CreateVolumeRequest, opts ...grpc.CallOption) (*CreateVolumeResponse, error) { func (c *dockerClient) CreateVolume(ctx context.Context, in *CreateVolumeRequest, opts ...grpc.CallOption) (*CreateVolumeResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...) cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(CreateVolumeResponse) out := new(CreateVolumeResponse)
@@ -249,11 +297,14 @@ type DockerServer interface {
StopContainer(context.Context, *StopContainerRequest) (*emptypb.Empty, error) StopContainer(context.Context, *StopContainerRequest) (*emptypb.Empty, error)
ListContainers(context.Context, *ListContainersRequest) (*ListContainersResponse, error) ListContainers(context.Context, *ListContainersRequest) (*ListContainersResponse, error)
RemoveContainer(context.Context, *RemoveContainerRequest) (*emptypb.Empty, error) RemoveContainer(context.Context, *RemoveContainerRequest) (*emptypb.Empty, error)
ExecContainer(grpc.BidiStreamingServer[ExecContainerRequest, ExecContainerResponse]) error
ContainerLogs(*ContainerLogsRequest, grpc.ServerStreamingServer[ContainerLogEntry]) error
PullImage(*PullImageRequest, grpc.ServerStreamingServer[JSONMessage]) error PullImage(*PullImageRequest, grpc.ServerStreamingServer[JSONMessage]) error
InspectImage(context.Context, *InspectImageRequest) (*InspectImageResponse, error) InspectImage(context.Context, *InspectImageRequest) (*InspectImageResponse, error)
// InspectRemoteImage returns the image metadata for an image in a remote registry using the machine's // InspectRemoteImage returns the image metadata for an image in a remote registry using the machine's
// Docker auth credentials if necessary. // Docker auth credentials if necessary.
InspectRemoteImage(context.Context, *InspectRemoteImageRequest) (*InspectRemoteImageResponse, error) InspectRemoteImage(context.Context, *InspectRemoteImageRequest) (*InspectRemoteImageResponse, error)
ListImages(context.Context, *ListImagesRequest) (*ListImagesResponse, error)
CreateVolume(context.Context, *CreateVolumeRequest) (*CreateVolumeResponse, error) CreateVolume(context.Context, *CreateVolumeRequest) (*CreateVolumeResponse, error)
ListVolumes(context.Context, *ListVolumesRequest) (*ListVolumesResponse, error) ListVolumes(context.Context, *ListVolumesRequest) (*ListVolumesResponse, error)
RemoveVolume(context.Context, *RemoveVolumeRequest) (*emptypb.Empty, error) RemoveVolume(context.Context, *RemoveVolumeRequest) (*emptypb.Empty, error)
@@ -289,6 +340,12 @@ func (UnimplementedDockerServer) ListContainers(context.Context, *ListContainers
func (UnimplementedDockerServer) RemoveContainer(context.Context, *RemoveContainerRequest) (*emptypb.Empty, error) { func (UnimplementedDockerServer) RemoveContainer(context.Context, *RemoveContainerRequest) (*emptypb.Empty, error) {
return nil, status.Errorf(codes.Unimplemented, "method RemoveContainer not implemented") return nil, status.Errorf(codes.Unimplemented, "method RemoveContainer not implemented")
} }
func (UnimplementedDockerServer) ExecContainer(grpc.BidiStreamingServer[ExecContainerRequest, ExecContainerResponse]) error {
return status.Errorf(codes.Unimplemented, "method ExecContainer not implemented")
}
func (UnimplementedDockerServer) ContainerLogs(*ContainerLogsRequest, grpc.ServerStreamingServer[ContainerLogEntry]) error {
return status.Errorf(codes.Unimplemented, "method ContainerLogs not implemented")
}
func (UnimplementedDockerServer) PullImage(*PullImageRequest, grpc.ServerStreamingServer[JSONMessage]) error { func (UnimplementedDockerServer) PullImage(*PullImageRequest, grpc.ServerStreamingServer[JSONMessage]) error {
return status.Errorf(codes.Unimplemented, "method PullImage not implemented") return status.Errorf(codes.Unimplemented, "method PullImage not implemented")
} }
@@ -298,6 +355,9 @@ func (UnimplementedDockerServer) InspectImage(context.Context, *InspectImageRequ
func (UnimplementedDockerServer) InspectRemoteImage(context.Context, *InspectRemoteImageRequest) (*InspectRemoteImageResponse, error) { func (UnimplementedDockerServer) InspectRemoteImage(context.Context, *InspectRemoteImageRequest) (*InspectRemoteImageResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method InspectRemoteImage not implemented") return nil, status.Errorf(codes.Unimplemented, "method InspectRemoteImage not implemented")
} }
func (UnimplementedDockerServer) ListImages(context.Context, *ListImagesRequest) (*ListImagesResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method ListImages not implemented")
}
func (UnimplementedDockerServer) CreateVolume(context.Context, *CreateVolumeRequest) (*CreateVolumeResponse, error) { func (UnimplementedDockerServer) CreateVolume(context.Context, *CreateVolumeRequest) (*CreateVolumeResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method CreateVolume not implemented") return nil, status.Errorf(codes.Unimplemented, "method CreateVolume not implemented")
} }
@@ -448,6 +508,24 @@ func _Docker_RemoveContainer_Handler(srv interface{}, ctx context.Context, dec f
return interceptor(ctx, in, info, handler) return interceptor(ctx, in, info, handler)
} }
func _Docker_ExecContainer_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(DockerServer).ExecContainer(&grpc.GenericServerStream[ExecContainerRequest, ExecContainerResponse]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Docker_ExecContainerServer = grpc.BidiStreamingServer[ExecContainerRequest, ExecContainerResponse]
func _Docker_ContainerLogs_Handler(srv interface{}, stream grpc.ServerStream) error {
m := new(ContainerLogsRequest)
if err := stream.RecvMsg(m); err != nil {
return err
}
return srv.(DockerServer).ContainerLogs(m, &grpc.GenericServerStream[ContainerLogsRequest, ContainerLogEntry]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Docker_ContainerLogsServer = grpc.ServerStreamingServer[ContainerLogEntry]
func _Docker_PullImage_Handler(srv interface{}, stream grpc.ServerStream) error { func _Docker_PullImage_Handler(srv interface{}, stream grpc.ServerStream) error {
m := new(PullImageRequest) m := new(PullImageRequest)
if err := stream.RecvMsg(m); err != nil { if err := stream.RecvMsg(m); err != nil {
@@ -495,6 +573,24 @@ func _Docker_InspectRemoteImage_Handler(srv interface{}, ctx context.Context, de
return interceptor(ctx, in, info, handler) return interceptor(ctx, in, info, handler)
} }
func _Docker_ListImages_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ListImagesRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(DockerServer).ListImages(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Docker_ListImages_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(DockerServer).ListImages(ctx, req.(*ListImagesRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Docker_CreateVolume_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { func _Docker_CreateVolume_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(CreateVolumeRequest) in := new(CreateVolumeRequest)
if err := dec(in); err != nil { if err := dec(in); err != nil {
@@ -660,6 +756,10 @@ var Docker_ServiceDesc = grpc.ServiceDesc{
MethodName: "InspectRemoteImage", MethodName: "InspectRemoteImage",
Handler: _Docker_InspectRemoteImage_Handler, Handler: _Docker_InspectRemoteImage_Handler,
}, },
{
MethodName: "ListImages",
Handler: _Docker_ListImages_Handler,
},
{ {
MethodName: "CreateVolume", MethodName: "CreateVolume",
Handler: _Docker_CreateVolume_Handler, Handler: _Docker_CreateVolume_Handler,
@@ -690,6 +790,17 @@ var Docker_ServiceDesc = grpc.ServiceDesc{
}, },
}, },
Streams: []grpc.StreamDesc{ Streams: []grpc.StreamDesc{
{
StreamName: "ExecContainer",
Handler: _Docker_ExecContainer_Handler,
ServerStreams: true,
ClientStreams: true,
},
{
StreamName: "ContainerLogs",
Handler: _Docker_ContainerLogs_Handler,
ServerStreams: true,
},
{ {
StreamName: "PullImage", StreamName: "PullImage",
Handler: _Docker_PullImage_Handler, Handler: _Docker_PullImage_Handler,
+330 -5
View File
@@ -1,17 +1,342 @@
package caddyconfig package caddyconfig
import ( import (
"bytes"
"cmp"
"context"
"fmt" "fmt"
"log/slog"
"maps"
"net"
"slices"
"strconv"
"strings"
"text/template"
"github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
) )
func GenerateCaddyfile(containers []api.ServiceContainer, verifyResponse string) (string, error) { const (
return fmt.Sprintf(`http:// { caddyfileHeader = `# This file is autogenerated by Uncloud based on the configuration of running services.
handle %s { # Do not edit manually. Any manual changes will be overwritten on the next update.
respond "%s" 200 `
caddyfileTemplate = `# Health check endpoint to verify Caddy reachability on this machine.
http:// {
handle {{.VerifyPath}} {
respond "{{.VerifyResponse}}" 200
} }
log log
} }
`, VerifyPath, verifyResponse), nil
(common_proxy) {
# Retry failed requests up to lb_retries times against other available upstreams.
lb_retries 3
# Upstreams are marked unhealthy for fail_duration after a failed request (passive health checking).
fail_duration 30s
}
{{- if or .HTTPHostUpstreams .HTTPSHostUpstreams }}
# Sites generated from service ports.{{end}}
{{- range $hostname, $upstreams := .HTTPHostUpstreams}}
http://{{$hostname}} {
reverse_proxy {{join $upstreams " "}} {
import common_proxy
}
log
}{{end}}
{{- range $hostname, $upstreams := .HTTPSHostUpstreams}}
https://{{$hostname}} {
reverse_proxy {{join $upstreams " "}} {
import common_proxy
}
log
}{{end}}
`
caddyfileUnavailabeFooter = `# NOTE: User-defined configs for services were skipped because Caddy is not running on this machine
# or the latest generated config is invalid. Please check the Caddy logs if it's running.
`
)
// CaddyfileGenerator generates a Caddyfile configuration for the Caddy reverse proxy.
type CaddyfileGenerator struct {
// machineID is the unique identifier of the machine where the controller is running.
machineID string
validator CaddyfileValidator
log *slog.Logger
}
// CaddyfileValidator is an interface for validating Caddyfile configurations.
type CaddyfileValidator interface {
Validate(ctx context.Context, caddyfile string) error
}
func NewCaddyfileGenerator(machineID string, validator CaddyfileValidator, log *slog.Logger) *CaddyfileGenerator {
if log == nil {
log = slog.Default()
}
return &CaddyfileGenerator{
machineID: machineID,
validator: validator,
log: log,
}
}
// Generate creates a Caddyfile configuration based on the provided service containers.
// The Caddyfile is generated from the service ports of the healthy containers.
// If a 'caddy' service container is running on this machine and defines a custom Caddy config (x-caddy) in its service
// spec, it will be validated and prepended to the generated Caddyfile. Custom Caddy configs (x-caddy) defined in other
// service specs are validated and appended to the generated Caddyfile. Invalid configs are logged and skipped to ensure
// the generated Caddyfile remains valid.
//
// The final Caddyfile structure includes:
//
// [caddy x-caddy (global config)]
// [generated Caddyfile from all service ports]
// [service-a x-caddy]
// ...
// [service-z x-caddy]
//
// If includeCustom is false, custom Caddy configs (x-caddy) are not included in the generated Caddyfile.
func (g *CaddyfileGenerator) Generate(
ctx context.Context, records []store.ContainerRecord, includeCustom bool,
) (string, error) {
containers := make([]api.ServiceContainer, len(records))
for i, cr := range records {
containers[i] = cr.Container
}
// Sort containers by service name and creation time to generate a stable Caddyfile.
slices.SortStableFunc(containers, func(a, b api.ServiceContainer) int {
return cmp.Or(
strings.Compare(a.ServiceName(), b.ServiceName()),
a.CreatedTime().Compare(b.CreatedTime()),
)
})
caddyfile, err := g.generateBaseFromPorts(containers)
if err != nil {
return "", fmt.Errorf("generate base Caddyfile from service ports: %w", err)
}
if !includeCustom {
return fmt.Sprintf("%s\n%s\n%s", caddyfileHeader, caddyfile, caddyfileUnavailabeFooter), nil
}
upstreams := serviceUpstreams(containers)
// Track validation errors for reporting.
var configErrors []string
// Find the 'caddy' service container on this machine. Use the most recent one if multiple exist.
var caddyCtr *api.ServiceContainer
for _, cr := range records {
if cr.MachineID == g.machineID && cr.Container.ServiceName() == CaddyServiceName &&
(caddyCtr == nil || cr.Container.CreatedTime().Compare(caddyCtr.CreatedTime()) > 0) {
caddyCtr = &cr.Container
}
}
// If the caddy container is running on this machine and has a custom Caddy config (global),
// prepend it to the generated Caddyfile and validate it.
if caddyCtr != nil && caddyCtr.ServiceSpec.CaddyConfig() != "" {
// Render the custom global Caddy config as a Go template with the upstreams.
tmplCtx := templateContext{
Name: caddyCtr.ServiceName(),
Upstreams: upstreams,
}
renderedConfig, err := renderCaddyfile(tmplCtx, caddyCtr.ServiceSpec.CaddyConfig())
if err != nil {
g.log.Error("Failed to render template directives in user-defined global Caddy config, skipping it.",
"service", caddyCtr.ServiceName(), "container", caddyCtr.ID, "err", err)
configErrors = append(configErrors,
fmt.Sprintf("service '%s': failed to render template: %v", caddyCtr.ServiceName(), err))
} else {
caddyfileCandidate := fmt.Sprintf("# User-defined global config from service '%s'.\n%s\n\n%s",
caddyCtr.ServiceName(), renderedConfig, caddyfile)
if err = g.validator.Validate(ctx, caddyfileCandidate); err != nil {
g.log.Error("User-defined global Caddy config is invalid, skipping it.",
"service", caddyCtr.ServiceName(), "container", caddyCtr.ID, "err", err)
configErrors = append(configErrors,
fmt.Sprintf("service '%s': validation failed: %v", caddyCtr.ServiceName(), err))
} else {
caddyfile = caddyfileCandidate
}
}
}
// There could be multiple service containers for the same service with different custom Caddy configs, for example,
// if the service has been partially updated. The most recent container for each service defines the current custom
// Caddy config for that service.
latestServiceContainers := make(map[string]api.ServiceContainer, len(containers))
for _, ctr := range containers {
if latest, ok := latestServiceContainers[ctr.ServiceName()]; ok {
if ctr.CreatedTime().Compare(latest.CreatedTime()) > 0 {
latestServiceContainers[ctr.ServiceName()] = ctr
}
} else {
latestServiceContainers[ctr.ServiceName()] = ctr
}
}
sortedServiceNames := slices.Sorted(maps.Keys(latestServiceContainers))
// Append a custom Caddy config for each service to the Caddyfile and validate it. If the config for a service
// is invalid, skip it but continue processing other services to ensure the Caddyfile remains valid.
for _, serviceName := range sortedServiceNames {
// Skip the caddy container as we already processed it.
if serviceName == CaddyServiceName {
continue
}
ctr := latestServiceContainers[serviceName]
if ctr.ServiceSpec.CaddyConfig() == "" {
continue
}
// Render the template actions in the service's Caddy config.
tmplCtx := templateContext{
Name: serviceName,
Upstreams: upstreams,
}
renderedConfig, err := renderCaddyfile(tmplCtx, ctr.ServiceSpec.CaddyConfig())
if err != nil {
g.log.Error("Failed to render template directives in user-defined Caddy config for service, skipping it.",
"service", serviceName, "err", err)
configErrors = append(configErrors,
fmt.Sprintf("service '%s': failed to render template: %v", serviceName, err))
continue
}
caddyfileCandidate := fmt.Sprintf("%s\n# User-defined config for service '%s'.\n%s\n",
caddyfile, serviceName, renderedConfig)
if err = g.validator.Validate(ctx, caddyfileCandidate); err != nil {
g.log.Error("User-defined Caddy config for service is invalid, skipping it.",
"service", serviceName, "err", err)
configErrors = append(configErrors, fmt.Sprintf("service '%s': validation failed: %v", serviceName, err))
} else {
caddyfile = caddyfileCandidate
}
}
// Append error summary as comment if there were any invalid configs.
if len(configErrors) > 0 {
errorsComment := "# Skipped invalid user-defined configs:\n"
for _, e := range configErrors {
errorsComment += fmt.Sprintf("# - %s\n", e)
}
caddyfile += "\n" + errorsComment
}
return caddyfileHeader + "\n" + caddyfile, nil
}
func (g *CaddyfileGenerator) generateBaseFromPorts(containers []api.ServiceContainer) (string, error) {
httpHostUpstreams, httpsHostUpstreams := httpUpstreamsFromPorts(containers)
funcs := template.FuncMap{"join": strings.Join}
tmpl, err := template.New("Caddyfile").Funcs(funcs).Parse(caddyfileTemplate)
if err != nil {
return "", fmt.Errorf("parse Caddyfile template: %w", err)
}
data := struct {
VerifyPath string
VerifyResponse string
HTTPHostUpstreams map[string][]string
HTTPSHostUpstreams map[string][]string
}{
VerifyPath: VerifyPath,
VerifyResponse: g.machineID,
HTTPHostUpstreams: httpHostUpstreams,
HTTPSHostUpstreams: httpsHostUpstreams,
}
var buf bytes.Buffer
if err = tmpl.Execute(&buf, data); err != nil {
return "", fmt.Errorf("execute Caddyfile template: %w", err)
}
return buf.String(), nil
}
// httpUpstreamsFromPorts extracts upstreams for HTTP and HTTPS protocols from the published ports of the provided
// service containers. It's expected that all containers are healthy.
func httpUpstreamsFromPorts(containers []api.ServiceContainer) (map[string][]string, map[string][]string) {
// Maps hostnames to lists of upstreams (container IP:port pairs).
httpHostUpstreams := make(map[string][]string)
httpsHostUpstreams := make(map[string][]string)
for _, ctr := range containers {
ip := ctr.UncloudNetworkIP()
if !ip.IsValid() {
// Container is not connected to the uncloud Docker network (could be host network).
continue
}
log := slog.With("container", ctr.ID)
ports, err := ctr.ServicePorts()
if err != nil {
log.Error("Failed to parse service ports for container.", "err", err)
continue
}
for _, port := range ports {
if port.Mode != api.PortModeIngress {
continue
}
switch port.Protocol {
case api.ProtocolHTTP:
upstream := net.JoinHostPort(ip.String(), strconv.Itoa(int(port.ContainerPort)))
httpHostUpstreams[port.Hostname] = append(httpHostUpstreams[port.Hostname], upstream)
case api.ProtocolHTTPS:
upstream := net.JoinHostPort(ip.String(), strconv.Itoa(int(port.ContainerPort)))
httpsHostUpstreams[port.Hostname] = append(httpsHostUpstreams[port.Hostname], upstream)
default:
// TODO: implement L4 ingress routing for TCP and UDP.
log.Error("Unsupported protocol for ingress port.", "port", port)
continue
}
}
}
return httpHostUpstreams, httpsHostUpstreams
}
// serviceUpstreams creates a map of service names to their container IPs.
// Only includes containers connected to the uncloud Docker network.
func serviceUpstreams(containers []api.ServiceContainer) map[string][]string {
upstreams := make(map[string][]string)
for _, ctr := range containers {
ip := ctr.UncloudNetworkIP()
if !ip.IsValid() {
// Container is not connected to the uncloud Docker network (could be host network).
continue
}
serviceName := ctr.ServiceName()
upstreams[serviceName] = append(upstreams[serviceName], ip.String())
}
return upstreams
}
// renderCaddyfile renders a Caddyfile template with the upstreams function and data.
func renderCaddyfile(tmplCtx templateContext, caddyfile string) (string, error) {
funcs := template.FuncMap{
"upstreams": upstreamsTemplateFn(tmplCtx),
}
tmpl, err := template.New("Caddyfile").Funcs(funcs).Parse(caddyfile)
if err != nil {
return "", fmt.Errorf("parse config as Go template: %w", err)
}
var buf bytes.Buffer
if err = tmpl.Execute(&buf, tmplCtx); err != nil {
return "", fmt.Errorf("execute template: %w", err)
}
return buf.String(), nil
} }
File diff suppressed because it is too large Load Diff
+142
View File
@@ -0,0 +1,142 @@
package caddyconfig
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"net/http"
"strings"
"time"
"github.com/caddyserver/caddy/v2"
)
// CaddyAdminClient is a client for interacting with the Caddy admin API over a Unix socket.
type CaddyAdminClient struct {
socketPath string
client *http.Client
}
func NewCaddyAdminClient(socketPath string) *CaddyAdminClient {
return &CaddyAdminClient{
socketPath: socketPath,
client: &http.Client{
Timeout: 5 * time.Second,
Transport: &http.Transport{
DialContext: func(_ context.Context, _, _ string) (net.Conn, error) {
return net.Dial("unix", socketPath)
},
},
},
}
}
// IsAvailable checks if the local Caddy instance is running and responding to admin API requests.
func (c *CaddyAdminClient) IsAvailable(ctx context.Context) bool {
// Caddy doesn't serve a /ping endpoint. It's a random endpoint we can use to check if Caddy is running.
req, err := http.NewRequestWithContext(ctx, "GET", "http://localhost/ping", nil)
if err != nil {
return false
}
resp, err := c.client.Do(req)
if err != nil {
return false
}
defer resp.Body.Close()
// Any HTTP response means Caddy is running and accessible.
return true
}
// Adapt converts a Caddyfile to JSON configuration without loading or running it.
func (c *CaddyAdminClient) Adapt(ctx context.Context, caddyfile string) (string, error) {
req, err := http.NewRequestWithContext(ctx, "POST", "http://localhost/adapt", strings.NewReader(caddyfile))
if err != nil {
return "", fmt.Errorf("create adapt request: %w", err)
}
req.Header.Set("Content-Type", "text/caddyfile")
resp, err := c.client.Do(req)
if err != nil {
return "", fmt.Errorf("send adapt request: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
return "", fmt.Errorf("read response body: %w", err)
}
if resp.StatusCode == http.StatusOK {
// Parse the response body to extract the result field.
var msg struct {
Result json.RawMessage `json:"result"`
}
if err = json.Unmarshal(body, &msg); err != nil {
return "", fmt.Errorf("parse adapt response: %w", err)
}
return string(msg.Result), nil
}
// If the response is a 400 Bad Request, try to parse the error message from it.
if resp.StatusCode == http.StatusBadRequest {
var apiError caddy.APIError
if err = json.Unmarshal(body, &apiError); err == nil {
return "", errors.New(apiError.Message)
}
}
return "", errors.New(string(body))
}
// Load loads a Caddyfile configuration into the Caddy instance running on the machine.
// Due to a Caddy bug (https://github.com/caddyserver/caddy/issues/7246), we first adapt the Caddyfile to JSON
// and then load the JSON config to get proper error handling.
func (c *CaddyAdminClient) Load(ctx context.Context, caddyfile string) error {
jsonConfig, err := c.Adapt(ctx, caddyfile)
if err != nil {
return fmt.Errorf("adapt Caddyfile to JSON config: %w", err)
}
req, err := http.NewRequestWithContext(ctx, "POST", "http://localhost/load", strings.NewReader(jsonConfig))
if err != nil {
return fmt.Errorf("create load request: %w", err)
}
req.Header.Set("Content-Type", "application/json")
resp, err := c.client.Do(req)
if err != nil {
return fmt.Errorf("send load request: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusOK {
return nil
}
body, _ := io.ReadAll(resp.Body)
// If the response is a 400 Bad Request, try to parse the error message from it.
if resp.StatusCode == http.StatusBadRequest {
var apiError caddy.APIError
if err = json.Unmarshal(body, &apiError); err == nil {
return fmt.Errorf("caddy responded with error: %s", apiError.Message)
}
}
return fmt.Errorf("caddy responded with error: HTTP %d: %s", resp.StatusCode, string(body))
}
// Validate checks if the provided Caddyfile can be adapted to Caddy JSON config using the running Caddy instance via
// its admin API. It doesn't guarantee that the Caddyfile is actually valid and can be loaded. For example, a tls
// directive with a missing certificate will pass the adaptation but will fail when Caddy tries to load it.
// But this is the best we can do over the admin API.
// TODO: run 'docker exec caddy-container caddy validate' to do proper validation or implement a Caddy module that
// exposes a validation endpoint.
func (c *CaddyAdminClient) Validate(ctx context.Context, caddyfile string) error {
_, err := c.Adapt(ctx, caddyfile)
return err
}
+89 -52
View File
@@ -14,6 +14,7 @@ import (
) )
const ( const (
CaddyServiceName = "caddy"
CaddyGroup = "uncloud" CaddyGroup = "uncloud"
VerifyPath = "/.uncloud-verify" VerifyPath = "/.uncloud-verify"
) )
@@ -22,13 +23,15 @@ const (
// proxy. The generated configuration allows Caddy to route external traffic to service containers across the internal // proxy. The generated configuration allows Caddy to route external traffic to service containers across the internal
// network. // network.
type Controller struct { type Controller struct {
machineID string
caddyfilePath string
generator *CaddyfileGenerator
client *CaddyAdminClient
store *store.Store store *store.Store
configDir string
verifyResponse string
log *slog.Logger log *slog.Logger
} }
func NewController(store *store.Store, configDir string, verifyResponse string) (*Controller, error) { func NewController(machineID, configDir, adminSock string, store *store.Store) (*Controller, error) {
if err := os.MkdirAll(configDir, 0o750); err != nil { if err := os.MkdirAll(configDir, 0o750); err != nil {
return nil, fmt.Errorf("create directory for Caddy configuration '%s': %w", configDir, err) return nil, fmt.Errorf("create directory for Caddy configuration '%s': %w", configDir, err)
} }
@@ -36,31 +39,33 @@ func NewController(store *store.Store, configDir string, verifyResponse string)
return nil, fmt.Errorf("change owner of directory for Caddy configuration '%s': %w", configDir, err) return nil, fmt.Errorf("change owner of directory for Caddy configuration '%s': %w", configDir, err)
} }
log := slog.With("component", "caddy-controller")
client := NewCaddyAdminClient(adminSock)
generator := NewCaddyfileGenerator(machineID, client, log)
return &Controller{ return &Controller{
machineID: machineID,
caddyfilePath: filepath.Join(configDir, "Caddyfile"),
generator: generator,
client: client,
store: store, store: store,
configDir: configDir, log: log,
verifyResponse: verifyResponse,
log: slog.With("component", "caddy-controller"),
}, nil }, nil
} }
func (c *Controller) Run(ctx context.Context) error { func (c *Controller) Run(ctx context.Context) error {
containerRecords, changes, err := c.store.SubscribeContainers(ctx) containers, changes, err := c.store.SubscribeContainers(ctx)
if err != nil { if err != nil {
return fmt.Errorf("subscribe to container changes: %w", err) return fmt.Errorf("subscribe to container changes: %w", err)
} }
c.log.Info("Subscribed to container changes in the cluster to generate Caddy configuration.") c.log.Info("Subscribed to container changes in the cluster to generate Caddy configuration.")
containers, err := c.filterAvailableContainers(containerRecords) containers = filterHealthyContainers(containers)
if err != nil { c.generateAndLoadCaddyfile(ctx, containers)
return fmt.Errorf("filter available containers: %w", err)
}
if err = c.generateCaddyfile(containers); err != nil { // TODO: left for backward compatibility, remove later.
return fmt.Errorf("generate Caddyfile configuration: %w", err)
}
if err = c.generateJSONConfig(containers); err != nil { if err = c.generateJSONConfig(containers); err != nil {
return fmt.Errorf("generate Caddy JSON configuration: %w", err) c.log.Error("Failed to generate Caddy JSON configuration to disk.", "err", err)
} }
for { for {
@@ -71,63 +76,95 @@ func (c *Controller) Run(ctx context.Context) error {
} }
c.log.Info("Cluster containers changed, updating Caddy configuration.") c.log.Info("Cluster containers changed, updating Caddy configuration.")
containerRecords, err = c.store.ListContainers(ctx, store.ListOptions{}) containers, err = c.store.ListContainers(ctx, store.ListOptions{})
if err != nil { if err != nil {
c.log.Info("Failed to list containers.", "err", err) c.log.Error("Failed to list containers.", "err", err)
continue
}
containers, err = c.filterAvailableContainers(containerRecords)
if err != nil {
c.log.Info("Failed to filter available containers.", "err", err)
continue continue
} }
containers = filterHealthyContainers(containers)
c.generateAndLoadCaddyfile(ctx, containers)
if err = c.generateCaddyfile(containers); err != nil { // TODO: left for backward compatibility, remove later.
c.log.Info("Failed to generate Caddyfile configuration.", "err", err)
}
if err = c.generateJSONConfig(containers); err != nil { if err = c.generateJSONConfig(containers); err != nil {
c.log.Info("Failed to generate Caddy JSON configuration.", "err", err) c.log.Error("Failed to generate Caddy JSON configuration to disk.", "err", err)
} }
c.log.Info("Updated Caddy configuration.", "dir", c.configDir)
case <-ctx.Done(): case <-ctx.Done():
return nil return nil
} }
} }
} }
// filterAvailableContainers filters out containers from this machine that are likely unavailable. The availability // filterHealthyContainers filters out containers that are not healthy.
// is determined by the cluster membership state of the machine that the container is running on. // TODO: Filters out containers from this machine that are likely unavailable. The availability can be determined
// TODO: implement machine membership check using Corrossion Admin client. // by the cluster membership state of the machine that the container is running on. Implement machine membership
func (c *Controller) filterAvailableContainers( // check using Corrossion Admin client.
containerRecords []store.ContainerRecord, func filterHealthyContainers(containers []store.ContainerRecord) []store.ContainerRecord {
) ([]api.ServiceContainer, error) { healthy := make([]store.ContainerRecord, 0, len(containers))
containers := make([]api.ServiceContainer, len(containerRecords)) for _, cr := range containers {
for i, cr := range containerRecords { if cr.Container.Healthy() {
containers[i] = cr.Container healthy = append(healthy, cr)
} }
return containers, nil }
return healthy
} }
func (c *Controller) generateCaddyfile(containers []api.ServiceContainer) error { func (c *Controller) generateAndLoadCaddyfile(ctx context.Context, containers []store.ContainerRecord) {
caddyfile, err := GenerateCaddyfile(containers, c.verifyResponse) // Check if Caddy is available before attempting to generate and load config.
caddyAvailable := c.client.IsAvailable(ctx)
caddyfile, err := c.generator.Generate(ctx, containers, caddyAvailable)
if err != nil { if err != nil {
return fmt.Errorf("generate Caddyfile: %w", err) c.log.Error("Failed to generate Caddyfile configuration.", "err", err)
} return
caddyfilePath := filepath.Join(c.configDir, "Caddyfile")
if err = os.WriteFile(caddyfilePath, []byte(caddyfile), 0o640); err != nil {
return fmt.Errorf("write Caddyfile to file '%s': %w", caddyfilePath, err)
}
if err = fs.Chown(caddyfilePath, "", CaddyGroup); err != nil {
return fmt.Errorf("change owner of Caddyfile '%s': %w", caddyfilePath, err)
} }
if !caddyAvailable {
// Caddy is not running so the generated Caddyfile should not include user-defined configs thus must be valid.
// It's safe to write the config to disk so that when Caddy is deployed on this machine, it can pick it up.
if err = c.writeCaddyfile(caddyfile); err != nil {
c.log.Error("Failed to write Caddyfile to disk.", "err", err)
return
}
c.log.Debug("Caddy is not running on this machine, skipping configuration load.", "path", c.caddyfilePath)
return
}
// Caddy is available, try to load the config which may fail if the config is invalid. Generally, a config can
// pass the adaptation/validation step but still fail to load, for example, if it references resources that are
// not available.
if err = c.client.Load(ctx, caddyfile); err != nil {
c.log.Error("Failed to load new Caddy configuration into local Caddy instance.",
"err", err, "path", c.caddyfilePath)
// Don't write invalid config to disk.
return
}
// Config loaded successfully, now write it to disk.
if err = c.writeCaddyfile(caddyfile); err != nil {
c.log.Error("Failed to write Caddyfile to disk after successful load.", "err", err)
// Config is already loaded in Caddy, so this is not critical.
}
c.log.Info("New Caddy configuration loaded into local Caddy instance.", "path", c.caddyfilePath)
}
// writeCaddyfile writes the Caddyfile content to disk with proper permissions.
func (c *Controller) writeCaddyfile(caddyfile string) error {
if err := os.WriteFile(c.caddyfilePath, []byte(caddyfile), 0o640); err != nil {
return fmt.Errorf("write Caddyfile to file '%s': %w", c.caddyfilePath, err)
}
if err := fs.Chown(c.caddyfilePath, "", CaddyGroup); err != nil {
return fmt.Errorf("change owner of Caddyfile '%s': %w", c.caddyfilePath, err)
}
return nil return nil
} }
func (c *Controller) generateJSONConfig(containers []api.ServiceContainer) error { func (c *Controller) generateJSONConfig(containers []store.ContainerRecord) error {
config, err := GenerateJSONConfig(containers, c.verifyResponse) serviceContainers := make([]api.ServiceContainer, len(containers))
for i, cr := range containers {
serviceContainers[i] = cr.Container
}
config, err := GenerateJSONConfig(serviceContainers, c.machineID)
if err != nil { if err != nil {
return err return err
} }
@@ -136,7 +173,7 @@ func (c *Controller) generateJSONConfig(containers []api.ServiceContainer) error
if err != nil { if err != nil {
return fmt.Errorf("marshal Caddy configuration: %w", err) return fmt.Errorf("marshal Caddy configuration: %w", err)
} }
configPath := filepath.Join(c.configDir, "caddy.json") configPath := filepath.Join(filepath.Dir(c.caddyfilePath), "caddy.json")
if err = os.WriteFile(configPath, configBytes, 0o640); err != nil { if err = os.WriteFile(configPath, configBytes, 0o640); err != nil {
return fmt.Errorf("write Caddy configuration to file '%s': %w", configPath, err) return fmt.Errorf("write Caddy configuration to file '%s': %w", configPath, err)
+1 -42
View File
@@ -4,9 +4,7 @@ import (
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"log/slog"
"maps" "maps"
"net"
"net/http" "net/http"
"slices" "slices"
"strconv" "strconv"
@@ -20,46 +18,7 @@ import (
) )
func GenerateJSONConfig(containers []api.ServiceContainer, verifyResponse string) (*caddy.Config, error) { func GenerateJSONConfig(containers []api.ServiceContainer, verifyResponse string) (*caddy.Config, error) {
// Maps hostnames to lists of upstreams (container IP:port pairs). httpHostUpstreams, httpsHostUpstreams := httpUpstreamsFromPorts(containers)
httpHostUpstreams := make(map[string][]string)
httpsHostUpstreams := make(map[string][]string)
for _, ctr := range containers {
if !ctr.Healthy() {
continue
}
ip := ctr.UncloudNetworkIP()
if !ip.IsValid() {
// Container is not connected to the uncloud Docker network (could be host network).
continue
}
log := slog.With("container", ctr.ID)
ports, err := ctr.ServicePorts()
if err != nil {
log.Error("Failed to parse service ports for container.", "err", err)
continue
}
for _, port := range ports {
if port.Mode != api.PortModeIngress {
continue
}
switch port.Protocol {
case api.ProtocolHTTP:
upstream := net.JoinHostPort(ip.String(), strconv.Itoa(int(port.ContainerPort)))
httpHostUpstreams[port.Hostname] = append(httpHostUpstreams[port.Hostname], upstream)
case api.ProtocolHTTPS:
upstream := net.JoinHostPort(ip.String(), strconv.Itoa(int(port.ContainerPort)))
httpsHostUpstreams[port.Hostname] = append(httpsHostUpstreams[port.Hostname], upstream)
default:
// TODO: implement L4 ingress routing for TCP and UDP.
log.Error("Unsupported protocol for ingress port.", "port", port)
continue
}
}
}
var warnings []caddyconfig.Warning var warnings []caddyconfig.Warning
servers := make(map[string]*caddyhttp.Server) servers := make(map[string]*caddyhttp.Server)
@@ -4,7 +4,6 @@ import (
"strings" "strings"
"testing" "testing"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/network" "github.com/docker/docker/api/types/network"
"github.com/psviderski/uncloud/internal/machine/docker" "github.com/psviderski/uncloud/internal/machine/docker"
@@ -13,7 +12,7 @@ import (
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
func TestGenerateConfig(t *testing.T) { func TestGenerateJSONConfig(t *testing.T) {
configWithoutServices := `{ configWithoutServices := `{
"servers": { "servers": {
"http": { "http": {
@@ -312,68 +311,6 @@ func TestGenerateConfig(t *testing.T) {
want: configWithoutServices, want: configWithoutServices,
wantErr: false, wantErr: false,
}, },
{
name: "restarting container ignored",
containers: []api.ServiceContainer{
newRestartingContainer("10.210.0.2", "app.example.com:8080/http"),
},
want: configWithoutServices,
wantErr: false,
},
{
name: "stopped container ignored",
containers: []api.ServiceContainer{
newStoppedContainer("10.210.0.2", "app.example.com:8080/http"),
},
want: configWithoutServices,
wantErr: false,
},
{
name: "mix of running, restarting, and stopped containers",
containers: []api.ServiceContainer{
newContainer("10.210.0.2", "app.example.com:8080/http"),
newRestartingContainer("10.210.0.3", "app.example.com:8080/http"),
newStoppedContainer("10.210.0.4", "app.example.com:8080/http"),
},
want: `{
"servers": {
"http": {
"listen": [":80"],
"routes": [
{
"match": [{"host": ["app.example.com"]}],
"handle": [{
"handler": "reverse_proxy",
"health_checks": {
"passive": {
"fail_duration": 30000000000
}
},
"load_balancing": {
"retries": 3
},
"upstreams": [{"dial": "10.210.0.2:8080"}]
}]
},
{
"match": [{"path": ["/.uncloud-verify"]}],
"handle": [{
"body": "verification-response-body",
"handler": "static_response",
"status_code": 200
}]
}
],
"logs": {}
},
"https": {
"listen": [":443"],
"logs": {}
}
}
}`,
wantErr: false,
},
} }
for _, tt := range tests { for _, tt := range tests {
@@ -396,13 +333,13 @@ func TestGenerateConfig(t *testing.T) {
func newContainer(ip string, ports ...string) api.ServiceContainer { func newContainer(ip string, ports ...string) api.ServiceContainer {
portsLabel := strings.Join(ports, ",") portsLabel := strings.Join(ports, ",")
return api.ServiceContainer{Container: api.Container{ContainerJSON: types.ContainerJSON{ return api.ServiceContainer{Container: api.Container{InspectResponse: container.InspectResponse{
ContainerJSONBase: &types.ContainerJSONBase{ ContainerJSONBase: &container.ContainerJSONBase{
State: &types.ContainerState{ State: &container.State{
Running: true, Running: true,
}, },
}, },
NetworkSettings: &types.NetworkSettings{ NetworkSettings: &container.NetworkSettings{
Networks: map[string]*network.EndpointSettings{ Networks: map[string]*network.EndpointSettings{
docker.NetworkName: { docker.NetworkName: {
IPAddress: ip, IPAddress: ip,
@@ -419,13 +356,13 @@ func newContainer(ip string, ports ...string) api.ServiceContainer {
func newContainerWithoutNetwork(ports ...string) api.ServiceContainer { func newContainerWithoutNetwork(ports ...string) api.ServiceContainer {
portsLabel := strings.Join(ports, ",") portsLabel := strings.Join(ports, ",")
return api.ServiceContainer{Container: api.Container{ContainerJSON: types.ContainerJSON{ return api.ServiceContainer{Container: api.Container{InspectResponse: container.InspectResponse{
ContainerJSONBase: &types.ContainerJSONBase{ ContainerJSONBase: &container.ContainerJSONBase{
State: &types.ContainerState{ State: &container.State{
Running: true, Running: true,
}, },
}, },
NetworkSettings: &types.NetworkSettings{ NetworkSettings: &container.NetworkSettings{
Networks: map[string]*network.EndpointSettings{ Networks: map[string]*network.EndpointSettings{
"other-network": { "other-network": {
IPAddress: "172.17.0.2", IPAddress: "172.17.0.2",
@@ -439,15 +376,3 @@ func newContainerWithoutNetwork(ports ...string) api.ServiceContainer {
}, },
}}} }}}
} }
func newRestartingContainer(ip string, ports ...string) api.ServiceContainer {
ctr := newContainer(ip, ports...)
ctr.Container.State.Restarting = true
return ctr
}
func newStoppedContainer(ip string, ports ...string) api.ServiceContainer {
ctr := newContainer(ip, ports...)
ctr.Container.State.Running = false
return ctr
}
@@ -0,0 +1,95 @@
// Code generated by mockery; DO NOT EDIT.
// github.com/vektra/mockery
// template: testify
package caddyconfig
import (
"context"
mock "github.com/stretchr/testify/mock"
)
// NewMockCaddyfileValidator creates a new instance of MockCaddyfileValidator. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
// The first argument is typically a *testing.T value.
func NewMockCaddyfileValidator(t interface {
mock.TestingT
Cleanup(func())
}) *MockCaddyfileValidator {
mock := &MockCaddyfileValidator{}
mock.Mock.Test(t)
t.Cleanup(func() { mock.AssertExpectations(t) })
return mock
}
// MockCaddyfileValidator is an autogenerated mock type for the CaddyfileValidator type
type MockCaddyfileValidator struct {
mock.Mock
}
type MockCaddyfileValidator_Expecter struct {
mock *mock.Mock
}
func (_m *MockCaddyfileValidator) EXPECT() *MockCaddyfileValidator_Expecter {
return &MockCaddyfileValidator_Expecter{mock: &_m.Mock}
}
// Validate provides a mock function for the type MockCaddyfileValidator
func (_mock *MockCaddyfileValidator) Validate(ctx context.Context, caddyfile string) error {
ret := _mock.Called(ctx, caddyfile)
if len(ret) == 0 {
panic("no return value specified for Validate")
}
var r0 error
if returnFunc, ok := ret.Get(0).(func(context.Context, string) error); ok {
r0 = returnFunc(ctx, caddyfile)
} else {
r0 = ret.Error(0)
}
return r0
}
// MockCaddyfileValidator_Validate_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'Validate'
type MockCaddyfileValidator_Validate_Call struct {
*mock.Call
}
// Validate is a helper method to define mock.On call
// - ctx context.Context
// - caddyfile string
func (_e *MockCaddyfileValidator_Expecter) Validate(ctx interface{}, caddyfile interface{}) *MockCaddyfileValidator_Validate_Call {
return &MockCaddyfileValidator_Validate_Call{Call: _e.mock.On("Validate", ctx, caddyfile)}
}
func (_c *MockCaddyfileValidator_Validate_Call) Run(run func(ctx context.Context, caddyfile string)) *MockCaddyfileValidator_Validate_Call {
_c.Call.Run(func(args mock.Arguments) {
var arg0 context.Context
if args[0] != nil {
arg0 = args[0].(context.Context)
}
var arg1 string
if args[1] != nil {
arg1 = args[1].(string)
}
run(
arg0,
arg1,
)
})
return _c
}
func (_c *MockCaddyfileValidator_Validate_Call) Return(err error) *MockCaddyfileValidator_Validate_Call {
_c.Call.Return(err)
return _c
}
func (_c *MockCaddyfileValidator_Validate_Call) RunAndReturn(run func(ctx context.Context, caddyfile string) error) *MockCaddyfileValidator_Validate_Call {
_c.Call.Return(run)
return _c
}
+39
View File
@@ -0,0 +1,39 @@
package caddyconfig
import (
"context"
"os"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb"
"google.golang.org/protobuf/types/known/timestamppb"
"github.com/psviderski/uncloud/internal/machine/api/pb"
)
// Server implements the gRPC Caddy service.
type Server struct {
pb.UnimplementedCaddyServer
service *Service
}
func NewServer(service *Service) *Server {
return &Server{service: service}
}
// GetConfig retrieves the current Caddy configuration from the machine.
func (s *Server) GetConfig(ctx context.Context, _ *emptypb.Empty) (*pb.GetCaddyConfigResponse, error) {
caddyfile, modifiedAt, err := s.service.Caddyfile()
if err != nil {
if os.IsNotExist(err) {
return nil, status.Error(codes.NotFound, err.Error())
}
return nil, status.Error(codes.Internal, err.Error())
}
return &pb.GetCaddyConfigResponse{
Caddyfile: caddyfile,
ModifiedAt: timestamppb.New(modifiedAt),
}, nil
}
+35
View File
@@ -0,0 +1,35 @@
package caddyconfig
import (
"fmt"
"os"
"path/filepath"
"time"
)
// Service provides methods to interact with the Caddy configuration on the machine.
type Service struct {
configDir string
}
// NewService creates a new Service instance with the specified Caddy configuration directory.
func NewService(configDir string) *Service {
return &Service{configDir: configDir}
}
// Caddyfile retrieves the current Caddy configuration (Caddyfile) from the machine's config directory.
func (s *Service) Caddyfile() (string, time.Time, error) {
path := filepath.Join(s.configDir, "Caddyfile")
content, err := os.ReadFile(path)
if err != nil {
return "", time.Time{}, fmt.Errorf("read Caddyfile from file '%s': %w", path, err)
}
// Get the file modification time.
fileInfo, err := os.Stat(path)
if err != nil {
return "", time.Time{}, fmt.Errorf("get Caddyfile file info '%s': %w", path, err)
}
return string(content), fileInfo.ModTime(), nil
}
+77
View File
@@ -0,0 +1,77 @@
package caddyconfig
import (
"fmt"
"net"
"strconv"
"strings"
)
// templateContext holds the data available to Caddyfile templates.
type templateContext struct {
// Name is the current service name.
Name string
// Upstreams maps service names to their container IPs.
Upstreams map[string][]string
}
// upstreamsTemplateFn returns a template function that generates a space separated string of upstreams for the service.
// It optionally accepts a service name and a port number: {{upstreams [service-name] [port]}}.
func upstreamsTemplateFn(tmplCtx templateContext) func(args ...any) (string, error) {
return func(args ...any) (string, error) {
var serviceName string
var port int
// Parse arguments.
switch len(args) {
case 0:
// Current service, default port.
serviceName = tmplCtx.Name
case 1:
// Either port (int) for current service or service name (string).
switch arg := args[0].(type) {
case int:
serviceName = tmplCtx.Name
port = arg
case string:
serviceName = arg
port = 0
default:
return "", fmt.Errorf("upstreams function: invalid argument type: %T", arg)
}
case 2:
// Service name and port.
name, ok := args[0].(string)
if !ok {
return "", fmt.Errorf("upstreams function: first argument must be service name (string)")
}
serviceName = name
p, ok := args[1].(int)
if !ok {
return "", fmt.Errorf("upstreams function: second argument must be port (int)")
}
port = p
default:
return "", fmt.Errorf("upstreams function: too many arguments; expected 0-2, got %d", len(args))
}
ips, ok := tmplCtx.Upstreams[serviceName]
if !ok || len(ips) == 0 {
// No upstreams available.
return "", nil
}
// Build the space separated upstreams string.
var upstreams []string
for _, ip := range ips {
if port > 0 {
upstreams = append(upstreams, net.JoinHostPort(ip, strconv.Itoa(port)))
} else {
upstreams = append(upstreams, ip)
}
}
return strings.Join(upstreams, " "), nil
}
}
+38 -2
View File
@@ -21,6 +21,7 @@ import (
"github.com/psviderski/uncloud/internal/machine/firewall" "github.com/psviderski/uncloud/internal/machine/firewall"
"github.com/psviderski/uncloud/internal/machine/network" "github.com/psviderski/uncloud/internal/machine/network"
"github.com/psviderski/uncloud/internal/machine/store" "github.com/psviderski/uncloud/internal/machine/store"
"github.com/psviderski/unregistry"
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
"google.golang.org/grpc" "google.golang.org/grpc"
) )
@@ -45,6 +46,8 @@ type clusterController struct {
// dnsServer is the embedded internal DNS server for the cluster listening on the machine IP. // dnsServer is the embedded internal DNS server for the cluster listening on the machine IP.
dnsServer *dns.Server dnsServer *dns.Server
dnsResolver *dns.ClusterResolver dnsResolver *dns.ClusterResolver
// unregistry is the embedded container registry that uses the local Docker (containerd) image store as its backend.
unregistry *unregistry.Registry
// stopped is a channel that is closed when the controller is stopped. // stopped is a channel that is closed when the controller is stopped.
stopped chan struct{} stopped chan struct{}
@@ -60,6 +63,7 @@ func newClusterController(
caddyfileCtrl *caddyconfig.Controller, caddyfileCtrl *caddyconfig.Controller,
dnsServer *dns.Server, dnsServer *dns.Server,
dnsResolver *dns.ClusterResolver, dnsResolver *dns.ClusterResolver,
unregistry *unregistry.Registry,
) (*clusterController, error) { ) (*clusterController, error) {
slog.Info("Starting WireGuard network.") slog.Info("Starting WireGuard network.")
wgnet, err := network.NewWireGuardNetwork() wgnet, err := network.NewWireGuardNetwork()
@@ -80,6 +84,7 @@ func newClusterController(
caddyconfigCtrl: caddyfileCtrl, caddyconfigCtrl: caddyfileCtrl,
dnsServer: dnsServer, dnsServer: dnsServer,
dnsResolver: dnsResolver, dnsResolver: dnsResolver,
unregistry: unregistry,
stopped: make(chan struct{}), stopped: make(chan struct{}),
}, nil }, nil
} }
@@ -87,7 +92,7 @@ func newClusterController(
func (cc *clusterController) Run(ctx context.Context) error { func (cc *clusterController) Run(ctx context.Context) error {
defer close(cc.stopped) defer close(cc.stopped)
if err := firewall.ConfigureIptablesChains(); err != nil { if err := firewall.ConfigureIptablesChains(network.MachineIP(cc.state.Network.Subnet)); err != nil {
return fmt.Errorf("configure iptables chains: %w", err) return fmt.Errorf("configure iptables chains: %w", err)
} }
@@ -164,7 +169,7 @@ func (cc *clusterController) Run(ctx context.Context) error {
return nil return nil
}) })
// Watch for endpoint changes and update the machine state accordingly. // Watch for WireGuard peer endpoint changes and update the machine state accordingly.
errGroup.Go(func() error { errGroup.Go(func() error {
for { for {
select { select {
@@ -210,6 +215,16 @@ func (cc *clusterController) Run(ctx context.Context) error {
return nil return nil
}) })
if cc.unregistry != nil {
errGroup.Go(func() error {
slog.Info("Starting unregistry server.")
if err := cc.unregistry.ListenAndServe(); err != nil {
return fmt.Errorf("unregistry server failed: %w", err)
}
return nil
})
}
// Wait for the context to be done and stop the network API server. // Wait for the context to be done and stop the network API server.
<-ctx.Done() <-ctx.Done()
slog.Info("Stopping network API server.") slog.Info("Stopping network API server.")
@@ -217,6 +232,19 @@ func (cc *clusterController) Run(ctx context.Context) error {
cc.server.GracefulStop() cc.server.GracefulStop()
slog.Info("Network API server stopped.") slog.Info("Network API server stopped.")
// Stop the unregistry server with a timeout if it was started.
if cc.unregistry != nil {
unregTimeout := 30 * time.Second
slog.Info("Stopping unregistry server.", "timeout", unregTimeout)
unregCtx, cancel := context.WithTimeout(context.Background(), unregTimeout)
defer cancel()
if err = cc.unregistry.Shutdown(unregCtx); err != nil {
return fmt.Errorf("unregistry server forced to shutdown: %w", err)
}
slog.Info("Unregistry server stopped.")
}
// Wait for all controllers to finish. // Wait for all controllers to finish.
err = errGroup.Wait() err = errGroup.Wait()
@@ -330,6 +358,14 @@ func (cc *clusterController) handleMachineChanges(ctx context.Context) error {
slog.Error("Failed to list machines.", "err", err) slog.Error("Failed to list machines.", "err", err)
continue continue
} }
// Skip reconfiguration if the machines list is empty. This can happen when joining the cluster.
// Corrosion can notifies about table changes before the data is fully replicated.
// Reconfiguring with an empty list would remove all peers and lock this machine out of the cluster.
// See https://github.com/psviderski/uncloud/issues/155.
if len(machines) == 0 {
slog.Debug("Skipping peer reconfiguration: machines list in store is empty.")
continue
}
if err = cc.configurePeers(machines); err != nil { if err = cc.configurePeers(machines); err != nil {
slog.Error("Failed to configure peers.", "err", err) slog.Error("Failed to configure peers.", "err", err)
} }
+1 -1
View File
@@ -44,7 +44,7 @@ func (c *Cluster) ReserveDomain(ctx context.Context, req *pb.ReserveDomainReques
dnsClient := dns.NewClient() dnsClient := dns.NewClient()
name, token, err := dnsClient.ReserveDomain(req.Endpoint) name, token, err := dnsClient.ReserveDomain(req.Endpoint)
if err != nil { if err != nil {
return nil, status.Errorf(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
} }
domain := uncloudDNSDomain{ domain := uncloudDNSDomain{
+2
View File
@@ -3,4 +3,6 @@ package constants
const ( const (
// MachineAPIPort is the port for the Machine API service on the management WireGuard network. // MachineAPIPort is the port for the Machine API service on the management WireGuard network.
MachineAPIPort = 51000 MachineAPIPort = 51000
// UnregistryPort is the port for the embedded container registry listening on the machine IP.
UnregistryPort = 5000
) )
+3 -2
View File
@@ -8,6 +8,7 @@ import (
"path/filepath" "path/filepath"
"time" "time"
"github.com/containerd/errdefs"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/image" "github.com/docker/docker/api/types/image"
"github.com/docker/docker/api/types/mount" "github.com/docker/docker/api/types/mount"
@@ -30,7 +31,7 @@ type DockerService struct {
func (s *DockerService) Start(ctx context.Context) error { func (s *DockerService) Start(ctx context.Context) error {
_, err := s.Client.ContainerInspect(ctx, s.Name) _, err := s.Client.ContainerInspect(ctx, s.Name)
if err != nil { if err != nil {
if !client.IsErrNotFound(err) { if !errdefs.IsNotFound(err) {
return fmt.Errorf("inspect container %q: %w", s.Name, err) return fmt.Errorf("inspect container %q: %w", s.Name, err)
} }
if err = s.startNewContainer(ctx); err != nil { if err = s.startNewContainer(ctx); err != nil {
@@ -114,7 +115,7 @@ func (s *DockerService) hostConfig() *container.HostConfig {
func (s *DockerService) startNewContainer(ctx context.Context) error { func (s *DockerService) startNewContainer(ctx context.Context) error {
_, err := s.Client.ContainerCreate(ctx, s.containerConfig(), s.hostConfig(), nil, nil, s.Name) _, err := s.Client.ContainerCreate(ctx, s.containerConfig(), s.hostConfig(), nil, nil, s.Name)
if err != nil { if err != nil {
if !client.IsErrNotFound(err) { if !errdefs.IsNotFound(err) {
return fmt.Errorf("create container: %w", err) return fmt.Errorf("create container: %w", err)
} }
+6 -1
View File
@@ -91,6 +91,11 @@ func (r *ClusterResolver) updateServiceIPs(containers []store.ContainerRecord) {
newServiceIPs[ctr.ServiceName()] = append(newServiceIPs[ctr.ServiceName()], ip) newServiceIPs[ctr.ServiceName()] = append(newServiceIPs[ctr.ServiceName()], ip)
// Also add the service ID as a valid lookup. // Also add the service ID as a valid lookup.
newServiceIPs[ctr.ServiceID()] = append(newServiceIPs[ctr.ServiceID()], ip) newServiceIPs[ctr.ServiceID()] = append(newServiceIPs[ctr.ServiceID()], ip)
// Add <machine-id>.m.<service-name> as a lookup
serviceNameWithMachineID := record.MachineID + ".m." + ctr.ServiceName()
newServiceIPs[serviceNameWithMachineID] = append(newServiceIPs[serviceNameWithMachineID], ip)
containersCount++ containersCount++
} }
@@ -99,7 +104,7 @@ func (r *ClusterResolver) updateServiceIPs(containers []store.ContainerRecord) {
r.serviceIPs = newServiceIPs r.serviceIPs = newServiceIPs
r.mu.Unlock() r.mu.Unlock()
r.log.Debug("DNS records updated.", "services", len(newServiceIPs)/2, "containers", containersCount) r.log.Debug("DNS records updated.", "services", len(newServiceIPs)/3, "containers", containersCount)
} }
// Resolve returns IP addresses of the service containers. // Resolve returns IP addresses of the service containers.
+34 -3
View File
@@ -8,6 +8,7 @@ import (
"math/rand/v2" "math/rand/v2"
"net" "net"
"net/netip" "net/netip"
"slices"
"strconv" "strconv"
"strings" "strings"
"sync" "sync"
@@ -40,6 +41,7 @@ type Resolver interface {
// to upstream DNS servers. // to upstream DNS servers.
type Server struct { type Server struct {
listenAddr netip.Addr listenAddr netip.Addr
localSubnet netip.Prefix
resolver Resolver resolver Resolver
upstreamServers []netip.AddrPort upstreamServers []netip.AddrPort
@@ -53,7 +55,7 @@ type Server struct {
// NewServer creates a new DNS server with the given configuration. // NewServer creates a new DNS server with the given configuration.
// If upstreams is nil, nameservers from /etc/resolv.conf will be used. An empty upstreams list means to only resolve // If upstreams is nil, nameservers from /etc/resolv.conf will be used. An empty upstreams list means to only resolve
// internal DNS queries and not forward any external queries. // internal DNS queries and not forward any external queries.
func NewServer(listenAddr netip.Addr, resolver Resolver, upstreams []netip.AddrPort) (*Server, error) { func NewServer(listenAddr netip.Addr, localSubnet netip.Prefix, resolver Resolver, upstreams []netip.AddrPort) (*Server, error) {
if !listenAddr.IsValid() { if !listenAddr.IsValid() {
return nil, fmt.Errorf("invalid listen address: %s", listenAddr) return nil, fmt.Errorf("invalid listen address: %s", listenAddr)
} }
@@ -87,6 +89,7 @@ func NewServer(listenAddr netip.Addr, resolver Resolver, upstreams []netip.AddrP
return &Server{ return &Server{
listenAddr: listenAddr, listenAddr: listenAddr,
localSubnet: localSubnet,
resolver: resolver, resolver: resolver,
upstreamServers: upstreams, upstreamServers: upstreams,
forwardSemaphore: make(chan struct{}, maxConcurrentForwards), forwardSemaphore: make(chan struct{}, maxConcurrentForwards),
@@ -287,7 +290,7 @@ func (s *Server) forwardRequest(req *dns.Msg, proto string) (*dns.Msg, error) {
// handleAQuery processes an A query for the internal domain and returns A records for the requested name. // handleAQuery processes an A query for the internal domain and returns A records for the requested name.
// The internal domain suffix is already stripped from the name. An empty list is returned if no records are found. // The internal domain suffix is already stripped from the name. An empty list is returned if no records are found.
func (s *Server) handleAQuery(name string) []dns.RR { func (s *Server) handleAQuery(name string) []dns.RR {
serviceName := trimInternalDomain(name) serviceName, mode := extractModeFromDomain(trimInternalDomain(name))
ips := s.resolver.Resolve(serviceName) ips := s.resolver.Resolve(serviceName)
if len(ips) == 0 { if len(ips) == 0 {
s.log.Debug("Failed to resolve service name.", "service", serviceName) s.log.Debug("Failed to resolve service name.", "service", serviceName)
@@ -296,10 +299,28 @@ func (s *Server) handleAQuery(name string) []dns.RR {
s.log.Debug("Resolved service name.", "service", serviceName, "ips", ips) s.log.Debug("Resolved service name.", "service", serviceName, "ips", ips)
if len(ips) > 1 { if len(ips) > 1 {
// TODO: sort by proximity to the requesting container/machine. For now, just shuffle the IPs. // Shuffle the IPs to approximate round-robin.
// We want to do this as a baseline for "nearest" mode, as well.
rand.Shuffle(len(ips), func(i, j int) { rand.Shuffle(len(ips), func(i, j int) {
ips[i], ips[j] = ips[j], ips[i] ips[i], ips[j] = ips[j], ips[i]
}) })
// Default (mode == "") currently behaves the same as round-robin,
// and nothing additional to do for round-robin (mode == "rr").
if mode == "nearest" {
// Sort IPs on local subnet to the top.
slices.SortFunc(ips, func(a, b netip.Addr) int {
aIsLocal := s.localSubnet.Contains(a)
bIsLocal := s.localSubnet.Contains(b)
if aIsLocal && !bIsLocal {
return -1
} else if bIsLocal && !aIsLocal {
return 1
}
return 0
})
}
} }
// Create A records for each IP. // Create A records for each IP.
@@ -345,3 +366,13 @@ func trimInternalDomain(name string) string {
return strings.TrimSuffix(name, "."+InternalDomain) return strings.TrimSuffix(name, "."+InternalDomain)
} }
func extractModeFromDomain(name string) (string, string) {
modes := []string{"nearest", "rr"}
for _, mode := range modes {
if cut, found := strings.CutPrefix(name, mode+"."); found {
return cut, mode
}
}
return name, ""
}
+29 -34
View File
@@ -8,7 +8,6 @@ import (
"io" "io"
"github.com/distribution/reference" "github.com/distribution/reference"
"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container" "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/network" "github.com/docker/docker/api/types/network"
"github.com/docker/docker/api/types/volume" "github.com/docker/docker/api/types/volume"
@@ -16,6 +15,7 @@ import (
"github.com/docker/docker/pkg/jsonmessage" "github.com/docker/docker/pkg/jsonmessage"
regtypes "github.com/google/go-containerregistry/pkg/v1/types" regtypes "github.com/google/go-containerregistry/pkg/v1/types"
ocispec "github.com/opencontainers/image-spec/specs-go/v1" ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/psviderski/uncloud/internal/docker"
"github.com/psviderski/uncloud/internal/machine/api/pb" "github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/pkg/api" "github.com/psviderski/uncloud/pkg/api"
"google.golang.org/grpc" "google.golang.org/grpc"
@@ -24,16 +24,18 @@ import (
) )
// Client is a gRPC client for the Docker service that provides a similar interface to the Docker HTTP client. // Client is a gRPC client for the Docker service that provides a similar interface to the Docker HTTP client.
// TODO: it doesn't seem there is much value in having this intermediate Docker client.
// Consider merging it into the main pkg/client.
type Client struct { type Client struct {
conn *grpc.ClientConn conn *grpc.ClientConn
grpcClient pb.DockerClient GRPCClient pb.DockerClient
} }
// NewClient creates a new Docker gRPC client with the provided gRPC connection. // NewClient creates a new Docker gRPC client with the provided gRPC connection.
func NewClient(conn *grpc.ClientConn) *Client { func NewClient(conn *grpc.ClientConn) *Client {
return &Client{ return &Client{
conn: conn, conn: conn,
grpcClient: pb.NewDockerClient(conn), GRPCClient: pb.NewDockerClient(conn),
} }
} }
@@ -70,7 +72,7 @@ func (c *Client) CreateContainer(
return resp, fmt.Errorf("marshal platform: %w", err) return resp, fmt.Errorf("marshal platform: %w", err)
} }
grpcResp, err := c.grpcClient.CreateContainer(ctx, &pb.CreateContainerRequest{ grpcResp, err := c.GRPCClient.CreateContainer(ctx, &pb.CreateContainerRequest{
Config: configBytes, Config: configBytes,
HostConfig: hostConfigBytes, HostConfig: hostConfigBytes,
NetworkConfig: networkingConfigBytes, NetworkConfig: networkingConfigBytes,
@@ -91,10 +93,10 @@ func (c *Client) CreateContainer(
} }
// InspectContainer returns the container information for the given container ID. // InspectContainer returns the container information for the given container ID.
func (c *Client) InspectContainer(ctx context.Context, id string) (types.ContainerJSON, error) { func (c *Client) InspectContainer(ctx context.Context, id string) (container.InspectResponse, error) {
var resp types.ContainerJSON var resp container.InspectResponse
grpcResp, err := c.grpcClient.InspectContainer(ctx, &pb.InspectContainerRequest{Id: id}) grpcResp, err := c.GRPCClient.InspectContainer(ctx, &pb.InspectContainerRequest{Id: id})
if err != nil { if err != nil {
if status.Convert(err).Code() == codes.NotFound { if status.Convert(err).Code() == codes.NotFound {
return resp, errdefs.NotFound(err) return resp, errdefs.NotFound(err)
@@ -115,7 +117,7 @@ func (c *Client) StartContainer(ctx context.Context, id string, opts container.S
return fmt.Errorf("marshal options: %w", err) return fmt.Errorf("marshal options: %w", err)
} }
_, err = c.grpcClient.StartContainer(ctx, &pb.StartContainerRequest{ _, err = c.GRPCClient.StartContainer(ctx, &pb.StartContainerRequest{
Id: id, Id: id,
Options: optsBytes, Options: optsBytes,
}) })
@@ -134,7 +136,7 @@ func (c *Client) StopContainer(ctx context.Context, id string, opts container.St
return fmt.Errorf("marshal options: %w", err) return fmt.Errorf("marshal options: %w", err)
} }
_, err = c.grpcClient.StopContainer(ctx, &pb.StopContainerRequest{ _, err = c.GRPCClient.StopContainer(ctx, &pb.StopContainerRequest{
Id: id, Id: id,
Options: optsBytes, Options: optsBytes,
}) })
@@ -148,7 +150,7 @@ func (c *Client) StopContainer(ctx context.Context, id string, opts container.St
type MachineContainers struct { type MachineContainers struct {
Metadata *pb.Metadata Metadata *pb.Metadata
Containers []types.ContainerJSON Containers []container.InspectResponse
} }
func (c *Client) ListContainers(ctx context.Context, opts container.ListOptions) ([]MachineContainers, error) { func (c *Client) ListContainers(ctx context.Context, opts container.ListOptions) ([]MachineContainers, error) {
@@ -157,7 +159,7 @@ func (c *Client) ListContainers(ctx context.Context, opts container.ListOptions)
return nil, fmt.Errorf("marshal options: %w", err) return nil, fmt.Errorf("marshal options: %w", err)
} }
resp, err := c.grpcClient.ListContainers(ctx, &pb.ListContainersRequest{Options: optsBytes}) resp, err := c.GRPCClient.ListContainers(ctx, &pb.ListContainersRequest{Options: optsBytes})
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -184,7 +186,7 @@ func (c *Client) RemoveContainer(ctx context.Context, id string, opts container.
return fmt.Errorf("marshal options: %w", err) return fmt.Errorf("marshal options: %w", err)
} }
_, err = c.grpcClient.RemoveContainer(ctx, &pb.RemoveContainerRequest{ _, err = c.GRPCClient.RemoveContainer(ctx, &pb.RemoveContainerRequest{
Id: id, Id: id,
Options: optsBytes, Options: optsBytes,
}) })
@@ -205,23 +207,18 @@ type PullOptions struct {
Platform string Platform string
} }
type PullImageMessage struct { func (c *Client) PullImage(ctx context.Context, image string, opts PullOptions) (<-chan docker.PullPushImageMessage, error) {
Message jsonmessage.JSONMessage
Err error
}
func (c *Client) PullImage(ctx context.Context, image string, opts PullOptions) (<-chan PullImageMessage, error) {
optsBytes, err := json.Marshal(opts) optsBytes, err := json.Marshal(opts)
if err != nil { if err != nil {
return nil, fmt.Errorf("marshal options: %w", err) return nil, fmt.Errorf("marshal options: %w", err)
} }
stream, err := c.grpcClient.PullImage(ctx, &pb.PullImageRequest{Image: image, Options: optsBytes}) stream, err := c.GRPCClient.PullImage(ctx, &pb.PullImageRequest{Image: image, Options: optsBytes})
if err != nil { if err != nil {
return nil, err return nil, err
} }
ch := make(chan PullImageMessage) ch := make(chan docker.PullPushImageMessage)
go func() { go func() {
defer close(ch) defer close(ch)
@@ -232,16 +229,16 @@ func (c *Client) PullImage(ctx context.Context, image string, opts PullOptions)
return return
} }
if err != nil { if err != nil {
ch <- PullImageMessage{Err: err} ch <- docker.PullPushImageMessage{Err: err}
return return
} }
var jm jsonmessage.JSONMessage var jm jsonmessage.JSONMessage
if err = json.Unmarshal(msg.Message, &jm); err != nil { if err = json.Unmarshal(msg.Message, &jm); err != nil {
ch <- PullImageMessage{Err: fmt.Errorf("unmarshal JSON message: %w", err)} ch <- docker.PullPushImageMessage{Err: fmt.Errorf("unmarshal JSON message: %w", err)}
return return
} }
ch <- PullImageMessage{Message: jm} ch <- docker.PullPushImageMessage{Message: jm}
} }
}() }()
@@ -250,7 +247,7 @@ func (c *Client) PullImage(ctx context.Context, image string, opts PullOptions)
// InspectImage returns the image information for the given image ID. The request may be sent to multiple machines. // InspectImage returns the image information for the given image ID. The request may be sent to multiple machines.
func (c *Client) InspectImage(ctx context.Context, id string) ([]api.MachineImage, error) { func (c *Client) InspectImage(ctx context.Context, id string) ([]api.MachineImage, error) {
resp, err := c.grpcClient.InspectImage(ctx, &pb.InspectImageRequest{Id: id}) resp, err := c.GRPCClient.InspectImage(ctx, &pb.InspectImageRequest{Id: id})
if err != nil { if err != nil {
// If the request was sent to only one machine, err is an actual error from the machine. // If the request was sent to only one machine, err is an actual error from the machine.
if status.Convert(err).Code() == codes.NotFound { if status.Convert(err).Code() == codes.NotFound {
@@ -288,7 +285,7 @@ func (c *Client) InspectImage(ctx context.Context, id string) ([]api.MachineImag
// credentials if necessary. If the response from a machine doesn't contain an error, the api.RemoteImage will either // credentials if necessary. If the response from a machine doesn't contain an error, the api.RemoteImage will either
// contain an IndexManifest or an ImageManifest. // contain an IndexManifest or an ImageManifest.
func (c *Client) InspectRemoteImage(ctx context.Context, id string) ([]api.MachineRemoteImage, error) { func (c *Client) InspectRemoteImage(ctx context.Context, id string) ([]api.MachineRemoteImage, error) {
resp, err := c.grpcClient.InspectRemoteImage(ctx, &pb.InspectRemoteImageRequest{Id: id}) resp, err := c.GRPCClient.InspectRemoteImage(ctx, &pb.InspectRemoteImageRequest{Id: id})
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -365,7 +362,7 @@ func (c *Client) CreateVolume(ctx context.Context, opts volume.CreateOptions) (v
return vol, fmt.Errorf("marshal options: %w", err) return vol, fmt.Errorf("marshal options: %w", err)
} }
resp, err := c.grpcClient.CreateVolume(ctx, &pb.CreateVolumeRequest{Options: optsBytes}) resp, err := c.GRPCClient.CreateVolume(ctx, &pb.CreateVolumeRequest{Options: optsBytes})
if err != nil { if err != nil {
return vol, err return vol, err
} }
@@ -377,20 +374,18 @@ func (c *Client) CreateVolume(ctx context.Context, opts volume.CreateOptions) (v
return vol, nil return vol, nil
} }
// MachineVolumes represents a volume list response from a machine.
type MachineVolumes struct { type MachineVolumes struct {
Metadata *pb.Metadata Metadata *pb.Metadata
Response volume.ListResponse Response volume.ListResponse
} }
// ListVolumes returns a list of all volumes matching the filter.
func (c *Client) ListVolumes(ctx context.Context, opts volume.ListOptions) ([]MachineVolumes, error) { func (c *Client) ListVolumes(ctx context.Context, opts volume.ListOptions) ([]MachineVolumes, error) {
optsBytes, err := json.Marshal(opts) optsBytes, err := json.Marshal(opts)
if err != nil { if err != nil {
return nil, fmt.Errorf("marshal options: %w", err) return nil, fmt.Errorf("marshal options: %w", err)
} }
resp, err := c.grpcClient.ListVolumes(ctx, &pb.ListVolumesRequest{Options: optsBytes}) resp, err := c.GRPCClient.ListVolumes(ctx, &pb.ListVolumesRequest{Options: optsBytes})
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -412,7 +407,7 @@ func (c *Client) ListVolumes(ctx context.Context, opts volume.ListOptions) ([]Ma
// RemoveVolume removes a volume with the given ID. // RemoveVolume removes a volume with the given ID.
func (c *Client) RemoveVolume(ctx context.Context, id string, force bool) error { func (c *Client) RemoveVolume(ctx context.Context, id string, force bool) error {
_, err := c.grpcClient.RemoveVolume(ctx, &pb.RemoveVolumeRequest{ _, err := c.GRPCClient.RemoveVolume(ctx, &pb.RemoveVolumeRequest{
Id: id, Id: id,
Force: force, Force: force,
}) })
@@ -435,7 +430,7 @@ func (c *Client) CreateServiceContainer(
if err != nil { if err != nil {
return resp, fmt.Errorf("marshal service spec: %w", err) return resp, fmt.Errorf("marshal service spec: %w", err)
} }
grpcResp, err := c.grpcClient.CreateServiceContainer(ctx, &pb.CreateServiceContainerRequest{ grpcResp, err := c.GRPCClient.CreateServiceContainer(ctx, &pb.CreateServiceContainerRequest{
ServiceId: serviceID, ServiceId: serviceID,
ServiceSpec: specBytes, ServiceSpec: specBytes,
ContainerName: containerName, ContainerName: containerName,
@@ -458,7 +453,7 @@ func (c *Client) CreateServiceContainer(
func (c *Client) InspectServiceContainer(ctx context.Context, id string) (api.ServiceContainer, error) { func (c *Client) InspectServiceContainer(ctx context.Context, id string) (api.ServiceContainer, error) {
var resp api.ServiceContainer var resp api.ServiceContainer
grpcResp, err := c.grpcClient.InspectServiceContainer(ctx, &pb.InspectContainerRequest{Id: id}) grpcResp, err := c.GRPCClient.InspectServiceContainer(ctx, &pb.InspectContainerRequest{Id: id})
if err != nil { if err != nil {
if status.Convert(err).Code() == codes.NotFound { if status.Convert(err).Code() == codes.NotFound {
return resp, errdefs.NotFound(err) return resp, errdefs.NotFound(err)
@@ -491,7 +486,7 @@ func (c *Client) ListServiceContainers(
return nil, fmt.Errorf("marshal options: %w", err) return nil, fmt.Errorf("marshal options: %w", err)
} }
resp, err := c.grpcClient.ListServiceContainers(ctx, &pb.ListServiceContainersRequest{ resp, err := c.GRPCClient.ListServiceContainers(ctx, &pb.ListServiceContainersRequest{
ServiceId: serviceNameOrID, ServiceId: serviceNameOrID,
Options: optsBytes, Options: optsBytes,
}) })
@@ -530,7 +525,7 @@ func (c *Client) RemoveServiceContainer(ctx context.Context, id string, opts con
return fmt.Errorf("marshal options: %w", err) return fmt.Errorf("marshal options: %w", err)
} }
_, err = c.grpcClient.RemoveServiceContainer(ctx, &pb.RemoveContainerRequest{ _, err = c.GRPCClient.RemoveServiceContainer(ctx, &pb.RemoveContainerRequest{
Id: id, Id: id,
Options: optsBytes, Options: optsBytes,
}) })
+303
View File
@@ -0,0 +1,303 @@
package docker
import (
"context"
"encoding/json"
"fmt"
"io"
"log/slog"
"os"
"os/signal"
"github.com/moby/term"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/pkg/api"
"golang.org/x/sync/errgroup"
"golang.org/x/sys/unix"
)
// ExecConfig contains options for executing a command in a container.
type ExecConfig struct {
// Container ID to execute the command in.
ContainerID string
// Exec configuration.
Options api.ExecOptions
}
// sendResizeRequest sends a terminal resize request to the exec stream.
func sendResizeRequest(stream pb.Docker_ExecContainerClient, size *term.Winsize) error {
slog.Debug("sending resize request", "width", size.Width, "height", size.Height)
return stream.Send(
&pb.ExecContainerRequest{
Payload: &pb.ExecContainerRequest_Resize{
Resize: &pb.ResizeEvent{
Height: uint32(size.Height),
Width: uint32(size.Width),
},
},
})
}
// setupTerminal configures the terminal for interactive TTY sessions.
// It checks if stdin is a terminal, sets it to raw mode, and sets up resize handling.
// Returns a cleanup function to restore terminal state, or an error.
func setupTerminal(ctx context.Context, stream pb.Docker_ExecContainerClient) (func(), error) {
inFd, isTerminal := term.GetFdInfo(os.Stdin)
if !isTerminal {
return nil, fmt.Errorf("stdin is not a terminal")
}
// Set terminal to raw mode
oldState, err := term.SetRawTerminal(inFd)
if err != nil {
return nil, fmt.Errorf("set raw terminal: %w", err)
}
// Cleanup function
restoreFunc := func() {
_ = term.RestoreTerminal(inFd, oldState)
}
// Set up resize handling
if err := handleTerminalResize(ctx, inFd, stream); err != nil {
restoreFunc()
return nil, err
}
return restoreFunc, nil
}
// handleTerminalResize sends initial window size and handles window resize signals for TTY sessions.
func handleTerminalResize(ctx context.Context, inFd uintptr, stream pb.Docker_ExecContainerClient) error {
// Handle window resize signals
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, unix.SIGWINCH)
// Send initial window size
if size, err := term.GetWinsize(inFd); err == nil {
_ = sendResizeRequest(stream, size)
}
go func() {
defer signal.Stop(sigCh)
for {
select {
case <-ctx.Done():
return
case <-sigCh:
size, err := term.GetWinsize(inFd)
if err != nil {
slog.Debug("get window size", "error", err)
continue
}
if err = sendResizeRequest(stream, size); err != nil {
slog.Debug("send resize request", "error", err)
}
}
}
}()
return nil
}
// handleClientInputStream reads from stdin and sends data to the remote server.
// It also periodically checks for context cancellation to exit gracefully when e.g.
// the output stream is closed.
func handleClientInputStream(ctx context.Context, stream pb.Docker_ExecContainerClient, stdin io.Reader) error {
slog.Debug("Input goroutine started")
defer slog.Debug("Input goroutine exited")
defer stream.CloseSend()
// Channel to receive stdin data
stdinCh := make(chan []byte)
stdinErrCh := make(chan error, 1)
// Read from stdin in a separate goroutine
// Note: this goroutine may continue blocking on Read even after we exit from the function,
// but that's OK - it will eventually unblock when data arrives or stdin closes.
go func() {
buf := make([]byte, 32*1024) // 32KB buffer
for {
n, err := stdin.Read(buf)
if n > 0 {
data := make([]byte, n)
copy(data, buf[:n])
select {
case stdinCh <- data:
case <-ctx.Done():
slog.Debug("stdin reader exiting due to context done")
return
}
}
if err != nil {
if err == io.EOF {
slog.Debug("stdin reader: EOF received")
} else {
slog.Debug("stdin reader error", "error", err)
}
stdinErrCh <- err
return
}
}
}()
// Send stdin data to the server or exit when context is cancelled
for {
select {
case <-ctx.Done():
return nil
case data := <-stdinCh:
if err := stream.Send(&pb.ExecContainerRequest{
Payload: &pb.ExecContainerRequest_Stdin{Stdin: data},
}); err != nil {
return fmt.Errorf("send stdin: %w", err)
}
case err := <-stdinErrCh:
if err != io.EOF {
return fmt.Errorf("read stdin: %w", err)
}
return nil
}
}
}
// handleClientOutputStream receives output from the exec stream and writes to stdout/stderr.
// It also captures the exit code and signals completion via context cancellation.
func handleClientOutputStream(ctx context.Context, stream pb.Docker_ExecContainerClient, stdout, stderr io.Writer, exitCode *int) error {
slog.Debug("Output goroutine started")
defer slog.Debug("Output goroutine exited")
for {
resp, err := stream.Recv()
if err == io.EOF {
slog.Debug("output stream: EOF received")
return nil
}
if err != nil {
return fmt.Errorf("receive from stream: %w", err)
}
switch payload := resp.Payload.(type) {
case *pb.ExecContainerResponse_ExecId:
// This is sent first; we already processed it earlier, so just ignore duplicates.
case *pb.ExecContainerResponse_Stdout:
if _, err := stdout.Write(payload.Stdout); err != nil {
return fmt.Errorf("write stdout: %w", err)
}
case *pb.ExecContainerResponse_Stderr:
if _, err := stderr.Write(payload.Stderr); err != nil {
return fmt.Errorf("write stderr: %w", err)
}
case *pb.ExecContainerResponse_ExitCode:
slog.Debug("received exit code", "code", payload.ExitCode)
*exitCode = int(payload.ExitCode)
return nil
}
}
}
// ExecContainer executes a command in a running container with bidirectional streaming.
// TODO: This can be merged with pkg/client as it's an unnecessary logic split.
func (c *Client) ExecContainer(ctx context.Context, opts ExecConfig) (exitCode int, err error) {
// Note: In non-interactive mode (without TTY), signals like SIGTERM/SIGINT will terminate the client
// process without being forwarded to the remote container process. We could catch and forward these
// signals (useful for long-running commands), but for example "kubectl exec" doesn't do this either (as of November 2025),
// so we keep it simple for now.
slog.Debug("starting ExecContainer", "containerID", opts.ContainerID, "options", opts.Options)
// Initialize exit code to non-zero in case we have to return early
exitCode = 1
// Set up I/O streams - use custom streams if provided, otherwise default to os.Stdin/Stdout/Stderr
stdin := io.Reader(os.Stdin)
stdout := io.Writer(os.Stdout)
stderr := io.Writer(os.Stderr)
if opts.Options.Stdin != nil {
stdin = opts.Options.Stdin
}
if opts.Options.Stdout != nil {
stdout = opts.Options.Stdout
}
if opts.Options.Stderr != nil {
stderr = opts.Options.Stderr
}
// Marshal the exec config
configBytes, err := json.Marshal(opts.Options)
if err != nil {
return -1, fmt.Errorf("marshal exec config: %w", err)
}
// Create the bidirectional stream
stream, err := c.GRPCClient.ExecContainer(ctx)
if err != nil {
return -1, fmt.Errorf("create exec stream: %w", err)
}
// Send the initial configuration
if err := stream.Send(&pb.ExecContainerRequest{
Payload: &pb.ExecContainerRequest_Config{
Config: &pb.ExecConfig{
ContainerId: opts.ContainerID,
Options: configBytes,
},
},
}); err != nil {
return -1, fmt.Errorf("send exec config: %w", err)
}
// Receive the exec ID
resp, err := stream.Recv()
if err != nil {
return -1, fmt.Errorf("receive exec ID: %w", err)
}
execID := resp.GetExecId()
if execID == "" {
return -1, fmt.Errorf("expected exec ID in first response")
}
errGroup, ctx := errgroup.WithContext(ctx)
// Create cancellable context for goroutine coordination
ctx, cancel := context.WithCancel(ctx)
defer cancel()
// Handle terminal setup for interactive sessions
if opts.Options.AttachStdin && opts.Options.Tty {
restoreTerminal, err := setupTerminal(ctx, stream)
if err != nil {
return -1, fmt.Errorf("setup terminal: %w", err)
}
if restoreTerminal != nil {
defer restoreTerminal()
}
}
// Handle stdin stream if needed
if opts.Options.AttachStdin {
errGroup.Go(func() error {
return handleClientInputStream(ctx, stream, stdin)
})
} else {
// Close send direction immediately if not attaching stdin
stream.CloseSend()
}
// Handle output streams (stdout/stderr)
errGroup.Go(func() error {
defer cancel()
return handleClientOutputStream(ctx, stream, stdout, stderr, &exitCode)
})
err = errGroup.Wait()
if err == nil && opts.Options.Detach {
return 0, nil
}
return exitCode, err
}
+6 -6
View File
@@ -8,10 +8,10 @@ import (
"net/netip" "net/netip"
"strconv" "strconv"
"github.com/containerd/errdefs"
dockercontainer "github.com/docker/docker/api/types/container" dockercontainer "github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters" "github.com/docker/docker/api/types/filters"
dnetwork "github.com/docker/docker/api/types/network" dnetwork "github.com/docker/docker/api/types/network"
"github.com/docker/docker/client"
"github.com/docker/docker/libnetwork/iptables" "github.com/docker/docker/libnetwork/iptables"
"github.com/psviderski/uncloud/internal/machine/dns" "github.com/psviderski/uncloud/internal/machine/dns"
"github.com/psviderski/uncloud/internal/machine/firewall" "github.com/psviderski/uncloud/internal/machine/firewall"
@@ -27,7 +27,7 @@ func (c *Controller) EnsureUncloudNetwork(ctx context.Context, subnet netip.Pref
needsCreation := false needsCreation := false
nw, err := c.client.NetworkInspect(ctx, NetworkName, dnetwork.InspectOptions{}) nw, err := c.client.NetworkInspect(ctx, NetworkName, dnetwork.InspectOptions{})
if err != nil { if err != nil {
if !client.IsErrNotFound(err) { if !errdefs.IsNotFound(err) {
return fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err) return fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err)
} }
needsCreation = true needsCreation = true
@@ -187,7 +187,7 @@ func (c *Controller) Cleanup() error {
for _, ctr := range containers { for _, ctr := range containers {
err = c.client.ContainerStop(ctx, ctr.ID, dockercontainer.StopOptions{}) err = c.client.ContainerStop(ctx, ctr.ID, dockercontainer.StopOptions{})
if err != nil && !client.IsErrNotFound(err) { if err != nil && !errdefs.IsNotFound(err) {
errs = append(errs, fmt.Errorf("stop container '%s': %w", ctr.ID, err)) errs = append(errs, fmt.Errorf("stop container '%s': %w", ctr.ID, err))
} }
@@ -197,7 +197,7 @@ func (c *Controller) Cleanup() error {
}) })
if err == nil { if err == nil {
removed++ removed++
} else if !client.IsErrNotFound(err) { } else if !errdefs.IsNotFound(err) {
errs = append(errs, fmt.Errorf("remove container '%s': %w", ctr.ID, err)) errs = append(errs, fmt.Errorf("remove container '%s': %w", ctr.ID, err))
} }
} }
@@ -223,10 +223,10 @@ func (c *Controller) Cleanup() error {
if err = c.client.NetworkRemove(ctx, NetworkName); err == nil { if err = c.client.NetworkRemove(ctx, NetworkName); err == nil {
slog.Info("Docker network removed.", "name", NetworkName) slog.Info("Docker network removed.", "name", NetworkName)
} else if !client.IsErrNotFound(err) { } else if !errdefs.IsNotFound(err) {
errs = append(errs, fmt.Errorf("remove Docker network '%s': %w", NetworkName, err)) errs = append(errs, fmt.Errorf("remove Docker network '%s': %w", NetworkName, err))
} }
} else if !client.IsErrNotFound(err) { } else if !errdefs.IsNotFound(err) {
errs = append(errs, fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err)) errs = append(errs, fmt.Errorf("inspect Docker network '%s': %w", NetworkName, err))
} }
+578 -51
View File
@@ -1,19 +1,25 @@
package docker package docker
import ( import (
"archive/tar"
"bytes"
"context" "context"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt" "fmt"
"io" "io"
"log/slog" "log/slog"
"maps"
"net/netip" "net/netip"
"os" "os"
"path/filepath"
"regexp" "regexp"
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
"time"
"github.com/containerd/errdefs"
"github.com/distribution/reference" "github.com/distribution/reference"
dockercommand "github.com/docker/cli/cli/command" dockercommand "github.com/docker/cli/cli/command"
dockerconfig "github.com/docker/cli/cli/config" dockerconfig "github.com/docker/cli/cli/config"
@@ -25,6 +31,7 @@ import (
"github.com/docker/docker/api/types/network" "github.com/docker/docker/api/types/network"
"github.com/docker/docker/api/types/volume" "github.com/docker/docker/api/types/volume"
"github.com/docker/docker/client" "github.com/docker/docker/client"
"github.com/docker/docker/pkg/stdcopy"
"github.com/docker/go-connections/nat" "github.com/docker/go-connections/nat"
"github.com/google/go-containerregistry/pkg/authn" "github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name" "github.com/google/go-containerregistry/pkg/name"
@@ -40,6 +47,7 @@ import (
"google.golang.org/grpc/codes" "google.golang.org/grpc/codes"
"google.golang.org/grpc/status" "google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb" "google.golang.org/protobuf/types/known/emptypb"
"google.golang.org/protobuf/types/known/timestamppb"
) )
var fullDockerIDRegex = regexp.MustCompile(`^[a-f0-9]{64}$`) var fullDockerIDRegex = regexp.MustCompile(`^[a-f0-9]{64}$`)
@@ -53,41 +61,35 @@ type Server struct {
// internalDNSIP is a function that returns the IP address of the internal DNS server. It may return an empty // internalDNSIP is a function that returns the IP address of the internal DNS server. It may return an empty
// address if the address is unknown (e.g. when the machine is not initialised yet). // address if the address is unknown (e.g. when the machine is not initialised yet).
internalDNSIP func() netip.Addr internalDNSIP func() netip.Addr
// machineID is a function that returns the machine ID. It may return an empty string if the machine
// is not initialised yet.
machineID func() string
// networkReady is a function that returns true if the Docker network is ready for containers. // networkReady is a function that returns true if the Docker network is ready for containers.
networkReady func() bool networkReady func() bool
// waitForNetworkReady is a function that waits for the Docker network to be ready for containers. // waitForNetworkReady is a function that waits for the Docker network to be ready for containers.
waitForNetworkReady func(ctx context.Context) error waitForNetworkReady func(ctx context.Context) error
} }
// ServerOption configures the Docker server. type ServerOptions struct {
type ServerOption func(*Server) // TODO: verify if we still need the network readiness checks as the cluster controller ensures the network
// is ready before starting the network API server. It may still be needed when communicating with the local
// WithNetworkReady sets the network readiness check function. // API server but in this case we should probably fail until the cluster is initialised.
func WithNetworkReady(networkReady func() bool) ServerOption { NetworkReady func() bool
return func(s *Server) { WaitForNetworkReady func(ctx context.Context) error
s.networkReady = networkReady
}
}
// WithWaitForNetworkReady sets the network readiness wait function.
func WithWaitForNetworkReady(waitForNetworkReady func(ctx context.Context) error) ServerOption {
return func(s *Server) {
s.waitForNetworkReady = waitForNetworkReady
}
} }
// NewServer creates a new Docker gRPC server with the provided Docker service. // NewServer creates a new Docker gRPC server with the provided Docker service.
func NewServer(service *Service, db *sqlx.DB, internalDNSIP func() netip.Addr, opts ...ServerOption) *Server { func NewServer(service *Service, db *sqlx.DB, internalDNSIP func() netip.Addr, machineID func() string, opts ServerOptions) *Server {
s := &Server{ s := &Server{
client: service.Client, client: service.Client,
service: service, service: service,
db: db, db: db,
internalDNSIP: internalDNSIP, internalDNSIP: internalDNSIP,
machineID: machineID,
} }
for _, opt := range opts { s.networkReady = opts.NetworkReady
opt(s) s.waitForNetworkReady = opts.WaitForNetworkReady
}
return s return s
} }
@@ -115,10 +117,10 @@ func (s *Server) CreateContainer(ctx context.Context, req *pb.CreateContainerReq
resp, err := s.client.ContainerCreate(ctx, &config, &hostConfig, &networkConfig, &platform, req.Name) resp, err := s.client.ContainerCreate(ctx, &config, &hostConfig, &networkConfig, &platform, req.Name)
if err != nil { if err != nil {
if client.IsErrNotFound(err) { if errdefs.IsNotFound(err) {
return nil, status.Errorf(codes.NotFound, err.Error()) return nil, status.Error(codes.NotFound, err.Error())
} }
return nil, status.Errorf(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
} }
respBytes, err := json.Marshal(resp) respBytes, err := json.Marshal(resp)
@@ -133,10 +135,10 @@ func (s *Server) CreateContainer(ctx context.Context, req *pb.CreateContainerReq
func (s *Server) InspectContainer(ctx context.Context, req *pb.InspectContainerRequest) (*pb.InspectContainerResponse, error) { func (s *Server) InspectContainer(ctx context.Context, req *pb.InspectContainerRequest) (*pb.InspectContainerResponse, error) {
resp, err := s.client.ContainerInspect(ctx, req.Id) resp, err := s.client.ContainerInspect(ctx, req.Id)
if err != nil { if err != nil {
if client.IsErrNotFound(err) { if errdefs.IsNotFound(err) {
return nil, status.Errorf(codes.NotFound, err.Error()) return nil, status.Error(codes.NotFound, err.Error())
} }
return nil, status.Errorf(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
} }
respBytes, err := json.Marshal(resp) respBytes, err := json.Marshal(resp)
@@ -166,10 +168,10 @@ func (s *Server) StartContainer(ctx context.Context, req *pb.StartContainerReque
} }
if err := s.client.ContainerStart(ctx, req.Id, opts); err != nil { if err := s.client.ContainerStart(ctx, req.Id, opts); err != nil {
if client.IsErrNotFound(err) { if errdefs.IsNotFound(err) {
return nil, status.Errorf(codes.NotFound, err.Error()) return nil, status.Error(codes.NotFound, err.Error())
} }
return nil, status.Errorf(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
} }
return &emptypb.Empty{}, nil return &emptypb.Empty{}, nil
@@ -185,10 +187,10 @@ func (s *Server) StopContainer(ctx context.Context, req *pb.StopContainerRequest
} }
if err := s.client.ContainerStop(ctx, req.Id, opts); err != nil { if err := s.client.ContainerStop(ctx, req.Id, opts); err != nil {
if client.IsErrNotFound(err) { if errdefs.IsNotFound(err) {
return nil, status.Errorf(codes.NotFound, err.Error()) return nil, status.Error(codes.NotFound, err.Error())
} }
return nil, status.Errorf(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
} }
return &emptypb.Empty{}, nil return &emptypb.Empty{}, nil
@@ -218,13 +220,13 @@ func (s *Server) ListContainers(ctx context.Context, req *pb.ListContainersReque
containerSummaries, err := s.client.ContainerList(ctx, opts) containerSummaries, err := s.client.ContainerList(ctx, opts)
if err != nil { if err != nil {
return nil, status.Errorf(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
} }
containers := make([]types.ContainerJSON, 0, len(containerSummaries)) containers := make([]container.InspectResponse, 0, len(containerSummaries))
for _, cs := range containerSummaries { for _, cs := range containerSummaries {
c, err := s.client.ContainerInspect(ctx, cs.ID) c, err := s.client.ContainerInspect(ctx, cs.ID)
if err != nil { if err != nil {
if client.IsErrNotFound(err) { if errdefs.IsNotFound(err) {
// The listed container may have been removed while we were inspecting other containers. // The listed container may have been removed while we were inspecting other containers.
continue continue
} }
@@ -257,10 +259,10 @@ func (s *Server) RemoveContainer(ctx context.Context, req *pb.RemoveContainerReq
} }
if err := s.client.ContainerRemove(ctx, req.Id, opts); err != nil { if err := s.client.ContainerRemove(ctx, req.Id, opts); err != nil {
if client.IsErrNotFound(err) { if errdefs.IsNotFound(err) {
return nil, status.Errorf(codes.NotFound, err.Error()) return nil, status.Error(codes.NotFound, err.Error())
} }
return nil, status.Errorf(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
} }
return &emptypb.Empty{}, nil return &emptypb.Empty{}, nil
@@ -286,7 +288,7 @@ func (s *Server) PullImage(req *pb.PullImageRequest, stream grpc.ServerStreaming
respBody, err := s.client.ImagePull(ctx, req.Image, opts) respBody, err := s.client.ImagePull(ctx, req.Image, opts)
if err != nil { if err != nil {
return status.Errorf(codes.Internal, err.Error()) return status.Error(codes.Internal, err.Error())
} }
defer respBody.Close() defer respBody.Close()
@@ -317,19 +319,19 @@ func (s *Server) PullImage(req *pb.PullImageRequest, stream grpc.ServerStreaming
case err = <-errCh: case err = <-errCh:
return err return err
case <-ctx.Done(): case <-ctx.Done():
return status.Errorf(codes.Canceled, ctx.Err().Error()) return status.Error(codes.Canceled, ctx.Err().Error())
} }
} }
} }
// InspectImage returns the image information for the given image ID. // InspectImage returns the image information for the given image ID.
func (s *Server) InspectImage(ctx context.Context, req *pb.InspectImageRequest) (*pb.InspectImageResponse, error) { func (s *Server) InspectImage(ctx context.Context, req *pb.InspectImageRequest) (*pb.InspectImageResponse, error) {
resp, _, err := s.client.ImageInspectWithRaw(ctx, req.Id) resp, err := s.client.ImageInspect(ctx, req.Id)
if err != nil { if err != nil {
if client.IsErrNotFound(err) { if errdefs.IsNotFound(err) {
return nil, status.Errorf(codes.NotFound, err.Error()) return nil, status.Error(codes.NotFound, err.Error())
} }
return nil, status.Errorf(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
} }
respBytes, err := json.Marshal(resp) respBytes, err := json.Marshal(resp)
@@ -385,6 +387,49 @@ func (s *Server) InspectRemoteImage(
}, nil }, nil
} }
// ListImages returns a list of all images matching the filter and indicates whether Docker is using the containerd
// image store.
func (s *Server) ListImages(ctx context.Context, req *pb.ListImagesRequest) (*pb.ListImagesResponse, error) {
var opts image.ListOptions
if len(req.Options) > 0 {
if err := json.Unmarshal(req.Options, &opts); err != nil {
return nil, status.Errorf(codes.InvalidArgument, "unmarshal options: %v", err)
}
// Handle filters separately because they implement custom JSON unmarshalling.
var raw map[string]json.RawMessage
if err := json.Unmarshal(req.Options, &raw); err != nil {
return nil, status.Errorf(codes.InvalidArgument, "unmarshal options to raw map: %v", err)
}
if filtersBytes, ok := raw["Filters"]; ok {
args, err := filters.FromJSON(string(filtersBytes))
if err != nil {
return nil, status.Errorf(codes.InvalidArgument, "unmarshal filters: %v", err)
}
opts.Filters = args
}
}
images, err := s.service.ListImages(ctx, opts)
if err != nil {
return nil, status.Error(codes.Internal, err.Error())
}
machineImages := pb.MachineImages{
ContainerdStore: images.ContainerdStore,
}
if len(images.Images) > 0 {
if machineImages.Images, err = json.Marshal(images.Images); err != nil {
return nil, status.Errorf(codes.Internal, "marshal Docker images: %v", err)
}
}
return &pb.ListImagesResponse{
Messages: []*pb.MachineImages{&machineImages},
}, nil
}
// CreateVolume creates a new volume with the given options. // CreateVolume creates a new volume with the given options.
func (s *Server) CreateVolume(ctx context.Context, req *pb.CreateVolumeRequest) (*pb.CreateVolumeResponse, error) { func (s *Server) CreateVolume(ctx context.Context, req *pb.CreateVolumeRequest) (*pb.CreateVolumeResponse, error) {
var opts volume.CreateOptions var opts volume.CreateOptions
@@ -456,7 +501,7 @@ func (s *Server) ListVolumes(ctx context.Context, req *pb.ListVolumesRequest) (*
// RemoveVolume removes a volume with the given ID. // RemoveVolume removes a volume with the given ID.
func (s *Server) RemoveVolume(ctx context.Context, req *pb.RemoveVolumeRequest) (*emptypb.Empty, error) { func (s *Server) RemoveVolume(ctx context.Context, req *pb.RemoveVolumeRequest) (*emptypb.Empty, error) {
if err := s.client.VolumeRemove(ctx, req.Id, req.Force); err != nil { if err := s.client.VolumeRemove(ctx, req.Id, req.Force); err != nil {
if client.IsErrNotFound(err) { if errdefs.IsNotFound(err) {
return nil, status.Error(codes.NotFound, err.Error()) return nil, status.Error(codes.NotFound, err.Error())
} }
return nil, status.Error(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
@@ -492,9 +537,21 @@ func (s *Server) CreateServiceContainer(
containerName = fmt.Sprintf("%s-%s", spec.Name, suffix) containerName = fmt.Sprintf("%s-%s", spec.Name, suffix)
} }
envVars := maps.Clone(spec.Container.Env)
if envVars == nil {
envVars = make(api.EnvVars)
}
// Inject the machine ID if available
if s.machineID != nil {
if machineID := s.machineID(); machineID != "" {
envVars["UNCLOUD_MACHINE_ID"] = machineID
}
}
config := &container.Config{ config := &container.Config{
Cmd: spec.Container.Command, Cmd: spec.Container.Command,
Env: spec.Container.Env.ToSlice(), Env: envVars.ToSlice(),
Entrypoint: spec.Container.Entrypoint, Entrypoint: spec.Container.Entrypoint,
Hostname: containerName, Hostname: containerName,
Image: spec.Container.Image, Image: spec.Container.Image,
@@ -557,6 +614,7 @@ func (s *Server) CreateServiceContainer(
NanoCPUs: spec.Container.Resources.CPU, NanoCPUs: spec.Container.Resources.CPU,
Memory: spec.Container.Resources.Memory, Memory: spec.Container.Resources.Memory,
MemoryReservation: spec.Container.Resources.MemoryReservation, MemoryReservation: spec.Container.Resources.MemoryReservation,
DeviceRequests: spec.Container.Resources.DeviceReservations,
}, },
// Restart service containers if they exit or a machine restarts unless they are explicitly stopped. // Restart service containers if they exit or a machine restarts unless they are explicitly stopped.
// For one-off containers and batch jobs we plan to use a different service type/mode. // For one-off containers and batch jobs we plan to use a different service type/mode.
@@ -590,12 +648,19 @@ func (s *Server) CreateServiceContainer(
resp, err := s.client.ContainerCreate(ctx, config, hostConfig, networkConfig, nil, containerName) resp, err := s.client.ContainerCreate(ctx, config, hostConfig, networkConfig, nil, containerName)
if err != nil { if err != nil {
if client.IsErrNotFound(err) { if errdefs.IsNotFound(err) {
return nil, status.Error(codes.NotFound, err.Error()) return nil, status.Error(codes.NotFound, err.Error())
} }
return nil, status.Error(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
} }
// Inject configs into the created container
if err = s.injectConfigs(ctx, resp.ID, spec.Configs, spec.Container.ConfigMounts); err != nil {
// Remove the container if config injection fails
_ = s.client.ContainerRemove(ctx, resp.ID, container.RemoveOptions{RemoveVolumes: true})
return nil, status.Errorf(codes.Internal, "inject configs: %v", err)
}
respBytes, err := json.Marshal(resp) respBytes, err := json.Marshal(resp)
if err != nil { if err != nil {
return nil, status.Errorf(codes.Internal, "marshal response: %v", err) return nil, status.Errorf(codes.Internal, "marshal response: %v", err)
@@ -672,6 +737,122 @@ func ToDockerMounts(volumes []api.VolumeSpec, mounts []api.VolumeMount) ([]mount
return dockerMounts, nil return dockerMounts, nil
} }
// injectConfigs writes config content directly into the container.
// It processes ConfigSpecs and ConfigMounts to mount configuration content into the container filesystem.
func (s *Server) injectConfigs(ctx context.Context, containerID string, configs []api.ConfigSpec, mounts []api.ConfigMount) error {
if len(configs) == 0 || len(mounts) == 0 {
return nil
}
if err := api.ValidateConfigsAndMounts(configs, mounts); err != nil {
return fmt.Errorf("validate configs and mounts: %w", err)
}
// Create a map of config name to config spec for quick lookup
configMap := make(map[string]api.ConfigSpec)
for _, config := range configs {
configMap[config.Name] = config
}
// Process each config mount
for _, m := range mounts {
config, exists := configMap[m.ConfigName]
if !exists {
return fmt.Errorf("config mount references a config that doesn't exist: '%s'", m.ConfigName)
}
// Determine target path in container
targetPath := m.ContainerPath
if targetPath == "" {
// This is the default from the Compose spec
targetPath = filepath.Join("/", m.ConfigName)
}
// Determine file mode
fileMode := os.FileMode(0o444) // Default permissions
if m.Mode != nil {
fileMode = *m.Mode
}
uid, err := m.GetNumericUid()
if err != nil {
return fmt.Errorf("invalid Uid: %w", err)
}
gid, err := m.GetNumericGid()
if err != nil {
return fmt.Errorf("invalid Gid: %w", err)
}
// Copy the config content directly into the container
if err := s.copyContentToContainer(
ctx, containerID, config.Content, targetPath, uid, gid, fileMode,
); err != nil {
return fmt.Errorf("copy config file '%s' to container: %w", config.Name, err)
}
slog.Debug("Injected config into container",
"config", config.Name,
"container", containerID[:12],
"target", targetPath)
}
return nil
}
// copyContentToContainer copies content directly to a file in the container using Docker's CopyToContainer API.
func (s *Server) copyContentToContainer(ctx context.Context, containerID string, content []byte, targetPath string, uid *uint64, gid *uint64, fileMode os.FileMode) error {
// Create a tar archive containing the file
var buf bytes.Buffer
tw := tar.NewWriter(&buf)
// Create tar header
header := &tar.Header{
Name: filepath.Base(targetPath),
Size: int64(len(content)),
Mode: int64(fileMode),
ModTime: time.Now(),
Typeflag: tar.TypeReg,
}
// Set ownership if specified
if uid != nil {
header.Uid = int(*uid)
}
if gid != nil {
header.Gid = int(*gid)
}
// Write header and content to tar archive
if err := tw.WriteHeader(header); err != nil {
return fmt.Errorf("write tar header: %w", err)
}
if _, err := tw.Write(content); err != nil {
return fmt.Errorf("write content to tar: %w", err)
}
if err := tw.Close(); err != nil {
return fmt.Errorf("close tar writer: %w", err)
}
// Copy the tar archive to the container
targetDir := filepath.Dir(targetPath)
if targetDir == "." {
targetDir = "/"
}
if err := s.client.CopyToContainer(
ctx,
containerID,
targetDir,
&buf,
container.CopyToContainerOptions{CopyUIDGID: true},
); err != nil {
return fmt.Errorf("copy to container: %w", err)
}
return nil
}
func toDockerBindOptions(opts *api.BindOptions) *mount.BindOptions { func toDockerBindOptions(opts *api.BindOptions) *mount.BindOptions {
if opts == nil { if opts == nil {
return nil return nil
@@ -703,7 +884,7 @@ func (s *Server) verifyDockerVolumesExist(ctx context.Context, mounts []mount.Mo
// TODO: non-local volume drivers should likely be handled differently (needs proper investigation). // TODO: non-local volume drivers should likely be handled differently (needs proper investigation).
if _, err := s.client.VolumeInspect(ctx, m.Source); err != nil { if _, err := s.client.VolumeInspect(ctx, m.Source); err != nil {
if client.IsErrNotFound(err) { if errdefs.IsNotFound(err) {
return status.Errorf(codes.NotFound, "volume '%s' not found", m.Source) return status.Errorf(codes.NotFound, "volume '%s' not found", m.Source)
} }
return status.Errorf(codes.Internal, "inspect volume '%s': %v", m.Source, err.Error()) return status.Errorf(codes.Internal, "inspect volume '%s': %v", m.Source, err.Error())
@@ -723,10 +904,10 @@ func (s *Server) InspectServiceContainer(
) (*pb.ServiceContainer, error) { ) (*pb.ServiceContainer, error) {
serviceCtr, err := s.service.InspectServiceContainer(ctx, req.Id) serviceCtr, err := s.service.InspectServiceContainer(ctx, req.Id)
if err != nil { if err != nil {
if client.IsErrNotFound(err) { if errdefs.IsNotFound(err) {
return nil, status.Errorf(codes.NotFound, err.Error()) return nil, status.Error(codes.NotFound, err.Error())
} }
return nil, status.Errorf(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
} }
ctrBytes, err := json.Marshal(serviceCtr.Container) ctrBytes, err := json.Marshal(serviceCtr.Container)
@@ -813,7 +994,7 @@ func (s *Server) RemoveServiceContainer(ctx context.Context, req *pb.RemoveConta
if !fullDockerIDRegex.MatchString(req.Id) { if !fullDockerIDRegex.MatchString(req.Id) {
ctr, err := s.client.ContainerInspect(ctx, req.Id) ctr, err := s.client.ContainerInspect(ctx, req.Id)
if err != nil { if err != nil {
if client.IsErrNotFound(err) { if errdefs.IsNotFound(err) {
return nil, status.Error(codes.NotFound, err.Error()) return nil, status.Error(codes.NotFound, err.Error())
} }
return nil, status.Error(codes.Internal, err.Error()) return nil, status.Error(codes.Internal, err.Error())
@@ -834,3 +1015,349 @@ func (s *Server) RemoveServiceContainer(ctx context.Context, req *pb.RemoveConta
return resp, nil return resp, nil
} }
// logsHeartbeatInterval is the interval at which heartbeat entries are sent when there are no logs to stream.
const logsHeartbeatInterval = 200 * time.Millisecond
// ContainerLogs streams logs from a container.
func (s *Server) ContainerLogs(
req *pb.ContainerLogsRequest, stream grpc.ServerStreamingServer[pb.ContainerLogEntry],
) error {
// Stream context is cancelled when the client has disconnected or the stream has ended.
ctx := stream.Context()
opts := ContainerLogsOptions{
ContainerID: req.ContainerId,
Follow: req.Follow,
Tail: int(req.Tail),
Since: req.Since,
Until: req.Until,
}
logsCh, err := s.service.ContainerLogs(ctx, opts)
if err != nil {
if errdefs.IsNotFound(err) {
return status.Error(codes.NotFound, err.Error())
}
return status.Errorf(codes.Internal, "get container logs: %v", err)
}
log := slog.With("container_id", req.ContainerId, "stream_id", fmt.Sprintf("%p", stream)[2:])
log.Debug("Starting container logs streaming.",
"follow", req.Follow, "tail", req.Tail, "since", req.Since, "until", req.Until)
// Heartbeats are needed only when following logs to let the client know when there are no new log entries
// to allow it to advance the watermark of last received log timestamp.
var heartbeatCh <-chan time.Time
if req.Follow {
heartbeatTicker := time.NewTicker(logsHeartbeatInterval)
defer heartbeatTicker.Stop()
heartbeatCh = heartbeatTicker.C
}
started := time.Now()
lastSent := time.Time{}
for {
select {
case entry, ok := <-logsCh:
if !ok {
// Channel closed, no more log entries.
return nil
}
if entry.Err != nil {
return status.Error(codes.Internal, entry.Err.Error())
}
pbEntry := &pb.ContainerLogEntry{
Stream: api.LogStreamTypeToProto(entry.Stream),
Timestamp: timestamppb.New(entry.Timestamp),
Message: entry.Message,
}
if err = stream.Send(pbEntry); err != nil {
return status.Errorf(codes.Internal, "send log entry: %v", err)
}
lastSent = entry.Timestamp
case now := <-heartbeatCh:
// Only send heartbeat if no log entries have been sent since the last heartbeat interval or
// if no log entries have been sent at all for at least a heartbeat interval since starting.
if now.Sub(lastSent) < logsHeartbeatInterval ||
(lastSent.IsZero() && now.Sub(started) < logsHeartbeatInterval) {
continue
}
// Use the timestamp one heartbeat in the past to be conservative. This reduces the chance of sending
// a timestamp that is greater than a log entry currently being parsed but not yet sent, which would
// cause the client to incorrectly believe it has received all logs up to that point.
heartbeat := &pb.ContainerLogEntry{
Stream: pb.ContainerLogEntry_HEARTBEAT,
Timestamp: timestamppb.New(now.Add(-logsHeartbeatInterval)),
}
if err = stream.Send(heartbeat); err != nil {
return status.Errorf(codes.Internal, "send log stream heartbeat: %v", err)
}
lastSent = heartbeat.Timestamp.AsTime()
log.Debug("Sent log stream heartbeat.", "timestamp", lastSent)
case <-ctx.Done():
return status.Error(codes.Canceled, ctx.Err().Error())
}
}
}
// receiveExecConfig receives and validates the initial exec configuration from the stream.
func (s *Server) receiveExecConfig(stream pb.Docker_ExecContainerServer) (*pb.ExecConfig, api.ExecOptions, error) {
req, err := stream.Recv()
if err != nil {
return nil, api.ExecOptions{}, status.Errorf(codes.InvalidArgument, "receive config: %v", err)
}
execConfig := req.GetConfig()
if execConfig == nil {
return nil, api.ExecOptions{}, status.Error(codes.InvalidArgument, "first message must contain exec config")
}
// Unmarshal the Uncloud's execOpts
var execOpts api.ExecOptions
if err := json.Unmarshal(execConfig.Options, &execOpts); err != nil {
return nil, api.ExecOptions{}, status.Errorf(codes.InvalidArgument, "unmarshal exec config: %v", err)
}
return execConfig, execOpts, nil
}
// handleServerExecInput reads from the gRPC stream and writes to Docker stdin, handling resize requests.
func (s *Server) handleServerExecInput(
ctx context.Context,
stream pb.Docker_ExecContainerServer,
attachConn types.HijackedResponse,
execID string,
tty bool,
) error {
slog.Debug("Input goroutine started", "exec_id", execID, "tty", tty)
defer slog.Debug("Input goroutine exited", "exec_id", execID)
defer attachConn.CloseWrite()
for {
select {
case <-ctx.Done():
slog.Debug("Input goroutine context canceled via context", "exec_id", execID)
return nil
default:
}
req, err := stream.Recv()
switch {
case errors.Is(err, io.EOF):
slog.Debug("Input goroutine received EOF", "exec_id", execID)
return nil
case status.Code(err) == codes.Canceled:
// Can be the case when the output goroutine ends and the stream context is canceled.
slog.Debug("Input goroutine context canceled", "exec_id", execID)
return nil
case err == nil:
// continue processing
default:
return fmt.Errorf("receive from stream: %w", err)
}
switch payload := req.Payload.(type) {
case *pb.ExecContainerRequest_Stdin:
if _, err := attachConn.Conn.Write(payload.Stdin); err != nil {
return fmt.Errorf("write to stdin: %w", err)
}
case *pb.ExecContainerRequest_Resize:
if tty {
resizeOpts := container.ResizeOptions{
Height: uint(payload.Resize.Height),
Width: uint(payload.Resize.Width),
}
if err := s.client.ContainerExecResize(ctx, execID, resizeOpts); err != nil {
slog.Warn("Failed to resize TTY", "err", err, "exec_id", execID)
}
}
}
}
}
// handleServerExecOutput reads from Docker stdout/stderr and writes to the gRPC stream.
func (s *Server) handleServerExecOutput(
stream pb.Docker_ExecContainerServer,
attachResp types.HijackedResponse,
execID string,
tty bool,
) error {
slog.Debug("Output goroutine started", "exec_id", execID, "tty", tty)
defer slog.Debug("Output goroutine exited", "exec_id", execID)
if tty {
// In TTY mode, all output is stdout - copy directly to stream
stdoutWriter := &grpcStreamWriter{stream: stream, isStderr: false}
_, err := io.Copy(stdoutWriter, attachResp.Reader)
if err != nil && err != io.EOF {
return fmt.Errorf("copy tty output: %w", err)
}
return nil
} else {
// In non-TTY mode, Docker multiplexes stdout/stderr with headers
// Use stdcopy to demultiplex
slog.Debug("Starting StdCopy for non-TTY", "exec_id", execID)
stdoutWriter := &grpcStreamWriter{stream: stream, isStderr: false}
stderrWriter := &grpcStreamWriter{stream: stream, isStderr: true}
written, err := stdcopy.StdCopy(stdoutWriter, stderrWriter, attachResp.Reader)
slog.Debug("StdCopy completed", "exec_id", execID, "bytes", written, "err", err)
if err != nil && err != io.EOF {
return fmt.Errorf("demultiplex docker output: %w", err)
}
return nil
}
}
// grpcStreamWriter is a writer that sends data to a gRPC stream as stdout or stderr.
type grpcStreamWriter struct {
stream pb.Docker_ExecContainerServer
isStderr bool
}
func (w *grpcStreamWriter) Write(p []byte) (n int, err error) {
data := make([]byte, len(p))
copy(data, p)
var resp *pb.ExecContainerResponse
if w.isStderr {
resp = &pb.ExecContainerResponse{
Payload: &pb.ExecContainerResponse_Stderr{Stderr: data},
}
} else {
resp = &pb.ExecContainerResponse{
Payload: &pb.ExecContainerResponse_Stdout{Stdout: data},
}
}
if err := w.stream.Send(resp); err != nil {
return 0, err
}
return len(p), nil
}
// ExecContainer executes a command in a running container with bidirectional streaming for stdin/stdout/stderr.
func (s *Server) ExecContainer(stream pb.Docker_ExecContainerServer) error {
slog.Debug("ExecContainer server-side called")
defer slog.Debug("ExecContainer server-side ended")
ctx := stream.Context()
// Receive and validate configuration
execConfig, execOpts, err := s.receiveExecConfig(stream)
if err != nil {
return err
}
// Convert to Docker's ExecOptions
dockerExecOpts := container.ExecOptions{
Cmd: execOpts.Command,
AttachStdin: execOpts.AttachStdin,
AttachStdout: execOpts.AttachStdout,
AttachStderr: execOpts.AttachStderr,
Tty: execOpts.Tty,
User: execOpts.User,
Privileged: execOpts.Privileged,
WorkingDir: execOpts.WorkingDir,
Env: execOpts.Env,
}
// Create the exec instance
execResp, err := s.client.ContainerExecCreate(ctx, execConfig.ContainerId, dockerExecOpts)
if err != nil {
if errdefs.IsNotFound(err) {
return status.Error(codes.NotFound, err.Error())
}
return status.Errorf(codes.Internal, "create exec: %v", err)
}
// Send the exec ID back to the client
if err := stream.Send(&pb.ExecContainerResponse{
Payload: &pb.ExecContainerResponse_ExecId{ExecId: execResp.ID},
}); err != nil {
return status.Errorf(codes.Internal, "send exec ID: %v", err)
}
slog.Debug("Sent exec ID to the client", "exec_id", execResp.ID)
// For detached mode, start without attaching
if execOpts.Detach {
dockerStartOpts := container.ExecStartOptions{
Tty: dockerExecOpts.Tty,
Detach: true,
}
if err := s.client.ContainerExecStart(ctx, execResp.ID, dockerStartOpts); err != nil {
return status.Errorf(codes.Internal, "start exec: %v", err)
}
return nil
}
// For attached mode, attach to the exec instance
attachOpts := container.ExecAttachOptions{
Tty: dockerExecOpts.Tty,
}
attachConn, err := s.client.ContainerExecAttach(ctx, execResp.ID, attachOpts)
if err != nil {
return status.Errorf(codes.Internal, "attach to exec: %v", err)
}
defer attachConn.Close()
// Create a cancelable context for the input handler
handlerCtx, cancelInput := context.WithCancel(ctx)
defer cancelInput()
// Create a channel to wait for output completion
outputDone := make(chan error, 1)
// Start stdin handler if stdin is attached
if dockerExecOpts.AttachStdin {
go func() {
err := s.handleServerExecInput(handlerCtx, stream, attachConn, execResp.ID, dockerExecOpts.Tty)
if err != nil {
slog.Warn("Error in exec input handler", "err", err, "exec_id", execResp.ID)
}
}()
} else {
// If not attaching stdin, close the write side immediately
attachConn.CloseWrite()
}
// Start output handler
// We only wait for this goroutine to complete - it signals when the exec process finishes
go func() {
outputDone <- s.handleServerExecOutput(stream, attachConn, execResp.ID, dockerExecOpts.Tty)
}()
// Wait for the output goroutine to complete (it signals when done)
// We only wait for the output handler goroutine, not for the stdin one.
if err := <-outputDone; err != nil {
slog.Warn("Error in exec output handler", "err", err, "exec_id", execResp.ID)
}
// Do a best-effort cancellation of the stdin handler.
// We can't guarantee immediate exit because it may be blocked on stream.Recv(), but at
// least we want to send a cancel signal explicitly.
cancelInput()
inspectResp, err := s.client.ContainerExecInspect(ctx, execResp.ID)
if err != nil {
slog.Error("Failed to inspect exec after completion", "err", err, "exec_id", execResp.ID)
return status.Errorf(codes.Internal, "inspect exec: %v", err)
}
// Send the exit code
slog.Debug("Sending exec exit code", "exec_id", execResp.ID, "exit_code", inspectResp.ExitCode)
if err := stream.Send(&pb.ExecContainerResponse{
Payload: &pb.ExecContainerResponse_ExitCode{ExitCode: int32(inspectResp.ExitCode)},
}); err != nil {
slog.Error("Failed to send exec exit code", "err", err, "exec_id", execResp.ID)
return status.Errorf(codes.Internal, "send exit code: %v", err)
}
return nil
}

Some files were not shown because too many files have changed in this diff Show More