Compare commits

...
48 Commits
Author SHA1 Message Date
Pasha Sviderski 796363f23c fix: TestCollectContainers unit tests 2025-12-12 21:02:22 +10:00
Pasha Sviderski cb0df2169f feat: add IP ADDRESS column to 'uc ps' and 'uc inspect' 2025-12-12 20:23:43 +10:00
Pasha Sviderski 95afc7f289 chore: sort 'uc inspect' containers by CREATED 2025-12-12 20:13:02 +10:00
Pasha Sviderski bc65c73c51 chore: add CREATED column to 'uc ps', fallback sorting by CREATED 2025-12-12 20:01:49 +10:00
Pasha Sviderski 596bc561b8 chore: 'uc stop' - make --timeout default explicit, update flag and long descriptions 2025-12-12 19:16:17 +10:00
Pasha Sviderski f215fae6c7 chore: group deploy and service management commands in uc --help 2025-12-12 18:52:08 +10:00
Pasha Sviderski 5707c37b63 docs: replace managed DNS domain: cluster.uncloud.run -> uncld.dev 2025-12-12 18:21:11 +10:00
Pasha Sviderski 5780d3a8a4 docs: regenereate CLI reference 2025-12-10 20:48:28 +10:00
Pasha Sviderski ac18ff3e8b docs: add examples section to uc logs command 2025-12-10 20:47:42 +10:00
a9a720f3c3 feat: Add uc service start/stop commands (#195)
* feat: Add `uc service start/stop` commands

* Add nolint:dupl for service start/stop commands

Co-authored-by: Pasha Sviderski <me@psviderski.name>

* Add --signal and --timeout options to uc service stop

* Check whether service stop --timeout flag was used to set options properly

---------

Co-authored-by: Pasha Sviderski <me@psviderski.name>
2025-12-09 15:58:09 +10:00
4b4b721d49 feat(uc/caddy): expose 443/udp port for HTTP3 (#204)
* feat(caddy): expose 443/udp port for HTTP3

* expose udp port 443 for caddy to support new HTTP3

* chore: Add comment for HTTP/3 (QUIC) port configuration

Added a comment to clarify the purpose of the UDP port configuration.

---------

Co-authored-by: Anton Ovchinnikov <anton@tonyo.info>
2025-12-08 19:21:30 +10:00
Justin BradfordandGitHub 6373d4097c fix: volume driver options are not used if driver if not specified [bug #210] (#211) 2025-12-08 15:58:43 +10:00
Pasha Sviderski dfe12ca644 docs: add #1 repo of the day badge to README 2025-12-08 12:23:50 +10:00
Pasha Sviderski 2ef8a6fd20 docs: update landing value section 2025-12-05 00:28:37 +10:00
Pasha Sviderski 27cfe1c4f3 feat: 'uc logs' without argements streams logs for services in the local Compose file 2025-12-04 15:40:24 +10:00
Pasha Sviderski ebd4622592 lint 2025-12-04 14:26:25 +10:00
Pasha Sviderski 4fa30ab8cb test: e2e test for client.ServiceLogs 2025-12-04 14:24:10 +10:00
Pasha Sviderski 1e6aaf451e feat: support filtering service logs by machine (-m/--machine) 2025-12-03 15:01:23 +10:00
Pasha Sviderski bf65fbaca5 feat --utc flag for 'uc logs' command to print timestamps in UTC 2025-12-03 12:52:20 +10:00
79dc05cb66 feat: service logs command to stream service logs (#196)
* chore: fix landing logo shadow

* feat: implement service logs command with colored output and strict ordering

* revert Makefile

* fix after rebase

* move logs command under services with root shortcut

* simplify server options for streaming logs and CLI flags

* refactor ContainerLogs grpc server

* update --tail flag

* minor docker.proto

* refactor client ServiceLogs and ContainerLogs

* move ProxyMachinesContext from api to client pkg

* minor refactor proto Stream

* add api/logs

* implement LogMerger

* fix LogMerger to correctly use semaphore

* increate inflish entries to 100 per stream

* send heartbeats

* refactor ContainerLogs to synchronise Send of entries and heartbeats to the steram

* minor logmerege

* remove ContainerName form ServiceLogEntryMetadata

* refactor ContainerLogs into docker.Service

* detect stalled container logs streams

* update logmerger tests

* fix comment in test

* refactor LogMerger with options

* uc logs: format one or multiple services

* make LogMerger emit heartbeats, emit entries <= watermark, rewrite tests

* update uc logs with new LogMerger

* go mod tidy

* fix after merge

---------

Co-authored-by: Evgenii Orlov <evgenii.orlov@semrush.com>
2025-12-02 19:50:48 +10:00
Anton OvchinnikovandGitHub 234985b57d feat(install): Add install-only mode to the install script (#194) 2025-11-30 20:00:33 +01:00
Justin BradfordandGitHub d89bfb8e03 feat: add uc ps command to list all containers in the cluster (#165)
* feat: Add `uc ps` command to list all containers in the cluster

* Fix indentation

* Improvements to the container state highlights code (use enum, better var naming)

* Add group/sort option by "health"

* Rename `uc ps --group-by` flag to `--sort`. Also, remove `--context` flag.

* Use huh.spinner with standard style/type

* Remove pointless TrimPrefix on container name

* Move machine lookup out of services loop

* Use more efficient `ListServiceContainers`

* Use consistent header other

* Use api.ProxyMachinesContext helper for context to query all machines for service containers

* Make errored MachineServiceContainer entries a warning, not return an error immediately, in `uc ps`

* Use lipgloss table instead of tabwriter for "uc ps" command

* Handle nil metadata in `uc ps` on container when cluster only has one machine
2025-11-27 20:05:58 +10:00
91a09d5d37 feat: add CLI connector for unix domain socket (#186)
* feat: Add cli connector for unix domain socket

* Update cmd/uncloud/main.go

---------

Co-authored-by: Pasha Sviderski <me@psviderski.name>
2025-11-25 21:45:07 +10:00
Pasha Sviderski e87207eae3 chore: fix landing logo shadow 2025-11-24 09:40:36 +10:00
Anton Ovchinnikov 3c33973269 chore(exec): Update TODO regarding signal forwarding 2025-11-23 23:58:53 +01:00
Pasha Sviderski 3019377310 chore: update logo for docs 2025-11-21 22:41:17 +10:00
Pasha Sviderski 53def3a624 chore: update white color in logo-title-dark.svg 2025-11-21 22:38:13 +10:00
Pasha Sviderski eb23e956cb docs: README subtitle minor 2025-11-21 22:33:09 +10:00
Pasha Sviderski d029d4d0bf docs: update uncloud logo with better spacing 2025-11-21 22:26:29 +10:00
Pasha Sviderski 4433095045 chore: show links to docs and Discord for the root 'uc --help' command 2025-11-21 22:08:32 +10:00
Pasha Sviderski 05c8117f7b chore: add redirect from uncloud.run/discord to discord invite 2025-11-21 21:03:48 +10:00
Pasha Sviderski 3fbba6d540 chore: sort services in 'uc ls' output by name 2025-11-21 18:11:20 +10:00
Pasha Sviderski 439d956ac2 docs: Deploy an app: build secrets are not supported (resolves #181) 2025-11-20 17:32:37 +10:00
Pasha Sviderski 055cfae272 docs: Compose compatibility matrix: deploy.rollback_config and deploy.update_config are not supported (resolves #179) 2025-11-20 17:15:40 +10:00
Justin BradfordandGitHub 76b4369aa6 feat: store machine id on connection entries in Uncloud config so it can be removed with machine (#182)
* fix: Store machine id on connection entries so it can be removed when machine is

* Add "GetContextOverrideOrCurrent" helper method to cli and use in `uc machine rm` to clean up connections
2025-11-20 16:49:37 +10:00
Pasha Sviderski 74c7738bc3 docs: update 'uc service rm' description that volumes are preserved 2025-11-20 16:36:36 +10:00
Pasha Sviderski f6d75d507c lint 2025-11-20 16:04:06 +10:00
Pasha Sviderski 6a1cd7bf87 refactor: replace loadProjectFromContent with compose.LoadProjectFromContent 2025-11-20 16:03:47 +10:00
Pasha Sviderski 57205e1149 fix: new Compose deployment with volumes and --recreate flag (fixes #176) 2025-11-20 15:19:44 +10:00
Pasha Sviderski ce4a8cce72 docs: regenereate CLI reference 2025-11-20 10:53:37 +10:00
Justin BradfordandGitHub 75085b3929 refactor: Simplify dns e2e test using new ExecContainer functionality (#178)
* refactor: Simplify dns e2e test using new ExecContainer functionality

* Fix linter errors
2025-11-20 08:43:58 +10:00
Pasha Sviderski c489fcc63c docs: typo 2025-11-20 08:06:26 +10:00
Pasha Sviderski e30ea89990 chore: enable CGO to be able to build fsevent required by updated compose on macOS 2025-11-20 08:06:26 +10:00
Pasha Sviderski 9ddcbd6de0 docs: render CLI reference 2025-11-20 08:06:26 +10:00
Justin BradfordandGitHub 939645e144 refactor: Make --context cli flag a global option (#174)
* Make `--context` flag a global option and remove duplication

* Remove unused dns showOptions struct

* First pass of refactoring "active context" determination and validation

* Undo some refactoring in cli.go to keep diff more focused
2025-11-19 20:30:32 +10:00
Justin BradfordandGitHub 1d413734d8 feat: Interactive 'uc ctx connection` command to select preferred cluster connection (#170)
* feat: Interactive uc command to select preferred cluster connection

https://github.com/psviderski/uncloud/issues/125

* Simplify connection selection and reordering by using the index instead of a pointer

* Move connection reordering logic to a new SetDefaultConnection on context
2025-11-19 13:53:29 +10:00
JiraliteandGitHub 2e655d870c chore: extract version from latest redirect in install_cli.sh (#180) 2025-11-18 13:31:35 +10:00
Luis LavenaandGitHub eea81723b8 feat(ssh+cli): Support SSH CLI on machine init and add commands (#173)
* Add SSHCLIRemote type with buildSSHArgs

Implement's Executor interface (Run, Stream, Close)

* Add UseSSHCLI field to RemoteMachine struct

Allow us to differentiate regular ssh from ssh+cli.

* Parse ssh+cli:// scheme on machine init/add

* Support SSH CLI when provisioning

* Update docs to reflect ssh+cli:// support

* Revert removal of `init` examples

Fixes copy & pasta mistake when documenting the new ssh+cli connection
methods.
2025-11-17 20:11:31 +10:00
146 changed files with 6308 additions and 2637 deletions
+2 -1
View File
@@ -16,7 +16,8 @@ builds:
main: ./cmd/uncloud
binary: uncloud
env:
- CGO_ENABLED=0
# Use CGO to be able to build fsevent required by compose on macOS.
- CGO_ENABLED={{ if eq .Os "darwin" }}1{{ else }}0{{ end }}
goos:
- darwin
- linux
+1 -1
View File
@@ -28,7 +28,7 @@ This document provides comprehensive information about the Uncloud project for A
- **Docker Compose compatibility**: Uses familiar Docker Compose format
- **Zero-downtime deployments**: Rolling updates without service interruption
- **Automatic HTTPS**: Caddy reverse proxy with Let's Encrypt integration
- **Managed DNS**: Free `*.cluster.uncloud.run` subdomains via Uncloud DNS service
- **Managed DNS**: Free `*.xxxxxx.uncld.dev` subdomains via Uncloud DNS service
- **Cross-machine scaling**: Run containers across multiple machines
### 🔧 Developer Experience
+8 -4
View File
@@ -1,7 +1,7 @@
<div align="center">
<img src="./website/landing/images/logo.svg" height="100" width="100" alt="Uncloud logo"/>
<h1>Uncloud</h1>
<p><strong>Docker simplicity. Multi-machine power.</strong></p>
<img src="./website/landing/images/logo-title.svg#gh-light-mode-only" alt="Uncloud logo"/>
<img src="./website/landing/images/logo-title-dark.svg#gh-dark-mode-only" alt="Uncloud logo"/>
<p><strong>Docker simplicity. Multi-machine power</strong></p>
<p>
<a href="https://uncloud.run/docs"><img src="https://img.shields.io/badge/Docs-blue.svg?style=for-the-badge&logo=gitbook&logoColor=white" alt="Documentation"></a>
@@ -40,7 +40,7 @@ complexity of Kubernetes.
* **No control plane**: Fully decentralised design eliminates single points of failure and reduces operational overhead.
* **Imperative over declarative**: Favoring imperative operations over state reconciliation simplifies both the mental
model and troubleshooting.
* **Managed DNS**: Automatic DNS records `*.<id>.cluster.uncloud.run` for services with public access via managed
* **Managed DNS**: Automatic DNS records `*.xxxxxx.uncld.dev` for services with public access via managed
[Uncloud DNS](https://github.com/psviderski/uncloud-dns) service.
* **Automatic HTTPS**: Built-in Caddy reverse proxy handles TLS certificate provisioning and renewal using Let's
Encrypt.
@@ -337,6 +337,10 @@ These companies and projects are helping Uncloud with their generous sponsorship
## ❤️ Contributors
<a href="https://trendshift.io/repositories/14069">
<img alt="Trendshift" src="https://trendshift.io/api/badge/repositories/14069" />
</a>
Thank you [@cedws](https://github.com/cedws) for being the first contributor to Uncloud! 🎉
<a href="https://github.com/psviderski/uncloud/graphs/contributors">
+1 -2
View File
@@ -49,6 +49,7 @@ to cluster machines or --push-registry to upload them to external registries.`,
return runBuild(cmd.Context(), uncli, opts)
},
GroupID: "service",
}
cmd.Flags().StringArrayVar(&opts.BuildArgs, "build-arg", nil,
@@ -74,8 +75,6 @@ to cluster machines or --push-registry to upload them to external registries.`,
"Use --machine to specify which machines. (default is all machines)")
cmd.Flags().BoolVar(&opts.PushRegistry, "push-registry", false,
"Upload the built images to external registries (e.g., Docker Hub) after building.")
cmd.Flags().StringVarP(&opts.Context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd
}
+2 -8
View File
@@ -7,14 +7,12 @@ import (
"github.com/alecthomas/chroma/v2/quick"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type configOptions struct {
machine string
noColor bool
context string
}
func NewConfigCommand() *cobra.Command {
@@ -34,16 +32,12 @@ func NewConfigCommand() *cobra.Command {
"Name or ID of the machine to get the configuration from. (default is connected machine)")
cmd.Flags().BoolVar(&opts.noColor, "no-color", false,
"Disable syntax highlighting for the output.")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func runConfig(ctx context.Context, uncli *cli.CLI, opts configOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
@@ -51,7 +45,7 @@ func runConfig(ctx context.Context, uncli *cli.CLI, opts configOptions) error {
if opts.machine != "" {
// If a specific machine is requested, use it to get the Caddy configuration.
ctx, _, err = api.ProxyMachinesContext(ctx, clusterClient, []string{opts.machine})
ctx, _, err = clusterClient.ProxyMachinesContext(ctx, []string{opts.machine})
if err != nil {
return err
}
+1 -6
View File
@@ -22,7 +22,6 @@ type deployOptions struct {
caddyfile string
image string
machines []string
context string
}
func NewDeployCommand() *cobra.Command {
@@ -46,10 +45,6 @@ func NewDeployCommand() *cobra.Command {
cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil,
"Machine names or IDs to deploy to. Can be specified multiple times or as a comma-separated "+
"list. (default is all machines)")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context to deploy to. (default is the current context)",
)
return cmd
}
@@ -64,7 +59,7 @@ func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
caddyfile = strings.TrimSpace(string(data))
}
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+78
View File
@@ -0,0 +1,78 @@
package context
import (
"fmt"
"github.com/charmbracelet/huh"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/internal/cli/config"
"github.com/spf13/cobra"
)
func NewConnectionCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "connection",
Short: "Choose a new default connection for the current context.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return selectConnection(uncli)
},
}
return cmd
}
func selectConnection(uncli *cli.CLI) error {
if uncli.Config == nil {
return fmt.Errorf("connection management is not available: Uncloud configuration file is not being used")
}
if len(uncli.Config.Contexts) == 0 {
return fmt.Errorf("no contexts found in Uncloud config (%s)", uncli.Config.Path())
}
currentCtxName := uncli.Config.CurrentContext
currentCtx, ok := uncli.Config.Contexts[currentCtxName]
if !ok {
return fmt.Errorf("current context '%s' not found", currentCtxName)
}
if len(currentCtx.Connections) == 0 {
return fmt.Errorf("no connections found in context '%s'", currentCtxName)
}
var selectedConnection int
form := huh.NewForm(
huh.NewGroup(
huh.NewSelect[int]().
Title("Select a default connection").
Options(buildConnectionOptions(currentCtx.Connections)...).
Value(&selectedConnection),
),
)
if err := form.Run(); err != nil {
return fmt.Errorf("select connection: %w", err)
}
currentCtx.SetDefaultConnection(selectedConnection)
selected := currentCtx.Connections[0]
if err := uncli.Config.Save(); err != nil {
return fmt.Errorf("save config: %w", err)
}
fmt.Printf("Default connection for context '%s' is now '%s'.\n", currentCtxName, selected)
return nil
}
func buildConnectionOptions(connections []config.MachineConnection) []huh.Option[int] {
options := make([]huh.Option[int], len(connections))
for i, conn := range connections {
key := conn.String()
opt := huh.NewOption(key, i)
if i == 0 {
opt.Key += " (default)"
opt = opt.Selected(true)
}
options[i] = opt
}
return options
}
+1
View File
@@ -19,6 +19,7 @@ func NewRootCommand() *cobra.Command {
cmd.AddCommand(
NewListCommand(),
NewUseCommand(),
NewConnectionCommand(),
)
return cmd
+2 -5
View File
@@ -26,8 +26,6 @@ type deployOptions struct {
noBuild bool
recreate bool
yes bool
context string
}
// NewDeployCommand creates a new command to deploy services from a Compose file.
@@ -44,6 +42,7 @@ func NewDeployCommand() *cobra.Command {
return runDeploy(cmd.Context(), uncli, opts)
},
GroupID: "service",
}
cmd.Flags().StringArrayVar(&opts.BuildServicesOptions.BuildArgs, "build-arg", nil,
@@ -51,8 +50,6 @@ func NewDeployCommand() *cobra.Command {
"Can be specified multiple times. Format: --build-arg VAR=VALUE")
cmd.Flags().BoolVar(&opts.BuildServicesOptions.Pull, "build-pull", false,
"Always attempt to pull newer versions of base images before building service images.")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context to deploy to (default is the current context)")
cmd.Flags().StringSliceVarP(&opts.files, "file", "f", nil,
"One or more Compose files to deploy services from. (default compose.yaml)")
cmd.Flags().BoolVar(&opts.noBuild, "no-build", false,
@@ -108,7 +105,7 @@ func runDeploy(ctx context.Context, uncli *cli.CLI, opts deployOptions) error {
fmt.Println()
}
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+3 -14
View File
@@ -12,32 +12,21 @@ import (
"google.golang.org/protobuf/types/known/emptypb"
)
type releaseOptions struct {
context string
}
func NewReleaseCommand() *cobra.Command {
opts := releaseOptions{}
cmd := &cobra.Command{
Use: "release",
Short: "Release the reserved cluster domain.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return release(cmd.Context(), uncli, opts)
return release(cmd.Context(), uncli)
},
}
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func release(ctx context.Context, uncli *cli.CLI, opts releaseOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
func release(ctx context.Context, uncli *cli.CLI) error {
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+1 -6
View File
@@ -19,7 +19,6 @@ const DefaultUncloudDNSAPIEndpoint = "https://dns.uncloud.run/v1"
type reserveOptions struct {
endpoint string
context string
}
func NewReserveCommand() *cobra.Command {
@@ -36,16 +35,12 @@ func NewReserveCommand() *cobra.Command {
cmd.Flags().StringVar(&opts.endpoint, "endpoint", DefaultUncloudDNSAPIEndpoint,
"API endpoint for the Uncloud DNS service.")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func reserve(ctx context.Context, uncli *cli.CLI, opts reserveOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+1 -1
View File
@@ -9,7 +9,7 @@ func NewRootCommand() *cobra.Command {
Use: "dns",
Short: "Manage cluster domain in Uncloud DNS.",
Long: "Manage cluster domain in Uncloud DNS.\n" +
"DNS commands allow you to reserve or release a unique '<id>.cluster.uncloud.run' domain for your " +
"DNS commands allow you to reserve or release a unique 'xxxxxx.uncld.dev' domain for your " +
"cluster. When reserved, Caddy service deployments will automatically update DNS records to route " +
"traffic to the services in the cluster.",
}
+3 -14
View File
@@ -10,32 +10,21 @@ import (
"github.com/spf13/cobra"
)
type showOptions struct {
context string
}
func NewShowCommand() *cobra.Command {
opts := showOptions{}
cmd := &cobra.Command{
Use: "show",
Short: "Print the cluster domain name.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return show(cmd.Context(), uncli, opts)
return show(cmd.Context(), uncli)
},
}
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func show(ctx context.Context, uncli *cli.CLI, opts showOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
func show(ctx context.Context, uncli *cli.CLI) error {
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+1 -6
View File
@@ -22,7 +22,6 @@ import (
type listOptions struct {
machines []string
nameFilter string
context string
}
func NewListCommand() *cobra.Command {
@@ -60,10 +59,6 @@ func NewListCommand() *cobra.Command {
cmd.Flags().StringSliceVarP(&opts.machines, "machine", "m", nil,
"Filter images by machine name or ID. Can be specified multiple times or as a comma-separated list. "+
"(default is include all machines)")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
@@ -82,7 +77,7 @@ type imageRow struct {
}
func list(ctx context.Context, uncli *cli.CLI, opts listOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+1 -6
View File
@@ -15,7 +15,6 @@ type pushOptions struct {
image string
machines []string
platform string
context string
}
func NewPushCommand() *cobra.Command {
@@ -54,16 +53,12 @@ The image is uploaded to all cluster machines (default) or the specified machine
"Push a specific platform of a multi-platform image (e.g., linux/amd64, linux/arm64).\n"+
"Local Docker must be configured to use containerd image store to support multi-platform images.",
)
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func push(ctx context.Context, uncli *cli.CLI, opts pushOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+19 -12
View File
@@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"net/netip"
"strings"
"time"
"github.com/cenkalti/backoff/v4"
@@ -25,7 +26,6 @@ type addOptions struct {
noInstall bool
publicIP string
sshKey string
context string
version string
}
@@ -34,19 +34,31 @@ func NewAddCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "add [USER@]HOST[:PORT]",
Short: "Add a remote machine to a cluster.",
Args: cobra.ExactArgs(1),
Long: `Add a new machine to an existing Uncloud cluster.
Connection methods:
ssh://user@host - Use built-in SSH library (default, no prefix required)
ssh+cli://user@host - Use system SSH command (supports ProxyJump, SSH config)`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
user, host, port, err := config.SSHDestination(args[0]).Parse()
// Determine if SSH CLI needs to be used and strip scheme
destination := args[0]
useSSHCLI := strings.HasPrefix(destination, "ssh+cli://")
destination = strings.TrimPrefix(destination, "ssh+cli://")
destination = strings.TrimPrefix(destination, "ssh://")
user, host, port, err := config.SSHDestination(destination).Parse()
if err != nil {
return fmt.Errorf("parse remote machine: %w", err)
}
remoteMachine := &cli.RemoteMachine{
User: user,
Host: host,
Port: port,
KeyPath: opts.sshKey,
User: user,
Host: host,
Port: port,
KeyPath: opts.sshKey,
UseSSHCLI: useSSHCLI,
}
return add(cmd.Context(), uncli, remoteMachine, opts)
@@ -76,10 +88,6 @@ func NewAddCommand() *cobra.Command {
&opts.version, "version", "latest",
"Version of the Uncloud daemon to install on the machine.",
)
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context to add the machine to. (default is the current context)",
)
return cmd
}
@@ -100,7 +108,6 @@ func add(ctx context.Context, uncli *cli.CLI, remoteMachine *cli.RemoteMachine,
}
clusterClient, machineClient, err := uncli.AddMachine(ctx, cli.AddMachineOptions{
Context: opts.context,
MachineName: opts.name,
PublicIP: publicIP,
RemoteMachine: remoteMachine,
+20 -8
View File
@@ -4,6 +4,7 @@ import (
"context"
"fmt"
"net/netip"
"strings"
"github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/cmd/uncloud/caddy"
@@ -34,8 +35,12 @@ func NewInitCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "init [USER@HOST:PORT]",
Short: "Initialise a new cluster with a remote machine as the first member.",
Long: "Initialise a new cluster by setting up a remote machine as the first member.\n" +
"This command creates a new context in your Uncloud config to manage the cluster.",
Long: `Initialise a new cluster by setting up a remote machine as the first member.
This command creates a new context in your Uncloud config to manage the cluster.
Connection methods:
ssh://user@host - Use built-in SSH library (default, no prefix required)
ssh+cli://user@host - Use system SSH command (supports ProxyJump, SSH config)`,
Example: ` # Initialise a new cluster with default settings.
uc machine init root@<your-server-ip>
@@ -45,7 +50,7 @@ func NewInitCommand() *cobra.Command {
# Initialise with a non-root user and custom SSH port and key.
uc machine init ubuntu@<your-server-ip>:2222 -i ~/.ssh/mykey
# Initialise without Caddy (no reverse proxy) and without an automatically managed domain name (xxxxxx.cluster.uncloud.run).
# Initialise without Caddy (no reverse proxy) and without an automatically managed domain name (xxxxxx.uncld.dev).
# You can deploy Caddy with 'uc caddy deploy' and reserve a domain with 'uc dns reserve' later.
uc machine init root@<your-server-ip> --no-caddy --no-dns`,
// TODO: support initialising a cluster on the local machine.
@@ -55,15 +60,22 @@ func NewInitCommand() *cobra.Command {
var remoteMachine *cli.RemoteMachine
if len(args) > 0 {
user, host, port, err := config.SSHDestination(args[0]).Parse()
// Determine if SSH CLI is requested and strip scheme
destination := args[0]
useSSHCLI := strings.HasPrefix(destination, "ssh+cli://")
destination = strings.TrimPrefix(destination, "ssh+cli://")
destination = strings.TrimPrefix(destination, "ssh://")
user, host, port, err := config.SSHDestination(destination).Parse()
if err != nil {
return fmt.Errorf("parse remote machine: %w", err)
}
remoteMachine = &cli.RemoteMachine{
User: user,
Host: host,
Port: port,
KeyPath: opts.sshKey,
User: user,
Host: host,
Port: port,
KeyPath: opts.sshKey,
UseSSHCLI: useSSHCLI,
}
}
+3 -8
View File
@@ -14,25 +14,20 @@ import (
)
func NewListCommand() *cobra.Command {
var contextName string
cmd := &cobra.Command{
Use: "ls",
Aliases: []string{"list"},
Short: "List machines in a cluster.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return list(cmd.Context(), uncli, contextName)
return list(cmd.Context(), uncli)
},
}
cmd.Flags().StringVarP(
&contextName, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func list(ctx context.Context, uncli *cli.CLI, clusterName string) error {
client, err := uncli.ConnectCluster(ctx, clusterName)
func list(ctx context.Context, uncli *cli.CLI) error {
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+3 -8
View File
@@ -9,7 +9,6 @@ import (
)
func NewRenameCommand() *cobra.Command {
var contextName string
cmd := &cobra.Command{
Use: "rename OLD_NAME NEW_NAME",
Short: "Rename a machine in the cluster.",
@@ -20,18 +19,14 @@ configuration including network settings, public IP, and cluster membership.`,
Args: cobra.ExactArgs(2),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return rename(cmd.Context(), uncli, contextName, args[0], args[1])
return rename(cmd.Context(), uncli, args[0], args[1])
},
}
cmd.Flags().StringVarP(
&contextName, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func rename(ctx context.Context, uncli *cli.CLI, contextName, oldName, newName string) error {
client, err := uncli.ConnectCluster(ctx, contextName)
func rename(ctx context.Context, uncli *cli.CLI, oldName, newName string) error {
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return err
}
+17 -7
View File
@@ -22,7 +22,6 @@ import (
type removeOptions struct {
noReset bool
yes bool
context string
}
func NewRmCommand() *cobra.Command {
@@ -39,8 +38,6 @@ func NewRmCommand() *cobra.Command {
},
}
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
"Do not prompt for confirmation before removing the machine.")
cmd.Flags().BoolVar(&opts.noReset, "no-reset", false,
@@ -51,14 +48,14 @@ func NewRmCommand() *cobra.Command {
func remove(ctx context.Context, uncli *cli.CLI, nameOrID string, opts removeOptions) error {
// TODO: automatically choose a connection to the machine that is not being removed.
client, err := uncli.ConnectCluster(ctx, opts.context)
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
// Verify the machine exists and list all service containers on it including stopped ones.
mctx, machines, err := api.ProxyMachinesContext(ctx, client, []string{nameOrID})
mctx, machines, err := client.ProxyMachinesContext(ctx, []string{nameOrID})
if err != nil {
return err
}
@@ -155,8 +152,21 @@ func remove(ctx context.Context, uncli *cli.CLI, nameOrID string, opts removeOpt
}
}
// TODO: remove the connection to the machine from the uncloud config if it exists. We need a way to associate
// the machine with its connection in the config, e.g. by storing the machine name in the connection metadata.
// Remove the connection to the machine from the uncloud config if it exists.
if uncli.Config != nil {
contextName := uncli.GetContextOverrideOrCurrent()
if context, ok := uncli.Config.Contexts[contextName]; ok {
for i, c := range context.Connections {
if c.MachineID == m.Id {
context.Connections = slices.Delete(context.Connections, i, i+1)
break
}
}
if err := uncli.Config.Save(); err != nil {
return fmt.Errorf("save config: %w", err)
}
}
}
// TODO: If Caddy was running on this machine and a cluster domain is reserved,
// let the user know that the DNS records should be updated.
+1 -1
View File
@@ -8,7 +8,7 @@ func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "machine",
Aliases: []string{"m"},
Short: "Manage machines in an Uncloud cluster.",
Short: "Manage machines in the cluster.",
}
cmd.AddCommand(
NewAddCommand(),
+1 -6
View File
@@ -13,7 +13,6 @@ import (
type updateOptions struct {
name string
publicIP string
context string
}
func NewUpdateCommand() *cobra.Command {
@@ -43,10 +42,6 @@ At least one flag must be specified to perform an update operation.`,
&opts.publicIP, "public-ip", "",
fmt.Sprintf("Public IP address of the machine for ingress configuration. Use '%s' or '' to remove the public IP.", PublicIPNone),
)
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
@@ -57,7 +52,7 @@ func update(ctx context.Context, uncli *cli.CLI, cmd *cobra.Command, opts update
return fmt.Errorf("at least one update flag must be specified (--name, --public-ip)")
}
client, err := uncli.ConnectCluster(ctx, opts.context)
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return err
}
+46 -14
View File
@@ -6,6 +6,7 @@ import (
"net/netip"
"strings"
"github.com/charmbracelet/lipgloss"
"github.com/psviderski/uncloud/cmd/uncloud/caddy"
cmdcontext "github.com/psviderski/uncloud/cmd/uncloud/context"
"github.com/psviderski/uncloud/cmd/uncloud/dns"
@@ -24,6 +25,7 @@ import (
type globalOptions struct {
configPath string
connect string
context string
}
func main() {
@@ -39,6 +41,7 @@ func main() {
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
cli.BindEnvToFlag(cmd, "connect", "UNCLOUD_CONNECT")
cli.BindEnvToFlag(cmd, "uncloud-config", "UNCLOUD_CONFIG")
cli.BindEnvToFlag(cmd, "context", "UNCLOUD_CONTEXT")
var conn *config.MachineConnection
if opts.connect != "" {
@@ -55,11 +58,12 @@ func main() {
conn = &config.MachineConnection{
SSHCLI: config.SSHDestination(dest),
}
} else {
dest := opts.connect
if strings.HasPrefix(dest, "ssh://") {
dest = dest[len("ssh://"):]
} else if strings.HasPrefix(opts.connect, "unix://") {
conn = &config.MachineConnection{
Unix: opts.connect[len("unix://"):],
}
} else {
dest := strings.TrimPrefix(opts.connect, "ssh://")
conn = &config.MachineConnection{
SSH: config.SSHDestination(dest),
}
@@ -67,7 +71,7 @@ func main() {
}
configPath := fs.ExpandHomeDir(opts.configPath)
uncli, err := cli.New(configPath, conn)
uncli, err := cli.New(configPath, conn, opts.context)
if err != nil {
return fmt.Errorf("initialise CLI: %w", err)
}
@@ -78,29 +82,57 @@ func main() {
cmd.PersistentFlags().StringVar(&opts.connect, "connect", "",
"Connect to a remote cluster machine without using the Uncloud configuration file. [$UNCLOUD_CONNECT]\n"+
"Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], or tcp://host:port")
"Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], tcp://host:port, or unix:///path/to/uncloud.sock")
cmd.PersistentFlags().StringVar(&opts.configPath, "uncloud-config", "~/.config/uncloud/config.yaml",
"Path to the Uncloud configuration file. [$UNCLOUD_CONFIG]")
_ = cmd.MarkPersistentFlagFilename("uncloud-config", "yaml", "yml")
// TODO: make --context a global flag and pass it as a value of the command context.
cmd.PersistentFlags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context to use (default is the current context). [$UNCLOUD_CONTEXT]")
// Set custom help function to show links to docs and Discord only for the root 'uc' command.
defaultHelpFunc := cmd.HelpFunc()
cmd.SetHelpFunc(func(c *cobra.Command, args []string) {
defaultHelpFunc(c, args)
// Only show links for the root 'uc' command.
if c.Name() == "uc" {
urlStyle := lipgloss.NewStyle().
Underline(true).
Foreground(lipgloss.Color("12")) // light blue
fmt.Fprintln(c.OutOrStdout())
fmt.Fprintf(c.OutOrStdout(), "Learn more about Uncloud: %s\n",
urlStyle.Render("https://uncloud.run/docs"))
fmt.Fprintf(c.OutOrStdout(), "Join our Discord community: %s\n",
urlStyle.Render("https://uncloud.run/discord"))
}
})
cmd.AddGroup(&cobra.Group{
ID: "service",
Title: "Deploy and manage services:",
})
cmd.AddCommand(
NewBuildCommand(),
NewDeployCommand(),
NewDocsCommand(),
NewBuildCommand(),
NewImagesCommand(),
NewPsCommand(),
caddy.NewRootCommand(),
cmdcontext.NewRootCommand(),
dns.NewRootCommand(),
image.NewRootCommand(),
machine.NewRootCommand(),
service.NewRootCommand(),
service.NewExecCommand(),
service.NewInspectCommand(),
service.NewListCommand(),
service.NewRmCommand(),
service.NewRunCommand(),
service.NewScaleCommand(),
service.NewExecCommand("service"),
service.NewInspectCommand("service"),
service.NewListCommand("service"),
service.NewLogsCommand("service"),
service.NewRmCommand("service"),
service.NewRunCommand("service"),
service.NewScaleCommand("service"),
service.NewStartCommand("service"),
service.NewStopCommand("service"),
volume.NewRootCommand(),
)
cobra.CheckErr(cmd.Execute())
+284
View File
@@ -0,0 +1,284 @@
package main
import (
"context"
"fmt"
"sort"
"time"
"github.com/charmbracelet/huh/spinner"
"github.com/charmbracelet/lipgloss"
"github.com/charmbracelet/lipgloss/table"
"github.com/docker/docker/api/types/container"
"github.com/docker/go-units"
"github.com/spf13/cobra"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/client"
)
const (
sortByService = "service"
sortByMachine = "machine"
sortByHealth = "health"
)
type containerHighlight int
const (
highlightDanger containerHighlight = iota
highlightWarning
highlightSuccess
highlightNormal
)
type psOptions struct {
sortBy string
}
func NewPsCommand() *cobra.Command {
opts := psOptions{}
cmd := &cobra.Command{
Use: "ps",
Short: "List all service containers.",
Long: `List all service containers across all machines in the cluster.
This command provides a comprehensive overview of all running containers that are part of a service,
making it easy to see the distribution and status of containers across the cluster.`,
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
if opts.sortBy != sortByService && opts.sortBy != sortByMachine && opts.sortBy != sortByHealth {
return fmt.Errorf("invalid value for --sort: %q, must be one of '%s', '%s' or '%s'", opts.sortBy,
sortByService, sortByMachine, sortByHealth)
}
return runPs(cmd.Context(), uncli, opts)
},
GroupID: "service",
}
cmd.Flags().StringVarP(&opts.sortBy, "sort", "s", sortByService,
"Sort containers by 'service', 'machine', or 'health'.")
return cmd
}
type containerInfo struct {
serviceName string
machineName string
id string
name string
image string
status string
highlight containerHighlight
created time.Time
ip string
}
func runPs(ctx context.Context, uncli *cli.CLI, opts psOptions) error {
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer clusterClient.Close()
var containers []containerInfo
err = spinner.New().
Title(" Collecting container info...").
Type(spinner.MiniDot).
Style(lipgloss.NewStyle().Foreground(lipgloss.Color("3"))).
ActionWithErr(func(ctx context.Context) error {
containers, err = collectContainers(ctx, clusterClient)
return err
}).
Run()
if err != nil {
return fmt.Errorf("collect containers: %w", err)
}
// Sort the containers based on the sorting option.
sort.SliceStable(containers, func(i, j int) bool {
a, b := containers[i], containers[j]
switch opts.sortBy {
case sortByHealth:
if a.highlight != b.highlight {
return a.highlight < b.highlight
}
if a.serviceName != b.serviceName {
return a.serviceName < b.serviceName
}
case sortByMachine:
if a.machineName != b.machineName {
return a.machineName < b.machineName
}
if a.serviceName != b.serviceName {
return a.serviceName < b.serviceName
}
default: // sortByService
if a.serviceName != b.serviceName {
return a.serviceName < b.serviceName
}
}
// Fallback to creation time (newest first).
return a.created.After(b.created)
})
return printContainers(containers)
}
func printContainers(containers []containerInfo) error {
t := table.New().
// Remove the default border.
Border(lipgloss.Border{}).
BorderTop(false).
BorderBottom(false).
BorderLeft(false).
BorderRight(false).
BorderHeader(false).
BorderColumn(false).
StyleFunc(func(row, col int) lipgloss.Style {
if row == table.HeaderRow {
return lipgloss.NewStyle().Bold(true).PaddingRight(3)
}
// Regular style for data rows with padding.
return lipgloss.NewStyle().PaddingRight(3)
})
t.Headers("SERVICE", "CONTAINER ID", "CONTAINER NAME", "IMAGE", "CREATED", "STATUS", "IP ADDRESS", "MACHINE")
for _, ctr := range containers {
id := ctr.id
if len(id) > 12 {
id = id[:12]
}
created := units.HumanDuration(time.Now().UTC().Sub(ctr.created)) + " ago"
var statusStyle lipgloss.Style
switch ctr.highlight {
case highlightSuccess:
statusStyle = lipgloss.NewStyle().Foreground(lipgloss.Color("2")) // Green
case highlightDanger:
statusStyle = lipgloss.NewStyle().Foreground(lipgloss.Color("1")) // Red
case highlightWarning:
statusStyle = lipgloss.NewStyle().Foreground(lipgloss.Color("3")) // Yellow
default:
statusStyle = lipgloss.NewStyle() // Default
}
t.Row(
ctr.serviceName,
id,
ctr.name,
ctr.image,
created,
statusStyle.Render(ctr.status),
ctr.ip,
ctr.machineName,
)
}
fmt.Println(t)
return nil
}
func collectContainers(ctx context.Context, cli *client.Client) ([]containerInfo, error) {
listCtx, machines, err := cli.ProxyMachinesContext(ctx, nil)
if err != nil {
return nil, fmt.Errorf("proxy machines context: %w", err)
}
// Create a map of IP to machine name for resolving response metadata
machinesNamesByIP := make(map[string]string)
for _, m := range machines {
if addr, err := m.Machine.Network.ManagementIp.ToAddr(); err == nil {
machinesNamesByIP[addr.String()] = m.Machine.Name
}
}
// List all service containers across all machines in the cluster.
machineContainers, err := cli.Docker.ListServiceContainers(
listCtx, "", container.ListOptions{All: true},
)
if err != nil {
return nil, fmt.Errorf("list service containers: %w", err)
}
var containers []containerInfo
for _, msc := range machineContainers {
machineName := "unknown"
// Metadata can be nil if the request was broadcasted to only one machine.
if msc.Metadata == nil && len(machineContainers) > 1 {
return nil, fmt.Errorf("something went wrong with gRPC proxy: metadata is missing for a machine response")
}
if msc.Metadata != nil && msc.Metadata.Error != "" {
client.PrintWarning(fmt.Sprintf("failed to list containers on machine %s: %s", machineName,
msc.Metadata.Error))
continue
}
if msc.Metadata != nil {
var ok bool
machineName, ok = machinesNamesByIP[msc.Metadata.Machine]
if !ok {
// Fallback to machine's IP as name.
machineName = msc.Metadata.Machine
}
} else {
// Fallback to the first available machine name.
if len(machines) > 0 {
machineName = machines[0].Machine.Name
}
}
for _, ctr := range msc.Containers {
if ctr.Container.State == nil || ctr.Container.Config == nil {
continue
}
status, err := ctr.Container.HumanState()
if err != nil {
return nil, fmt.Errorf("get human state for container %s: %w", ctr.Container.ID, err)
}
var highlight containerHighlight
healthStatus := ""
if ctr.Container.State.Health != nil {
healthStatus = ctr.Container.State.Health.Status
}
if healthStatus == container.Unhealthy || ctr.Container.State.Status == "dead" || ctr.Container.State.OOMKilled || ctr.Container.State.Dead {
highlight = highlightDanger
} else if healthStatus == container.Healthy {
highlight = highlightSuccess
} else if ctr.Container.State.Status == "running" {
highlight = highlightNormal
} else { // Other non-critical but noteworthy states
highlight = highlightWarning
}
created, _ := time.Parse(time.RFC3339Nano, ctr.Container.Created)
ip := ctr.Container.UncloudNetworkIP()
ipStr := ""
// The container might not have an IP if it's not running or uses the host network.
if ip.IsValid() {
ipStr = ip.String()
}
info := containerInfo{
serviceName: ctr.ServiceName(),
machineName: machineName,
id: ctr.Container.ID,
name: ctr.Container.Name,
image: ctr.Container.Config.Image,
status: status,
highlight: highlight,
created: created,
ip: ipStr,
}
containers = append(containers, info)
}
}
return containers, nil
}
+442
View File
@@ -0,0 +1,442 @@
package main
import (
"context"
"encoding/json"
"net/netip"
"testing"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/internal/machine/docker"
"github.com/psviderski/uncloud/pkg/client"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"google.golang.org/grpc"
"google.golang.org/protobuf/types/known/emptypb"
)
// mockDockerClient implements pb.DockerClient
type mockDockerClient struct {
pb.DockerClient // Embed to avoid implementing all methods
listResp *pb.ListServiceContainersResponse
listErr error
}
func (m *mockDockerClient) ListServiceContainers(ctx context.Context, in *pb.ListServiceContainersRequest, opts ...grpc.CallOption) (*pb.ListServiceContainersResponse, error) {
return m.listResp, m.listErr
}
// mockClusterClient implements pb.ClusterClient
type mockClusterClient struct {
pb.ClusterClient // Embed to avoid implementing all methods
machinesResp *pb.ListMachinesResponse
machinesErr error
}
func (m *mockClusterClient) ListMachines(ctx context.Context, in *emptypb.Empty, opts ...grpc.CallOption) (*pb.ListMachinesResponse, error) {
return m.machinesResp, m.machinesErr
}
func TestCollectContainers_NilMetadata(t *testing.T) {
// Setup container data
containerData := map[string]interface{}{
"Id": "container1",
"Name": "test-container",
"Config": map[string]interface{}{
"Image": "test-image",
},
"State": map[string]interface{}{
"Status": "running",
"StartedAt": "2023-01-01T12:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
},
"NetworkSettings": map[string]interface{}{
"Networks": map[string]interface{}{},
},
}
containerJSON, err := json.Marshal(containerData)
require.NoError(t, err)
serviceSpecJSON, err := json.Marshal(map[string]interface{}{})
require.NoError(t, err)
// Setup mocks
mockDocker := &mockDockerClient{
listResp: &pb.ListServiceContainersResponse{
Messages: []*pb.MachineServiceContainers{
{
Metadata: nil, // Simulating the issue: nil metadata
Containers: []*pb.ServiceContainer{
{
Container: containerJSON,
ServiceSpec: serviceSpecJSON,
},
},
},
},
},
}
machineIP := "10.0.0.1"
mockCluster := &mockClusterClient{
machinesResp: &pb.ListMachinesResponse{
Machines: []*pb.MachineMember{
{
Machine: &pb.MachineInfo{
Name: "machine-1",
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(netip.MustParseAddr(machineIP)),
},
},
State: pb.MachineMember_UP,
},
},
},
}
// Construct client with mocks
cli := &client.Client{
Docker: &docker.Client{
GRPCClient: mockDocker,
},
ClusterClient: mockCluster,
}
// Execute
containers, err := collectContainers(context.Background(), cli)
require.NoError(t, err)
// Verify
assert.Len(t, containers, 1)
if len(containers) > 0 {
c := containers[0]
assert.Equal(t, "container1", c.id)
assert.Equal(t, "test-container", c.name)
assert.Equal(t, "machine-1", c.machineName, "Should fall back to the single machine name when metadata is nil")
}
}
func TestCollectContainers_NilMetadata_MultipleMachines_Error(t *testing.T) {
// If we have multiple machines but receive nil metadata, it should return an error as it is ambiguous
// Setup container data
containerData1 := map[string]interface{}{
"Id": "container1",
}
containerJSON1, _ := json.Marshal(containerData1)
containerData2 := map[string]interface{}{
"Id": "container2",
"Config": map[string]interface{}{
"Image": "test-image",
},
"State": map[string]interface{}{
"Status": "running",
"StartedAt": "2023-01-01T12:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
},
"NetworkSettings": map[string]interface{}{
"Networks": map[string]interface{}{},
},
}
containerJSON2, _ := json.Marshal(containerData2)
serviceSpecJSON, _ := json.Marshal(map[string]interface{}{})
// Setup mocks
mockDocker := &mockDockerClient{
listResp: &pb.ListServiceContainersResponse{
Messages: []*pb.MachineServiceContainers{
{
Metadata: nil, // Nil metadata
Containers: []*pb.ServiceContainer{
{
Container: containerJSON1,
ServiceSpec: serviceSpecJSON,
},
},
},
{
Metadata: &pb.Metadata{Machine: "10.0.0.2"},
Containers: []*pb.ServiceContainer{
{
Container: containerJSON2,
ServiceSpec: serviceSpecJSON,
},
},
},
},
},
}
mockCluster := &mockClusterClient{
machinesResp: &pb.ListMachinesResponse{
Machines: []*pb.MachineMember{
{
Machine: &pb.MachineInfo{
Name: "machine-1",
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(netip.MustParseAddr("10.0.0.1")),
},
},
},
{
Machine: &pb.MachineInfo{
Name: "machine-2",
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(netip.MustParseAddr("10.0.0.2")),
},
},
},
},
},
}
// Construct client with mocks
cli := &client.Client{
Docker: &docker.Client{
GRPCClient: mockDocker,
},
ClusterClient: mockCluster,
}
// Execute
_, err := collectContainers(context.Background(), cli)
require.Error(t, err)
assert.Contains(t, err.Error(), "metadata is missing for a machine response")
}
func TestCollectContainers_MetadataPresent_MultipleMachines(t *testing.T) {
// Verify correct mapping of containers to machines when metadata is present
// Setup container data
containerData1 := map[string]interface{}{
"Id": "container1",
"Name": "container-1",
"Config": map[string]interface{}{
"Image": "image-1",
},
"State": map[string]interface{}{
"Status": "running",
"StartedAt": "2023-01-01T12:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
},
"NetworkSettings": map[string]interface{}{
"Networks": map[string]interface{}{},
},
}
containerJSON1, _ := json.Marshal(containerData1)
containerData2 := map[string]interface{}{
"Id": "container2",
"Name": "container-2",
"Config": map[string]interface{}{
"Image": "image-2",
},
"State": map[string]interface{}{
"Status": "running",
"StartedAt": "2023-01-01T12:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
},
"NetworkSettings": map[string]interface{}{
"Networks": map[string]interface{}{},
},
}
containerJSON2, _ := json.Marshal(containerData2)
serviceSpecJSON, _ := json.Marshal(map[string]interface{}{})
// Setup mocks
mockDocker := &mockDockerClient{
listResp: &pb.ListServiceContainersResponse{
Messages: []*pb.MachineServiceContainers{
{
Metadata: &pb.Metadata{Machine: "10.0.0.1"},
Containers: []*pb.ServiceContainer{
{
Container: containerJSON1,
ServiceSpec: serviceSpecJSON,
},
},
},
{
Metadata: &pb.Metadata{Machine: "10.0.0.2"},
Containers: []*pb.ServiceContainer{
{
Container: containerJSON2,
ServiceSpec: serviceSpecJSON,
},
},
},
},
},
}
mockCluster := &mockClusterClient{
machinesResp: &pb.ListMachinesResponse{
Machines: []*pb.MachineMember{
{
Machine: &pb.MachineInfo{
Name: "machine-1",
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(netip.MustParseAddr("10.0.0.1")),
},
},
},
{
Machine: &pb.MachineInfo{
Name: "machine-2",
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(netip.MustParseAddr("10.0.0.2")),
},
},
},
},
},
}
cli := &client.Client{
Docker: &docker.Client{GRPCClient: mockDocker},
ClusterClient: mockCluster,
}
containers, err := collectContainers(context.Background(), cli)
require.NoError(t, err)
assert.Len(t, containers, 2)
// Order is not guaranteed by the map iteration in logic or parallel fetch (though here it's mocked sequential),
// but collectContainers just appends.
// We'll find them by ID.
for _, c := range containers {
if c.id == "container1" {
assert.Equal(t, "machine-1", c.machineName)
} else if c.id == "container2" {
assert.Equal(t, "machine-2", c.machineName)
} else {
t.Errorf("unexpected container id: %s", c.id)
}
}
}
func TestCollectContainers_NilMetadata_NoMachines(t *testing.T) {
// Case: 1 msc with nil metadata but no machines at all
containerData := map[string]interface{}{
"Id": "container1",
"Config": map[string]interface{}{
"Image": "test-image",
},
"State": map[string]interface{}{
"Status": "running",
"StartedAt": "2023-01-01T12:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
},
"NetworkSettings": map[string]interface{}{
"Networks": map[string]interface{}{},
},
}
containerJSON, _ := json.Marshal(containerData)
serviceSpecJSON, _ := json.Marshal(map[string]interface{}{})
mockDocker := &mockDockerClient{
listResp: &pb.ListServiceContainersResponse{
Messages: []*pb.MachineServiceContainers{
{
Metadata: nil,
Containers: []*pb.ServiceContainer{
{
Container: containerJSON,
ServiceSpec: serviceSpecJSON,
},
},
},
},
},
}
// No machines in cluster response
mockCluster := &mockClusterClient{
machinesResp: &pb.ListMachinesResponse{
Machines: []*pb.MachineMember{},
},
}
cli := &client.Client{
Docker: &docker.Client{GRPCClient: mockDocker},
ClusterClient: mockCluster,
}
containers, err := collectContainers(context.Background(), cli)
require.NoError(t, err)
assert.Len(t, containers, 1)
if len(containers) > 0 {
assert.Equal(t, "unknown", containers[0].machineName)
}
}
func TestCollectContainers_MetadataPresent_NotInMapping(t *testing.T) {
// Case: msc with metadata that is not in the IP-to-name mapping
containerData := map[string]interface{}{
"Id": "container1",
"Config": map[string]interface{}{
"Image": "test-image",
},
"State": map[string]interface{}{
"Status": "running",
"StartedAt": "2023-01-01T12:00:00Z",
"FinishedAt": "0001-01-01T00:00:00Z",
},
"NetworkSettings": map[string]interface{}{
"Networks": map[string]interface{}{},
},
}
containerJSON, _ := json.Marshal(containerData)
serviceSpecJSON, _ := json.Marshal(map[string]interface{}{})
mockDocker := &mockDockerClient{
listResp: &pb.ListServiceContainersResponse{
Messages: []*pb.MachineServiceContainers{
{
Metadata: &pb.Metadata{Machine: "10.0.0.99"}, // Unknown IP
Containers: []*pb.ServiceContainer{
{
Container: containerJSON,
ServiceSpec: serviceSpecJSON,
},
},
},
},
},
}
mockCluster := &mockClusterClient{
machinesResp: &pb.ListMachinesResponse{
Machines: []*pb.MachineMember{
{
Machine: &pb.MachineInfo{
Name: "machine-1",
Network: &pb.NetworkConfig{
ManagementIp: pb.NewIP(netip.MustParseAddr("10.0.0.1")),
},
},
},
},
},
}
cli := &client.Client{
Docker: &docker.Client{GRPCClient: mockDocker},
ClusterClient: mockCluster,
}
containers, err := collectContainers(context.Background(), cli)
require.NoError(t, err)
assert.Len(t, containers, 1)
if len(containers) > 0 {
// Should fallback to the IP/string in metadata
assert.Equal(t, "10.0.0.99", containers[0].machineName)
}
}
+4 -7
View File
@@ -15,18 +15,17 @@ type execCliOptions struct {
detach bool
interactive bool
noTty bool
context string
containerId string
}
var DEFAULT_COMMAND = []string{"sh", "-c", "command -v bash >/dev/null 2>&1 && exec bash || exec sh"}
func NewExecCommand() *cobra.Command {
func NewExecCommand(groupID string) *cobra.Command {
opts := execCliOptions{}
execCmd := &cobra.Command{
Use: "exec [OPTIONS] SERVICE [COMMAND ARGS...]",
Short: "Execute a command in a running service container",
Short: "Execute a command in a running service container.",
Long: `Execute a command (interactive shell by default) in a running container within a service.
If the service has multiple replicas and no container ID is specified, the command will be executed in a random container.
`,
@@ -55,6 +54,7 @@ If the service has multiple replicas and no container ID is specified, the comma
}
return runExec(cmd.Context(), uncli, serviceName, command, opts)
},
GroupID: groupID,
}
execCmd.Flags().BoolVarP(&opts.detach, "detach", "d", false, "Detached mode: run command in the background")
@@ -69,9 +69,6 @@ If the service has multiple replicas and no container ID is specified, the comma
execCmd.Flags().BoolP("tty", "t", false, "Allocate a pseudo-TTY")
execCmd.Flags().MarkHidden("tty")
execCmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
// Common flags
execCmd.Flags().StringVar(&opts.containerId, "container", "",
"ID of the container to exec into. Accepts full ID or a unique prefix "+
@@ -95,7 +92,7 @@ func runExec(ctx context.Context, uncli *cli.CLI, serviceName string, command []
}
}
client, err := uncli.ConnectClusterWithOptions(ctx, opts.context, cli.ConnectOptions{
client, err := uncli.ConnectClusterWithOptions(ctx, cli.ConnectOptions{
ShowProgress: false,
})
if err != nil {
+28 -14
View File
@@ -4,21 +4,22 @@ import (
"context"
"fmt"
"os"
"slices"
"text/tabwriter"
"time"
"github.com/docker/docker/pkg/stringid"
"github.com/docker/go-units"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
type inspectOptions struct {
service string
context string
}
func NewInspectCommand() *cobra.Command {
func NewInspectCommand(groupID string) *cobra.Command {
opts := inspectOptions{}
cmd := &cobra.Command{
Use: "inspect SERVICE",
@@ -29,16 +30,13 @@ func NewInspectCommand() *cobra.Command {
opts.service = args[0]
return inspect(cmd.Context(), uncli, opts)
},
GroupID: groupID,
}
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func inspect(ctx context.Context, uncli *cli.CLI, opts inspectOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
@@ -63,18 +61,26 @@ func inspect(ctx context.Context, uncli *cli.CLI, opts inspectOptions) error {
fmt.Printf("Mode: %s\n", svc.Mode)
fmt.Println()
// Parse created times for sorting and display.
createdTimes := make(map[string]time.Time, len(svc.Containers))
for _, ctr := range svc.Containers {
createdTimes[ctr.Container.ID], _ = time.Parse(time.RFC3339Nano, ctr.Container.Created)
}
// Sort containers by created time (newest first).
slices.SortFunc(svc.Containers, func(a, b api.MachineServiceContainer) int {
return createdTimes[b.Container.ID].Compare(createdTimes[a.Container.ID])
})
// Print the list of containers in a table format.
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
if _, err = fmt.Fprintln(tw, "CONTAINER ID\tIMAGE\tCREATED\tSTATUS\tMACHINE"); err != nil {
if _, err = fmt.Fprintln(tw, "CONTAINER ID\tIMAGE\tCREATED\tSTATUS\tIP ADDRESS\tMACHINE"); err != nil {
return fmt.Errorf("write header: %w", err)
}
now := time.Now().UTC()
for _, ctr := range svc.Containers {
createdAt, err := time.Parse(time.RFC3339Nano, ctr.Container.Created)
if err != nil {
return fmt.Errorf("parse created time: %w", err)
}
created := units.HumanDuration(time.Now().UTC().Sub(createdAt)) + " ago"
created := units.HumanDuration(now.Sub(createdTimes[ctr.Container.ID])) + " ago"
machine := machinesNamesByID[ctr.MachineID]
if machine == "" {
@@ -85,13 +91,21 @@ func inspect(ctx context.Context, uncli *cli.CLI, opts inspectOptions) error {
return fmt.Errorf("get human state: %w", err)
}
ip := ctr.Container.UncloudNetworkIP()
ipStr := ""
// The container might not have an IP if it's not running or uses the host network.
if ip.IsValid() {
ipStr = ip.String()
}
_, err = fmt.Fprintf(
tw,
"%s\t%s\t%s\t%s\t%s\n",
"%s\t%s\t%s\t%s\t%s\t%s\n",
stringid.TruncateID(ctr.Container.ID),
ctr.Container.Config.Image,
created,
state,
ipStr,
machine,
)
if err != nil {
+336
View File
@@ -0,0 +1,336 @@
package service
import (
"context"
"errors"
"fmt"
"os"
"slices"
"strconv"
"strings"
"time"
"github.com/charmbracelet/lipgloss"
mapset "github.com/deckarep/golang-set/v2"
"github.com/docker/docker/pkg/stringid"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client"
"github.com/psviderski/uncloud/pkg/client/compose"
"github.com/spf13/cobra"
)
type logsOptions struct {
files []string
follow bool
tail string
since string
until string
utc bool
machines []string
}
func NewLogsCommand(groupID string) *cobra.Command {
var options logsOptions
cmd := &cobra.Command{
Use: "logs [SERVICE...]",
Aliases: []string{"log"},
Short: "View service logs.",
Long: `View logs from all replicas of the specified service(s) across all machines in the cluster.
If no services are specified, streams logs from all services defined in the Compose file
(compose.yaml by default or the file(s) specified with --file).`,
Example: ` # View recent logs for a service.
uc logs web
# Stream logs in real-time (follow mode).
uc logs -f web
# View logs from multiple services.
uc logs web api db
# View logs from all services in compose.yaml.
uc logs
# Show last 20 lines per replica (default is 100).
uc logs -n 20 web
# Show all logs without line limit.
uc logs -n all web
# View logs from a specific time range.
uc logs --since 3h --until 1h30m web
# View logs only from replicas running on specific machines.
uc logs -m machine1,machine2 web api`,
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return runLogs(cmd.Context(), uncli, args, options)
},
GroupID: groupID,
}
cmd.Flags().StringSliceVar(&options.files, "file", nil,
"One or more Compose files to load service names from when no services are specified. (default compose.yaml)")
cmd.Flags().BoolVarP(&options.follow, "follow", "f", false,
"Continually stream new logs.")
cmd.Flags().StringSliceVarP(&options.machines, "machine", "m", nil,
"Filter logs by machine name or ID. Can be specified multiple times or as a comma-separated list.")
cmd.Flags().StringVar(&options.since, "since", "",
"Show logs generated on or after the given timestamp. Accepts relative duration, RFC 3339 date, or Unix timestamp.\n"+
"Examples:\n"+
" --since 2m30s Relative duration (2 minutes 30 seconds ago)\n"+
" --since 1h Relative duration (1 hour ago)\n"+
" --since 2025-11-24 RFC 3339 date only (midnight using local timezone)\n"+
" --since 2024-05-14T22:50:00 RFC 3339 date/time using local timezone\n"+
" --since 2024-01-31T10:30:00Z RFC 3339 date/time in UTC\n"+
" --since 1763953966 Unix timestamp (seconds since January 1, 1970)")
cmd.Flags().StringVarP(&options.tail, "tail", "n", "100",
"Show the most recent logs and limit the number of lines shown per replica. Use 'all' to show all logs.")
cmd.Flags().StringVar(&options.until, "until", "",
"Show logs generated before the given timestamp. Accepts relative duration, RFC 3339 date, or Unix timestamp.\n"+
"See --since for examples.")
cmd.Flags().BoolVar(&options.utc, "utc", false,
"Print timestamps in UTC instead of local timezone.")
return cmd
}
func runLogs(ctx context.Context, uncli *cli.CLI, serviceNames []string, opts logsOptions) error {
// If no services specified, try to load them from the Compose file(s).
if len(serviceNames) == 0 {
project, err := compose.LoadProject(ctx, opts.files)
if err != nil {
return fmt.Errorf("load compose file(s): %w", err)
}
// View logs for all services, including disabled by inactive profiles.
serviceNames = append(project.ServiceNames(), project.DisabledServiceNames()...)
if len(serviceNames) == 0 {
return errors.New("no services found in compose file(s)")
}
}
// Parse tail option.
tail := -1
if opts.tail != "all" {
tailInt, err := strconv.Atoi(opts.tail)
if err != nil {
return fmt.Errorf("invalid --tail value '%s': %w", opts.tail, err)
}
tail = tailInt
}
c, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer c.Close()
logsOpts := api.ServiceLogsOptions{
Follow: opts.follow,
Tail: tail,
Since: opts.since,
Until: opts.until,
Machines: cli.ExpandCommaSeparatedValues(opts.machines),
}
// Collect log streams from all services.
machineIDsSet := mapset.NewSet[string]()
svcStreams := make([]<-chan api.ServiceLogEntry, 0, len(serviceNames))
for _, serviceName := range serviceNames {
svc, ch, err := c.ServiceLogs(ctx, serviceName, logsOpts)
if err != nil {
return fmt.Errorf("stream logs for service '%s': %w", serviceName, err)
}
svcStreams = append(svcStreams, ch)
machineIDs := svc.MachineIDs()
machineIDsSet.Append(machineIDs...)
}
var stream <-chan api.ServiceLogEntry
if len(serviceNames) == 1 {
stream = svcStreams[0]
} else {
// Merge all service streams into a single sorted stream without stall detection as its handled per-service.
merger := client.NewLogMerger(svcStreams, client.LogMergerOptions{})
stream = merger.Stream()
}
// Fetch machine names for all machines (machineIDsSet) service containers are running on.
machines, err := c.ListMachines(ctx, &api.MachineFilter{NamesOrIDs: machineIDsSet.ToSlice()})
if err != nil {
return fmt.Errorf("list machines: %w", err)
}
machineNames := make([]string, 0, len(machines))
for _, m := range machines {
machineNames = append(machineNames, m.Machine.Name)
}
formatter := newLogFormatter(machineNames, serviceNames, opts.utc)
// Print merged logs.
for entry := range stream {
if entry.Err != nil {
formatter.printError(entry)
continue
}
formatter.printEntry(entry)
}
return nil
}
// Available colors for machine/service differentiation.
var colorPalette = []lipgloss.Color{
lipgloss.Color("10"), // Bright green
lipgloss.Color("11"), // Bright yellow
lipgloss.Color("12"), // Bright blue
lipgloss.Color("13"), // Bright magenta
lipgloss.Color("14"), // Bright cyan
lipgloss.Color("2"), // Green
lipgloss.Color("3"), // Yellow
lipgloss.Color("4"), // Blue
lipgloss.Color("5"), // Magenta
lipgloss.Color("6"), // Cyan
}
// logFormatter handles formatting and printing of log entries with dynamic column alignment.
type logFormatter struct {
machineNames []string
serviceNames []string
maxMachineWidth int
maxServiceWidth int
utc bool
}
func newLogFormatter(machineNames, serviceNames []string, utc bool) *logFormatter {
slices.Sort(machineNames)
slices.Sort(serviceNames)
maxMachineWidth := 0
for _, name := range machineNames {
if len(name) > maxMachineWidth {
maxMachineWidth = len(name)
}
}
maxServiceWidth := 0
for _, name := range serviceNames {
if len(name) > maxServiceWidth {
maxServiceWidth = len(name)
}
}
return &logFormatter{
machineNames: machineNames,
serviceNames: serviceNames,
maxMachineWidth: maxMachineWidth,
maxServiceWidth: maxServiceWidth,
utc: utc,
}
}
// formatTimestamp formats timestamp using local timezone or UTC if configured.
func (f *logFormatter) formatTimestamp(t time.Time) string {
if f.utc {
t = t.UTC()
} else {
t = t.In(time.Local)
}
dimStyle := lipgloss.NewStyle().Faint(true)
return dimStyle.Render(t.Format(time.StampMilli))
}
func (f *logFormatter) formatMachine(name string) string {
style := lipgloss.NewStyle().Bold(true).PaddingRight(f.maxMachineWidth - len(name))
if len(f.serviceNames) == 1 {
// Machine name is coloured for single-service logs.
i := slices.Index(f.machineNames, name)
if i == -1 {
f.machineNames = append(f.machineNames, name)
i = len(f.machineNames) - 1
}
style = style.Foreground(colorPalette[i%len(colorPalette)])
}
return style.Render(name)
}
func (f *logFormatter) formatServiceContainer(serviceName, containerID string) string {
styleService := lipgloss.NewStyle().Bold(true).PaddingRight(f.maxServiceWidth - len(serviceName))
styleContainer := lipgloss.NewStyle().Faint(true)
if len(f.serviceNames) > 1 {
// Service name is coloured for multi-service logs.
i := slices.Index(f.serviceNames, serviceName)
if i == -1 {
f.serviceNames = append(f.serviceNames, serviceName)
i = len(f.serviceNames) - 1
}
styleService = styleService.Foreground(colorPalette[i%len(colorPalette)])
}
return styleService.Render(serviceName) + styleContainer.Render("["+containerID[:5]+"]")
}
// printEntry prints a single log entry with proper formatting.
func (f *logFormatter) printEntry(entry api.ServiceLogEntry) {
if entry.Stream != api.LogStreamStdout && entry.Stream != api.LogStreamStderr {
return
}
var output strings.Builder
// Timestamp
output.WriteString(f.formatTimestamp(entry.Timestamp))
output.WriteString(" ")
// Machine name
output.WriteString(f.formatMachine(entry.Metadata.MachineName))
output.WriteString(" ")
// Service[container_id]
output.WriteString(f.formatServiceContainer(entry.Metadata.ServiceName, entry.Metadata.ContainerID))
output.WriteString(" ")
// Message
output.Write(entry.Message)
// Print to appropriate stream.
if entry.Stream == api.LogStreamStderr {
fmt.Fprint(os.Stderr, output.String())
} else {
fmt.Print(output.String())
}
}
// printError prints an error entry (e.g., stalled stream warning).
func (f *logFormatter) printError(entry api.ServiceLogEntry) {
if entry.Metadata.ContainerID != "" {
msg := fmt.Sprintf("WARNING: log stream from %s[%s] on machine '%s'",
entry.Metadata.ServiceName,
stringid.TruncateID(entry.Metadata.ContainerID),
entry.Metadata.MachineName)
if errors.Is(entry.Err, api.ErrLogStreamStalled) {
msg += " stopped responding"
} else {
msg += fmt.Sprintf(": %v", entry.Err)
}
style := lipgloss.NewStyle().Bold(true).Foreground(lipgloss.Color("11")) // Bold bright yellow
fmt.Fprintln(os.Stderr, style.Render(msg))
} else {
msg := fmt.Sprintf("ERROR: %v", entry.Err)
style := lipgloss.NewStyle().Bold(true).Foreground(lipgloss.Color("9")) // Bold bright red
fmt.Fprintln(os.Stderr, style.Render(msg))
}
}
+13 -15
View File
@@ -4,33 +4,31 @@ import (
"context"
"fmt"
"os"
"slices"
"strings"
"text/tabwriter"
"github.com/psviderski/uncloud/internal/cli"
"github.com/psviderski/uncloud/pkg/api"
"github.com/spf13/cobra"
)
func NewListCommand() *cobra.Command {
var contextName string
func NewListCommand(groupID string) *cobra.Command {
cmd := &cobra.Command{
Use: "ls",
Aliases: []string{"list"},
Short: "List services.",
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
return list(cmd.Context(), uncli, contextName)
return list(cmd.Context(), uncli)
},
GroupID: groupID,
}
cmd.Flags().StringVarP(
&contextName, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func list(ctx context.Context, uncli *cli.CLI, contextName string) error {
client, err := uncli.ConnectCluster(ctx, contextName)
func list(ctx context.Context, uncli *cli.CLI) error {
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
@@ -41,15 +39,15 @@ func list(ctx context.Context, uncli *cli.CLI, contextName string) error {
return fmt.Errorf("list services: %w", err)
}
serviceNames := make(map[string]struct{}, len(services))
// Sort services by name.
haveDuplicateNames := false
for _, svc := range services {
if _, exists := serviceNames[svc.Name]; exists {
slices.SortFunc(services, func(a, b api.Service) int {
if a.Name == b.Name {
haveDuplicateNames = true
break
return strings.Compare(a.ID, b.ID)
}
serviceNames[svc.Name] = struct{}{}
}
return strings.Compare(a.Name, b.Name)
})
// Print the list of services in a table format.
tw := tabwriter.NewWriter(os.Stdout, 0, 0, 3, ' ', 0)
+9 -8
View File
@@ -11,31 +11,32 @@ import (
type rmOptions struct {
services []string
context string
}
func NewRmCommand() *cobra.Command {
func NewRmCommand(groupID string) *cobra.Command {
opts := rmOptions{}
cmd := &cobra.Command{
Use: "rm SERVICE [SERVICE...]",
Aliases: []string{"remove", "delete"},
Short: "Remove one or more services.",
Args: cobra.MinimumNArgs(1),
Long: `Remove one or more services.
The volumes used by the services are preserved and should be removed separately
with 'uc volume rm'. Anonymous Docker volumes (automatically created from VOLUME
directives in image Dockerfiles) are automatically removed with their containers.`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.services = args
return rm(cmd.Context(), uncli, opts)
},
GroupID: groupID,
}
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
func rm(ctx context.Context, uncli *cli.CLI, opts rmOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+10 -7
View File
@@ -8,15 +8,18 @@ func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "service",
Aliases: []string{"svc"},
Short: "Manage services in an Uncloud cluster.",
Short: "Manage services in the cluster.",
}
cmd.AddCommand(
NewInspectCommand(),
NewListCommand(),
NewRmCommand(),
NewRunCommand(),
NewScaleCommand(),
NewExecCommand(),
NewExecCommand(""),
NewInspectCommand(""),
NewListCommand(""),
NewLogsCommand(""),
NewRmCommand(""),
NewRunCommand(""),
NewScaleCommand(""),
NewStartCommand(""),
NewStopCommand(""),
)
return cmd
}
+3 -9
View File
@@ -34,11 +34,9 @@ type runOptions struct {
replicas uint
user string
volumes []string
context string
}
func NewRunCommand() *cobra.Command {
func NewRunCommand(groupID string) *cobra.Command {
opts := runOptions{}
cmd := &cobra.Command{
@@ -56,6 +54,7 @@ func NewRunCommand() *cobra.Command {
return run(cmd.Context(), uncli, opts)
},
GroupID: groupID,
}
cmd.Flags().StringVar(&opts.caddyfile, "caddyfile", "",
@@ -112,11 +111,6 @@ func NewRunCommand() *cobra.Command {
" -v /data/uploads:/app/uploads Bind mount /data/uploads host directory to /app/uploads in container\n"+
" -v /host/path:/container/path:ro Bind mount a host directory or file as read-only")
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context to run the service in. (default is the current context)",
)
return cmd
}
@@ -126,7 +120,7 @@ func run(ctx context.Context, uncli *cli.CLI, opts runOptions) error {
return err
}
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+3 -8
View File
@@ -15,10 +15,9 @@ import (
type scaleOptions struct {
service string
replicas uint
context string
}
func NewScaleCommand() *cobra.Command {
func NewScaleCommand(groupID string) *cobra.Command {
opts := scaleOptions{}
cmd := &cobra.Command{
Use: "scale SERVICE REPLICAS",
@@ -37,13 +36,9 @@ func NewScaleCommand() *cobra.Command {
return scale(cmd.Context(), uncli, opts)
},
GroupID: groupID,
}
cmd.Flags().StringVarP(
&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)",
)
return cmd
}
@@ -57,7 +52,7 @@ func scale(ctx context.Context, uncli *cli.CLI, opts scaleOptions) error {
)
}
clusterClient, err := uncli.ConnectCluster(ctx, opts.context)
clusterClient, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+54
View File
@@ -0,0 +1,54 @@
//nolint:dupl
package service
import (
"context"
"fmt"
"github.com/docker/compose/v2/pkg/progress"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
type startOptions struct {
services []string
}
func NewStartCommand(groupID string) *cobra.Command {
opts := startOptions{}
cmd := &cobra.Command{
Use: "start SERVICE [SERVICE...]",
Short: "Start one or more services.",
Long: `Start one or more previously stopped services.
Starts all containers of the specified service(s) across all machines in the cluster.
Services can be specified by name or ID.`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.services = args
return start(cmd.Context(), uncli, opts)
},
GroupID: groupID,
}
return cmd
}
func start(ctx context.Context, uncli *cli.CLI, opts startOptions) error {
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
for _, s := range opts.services {
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
if err = client.StartService(ctx, s); err != nil {
return fmt.Errorf("start service '%s': %w", s, err)
}
return nil
}, uncli.ProgressOut(), "Starting service "+s)
}
return err
}
+68
View File
@@ -0,0 +1,68 @@
//nolint:dupl
package service
import (
"context"
"fmt"
"github.com/docker/compose/v2/pkg/progress"
"github.com/docker/docker/api/types/container"
"github.com/psviderski/uncloud/internal/cli"
"github.com/spf13/cobra"
)
type stopOptions struct {
services []string
signal string
timeout int
}
func NewStopCommand(groupID string) *cobra.Command {
opts := stopOptions{}
cmd := &cobra.Command{
Use: "stop SERVICE [SERVICE...]",
Short: "Stop one or more services.",
Long: `Stop one or more running services.
Gracefully stops all containers of the specified service(s) across all machines in the cluster.
Services can be specified by name or ID. Stopped services can be restarted with 'uc start'.`,
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
uncli := cmd.Context().Value("cli").(*cli.CLI)
opts.services = args
return stop(cmd.Context(), uncli, opts)
},
GroupID: groupID,
}
cmd.Flags().StringVarP(&opts.signal, "signal", "s", "",
"Signal to send to each container's main process.\n"+
"Can be a signal name (SIGTERM, SIGINT, SIGHUP, etc.) or a number. (default SIGTERM)")
cmd.Flags().IntVarP(&opts.timeout, "timeout", "t", 10,
"Seconds to wait for each container to stop gracefully before forcibly killing it with SIGKILL.\n"+
"Use -1 to wait indefinitely.")
return cmd
}
func stop(ctx context.Context, uncli *cli.CLI, opts stopOptions) error {
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
defer client.Close()
stopOpts := container.StopOptions{
Signal: opts.signal,
Timeout: &opts.timeout,
}
for _, s := range opts.services {
err = progress.RunWithTitle(ctx, func(ctx context.Context) error {
if err = client.StopService(ctx, s, stopOpts); err != nil {
return fmt.Errorf("stop service '%s': %w", s, err)
}
return nil
}, uncli.ProgressOut(), "Stopping service "+s)
}
return err
}
+1 -4
View File
@@ -19,7 +19,6 @@ type createOptions struct {
driverOpts []string
labels []string
machine string
context string
}
func NewCreateCommand() *cobra.Command {
@@ -50,14 +49,12 @@ func NewCreateCommand() *cobra.Command {
"Labels to assign to the volume in the form of 'key=value' pairs. Can be specified multiple times.")
cmd.Flags().StringVarP(&opts.machine, "machine", "m", "",
"Name or ID of the machine to create the volume on.")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd
}
func create(ctx context.Context, uncli *cli.CLI, name string, opts createOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+1 -4
View File
@@ -13,7 +13,6 @@ import (
type inspectOptions struct {
machine string
context string
}
func NewInspectCommand() *cobra.Command {
@@ -32,14 +31,12 @@ func NewInspectCommand() *cobra.Command {
cmd.Flags().StringVarP(&opts.machine, "machine", "m", "",
"Name or ID of the machine where the volume is located. "+
"If not specified, the volume will be searched across all machines.")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd
}
func inspect(ctx context.Context, uncli *cli.CLI, name string, opts inspectOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+1 -5
View File
@@ -16,7 +16,6 @@ import (
type listOptions struct {
machines []string
quiet bool
context string
}
func NewListCommand() *cobra.Command {
@@ -38,14 +37,11 @@ func NewListCommand() *cobra.Command {
cmd.Flags().BoolVarP(&opts.quiet, "quiet", "q", false,
"Only display volume names.")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd
}
func list(ctx context.Context, uncli *cli.CLI, opts listOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+1 -5
View File
@@ -14,7 +14,6 @@ type removeOptions struct {
force bool
machines []string
yes bool
context string
}
func NewRemoveCommand() *cobra.Command {
@@ -41,14 +40,11 @@ func NewRemoveCommand() *cobra.Command {
cmd.Flags().BoolVarP(&opts.yes, "yes", "y", false,
"Do not prompt for confirmation before removing the volume(s).")
cmd.Flags().StringVarP(&opts.context, "context", "c", "",
"Name of the cluster context. (default is the current context)")
return cmd
}
func remove(ctx context.Context, uncli *cli.CLI, names []string, opts removeOptions) error {
client, err := uncli.ConnectCluster(ctx, opts.context)
client, err := uncli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+1 -1
View File
@@ -7,7 +7,7 @@ import (
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "volume",
Short: "Manage volumes in an Uncloud cluster.",
Short: "Manage volumes in the cluster.",
}
cmd.AddCommand(
NewCreateCommand(),
+4 -3
View File
@@ -9,9 +9,10 @@ require (
github.com/alecthomas/chroma/v2 v2.20.0
github.com/caddyserver/caddy/v2 v2.8.4
github.com/cenkalti/backoff/v4 v4.3.0
github.com/charmbracelet/bubbles v0.20.0
github.com/charmbracelet/bubbletea v1.3.9
github.com/charmbracelet/bubbles v0.21.0
github.com/charmbracelet/bubbletea v1.3.10
github.com/charmbracelet/huh v0.6.0
github.com/charmbracelet/huh/spinner v0.0.0-20251110114415-25888d17260b
github.com/charmbracelet/lipgloss v1.1.0
github.com/compose-spec/compose-go/v2 v2.9.0
github.com/containerd/errdefs v1.0.0
@@ -149,7 +150,7 @@ require (
github.com/eiannone/keyboard v0.0.0-20220611211555-0d226195f203 // indirect
github.com/emicklei/go-restful/v3 v3.11.0 // indirect
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
github.com/fatih/color v1.17.0 // indirect
github.com/fatih/color v1.18.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fsnotify/fsevents v0.2.0 // indirect
github.com/fvbommel/sortorder v1.1.0 // indirect
+10 -8
View File
@@ -168,22 +168,24 @@ github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XL
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/charmbracelet/bubbles v0.20.0 h1:jSZu6qD8cRQ6k9OMfR1WlM+ruM8fkPWkHvQWD9LIutE=
github.com/charmbracelet/bubbles v0.20.0/go.mod h1:39slydyswPy+uVOHZ5x/GjwVAFkCsV8IIVy+4MhzwwU=
github.com/charmbracelet/bubbletea v1.3.9 h1:OBYdfRo6QnlIcXNmcoI2n1NNS65Nk6kI2L2FO1puS/4=
github.com/charmbracelet/bubbletea v1.3.9/go.mod h1:ORQfo0fk8U+po9VaNvnV95UPWA1BitP1E0N6xJPlHr4=
github.com/charmbracelet/bubbles v0.21.0 h1:9TdC97SdRVg/1aaXNVWfFH3nnLAwOXr8Fn6u6mfQdFs=
github.com/charmbracelet/bubbles v0.21.0/go.mod h1:HF+v6QUR4HkEpz62dx7ym2xc71/KBHg+zKwJtMw+qtg=
github.com/charmbracelet/bubbletea v1.3.10 h1:otUDHWMMzQSB0Pkc87rm691KZ3SWa4KUlvF9nRvCICw=
github.com/charmbracelet/bubbletea v1.3.10/go.mod h1:ORQfo0fk8U+po9VaNvnV95UPWA1BitP1E0N6xJPlHr4=
github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc h1:4pZI35227imm7yK2bGPcfpFEmuY1gc2YSTShr4iJBfs=
github.com/charmbracelet/colorprofile v0.2.3-0.20250311203215-f60798e515dc/go.mod h1:X4/0JoqgTIPSFcRA/P6INZzIuyqdFY5rm8tb41s9okk=
github.com/charmbracelet/huh v0.6.0 h1:mZM8VvZGuE0hoDXq6XLxRtgfWyTI3b2jZNKh0xWmax8=
github.com/charmbracelet/huh v0.6.0/go.mod h1:GGNKeWCeNzKpEOh/OJD8WBwTQjV3prFAtQPpLv+AVwU=
github.com/charmbracelet/huh/spinner v0.0.0-20251110114415-25888d17260b h1:oefmPctgff7OB4cDz3Ndp+ewwrh7i+fwth70qvZCieI=
github.com/charmbracelet/huh/spinner v0.0.0-20251110114415-25888d17260b/go.mod h1:OMqKat/mm9a/qOnpuNOPyYO9bPzRNnmzLnRZT5KYltg=
github.com/charmbracelet/lipgloss v1.1.0 h1:vYXsiLHVkK7fp74RkV7b2kq9+zDLoEU4MZoFqR/noCY=
github.com/charmbracelet/lipgloss v1.1.0/go.mod h1:/6Q8FR2o+kj8rz4Dq0zQc3vYf7X+B0binUUBwA0aL30=
github.com/charmbracelet/x/ansi v0.10.1 h1:rL3Koar5XvX0pHGfovN03f5cxLbCF2YvLeyz7D2jVDQ=
github.com/charmbracelet/x/ansi v0.10.1/go.mod h1:3RQDQ6lDnROptfpWuUVIUG64bD2g2BgntdxH0Ya5TeE=
github.com/charmbracelet/x/cellbuf v0.0.13-0.20250311204145-2c3ea96c31dd h1:vy0GVL4jeHEwG5YOXDmi86oYw2yuYUGqz6a8sLwg0X8=
github.com/charmbracelet/x/cellbuf v0.0.13-0.20250311204145-2c3ea96c31dd/go.mod h1:xe0nKWGd3eJgtqZRaN9RjMtK7xUYchjzPr7q6kcvCCs=
github.com/charmbracelet/x/exp/golden v0.0.0-20240815200342-61de596daa2b h1:MnAMdlwSltxJyULnrYbkZpp4k58Co7Tah3ciKhSNo0Q=
github.com/charmbracelet/x/exp/golden v0.0.0-20240815200342-61de596daa2b/go.mod h1:wDlXFlCrmJ8J+swcL/MnGUuYnqgQdW9rhSD61oNMb6U=
github.com/charmbracelet/x/exp/golden v0.0.0-20241011142426-46044092ad91 h1:payRxjMjKgx2PaCWLZ4p3ro9y97+TVLZNaRZgJwSVDQ=
github.com/charmbracelet/x/exp/golden v0.0.0-20241011142426-46044092ad91/go.mod h1:wDlXFlCrmJ8J+swcL/MnGUuYnqgQdW9rhSD61oNMb6U=
github.com/charmbracelet/x/exp/strings v0.0.0-20240919170804-a4978c8e603a h1:JMdM89Udp/cOl5tC3MuUJXTPE/nAdU1oyt9jRU44qq8=
github.com/charmbracelet/x/exp/strings v0.0.0-20240919170804-a4978c8e603a/go.mod h1:pBhA0ybfXv6hDjQUZ7hk1lVxBiUbupdw5R31yPUViVQ=
github.com/charmbracelet/x/term v0.2.1 h1:AQeHeLZ1OqSXhrAWpYUtZyX1T3zVxfpZuEQMIQaGIAQ=
@@ -344,8 +346,8 @@ github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97
github.com/erikstmartin/go-testdb v0.0.0-20160219214506-8d10e4a1bae5/go.mod h1:a2zkGnVExMxdzMo3M0Hi/3sEU+cWnZpSni0O6/Yb/P0=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/fatih/color v1.9.0/go.mod h1:eQcE1qtQxscV5RaZvpXrrb8Drkc3/DdQ+uUYCNjL+zU=
github.com/fatih/color v1.17.0 h1:GlRw1BRJxkpqUCBKzKOw098ed57fEsKeNjpTe3cSjK4=
github.com/fatih/color v1.17.0/go.mod h1:YZ7TlrGPkiz6ku9fK3TLD/pl3CpsiFyu8N92HLgmosI=
github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/flynn/noise v1.1.0 h1:KjPQoQCEFdZDiP03phOvGi11+SVVhBG2wOWAorLsstg=
+1 -3
View File
@@ -40,8 +40,6 @@ type BuildServicesOptions struct {
PushRegistry bool
// Cluster-specific options (only used if PushCluster is true).
// Context is the name of the cluster context.
Context string
// Machines is a list of machine names or IDs to push the image to. If empty, images are pushed to all machines.
Machines []string
}
@@ -95,7 +93,7 @@ func (cli *CLI) BuildServices(ctx context.Context, project *composetypes.Project
// Add a line break after the build output.
fmt.Fprintln(cli.ProgressOut())
clusterClient, err := cli.ConnectCluster(ctx, opts.Context)
clusterClient, err := cli.ConnectCluster(ctx)
if err != nil {
return fmt.Errorf("connect to cluster: %w", err)
}
+65 -19
View File
@@ -28,14 +28,16 @@ const (
)
type CLI struct {
Config *config.Config
conn *config.MachineConnection
Config *config.Config
conn *config.MachineConnection
contextOverride string
}
// New creates a new CLI instance with the given config path or remote machine connection.
// If the connection is provided, the config is ignored for all operations which is useful for interacting with
// a cluster without creating a config.
func New(configPath string, conn *config.MachineConnection) (*CLI, error) {
// If a non-empty context name is given, it will override the current default.
func New(configPath string, conn *config.MachineConnection, contextName string) (*CLI, error) {
if conn != nil {
return &CLI{conn: conn}, nil
}
@@ -46,7 +48,8 @@ func New(configPath string, conn *config.MachineConnection) (*CLI, error) {
}
return &CLI{
Config: cfg,
Config: cfg,
contextOverride: contextName,
}, nil
}
@@ -68,19 +71,28 @@ func (cli *CLI) SetCurrentContext(name string) error {
return cli.Config.Save()
}
// ConnectCluster connects to a cluster using the given context name or the current context if not specified.
func (cli *CLI) GetContextOverrideOrCurrent() string {
contextName := cli.contextOverride
if contextName == "" {
contextName = cli.Config.CurrentContext
}
return contextName
}
// ConnectCluster connects to a cluster using the context override or the current context if not specified.
// If the CLI was initialised with a machine connection, the config is ignored and the connection is used instead.
func (cli *CLI) ConnectCluster(ctx context.Context, contextName string) (*client.Client, error) {
return cli.ConnectClusterWithOptions(ctx, contextName, ConnectOptions{
func (cli *CLI) ConnectCluster(ctx context.Context) (*client.Client, error) {
return cli.ConnectClusterWithOptions(ctx, ConnectOptions{
// Default to showing progress for CLI usage.
ShowProgress: true,
})
}
// ConnectClusterWithOptions connects to a cluster using the given context name and options.
// ConnectClusterWithOptions connects to a cluster with the given options.
// If the CLI was initialised with a machine connection, the config is ignored and the connection is used instead.
// If the CLI has an override context, it is used instead of the current default.
// Options are useful when using the CLI as a library where you may want to disable visual feedback.
func (cli *CLI) ConnectClusterWithOptions(ctx context.Context, contextName string, opts ConnectOptions) (*client.Client, error) {
func (cli *CLI) ConnectClusterWithOptions(ctx context.Context, opts ConnectOptions) (*client.Client, error) {
if cli.conn != nil {
return ConnectCluster(ctx, *cli.conn, opts)
}
@@ -92,6 +104,8 @@ func (cli *CLI) ConnectClusterWithOptions(ctx context.Context, contextName strin
cli.Config.Path(),
)
}
contextName := cli.contextOverride
if contextName == "" {
// If the cluster is not specified, use the current cluster if set.
if cli.Config.CurrentContext == "" {
@@ -228,8 +242,13 @@ func (cli *CLI) initRemoteMachine(ctx context.Context, opts InitClusterOptions)
// Save the machine's SSH connection details in the context config.
connCfg := config.MachineConnection{
SSH: config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port),
SSHKeyFile: opts.RemoteMachine.KeyPath,
MachineID: resp.Machine.Id,
}
if opts.RemoteMachine.UseSSHCLI {
connCfg.SSHCLI = config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port)
} else {
connCfg.SSH = config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port)
}
cli.Config.Contexts[contextName].Connections = append(cli.Config.Contexts[contextName].Connections, connCfg)
if err = cli.Config.Save(); err != nil {
@@ -265,7 +284,6 @@ func (cli *CLI) newContextName(name string) (string, error) {
}
type AddMachineOptions struct {
Context string
MachineName string
PublicIP *netip.Addr
RemoteMachine *RemoteMachine
@@ -278,11 +296,8 @@ type AddMachineOptions struct {
// cluster. The machine client is connected to the new machine and can be used to interact with it.
// Both client should be closed after use by the caller.
func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client.Client, *client.Client, error) {
contextName := opts.Context
if contextName == "" {
contextName = cli.Config.CurrentContext
}
c, err := cli.ConnectCluster(ctx, contextName)
contextName := cli.GetContextOverrideOrCurrent()
c, err := cli.ConnectCluster(ctx)
if err != nil {
return nil, nil, fmt.Errorf("connect to cluster (context '%s'): %w", contextName, err)
}
@@ -396,11 +411,13 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
// Save the machine's SSH connection details in the context config.
connCfg := config.MachineConnection{
SSH: config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port),
SSHKeyFile: opts.RemoteMachine.KeyPath,
MachineID: addResp.Machine.Id,
}
if contextName == "" {
contextName = cli.Config.CurrentContext
if opts.RemoteMachine.UseSSHCLI {
connCfg.SSHCLI = config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port)
} else {
connCfg.SSH = config.NewSSHDestination(opts.RemoteMachine.User, opts.RemoteMachine.Host, opts.RemoteMachine.Port)
}
cli.Config.Contexts[contextName].Connections = append(cli.Config.Contexts[contextName].Connections, connCfg)
if err = cli.Config.Save(); err != nil {
@@ -420,6 +437,35 @@ func (cli *CLI) AddMachine(ctx context.Context, opts AddMachineOptions) (*client
func provisionOrConnectRemoteMachine(
ctx context.Context, remoteMachine *RemoteMachine, skipInstall bool, version string,
) (*client.Client, error) {
// Use SSH CLI
if remoteMachine.UseSSHCLI {
exec := sshexec.NewSSHCLIRemote(
remoteMachine.User,
remoteMachine.Host,
remoteMachine.Port,
remoteMachine.KeyPath,
)
if !skipInstall {
if err := provisionMachine(ctx, exec, version); err != nil {
return nil, fmt.Errorf("provision machine: %w", err)
}
}
sshConfig := &connector.SSHConnectorConfig{
User: remoteMachine.User,
Host: remoteMachine.Host,
Port: remoteMachine.Port,
KeyPath: remoteMachine.KeyPath,
}
machineClient, err := client.New(ctx, connector.NewSSHCLIConnector(sshConfig))
if err != nil {
return nil, fmt.Errorf("connect to remote machine: %w", err)
}
return machineClient, nil
}
// Use Go SSH
sshClient, err := sshexec.Connect(remoteMachine.User, remoteMachine.Host, remoteMachine.Port, remoteMachine.KeyPath)
// If the SSH connection using SSH agent fails and no key path is provided, try to use the default SSH key.
if err != nil && remoteMachine.KeyPath == "" {
+13 -5
View File
@@ -22,9 +22,12 @@ type MachineConnection struct {
SSHKeyFile string `yaml:"ssh_key_file,omitempty"`
// TCP is the address and port of the machine's API server.
// The pointer is used to omit the field when not set. Otherwise, yaml marshalling includes an empty object.
TCP *netip.AddrPort `yaml:"tcp,omitempty"`
Host string `yaml:"host,omitempty"`
PublicKey secret.Secret `yaml:"public_key,omitempty"`
TCP *netip.AddrPort `yaml:"tcp,omitempty"`
// Unix is the path to the machine's API unix socket.
Unix string `yaml:"unix,omitempty"`
Host string `yaml:"host,omitempty"`
PublicKey secret.Secret `yaml:"public_key,omitempty"`
MachineID string `yaml:"machine_id,omitempty"`
}
func (c MachineConnection) String() string {
@@ -34,6 +37,8 @@ func (c MachineConnection) String() string {
return "ssh+cli://" + string(c.SSHCLI)
} else if c.TCP != nil && c.TCP.IsValid() {
return fmt.Sprintf("tcp://%s", c.TCP)
} else if c.Unix != "" {
return fmt.Sprintf("unix://%s", c.Unix)
}
return "unknown connection"
}
@@ -49,12 +54,15 @@ func (c *MachineConnection) Validate() error {
if c.TCP != nil && c.TCP.IsValid() {
setCount++
}
if c.Unix != "" {
setCount++
}
if setCount == 0 {
return errors.New("no connection method specified (ssh, ssh_cli, or tcp required)")
return errors.New("no connection method specified (ssh, ssh_cli, tcp, or unix required)")
}
if setCount > 1 {
return errors.New("only one connection method allowed per connection (ssh, ssh_cli, or tcp)")
return errors.New("only one connection method allowed per connection (ssh, ssh_cli, tcp, or unix)")
}
return nil
+32
View File
@@ -53,11 +53,27 @@ func TestMachineConnection_String(t *testing.T) {
},
want: "tcp://10.0.0.1:8080",
},
{
name: "unix connection",
conn: MachineConnection{
Unix: "/run/uncloud/uncloud.sock",
},
want: "unix:///run/uncloud/uncloud.sock",
},
{
name: "no connection",
conn: MachineConnection{},
want: "unknown connection",
},
{
name: "ssh connection with machine id",
conn: MachineConnection{
SSH: "user@host.com",
MachineID: "ed98b9f7575308c340263cd279e3b568",
},
// String() does not use MachineID; just verifying entry with it is valid
want: "ssh://user@host.com",
},
}
for _, tt := range tests {
@@ -103,6 +119,13 @@ func TestMachineConnection_Validate(t *testing.T) {
},
wantErr: false,
},
{
name: "unix only - valid",
conn: MachineConnection{
Unix: "/path/to/socket",
},
wantErr: false,
},
{
name: "no connection method - error",
conn: MachineConnection{},
@@ -118,6 +141,15 @@ func TestMachineConnection_Validate(t *testing.T) {
wantErr: true,
errMsg: "only one connection method allowed",
},
{
name: "ssh and unix - error",
conn: MachineConnection{
SSH: "user@host",
Unix: "/path/to/socket",
},
wantErr: true,
errMsg: "only one connection method allowed",
},
{
name: "ssh and tcp - error",
conn: MachineConnection{
+18
View File
@@ -4,3 +4,21 @@ type Context struct {
Name string `yaml:"-"`
Connections []MachineConnection `yaml:"connections"`
}
func (c *Context) SetDefaultConnection(index int) {
// Do nothing if the index is out of range.
if index < 0 || index >= len(c.Connections) {
return
}
selected := c.Connections[index]
newConnections := make([]MachineConnection, 0, len(c.Connections))
newConnections = append(newConnections, selected)
for i, conn := range c.Connections {
if i != index {
newConnections = append(newConnections, conn)
}
}
c.Connections = newConnections
}
+2
View File
@@ -73,6 +73,8 @@ func connectCluster(ctx context.Context, conn config.MachineConnection) (*client
useSSHCLI = true
} else if conn.TCP != nil && conn.TCP.IsValid() {
return client.New(ctx, connector.NewTCPConnector(*conn.TCP))
} else if conn.Unix != "" {
return client.New(ctx, connector.NewUnixConnector(conn.Unix))
} else {
return nil, errors.New("connection configuration is invalid")
}
+5 -4
View File
@@ -21,10 +21,11 @@ const (
)
type RemoteMachine struct {
User string
Host string
Port int
KeyPath string
User string
Host string
Port int
KeyPath string
UseSSHCLI bool // indicates ssh+cli:// should be used
}
func installCmd(user string, version string) string {
File diff suppressed because it is too large Load Diff
+62 -38
View File
@@ -5,6 +5,7 @@ package api;
option go_package = "github.com/psviderski/uncloud/internal/machine/api/pb";
import "google/protobuf/empty.proto";
import "google/protobuf/timestamp.proto";
import "internal/machine/api/pb/common.proto";
service Docker {
@@ -15,6 +16,9 @@ service Docker {
rpc ListContainers(ListContainersRequest) returns (ListContainersResponse);
rpc RemoveContainer(RemoveContainerRequest) returns (google.protobuf.Empty);
rpc ExecContainer(stream ExecContainerRequest) returns (stream ExecContainerResponse);
rpc ContainerLogs(ContainerLogsRequest) returns (stream ContainerLogEntry);
rpc PullImage(PullImageRequest) returns (stream JSONMessage);
rpc InspectImage(InspectImageRequest) returns (InspectImageResponse);
// InspectRemoteImage returns the image metadata for an image in a remote registry using the machine's
@@ -30,8 +34,6 @@ service Docker {
rpc InspectServiceContainer(InspectContainerRequest) returns (ServiceContainer);
rpc ListServiceContainers(ListServiceContainersRequest) returns (ListServiceContainersResponse);
rpc RemoveServiceContainer(RemoveContainerRequest) returns (google.protobuf.Empty);
rpc ExecContainer(stream ExecContainerRequest) returns (stream ExecContainerResponse);
}
message CreateContainerRequest {
@@ -94,6 +96,64 @@ message RemoveContainerRequest {
bytes options = 2;
}
message ExecContainerRequest {
oneof payload {
// Initial configuration for the exec session. Must be sent as the first message.
ExecConfig config = 1;
// Raw stdin data to be written to the exec process.
bytes stdin = 2;
// TTY resize event (only used when TTY is enabled).
ResizeEvent resize = 3;
}
}
message ExecConfig {
// Container ID to execute the command in.
string container_id = 1;
// JSON serialised ExecOptions
bytes options = 2;
}
message ResizeEvent {
uint32 height = 1;
uint32 width = 2;
}
message ExecContainerResponse {
oneof payload {
// Exec instance ID returned after creating the exec.
string exec_id = 1;
// Raw stdout data from the exec process.
bytes stdout = 2;
// Raw stderr data from the exec process (only when TTY is disabled).
bytes stderr = 3;
// Exit code of the exec process. Sent as the final message.
int32 exit_code = 4;
}
}
message ContainerLogsRequest {
string container_id = 1;
// Options for logs retrieval.
bool follow = 2;
int32 tail = 3; // -1 means all
string since = 4; // https://www.rfc-editor.org/rfc/rfc3339.html timestamp or Go duration string
string until = 5; // https://www.rfc-editor.org/rfc/rfc3339.html timestamp or Go duration string
}
message ContainerLogEntry {
enum StreamType {
UNKNOWN = 0;
STDOUT = 1;
STDERR = 2;
HEARTBEAT = 3;
}
StreamType stream = 1;
google.protobuf.Timestamp timestamp = 2;
// Log line content. Empty for heartbeat entries.
bytes message = 3;
}
message PullImageRequest {
string image = 1;
// JSON serialised image.PullOptions.
@@ -216,39 +276,3 @@ message MachineServiceContainers {
Metadata metadata = 1;
repeated ServiceContainer containers = 2;
}
message ExecContainerRequest {
oneof payload {
// Initial configuration for the exec session. Must be sent as the first message.
ExecConfig config = 1;
// Raw stdin data to be written to the exec process.
bytes stdin = 2;
// TTY resize event (only used when TTY is enabled).
ResizeEvent resize = 3;
}
}
message ExecConfig {
// Container ID to execute the command in.
string container_id = 1;
// JSON serialised ExecOptions
bytes options = 2;
}
message ResizeEvent {
uint32 height = 1;
uint32 width = 2;
}
message ExecContainerResponse {
oneof payload {
// Exec instance ID returned after creating the exec.
string exec_id = 1;
// Raw stdout data from the exec process.
bytes stdout = 2;
// Raw stderr data from the exec process (only when TTY is disabled).
bytes stderr = 3;
// Exit code of the exec process. Sent as the final message.
int32 exit_code = 4;
}
}
+73 -32
View File
@@ -26,6 +26,8 @@ const (
Docker_StopContainer_FullMethodName = "/api.Docker/StopContainer"
Docker_ListContainers_FullMethodName = "/api.Docker/ListContainers"
Docker_RemoveContainer_FullMethodName = "/api.Docker/RemoveContainer"
Docker_ExecContainer_FullMethodName = "/api.Docker/ExecContainer"
Docker_ContainerLogs_FullMethodName = "/api.Docker/ContainerLogs"
Docker_PullImage_FullMethodName = "/api.Docker/PullImage"
Docker_InspectImage_FullMethodName = "/api.Docker/InspectImage"
Docker_InspectRemoteImage_FullMethodName = "/api.Docker/InspectRemoteImage"
@@ -37,7 +39,6 @@ const (
Docker_InspectServiceContainer_FullMethodName = "/api.Docker/InspectServiceContainer"
Docker_ListServiceContainers_FullMethodName = "/api.Docker/ListServiceContainers"
Docker_RemoveServiceContainer_FullMethodName = "/api.Docker/RemoveServiceContainer"
Docker_ExecContainer_FullMethodName = "/api.Docker/ExecContainer"
)
// DockerClient is the client API for Docker service.
@@ -50,6 +51,8 @@ type DockerClient interface {
StopContainer(ctx context.Context, in *StopContainerRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
ListContainers(ctx context.Context, in *ListContainersRequest, opts ...grpc.CallOption) (*ListContainersResponse, error)
RemoveContainer(ctx context.Context, in *RemoveContainerRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
ExecContainer(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[ExecContainerRequest, ExecContainerResponse], error)
ContainerLogs(ctx context.Context, in *ContainerLogsRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[ContainerLogEntry], error)
PullImage(ctx context.Context, in *PullImageRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[JSONMessage], error)
InspectImage(ctx context.Context, in *InspectImageRequest, opts ...grpc.CallOption) (*InspectImageResponse, error)
// InspectRemoteImage returns the image metadata for an image in a remote registry using the machine's
@@ -63,7 +66,6 @@ type DockerClient interface {
InspectServiceContainer(ctx context.Context, in *InspectContainerRequest, opts ...grpc.CallOption) (*ServiceContainer, error)
ListServiceContainers(ctx context.Context, in *ListServiceContainersRequest, opts ...grpc.CallOption) (*ListServiceContainersResponse, error)
RemoveServiceContainer(ctx context.Context, in *RemoveContainerRequest, opts ...grpc.CallOption) (*emptypb.Empty, error)
ExecContainer(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[ExecContainerRequest, ExecContainerResponse], error)
}
type dockerClient struct {
@@ -134,9 +136,41 @@ func (c *dockerClient) RemoveContainer(ctx context.Context, in *RemoveContainerR
return out, nil
}
func (c *dockerClient) ExecContainer(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[ExecContainerRequest, ExecContainerResponse], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Docker_ServiceDesc.Streams[0], Docker_ExecContainer_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[ExecContainerRequest, ExecContainerResponse]{ClientStream: stream}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Docker_ExecContainerClient = grpc.BidiStreamingClient[ExecContainerRequest, ExecContainerResponse]
func (c *dockerClient) ContainerLogs(ctx context.Context, in *ContainerLogsRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[ContainerLogEntry], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Docker_ServiceDesc.Streams[1], Docker_ContainerLogs_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[ContainerLogsRequest, ContainerLogEntry]{ClientStream: stream}
if err := x.ClientStream.SendMsg(in); err != nil {
return nil, err
}
if err := x.ClientStream.CloseSend(); err != nil {
return nil, err
}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Docker_ContainerLogsClient = grpc.ServerStreamingClient[ContainerLogEntry]
func (c *dockerClient) PullImage(ctx context.Context, in *PullImageRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[JSONMessage], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Docker_ServiceDesc.Streams[0], Docker_PullImage_FullMethodName, cOpts...)
stream, err := c.cc.NewStream(ctx, &Docker_ServiceDesc.Streams[2], Docker_PullImage_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
@@ -253,19 +287,6 @@ func (c *dockerClient) RemoveServiceContainer(ctx context.Context, in *RemoveCon
return out, nil
}
func (c *dockerClient) ExecContainer(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[ExecContainerRequest, ExecContainerResponse], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Docker_ServiceDesc.Streams[1], Docker_ExecContainer_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[ExecContainerRequest, ExecContainerResponse]{ClientStream: stream}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Docker_ExecContainerClient = grpc.BidiStreamingClient[ExecContainerRequest, ExecContainerResponse]
// DockerServer is the server API for Docker service.
// All implementations must embed UnimplementedDockerServer
// for forward compatibility.
@@ -276,6 +297,8 @@ type DockerServer interface {
StopContainer(context.Context, *StopContainerRequest) (*emptypb.Empty, error)
ListContainers(context.Context, *ListContainersRequest) (*ListContainersResponse, error)
RemoveContainer(context.Context, *RemoveContainerRequest) (*emptypb.Empty, error)
ExecContainer(grpc.BidiStreamingServer[ExecContainerRequest, ExecContainerResponse]) error
ContainerLogs(*ContainerLogsRequest, grpc.ServerStreamingServer[ContainerLogEntry]) error
PullImage(*PullImageRequest, grpc.ServerStreamingServer[JSONMessage]) error
InspectImage(context.Context, *InspectImageRequest) (*InspectImageResponse, error)
// InspectRemoteImage returns the image metadata for an image in a remote registry using the machine's
@@ -289,7 +312,6 @@ type DockerServer interface {
InspectServiceContainer(context.Context, *InspectContainerRequest) (*ServiceContainer, error)
ListServiceContainers(context.Context, *ListServiceContainersRequest) (*ListServiceContainersResponse, error)
RemoveServiceContainer(context.Context, *RemoveContainerRequest) (*emptypb.Empty, error)
ExecContainer(grpc.BidiStreamingServer[ExecContainerRequest, ExecContainerResponse]) error
mustEmbedUnimplementedDockerServer()
}
@@ -318,6 +340,12 @@ func (UnimplementedDockerServer) ListContainers(context.Context, *ListContainers
func (UnimplementedDockerServer) RemoveContainer(context.Context, *RemoveContainerRequest) (*emptypb.Empty, error) {
return nil, status.Errorf(codes.Unimplemented, "method RemoveContainer not implemented")
}
func (UnimplementedDockerServer) ExecContainer(grpc.BidiStreamingServer[ExecContainerRequest, ExecContainerResponse]) error {
return status.Errorf(codes.Unimplemented, "method ExecContainer not implemented")
}
func (UnimplementedDockerServer) ContainerLogs(*ContainerLogsRequest, grpc.ServerStreamingServer[ContainerLogEntry]) error {
return status.Errorf(codes.Unimplemented, "method ContainerLogs not implemented")
}
func (UnimplementedDockerServer) PullImage(*PullImageRequest, grpc.ServerStreamingServer[JSONMessage]) error {
return status.Errorf(codes.Unimplemented, "method PullImage not implemented")
}
@@ -351,9 +379,6 @@ func (UnimplementedDockerServer) ListServiceContainers(context.Context, *ListSer
func (UnimplementedDockerServer) RemoveServiceContainer(context.Context, *RemoveContainerRequest) (*emptypb.Empty, error) {
return nil, status.Errorf(codes.Unimplemented, "method RemoveServiceContainer not implemented")
}
func (UnimplementedDockerServer) ExecContainer(grpc.BidiStreamingServer[ExecContainerRequest, ExecContainerResponse]) error {
return status.Errorf(codes.Unimplemented, "method ExecContainer not implemented")
}
func (UnimplementedDockerServer) mustEmbedUnimplementedDockerServer() {}
func (UnimplementedDockerServer) testEmbeddedByValue() {}
@@ -483,6 +508,24 @@ func _Docker_RemoveContainer_Handler(srv interface{}, ctx context.Context, dec f
return interceptor(ctx, in, info, handler)
}
func _Docker_ExecContainer_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(DockerServer).ExecContainer(&grpc.GenericServerStream[ExecContainerRequest, ExecContainerResponse]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Docker_ExecContainerServer = grpc.BidiStreamingServer[ExecContainerRequest, ExecContainerResponse]
func _Docker_ContainerLogs_Handler(srv interface{}, stream grpc.ServerStream) error {
m := new(ContainerLogsRequest)
if err := stream.RecvMsg(m); err != nil {
return err
}
return srv.(DockerServer).ContainerLogs(m, &grpc.GenericServerStream[ContainerLogsRequest, ContainerLogEntry]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Docker_ContainerLogsServer = grpc.ServerStreamingServer[ContainerLogEntry]
func _Docker_PullImage_Handler(srv interface{}, stream grpc.ServerStream) error {
m := new(PullImageRequest)
if err := stream.RecvMsg(m); err != nil {
@@ -674,13 +717,6 @@ func _Docker_RemoveServiceContainer_Handler(srv interface{}, ctx context.Context
return interceptor(ctx, in, info, handler)
}
func _Docker_ExecContainer_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(DockerServer).ExecContainer(&grpc.GenericServerStream[ExecContainerRequest, ExecContainerResponse]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Docker_ExecContainerServer = grpc.BidiStreamingServer[ExecContainerRequest, ExecContainerResponse]
// Docker_ServiceDesc is the grpc.ServiceDesc for Docker service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
@@ -754,17 +790,22 @@ var Docker_ServiceDesc = grpc.ServiceDesc{
},
},
Streams: []grpc.StreamDesc{
{
StreamName: "PullImage",
Handler: _Docker_PullImage_Handler,
ServerStreams: true,
},
{
StreamName: "ExecContainer",
Handler: _Docker_ExecContainer_Handler,
ServerStreams: true,
ClientStreams: true,
},
{
StreamName: "ContainerLogs",
Handler: _Docker_ContainerLogs_Handler,
ServerStreams: true,
},
{
StreamName: "PullImage",
Handler: _Docker_PullImage_Handler,
ServerStreams: true,
},
},
Metadata: "internal/machine/api/pb/docker.proto",
}
+4 -4
View File
@@ -201,10 +201,10 @@ func handleClientOutputStream(ctx context.Context, stream pb.Docker_ExecContaine
// ExecContainer executes a command in a running container with bidirectional streaming.
// TODO: This can be merged with pkg/client as it's an unnecessary logic split.
func (c *Client) ExecContainer(ctx context.Context, opts ExecConfig) (exitCode int, err error) {
// TODO: We need to handle Ctrl-C and other signals here to forward them to the container process.
// Right now, Ctrl-C will just terminate the client process, which is not ideal.
// We should catch the signal, send it to the container process, and only exit
// when the container process exits.
// Note: In non-interactive mode (without TTY), signals like SIGTERM/SIGINT will terminate the client
// process without being forwarded to the remote container process. We could catch and forward these
// signals (useful for long-running commands), but for example "kubectl exec" doesn't do this either (as of November 2025),
// so we keep it simple for now.
slog.Debug("starting ExecContainer", "containerID", opts.ContainerID, "options", opts.Options)
+92
View File
@@ -47,6 +47,7 @@ import (
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"google.golang.org/protobuf/types/known/emptypb"
"google.golang.org/protobuf/types/known/timestamppb"
)
var fullDockerIDRegex = regexp.MustCompile(`^[a-f0-9]{64}$`)
@@ -1015,6 +1016,97 @@ func (s *Server) RemoveServiceContainer(ctx context.Context, req *pb.RemoveConta
return resp, nil
}
// logsHeartbeatInterval is the interval at which heartbeat entries are sent when there are no logs to stream.
const logsHeartbeatInterval = 200 * time.Millisecond
// ContainerLogs streams logs from a container.
func (s *Server) ContainerLogs(
req *pb.ContainerLogsRequest, stream grpc.ServerStreamingServer[pb.ContainerLogEntry],
) error {
// Stream context is cancelled when the client has disconnected or the stream has ended.
ctx := stream.Context()
opts := ContainerLogsOptions{
ContainerID: req.ContainerId,
Follow: req.Follow,
Tail: int(req.Tail),
Since: req.Since,
Until: req.Until,
}
logsCh, err := s.service.ContainerLogs(ctx, opts)
if err != nil {
if errdefs.IsNotFound(err) {
return status.Error(codes.NotFound, err.Error())
}
return status.Errorf(codes.Internal, "get container logs: %v", err)
}
log := slog.With("container_id", req.ContainerId, "stream_id", fmt.Sprintf("%p", stream)[2:])
log.Debug("Starting container logs streaming.",
"follow", req.Follow, "tail", req.Tail, "since", req.Since, "until", req.Until)
// Heartbeats are needed only when following logs to let the client know when there are no new log entries
// to allow it to advance the watermark of last received log timestamp.
var heartbeatCh <-chan time.Time
if req.Follow {
heartbeatTicker := time.NewTicker(logsHeartbeatInterval)
defer heartbeatTicker.Stop()
heartbeatCh = heartbeatTicker.C
}
started := time.Now()
lastSent := time.Time{}
for {
select {
case entry, ok := <-logsCh:
if !ok {
// Channel closed, no more log entries.
return nil
}
if entry.Err != nil {
return status.Error(codes.Internal, entry.Err.Error())
}
pbEntry := &pb.ContainerLogEntry{
Stream: api.LogStreamTypeToProto(entry.Stream),
Timestamp: timestamppb.New(entry.Timestamp),
Message: entry.Message,
}
if err = stream.Send(pbEntry); err != nil {
return status.Errorf(codes.Internal, "send log entry: %v", err)
}
lastSent = entry.Timestamp
case now := <-heartbeatCh:
// Only send heartbeat if no log entries have been sent since the last heartbeat interval or
// if no log entries have been sent at all for at least a heartbeat interval since starting.
if now.Sub(lastSent) < logsHeartbeatInterval ||
(lastSent.IsZero() && now.Sub(started) < logsHeartbeatInterval) {
continue
}
// Use the timestamp one heartbeat in the past to be conservative. This reduces the chance of sending
// a timestamp that is greater than a log entry currently being parsed but not yet sent, which would
// cause the client to incorrectly believe it has received all logs up to that point.
heartbeat := &pb.ContainerLogEntry{
Stream: pb.ContainerLogEntry_HEARTBEAT,
Timestamp: timestamppb.New(now.Add(-logsHeartbeatInterval)),
}
if err = stream.Send(heartbeat); err != nil {
return status.Errorf(codes.Internal, "send log stream heartbeat: %v", err)
}
lastSent = heartbeat.Timestamp.AsTime()
log.Debug("Sent log stream heartbeat.", "timestamp", lastSent)
case <-ctx.Done():
return status.Error(codes.Canceled, ctx.Err().Error())
}
}
}
// receiveExecConfig receives and validates the initial exec configuration from the stream.
func (s *Server) receiveExecConfig(stream pb.Docker_ExecContainerServer) (*pb.ExecConfig, api.ExecOptions, error) {
req, err := stream.Recv()
+105
View File
@@ -1,18 +1,22 @@
package docker
import (
"bytes"
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"log/slog"
"strconv"
"strings"
"time"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/filters"
"github.com/docker/docker/api/types/image"
"github.com/docker/docker/client"
"github.com/docker/docker/pkg/stdcopy"
"github.com/jmoiron/sqlx"
"github.com/psviderski/uncloud/pkg/api"
"google.golang.org/grpc/codes"
@@ -150,3 +154,104 @@ func (s *Service) ListImages(ctx context.Context, opts image.ListOptions) (Image
return imagesResp, nil
}
// ContainerLogsOptions specifies parameters for ContainerLogs.
type ContainerLogsOptions struct {
ContainerID string
Follow bool
Tail int
Since string
Until string
}
// ContainerLogs streams logs from a container and returns demultiplexed entries via a channel.
// The channel is closed when streaming completes or context is cancelled.
func (s *Service) ContainerLogs(ctx context.Context, opts ContainerLogsOptions) (<-chan api.ContainerLogEntry, error) {
dockerOpts := container.LogsOptions{
ShowStdout: true,
ShowStderr: true,
Follow: opts.Follow,
Tail: strconv.FormatInt(int64(opts.Tail), 10),
Since: opts.Since,
Until: opts.Until,
Timestamps: true,
}
reader, err := s.Client.ContainerLogs(ctx, opts.ContainerID, dockerOpts)
if err != nil {
return nil, err
}
outCh := make(chan api.ContainerLogEntry)
stdoutWriter := &logsChannelWriter{ctx: ctx, ch: outCh, isStderr: false}
stderrWriter := &logsChannelWriter{ctx: ctx, ch: outCh, isStderr: true}
// Wrap the context in a cancellable one to unblock the second goroutine below when StdCopy completes.
ctx, cancel := context.WithCancel(ctx)
// Run StdCopy in a goroutine to be able to handle context cancellation.
go func() {
defer close(outCh)
defer cancel()
// StdCopy is blocking and will return when the reader is closed in another goroutine below or on error.
if _, err := stdcopy.StdCopy(stdoutWriter, stderrWriter, reader); err != nil {
// Send error as the last entry.
select {
case outCh <- api.ContainerLogEntry{Err: fmt.Errorf("demultiplex container logs: %w", err)}:
case <-ctx.Done():
}
}
}()
// Close the reader when the context is done to cancel StdCopy if it's still running.
go func() {
<-ctx.Done()
reader.Close()
}()
return outCh, nil
}
// logsChannelWriter is a writer for stdcopy.StdCopy that sends demultiplexed container logs to a channel.
type logsChannelWriter struct {
ctx context.Context
ch chan<- api.ContainerLogEntry
isStderr bool
}
func (w *logsChannelWriter) Write(data []byte) (n int, err error) {
// Parse timestamp and message from the demultiplexed Docker log payload if the data looks like it contains one.
// Format: 2025-01-01T00:00:00.000000000Z message
timestamp := time.Time{}
message := data
if len(data) > 30 && data[4] == '-' && data[7] == '-' && data[10] == 'T' {
timestampPart, messagePart, found := bytes.Cut(data, []byte(" "))
if found {
timestamp, err = time.Parse(time.RFC3339Nano, string(timestampPart))
if err != nil {
timestamp = time.Time{}
}
message = messagePart
}
}
entry := api.ContainerLogEntry{
Timestamp: timestamp,
// Clone is required because message is a slice into data, which stdcopy.StdCopy may reuse
// after Write returns but before the entry is consumed from the channel.
Message: bytes.Clone(message),
}
if w.isStderr {
entry.Stream = api.LogStreamStderr
} else {
entry.Stream = api.LogStreamStdout
}
select {
case w.ch <- entry:
return len(data), nil
case <-w.ctx.Done():
return 0, w.ctx.Err()
}
}
+72
View File
@@ -0,0 +1,72 @@
package sshexec
import (
"context"
"fmt"
"io"
"os/exec"
"strconv"
"strings"
)
type SSHCLIRemote struct {
user string
host string
port int
keyPath string
}
func NewSSHCLIRemote(user, host string, port int, keyPath string) *SSHCLIRemote {
return &SSHCLIRemote{
user: user,
host: host,
port: port,
keyPath: keyPath,
}
}
// TODO: Refactor and reuse this with buildDialArgs and buildSSHArgs from
// SSHCLI Connector.
func (r *SSHCLIRemote) buildSSHArgs() []string {
args := []string{"-o", "ConnectTimeout=5"}
if r.port != 0 && r.port != 22 {
args = append(args, "-p", strconv.Itoa(r.port))
}
if r.keyPath != "" {
args = append(args, "-i", r.keyPath)
}
args = append(args, r.user+"@"+r.host)
return args
}
func (r *SSHCLIRemote) Run(ctx context.Context, cmd string) (string, error) {
args := r.buildSSHArgs()
args = append(args, cmd)
execCmd := exec.CommandContext(ctx, "ssh", args...)
output, err := execCmd.CombinedOutput()
if err != nil {
return strings.TrimSpace(string(output)),
fmt.Errorf("run command on remote host: %w: %s", err, string(output))
}
return strings.TrimSpace(string(output)), nil
}
func (r *SSHCLIRemote) Stream(ctx context.Context, cmd string, stdout, stderr io.Writer) error {
args := r.buildSSHArgs()
args = append(args, cmd)
execCmd := exec.CommandContext(ctx, "ssh", args...)
execCmd.Stdout = stdout
execCmd.Stderr = stderr
return execCmd.Run()
}
// no-op as there is no persistent connection.
func (r *SSHCLIRemote) Close() error {
return nil
}
+76
View File
@@ -0,0 +1,76 @@
package sshexec
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestSSHCLIRemote_buildSSHArgs(t *testing.T) {
t.Parallel()
tests := []struct {
name string
user string
host string
port int
keyPath string
expected []string
}{
{
name: "default port, no key",
user: "root",
host: "example.com",
port: 22,
keyPath: "",
expected: []string{"-o", "ConnectTimeout=5", "root@example.com"},
},
{
name: "custom port",
user: "ubuntu",
host: "192.168.1.10",
port: 2222,
keyPath: "",
expected: []string{"-o", "ConnectTimeout=5", "-p", "2222", "ubuntu@192.168.1.10"},
},
{
name: "with key path",
user: "root",
host: "example.com",
port: 22,
keyPath: "/path/to/key",
expected: []string{"-o", "ConnectTimeout=5", "-i", "/path/to/key", "root@example.com"},
},
{
name: "all options",
user: "admin",
host: "server.local",
port: 2222,
keyPath: "~/.ssh/id_rsa",
expected: []string{"-o", "ConnectTimeout=5", "-p", "2222", "-i", "~/.ssh/id_rsa", "admin@server.local"},
},
{
name: "port zero (default)",
user: "root",
host: "example.com",
port: 0,
keyPath: "",
expected: []string{"-o", "ConnectTimeout=5", "root@example.com"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
remote := &SSHCLIRemote{
user: tt.user,
host: tt.host,
port: tt.port,
keyPath: tt.keyPath,
}
result := remote.buildSSHArgs()
assert.Equal(t, tt.expected, result)
})
}
}
+1 -1
View File
@@ -22,7 +22,7 @@ Uncloud stores its configuration in `~/.config/uncloud/config.yaml`. If you wish
Begin by initialising the first node in your cluster with `uc machine init [USER@HOST:PORT]`. If you do not have a need for Caddy reverse proxy, you may disable this feature with `--no-caddy`. If you want to avoid using uncloud's managed DNS service, add the `--no-dns` flag.
This command will idempotently install Docker, uncloudd, uncloud-corrosion. If Caddy is enabled, it will set up a reverse proxy. If Uncloud DNS is enabled, it will create a DNS A record for the machine's public IP address under `*.[CLUSTER ID].cluster.uncloud.run`.
This command will idempotently install Docker, uncloudd, uncloud-corrosion. If Caddy is enabled, it will set up a reverse proxy. If Uncloud DNS is enabled, it will create a DNS A record for the machine's public IP address under `*.[CLUSTER ID].uncld.dev`.
If you wish to uninstall Uncloud and its components, run `uncloud-uninstall`.
+2 -41
View File
@@ -2,13 +2,10 @@ package api
import (
"context"
"fmt"
"strings"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/volume"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"google.golang.org/grpc/metadata"
)
type Client interface {
@@ -51,6 +48,8 @@ type ServiceClient interface {
RunService(ctx context.Context, spec ServiceSpec) (RunServiceResponse, error)
InspectService(ctx context.Context, id string) (Service, error)
RemoveService(ctx context.Context, id string) error
StopService(ctx context.Context, id string, opts container.StopOptions) error
StartService(ctx context.Context, id string) error
}
type VolumeClient interface {
@@ -58,41 +57,3 @@ type VolumeClient interface {
ListVolumes(ctx context.Context, filter *VolumeFilter) ([]MachineVolume, error)
RemoveVolume(ctx context.Context, machineNameOrID, volumeName string, force bool) error
}
// ProxyMachinesContext returns a new context that proxies gRPC requests to the specified machines.
// If namesOrIDs is nil, all machines are included.
func ProxyMachinesContext(
ctx context.Context, cli MachineClient, namesOrIDs []string,
) (context.Context, MachineMembersList, error) {
// TODO: move the machine IP resolution to the proxy router to allow setting machine names and IDs in the metadata.
machines, err := cli.ListMachines(ctx, nil)
if err != nil {
return nil, nil, fmt.Errorf("list machines: %w", err)
}
var proxiedMachines MachineMembersList
var notFound []string
for _, nameOrID := range namesOrIDs {
if m := machines.FindByNameOrID(nameOrID); m != nil {
proxiedMachines = append(proxiedMachines, m)
} else {
notFound = append(notFound, nameOrID)
}
}
if len(notFound) > 0 {
return nil, nil, fmt.Errorf("machines not found: %s", strings.Join(notFound, ", "))
}
if len(namesOrIDs) == 0 {
proxiedMachines = machines
}
md := metadata.New(nil)
for _, m := range proxiedMachines {
machineIP, _ := m.Machine.Network.ManagementIp.ToAddr()
md.Append("machines", machineIP.String())
}
return metadata.NewOutgoingContext(ctx, md), proxiedMachines, nil
}
+87
View File
@@ -0,0 +1,87 @@
package api
import (
"errors"
"time"
"github.com/psviderski/uncloud/internal/machine/api/pb"
)
const (
LogStreamUnknown LogStreamType = iota
LogStreamStdout
LogStreamStderr
// LogStreamHeartbeat represents a heartbeat log entry with a timestamp indicating that
// there are no older logs than this timestamp.
LogStreamHeartbeat
)
type LogStreamType int
// LogStreamTypeFromProto converts a protobuf ContainerLogEntry.StreamType to the internal LogStreamType.
func LogStreamTypeFromProto(s pb.ContainerLogEntry_StreamType) LogStreamType {
switch s {
case pb.ContainerLogEntry_STDOUT:
return LogStreamStdout
case pb.ContainerLogEntry_STDERR:
return LogStreamStderr
case pb.ContainerLogEntry_HEARTBEAT:
return LogStreamHeartbeat
default:
return LogStreamUnknown
}
}
// LogStreamTypeToProto converts LogStreamType to protobuf ContainerLogEntry.StreamType.
func LogStreamTypeToProto(s LogStreamType) pb.ContainerLogEntry_StreamType {
switch s {
case LogStreamStdout:
return pb.ContainerLogEntry_STDOUT
case LogStreamStderr:
return pb.ContainerLogEntry_STDERR
case LogStreamHeartbeat:
return pb.ContainerLogEntry_HEARTBEAT
default:
return pb.ContainerLogEntry_UNKNOWN
}
}
// ServiceLogsOptions specifies parameters for ServiceLogs.
type ServiceLogsOptions struct {
Follow bool
Tail int
Since string
Until string
// Machines filters logs to only include containers running on the specified machines (names or IDs).
// If empty, logs from all machines are included.
Machines []string
}
// ServiceLogEntry represents a single log entry from a service container.
type ServiceLogEntry struct {
// Metadata may not be set if an error occurred (Err is not nil).
Metadata ServiceLogEntryMetadata
ContainerLogEntry
}
// ServiceLogEntryMetadata contains metadata about the source of a log entry.
type ServiceLogEntryMetadata struct {
ServiceID string
ServiceName string
ContainerID string
MachineID string
MachineName string
}
// ContainerLogEntry represents a single log entry from a container.
type ContainerLogEntry struct {
Stream LogStreamType
Timestamp time.Time
Message []byte
// Err indicates that an error occurred while streaming logs from a container.
// Other fields are not set if Err is not nil.
Err error
}
// ErrLogStreamStalled indicates that a log stream stopped sending data and may be unresponsive.
var ErrLogStreamStalled = errors.New("log stream stopped responding")
+11
View File
@@ -8,6 +8,7 @@ import (
"slices"
"strings"
mapset "github.com/deckarep/golang-set/v2"
"github.com/distribution/reference"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
@@ -366,6 +367,16 @@ type MachineServiceContainer struct {
Container ServiceContainer
}
// MachineIDs returns a list of unique machine IDs where the service containers are running.
func (s *Service) MachineIDs() []string {
ids := mapset.NewSet[string]()
for _, mc := range s.Containers {
ids.Add(mc.MachineID)
}
return ids.ToSlice()
}
// Images returns a sorted list of unique images used by the service containers.
func (s *Service) Images() []string {
images := make(map[string]struct{})
+7
View File
@@ -71,6 +71,13 @@ func (cli *Client) NewCaddyDeployment(image, config string, placement api.Placem
Protocol: api.ProtocolTCP,
Mode: api.PortModeHost,
},
// Needed for HTTP/3 (QUIC)
{
PublishedPort: 443,
ContainerPort: 443,
Protocol: api.ProtocolUDP,
Mode: api.PortModeHost,
},
},
Volumes: []api.VolumeSpec{
{
+39
View File
@@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"os"
"strings"
"github.com/docker/cli/cli/streams"
"github.com/psviderski/uncloud/internal/machine/api/pb"
@@ -76,3 +77,41 @@ func proxyToMachine(ctx context.Context, machine *pb.MachineInfo) context.Contex
md := metadata.Pairs("machines", machineIP.String())
return metadata.NewOutgoingContext(ctx, md)
}
// ProxyMachinesContext returns a new context that proxies gRPC requests to the specified machines.
// If namesOrIDs is nil, all machines are included.
func (cli *Client) ProxyMachinesContext(
ctx context.Context, namesOrIDs []string,
) (context.Context, api.MachineMembersList, error) {
// TODO: move the machine IP resolution to the proxy router to allow setting machine names and IDs in the metadata.
machines, err := cli.ListMachines(ctx, nil)
if err != nil {
return nil, nil, fmt.Errorf("list machines: %w", err)
}
var proxiedMachines api.MachineMembersList
var notFound []string
for _, nameOrID := range namesOrIDs {
if m := machines.FindByNameOrID(nameOrID); m != nil {
proxiedMachines = append(proxiedMachines, m)
} else {
notFound = append(notFound, nameOrID)
}
}
if len(notFound) > 0 {
return nil, nil, fmt.Errorf("machines not found: %s", strings.Join(notFound, ", "))
}
if len(namesOrIDs) == 0 {
proxiedMachines = machines
}
md := metadata.New(nil)
for _, m := range proxiedMachines {
machineIP, _ := m.Machine.Network.ManagementIp.ToAddr()
md.Append("machines", machineIP.String())
}
return metadata.NewOutgoingContext(ctx, md), proxiedMachines, nil
}
+12 -1
View File
@@ -1,8 +1,11 @@
package compose
import (
"context"
"os"
"testing"
composecli "github.com/compose-spec/compose-go/v2/cli"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -178,7 +181,15 @@ services:
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
project, err := loadProjectFromContent(t, tt.composeYAML)
// Get current working directory for relative path resolution to testdata directory.
wd, err := os.Getwd()
require.NoError(t, err)
project, err := LoadProjectFromContent(
context.Background(),
tt.composeYAML,
composecli.WithWorkingDirectory(wd),
)
if tt.wantErr != "" {
require.ErrorContains(t, err, tt.wantErr)
+3 -6
View File
@@ -49,10 +49,6 @@ func NewDeploymentWithStrategy(ctx context.Context, cli Client, project *types.P
ClusterDomain: domain,
}
if strategy == nil {
strategy = &deploy.RollingStrategy{State: state}
}
return &Deployment{
Client: cli,
Project: project,
@@ -88,6 +84,7 @@ func (d *Deployment) Plan(ctx context.Context) (deploy.SequenceOperation, error)
}
// Check external volumes and plan the creation of missing volumes before deploying services.
// Updates the cluster state (d.state) with the scheduled volumes.
volumeOps, err := d.planVolumes(serviceSpecs)
if err != nil {
return plan, err
@@ -98,8 +95,8 @@ func (d *Deployment) Plan(ctx context.Context) (deploy.SequenceOperation, error)
for _, spec := range serviceSpecs {
// TODO: properly handle depends_on conditions in the service deployment plan as the first operation.
// Pass the update cluster state with scheduled volumes to the deployment.
deployment := deploy.NewDeployment(d.Client, spec, d.Strategy)
// Pass the updated cluster state with the scheduled volumes to the deployment.
deployment := deploy.NewDeploymentWithClusterState(d.Client, spec, d.Strategy, d.state)
servicePlan, err := deployment.Plan(ctx)
if err != nil {
return plan, fmt.Errorf("create deployment plan for service '%s': %w", spec.Name, err)
+4 -3
View File
@@ -1,6 +1,7 @@
package compose
import (
"context"
"net/netip"
"testing"
@@ -301,7 +302,7 @@ services:
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
project, err := loadProjectFromContent(t, tt.content)
project, err := LoadProjectFromContent(context.Background(), tt.content)
if tt.wantErr != "" {
require.Error(t, err)
assert.Contains(t, err.Error(), tt.wantErr)
@@ -375,7 +376,7 @@ services:
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
project, err := loadProjectFromContent(t, tt.content)
project, err := LoadProjectFromContent(context.Background(), tt.content)
require.NoError(t, err)
service, err := project.GetService("web")
@@ -402,7 +403,7 @@ services:
- "9090:9090/tcp@host"
`
project, err := loadProjectFromContent(t, content)
project, err := LoadProjectFromContent(context.Background(), content)
require.NoError(t, err)
service, err := project.GetService("web")
+23
View File
@@ -3,12 +3,15 @@ package compose
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
composecli "github.com/compose-spec/compose-go/v2/cli"
"github.com/compose-spec/compose-go/v2/types"
)
// LoadProject loads a Compose project from the default locations or the given paths.
func LoadProject(ctx context.Context, paths []string, opts ...composecli.ProjectOptionsFn) (*types.Project, error) {
defaultOpts := []composecli.ProjectOptionsFn{
// First apply os.Environment, always wins.
@@ -61,6 +64,26 @@ func LoadProject(ctx context.Context, paths []string, opts ...composecli.Project
return project, nil
}
// LoadProjectFromContent loads a Compose project from the given YAML content.
func LoadProjectFromContent(
ctx context.Context, content string, opts ...composecli.ProjectOptionsFn,
) (*types.Project, error) {
// Create a temporary directory for the compose file.
tmpDir, err := os.MkdirTemp("", "uncloud-compose-*")
if err != nil {
return nil, fmt.Errorf("create temporary directory: %w", err)
}
defer os.RemoveAll(tmpDir)
// Write the YAML content to compose.yaml in the temporary directory.
composePath := filepath.Join(tmpDir, "compose.yaml")
if err := os.WriteFile(composePath, []byte(content), 0o644); err != nil {
return nil, fmt.Errorf("write compose file: %w", err)
}
return LoadProject(ctx, []string{composePath}, opts...)
}
// removeProjectPrefixFromNames removes the project name prefix from volume names.
func removeProjectPrefixFromNames(project *types.Project) {
prefix := project.Name + "_"
+1 -1
View File
@@ -249,7 +249,7 @@ func dockerVolumeSpecFromCompose(serviceVolume types.ServiceVolumeConfig, volume
}
if !volume.External {
if volume.Driver != "" {
if volume.Driver != "" || len(volume.DriverOpts) > 0 {
spec.VolumeOptions.Driver = &mount.Driver{
Name: volume.Driver,
Options: volume.DriverOpts,
+121 -56
View File
@@ -3,13 +3,13 @@ package compose
import (
"context"
"net/netip"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"github.com/compose-spec/compose-go/v2/loader"
"github.com/compose-spec/compose-go/v2/types"
composecli "github.com/compose-spec/compose-go/v2/cli"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/mount"
"github.com/docker/go-units"
@@ -20,57 +20,6 @@ import (
"github.com/stretchr/testify/require"
)
// loadProjectFromContent loads a compose project from YAML content.
// Keep the implementation in sync with LoadProject.
func loadProjectFromContent(t *testing.T, content string) (*types.Project, error) {
t.Helper()
ctx := context.Background()
configDetails := types.ConfigDetails{
ConfigFiles: []types.ConfigFile{
{
Filename: "docker-compose.yml",
Content: []byte(content),
},
},
}
project, err := loader.LoadWithContext(ctx, configDetails, func(o *loader.Options) {
o.SetProjectName("test", true)
// Register our custom extensions
if o.KnownExtensions == nil {
o.KnownExtensions = map[string]any{}
}
o.KnownExtensions[CaddyExtensionKey] = Caddy{}
o.KnownExtensions[PortsExtensionKey] = PortsSource{}
o.KnownExtensions[MachinesExtensionKey] = MachinesSource{}
})
if err != nil {
return nil, err
}
removeProjectPrefixFromNames(project)
// Apply extension transformations since we're not using LoadProject.
if project, err = transformServicesCaddyExtension(project); err != nil {
return nil, err
}
if project, err = transformServicesPortsExtension(project); err != nil {
return nil, err
}
// Validate extension combinations after all transformations.
if err = validateServicesExtensions(project); err != nil {
return nil, err
}
// Process image templates in services to expand Go template expressions using git repo state.
if project, err = ProcessImageTemplates(project); err != nil {
return nil, err
}
return project, nil
}
func TestServiceSpecFromCompose(t *testing.T) {
t.Parallel()
@@ -431,7 +380,15 @@ services:
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
project, err := loadProjectFromContent(t, tt.composeYAML)
// Get current working directory for relative path resolution to testdata directory.
wd, err := os.Getwd()
require.NoError(t, err)
project, err := LoadProjectFromContent(
context.Background(),
tt.composeYAML,
composecli.WithWorkingDirectory(wd),
)
require.NoError(t, err)
spec, err := ServiceSpecFromCompose(project, "web")
@@ -617,7 +574,7 @@ services:
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
project, err := loadProjectFromContent(t, tt.composeYAML)
project, err := LoadProjectFromContent(context.Background(), tt.composeYAML)
require.NoError(t, err)
spec, err := ServiceSpecFromCompose(project, "ai")
@@ -629,6 +586,114 @@ services:
}
}
func TestServiceSpecFromCompose_VolumeDriverOpts(t *testing.T) {
tests := []struct {
name string
composeYAML string
expectedVolume api.VolumeSpec
}{
{
name: "volume with driver_opts only (no driver specified)",
composeYAML: `
services:
test:
image: nginx
volumes:
- nfsmount:/data/nfs
volumes:
nfsmount:
driver_opts:
type: "nfs"
o: "addr=192.168.1.100,nolock,soft"
device: ":/mnt/share"
`,
expectedVolume: api.VolumeSpec{
Name: "nfsmount",
Type: api.VolumeTypeVolume,
VolumeOptions: &api.VolumeOptions{
Name: "nfsmount",
Driver: &mount.Driver{
Options: map[string]string{
"type": "nfs",
"o": "addr=192.168.1.100,nolock,soft",
"device": ":/mnt/share",
},
},
},
},
},
{
name: "volume with driver and driver_opts",
composeYAML: `
services:
test:
image: nginx
volumes:
- nfsmount:/data/nfs
volumes:
nfsmount:
driver: local
driver_opts:
type: "nfs"
o: "addr=192.168.1.100,nolock,soft"
device: ":/mnt/share"
`,
expectedVolume: api.VolumeSpec{
Name: "nfsmount",
Type: api.VolumeTypeVolume,
VolumeOptions: &api.VolumeOptions{
Name: "nfsmount",
Driver: &mount.Driver{
Name: "local",
Options: map[string]string{
"type": "nfs",
"o": "addr=192.168.1.100,nolock,soft",
"device": ":/mnt/share",
},
},
},
},
},
{
name: "external volume ignores driver_opts",
composeYAML: `
services:
test:
image: nginx
volumes:
- extvolume:/data
volumes:
extvolume:
external: true
`,
expectedVolume: api.VolumeSpec{
Name: "extvolume",
Type: api.VolumeTypeVolume,
VolumeOptions: &api.VolumeOptions{
Name: "extvolume",
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
project, err := LoadProjectFromContent(context.Background(), tt.composeYAML)
require.NoError(t, err)
spec, err := ServiceSpecFromCompose(project, "test")
require.NoError(t, err)
require.Len(t, spec.Volumes, 1, "expected exactly one volume")
actualVolume := spec.Volumes[0]
cmpOpts := cmp.Options{cmpopts.EquateEmpty()}
assert.True(t, cmp.Equal(actualVolume, tt.expectedVolume, cmpOpts...),
"Volume mismatch:\n%s", cmp.Diff(actualVolume, tt.expectedVolume, cmpOpts...))
})
}
}
func TestServiceSpecFromCompose_XMachinesPlacement(t *testing.T) {
tests := []struct {
name string
@@ -765,7 +830,7 @@ services:
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
project, err := loadProjectFromContent(t, tt.composeYAML)
project, err := LoadProjectFromContent(context.Background(), tt.composeYAML)
if tt.expectError {
assert.Error(t, err)
+41
View File
@@ -0,0 +1,41 @@
package connector
import (
"context"
"fmt"
"golang.org/x/net/proxy"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
)
// UnixConnector establishes a connection to the machine API through a unix domain socket.
type UnixConnector struct {
socketPath string
}
func NewUnixConnector(socketPath string) *UnixConnector {
return &UnixConnector{socketPath: socketPath}
}
func (c *UnixConnector) Connect(_ context.Context) (*grpc.ClientConn, error) {
// gRPC uses "unix:path" syntax for unix sockets.
target := "unix:" + c.socketPath
conn, err := grpc.NewClient(
target,
grpc.WithTransportCredentials(insecure.NewCredentials()),
)
if err != nil {
return nil, fmt.Errorf("create machine API client: %w", err)
}
return conn, nil
}
func (c *UnixConnector) Dialer() (proxy.ContextDialer, error) {
return nil, fmt.Errorf("proxy connections are not supported over a unix connection")
}
func (c *UnixConnector) Close() error {
return nil
}
+21 -1
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"github.com/psviderski/uncloud/pkg/api"
"github.com/psviderski/uncloud/pkg/client/deploy/scheduler"
)
type Client interface {
@@ -25,6 +26,8 @@ type Deployment struct {
Strategy Strategy
cli Client
plan *Plan
// state is an optional current and planned cluster state used for scheduling decisions.
state *scheduler.ClusterState
}
type Plan struct {
@@ -47,6 +50,16 @@ func NewDeployment(cli Client, spec api.ServiceSpec, strategy Strategy) *Deploym
}
}
// NewDeploymentWithClusterState creates a new deployment like NewDeployment but also with a provided current cluster
// state used for scheduling decisions.
func NewDeploymentWithClusterState(
cli Client, spec api.ServiceSpec, strategy Strategy, state *scheduler.ClusterState,
) *Deployment {
d := NewDeployment(cli, spec, strategy)
d.state = state
return d
}
// Plan returns a plan of operations to reconcile the service to the desired state.
// If a plan has already been created, the same plan will be returned.
func (d *Deployment) Plan(ctx context.Context) (Plan, error) {
@@ -73,7 +86,14 @@ func (d *Deployment) Plan(ctx context.Context) (Plan, error) {
return Plan{}, fmt.Errorf("resolve service spec: %w", err)
}
plan, err := d.Strategy.Plan(ctx, d.cli, d.Service, resolvedSpec)
if d.state == nil {
d.state, err = scheduler.InspectClusterState(ctx, d.cli)
if err != nil {
return Plan{}, fmt.Errorf("inspect cluster state: %w", err)
}
}
plan, err := d.Strategy.Plan(d.state, d.Service, resolvedSpec)
if err != nil {
return Plan{}, fmt.Errorf("create plan using %s strategy: %w", d.Strategy.Type(), err)
}
+2 -1
View File
@@ -17,7 +17,7 @@ import (
// - If a volume already exists on a machine, it must be used instead of creating a new one.
// - A missing volume must only be created on one machine.
type VolumeScheduler struct {
// state is the current state of machines and their resources in the cluster.
// state is the current and planned state of machines and their resources in the cluster.
state *ClusterState
// serviceSpecs is a list of service specifications included in the deployment.
serviceSpecs []api.ServiceSpec
@@ -106,6 +106,7 @@ func NewVolumeScheduler(state *ClusterState, specs []api.ServiceSpec) (*VolumeSc
// Schedule determines what missing volumes should be created and where for services in the multi-service deployment.
// It returns a map of machine IDs to a list of api.VolumeSpec that should be created on that machine,
// or an error if services can't be scheduled due to scheduling constraints.
// It also updates the state of the machines in the cluster state to reflect the scheduled volumes.
func (s *VolumeScheduler) Schedule() (map[string][]api.VolumeSpec, error) {
if len(s.serviceSpecs) == 0 {
// No services with volume mounts, nothing to schedule.
+14 -15
View File
@@ -1,7 +1,6 @@
package deploy
import (
"context"
"fmt"
"math/rand/v2"
"slices"
@@ -18,31 +17,31 @@ type Strategy interface {
// Type returns the type of the deployment strategy, e.g. "rolling", "blue-green".
Type() string
// Plan returns the operation to reconcile the service to the desired state.
// If the service does not exist (new deployment), svc will be nil.
Plan(ctx context.Context, cli scheduler.Client, svc *api.Service, spec api.ServiceSpec) (Plan, error)
// If the service does not exist (new deployment), svc will be nil. state provides the current and planned state
// of the cluster for scheduling decisions.
Plan(state *scheduler.ClusterState, svc *api.Service, spec api.ServiceSpec) (Plan, error)
}
// RollingStrategy implements a rolling update deployment pattern where containers are updated one at a time
// to minimize service disruption.
type RollingStrategy struct {
State *scheduler.ClusterState
// ForceRecreate indicates whether all containers should be recreated during the deployment,
// regardless of whether their specifications have changed.
ForceRecreate bool
// state is the current and planned state of the cluster used for scheduling decisions.
state *scheduler.ClusterState
}
func (s *RollingStrategy) Type() string {
return "rolling"
}
func (s *RollingStrategy) Plan(
ctx context.Context, cli scheduler.Client, svc *api.Service, spec api.ServiceSpec,
) (Plan, error) {
if s.State == nil {
state, err := scheduler.InspectClusterState(ctx, cli)
if err != nil {
return Plan{}, fmt.Errorf("inspect cluster state: %w", err)
}
s.State = state
func (s *RollingStrategy) Plan(state *scheduler.ClusterState, svc *api.Service, spec api.ServiceSpec) (Plan, error) {
if state == nil {
return Plan{}, fmt.Errorf("cluster state must be provided")
}
s.state = state
// We can assume that the spec is valid at this point because it has been validated by the deployment.
switch spec.Mode {
@@ -65,7 +64,7 @@ func (s *RollingStrategy) planReplicated(svc *api.Service, spec api.ServiceSpec)
return plan, err
}
sched := scheduler.NewServiceScheduler(s.State, spec)
sched := scheduler.NewServiceScheduler(s.state, spec)
// TODO: return a detailed report on required constraints and which ones are satisfied?
availableMachines, err := sched.EligibleMachines()
if err != nil {
@@ -221,7 +220,7 @@ func (s *RollingStrategy) planGlobal(svc *api.Service, spec api.ServiceSpec) (Pl
}
}
sched := scheduler.NewServiceScheduler(s.State, spec)
sched := scheduler.NewServiceScheduler(s.state, spec)
availableMachines, err := sched.EligibleMachines()
if err != nil {
return plan, err
+1 -13
View File
@@ -48,7 +48,7 @@ func (cli *Client) InspectRemoteImage(ctx context.Context, id string) ([]api.Mac
// it lists images on all machines.
func (cli *Client) ListImages(ctx context.Context, filter api.ImageFilter) ([]api.MachineImages, error) {
// Broadcast the image list request to the specified machines or all machines if none specified.
listCtx, machines, err := api.ProxyMachinesContext(ctx, cli, filter.Machines)
listCtx, machines, err := cli.ProxyMachinesContext(ctx, filter.Machines)
if err != nil {
return nil, fmt.Errorf("create request context to broadcast to machines: %w", err)
}
@@ -149,18 +149,6 @@ func (cli *Client) PushImage(ctx context.Context, image string, opts PushImageOp
return fmt.Errorf("list machines: %w", err)
}
// Check if all specified machines were found.
if len(machineMembers) != len(opts.Machines) {
var notFound []string
for _, nameOrID := range opts.Machines {
if machineMembers.FindByNameOrID(nameOrID) == nil {
notFound = append(notFound, nameOrID)
}
}
return fmt.Errorf("machines not found: %s", strings.Join(notFound, ", "))
}
for _, mm := range machineMembers {
machines = append(machines, mm.Machine)
}
+310
View File
@@ -0,0 +1,310 @@
package client
import (
"container/heap"
"sync"
"time"
"github.com/psviderski/uncloud/pkg/api"
)
// logMergerMaxInFlightPerStream limits how many entries each input stream can have in the processing queue before being
// throttled. This ensures fair interleaving between streams and prevents one fast stream from causing unbounded
// buffering while waiting for slower streams.
const (
logMergerMaxInFlightPerStream = 100
// logMergerHeartbeatDebounceInterval defines the minimum interval between deduplicated heartbeat entries emitted
// by the LogMerger. Keep it in sync with logsHeartbeatInterval internal/machine/docker/server.go.
logMergerHeartbeatDebounceInterval = 200 * time.Millisecond
)
// LogMergerOptions configures the behavior of LogMerger.
type LogMergerOptions struct {
// StallTimeout specifies how long a stream can go without receiving any data before it's considered
// stalled and excluded from watermark calculation. A zero timeout disables stall detection.
StallTimeout time.Duration
// StallCheckInterval specifies how often to check for stalled streams.
StallCheckInterval time.Duration
}
// DefaultLogMergerOptions provides sensible default options that enable stall detection for LogMerger.
var DefaultLogMergerOptions = LogMergerOptions{
StallTimeout: 10 * time.Second,
StallCheckInterval: 1 * time.Second,
}
// LogMerger merges multiple log streams into a single chronologically ordered stream based on timestamps.
// It uses a low watermark algorithm to ensure proper ordering across streams.
// Heartbeat entries from streams advance the watermark to enable timely emission of buffered logs.
type LogMerger struct {
streams []*mergerStream
queue logsHeap
// watermark is min(latest_timestamp for each stream).
watermark time.Time
output chan api.ServiceLogEntry
// lastEmitted is the timestamp of the last emitted log entry or heartbeat.
lastEmitted time.Time
stallTimeout time.Duration
stallCheckInterval time.Duration
}
// NewLogMerger creates a new LogMerger for the given input streams with the specified options.
func NewLogMerger(streams []<-chan api.ServiceLogEntry, opts LogMergerOptions) *LogMerger {
mergerStreams := make([]*mergerStream, len(streams))
now := time.Now()
for i, ch := range streams {
mergerStreams[i] = &mergerStream{
stream: ch,
semaphore: make(chan struct{}, logMergerMaxInFlightPerStream),
lastActivity: now,
}
}
return &LogMerger{
streams: mergerStreams,
output: make(chan api.ServiceLogEntry),
stallTimeout: opts.StallTimeout,
stallCheckInterval: opts.StallCheckInterval,
}
}
// Stream starts the merge process and returns a channel that emits log entries in chronological order.
// The returned channel is closed when all input streams are closed.
func (m *LogMerger) Stream() <-chan api.ServiceLogEntry {
if len(m.streams) == 0 {
close(m.output)
return m.output
}
go m.run()
return m.output
}
// mergerStream combines a stream channel with its state and flow control.
type mergerStream struct {
stream <-chan api.ServiceLogEntry
semaphore chan struct{}
// Latest timestamp seen from this stream (log or heartbeat).
lastSeen time.Time
// Wall clock time when we last received any data from this stream.
lastActivity time.Time
// Metadata associated with this stream. It's populated from the first log entry received.
metadata *api.ServiceLogEntryMetadata
// Whether the stream channel has closed.
closed bool
// Whether the stream is considered stalled (no data received within timeout).
stalled bool
}
// streamEvent represents an event from a stream (entry received or stream closed).
type streamEvent struct {
stream *mergerStream
entry api.ServiceLogEntry
closed bool
}
// queuedEntry wraps a log entry with its source semaphore for release tracking.
type queuedEntry struct {
entry api.ServiceLogEntry
semaphore chan struct{}
}
// run is the main processing loop that merges all streams.
func (m *LogMerger) run() {
defer close(m.output)
// Fan-in channel for stream events.
events := make(chan streamEvent)
// Start a reader goroutine for each stream to send entries to the events channel with flow control.
var wg sync.WaitGroup
for _, stream := range m.streams {
wg.Go(func() {
for entry := range stream.stream {
// Acquire semaphore slot before sending the entry to limit in-flight unprocessed entries per stream.
stream.semaphore <- struct{}{}
events <- streamEvent{stream: stream, entry: entry}
}
events <- streamEvent{stream: stream, closed: true}
})
}
// Close events channel when all readers finish.
go func() {
wg.Wait()
close(events)
}()
// Set up stall detection timer if enabled.
var stallCh <-chan time.Time
if m.stallTimeout > 0 && m.stallCheckInterval > 0 {
stallTicker := time.NewTicker(m.stallCheckInterval)
stallCh = stallTicker.C
defer stallTicker.Stop()
}
// Process events and emit entries.
for {
select {
case e, ok := <-events:
if !ok {
// All streams closed: flush remaining entries in order.
for m.queue.Len() > 0 {
qe := heap.Pop(&m.queue).(queuedEntry)
m.output <- qe.entry
<-qe.semaphore
}
return
}
e.stream.lastActivity = time.Now()
if e.stream.stalled {
e.stream.stalled = false
}
if e.stream.metadata == nil {
e.stream.metadata = &e.entry.Metadata
}
if e.closed {
e.stream.closed = true
m.updateWatermark()
m.emitReadyEntries()
continue
}
// Forward errors immediately and release semaphore.
if e.entry.Err != nil {
m.output <- e.entry
<-e.stream.semaphore
continue
}
if e.entry.Timestamp.After(e.stream.lastSeen) {
e.stream.lastSeen = e.entry.Timestamp
}
if e.entry.Stream == api.LogStreamStdout || e.entry.Stream == api.LogStreamStderr {
heap.Push(&m.queue, queuedEntry{entry: e.entry, semaphore: e.stream.semaphore})
}
m.updateWatermark()
m.emitReadyEntries()
// When merging streams, each input emits its own heartbeats. We want to debounce them and emit our own
// heartbeats at the same rate as a single input stream. Note that we need to adjust the heartbeat timestamp
// to the current watermark to not violate ordering guarantees.
if e.entry.Stream == api.LogStreamHeartbeat {
if m.watermark.Sub(m.lastEmitted) >= logMergerHeartbeatDebounceInterval {
heartbeat := e.entry
heartbeat.Timestamp = m.watermark
m.output <- heartbeat
m.lastEmitted = m.watermark
}
// Heartbeat processed, release semaphore.
<-e.stream.semaphore
}
case <-stallCh:
stalled := m.checkStalledStreams()
if len(stalled) == 0 {
continue
}
for _, s := range stalled {
errEntry := api.ServiceLogEntry{
ContainerLogEntry: api.ContainerLogEntry{
Err: api.ErrLogStreamStalled,
},
}
if s.metadata != nil {
errEntry.Metadata = *s.metadata
}
m.output <- errEntry
}
m.updateWatermark()
m.emitReadyEntries()
}
}
}
// checkStalledStreams marks streams as stalled if they haven't received any data within the timeout.
// Returns true if any stream's stalled state changed.
func (m *LogMerger) checkStalledStreams() []*mergerStream {
var stalled []*mergerStream
now := time.Now()
for _, s := range m.streams {
if s.closed || s.stalled {
continue
}
if now.Sub(s.lastActivity) > m.stallTimeout {
s.stalled = true
stalled = append(stalled, s)
}
}
return stalled
}
// updateWatermark recalculates the low watermark based on the lastSeen timestamps of all active streams.
func (m *LogMerger) updateWatermark() {
first := true
for _, s := range m.streams {
if s.closed || s.stalled {
// Closed and stalled streams don't affect watermark.
continue
}
if first || s.lastSeen.Before(m.watermark) {
m.watermark = s.lastSeen
first = false
}
}
}
// emitReadyEntries pops and emits all buffered entries from the queue with timestamp before the watermark.
func (m *LogMerger) emitReadyEntries() {
if m.watermark.IsZero() {
// No entries received yet.
return
}
for m.queue.Len() > 0 && m.queue[0].entry.Timestamp.Compare(m.watermark) <= 0 {
qe := heap.Pop(&m.queue).(queuedEntry)
m.output <- qe.entry
m.lastEmitted = qe.entry.Timestamp
<-qe.semaphore
}
}
// logsHeap is a min-heap (heap.Interface) of queued entries ordered by timestamp.
type logsHeap []queuedEntry
func (h *logsHeap) Len() int {
return len(*h)
}
func (h *logsHeap) Less(i, j int) bool {
return (*h)[i].entry.Timestamp.Before((*h)[j].entry.Timestamp)
}
func (h *logsHeap) Swap(i, j int) {
(*h)[i], (*h)[j] = (*h)[j], (*h)[i]
}
func (h *logsHeap) Push(x any) {
*h = append(*h, x.(queuedEntry))
}
func (h *logsHeap) Pop() any {
old := *h
n := len(old)
x := old[n-1]
*h = old[:n-1]
return x
}
+383
View File
@@ -0,0 +1,383 @@
package client
import (
"testing"
"time"
"github.com/psviderski/uncloud/pkg/api"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// testEntry creates a ServiceLogEntry for testing.
func testEntry(stream api.LogStreamType, ts time.Time, msg string) api.ServiceLogEntry {
return api.ServiceLogEntry{
ContainerLogEntry: api.ContainerLogEntry{
Stream: stream,
Timestamp: ts,
Message: []byte(msg),
},
}
}
// collectEntries collects up to maxCount entries from the channel or until it is closed.
func collectEntries(t *testing.T, ch <-chan api.ServiceLogEntry, maxCount int) []api.ServiceLogEntry {
t.Helper()
var entries []api.ServiceLogEntry
for i := 0; i < maxCount || maxCount <= 0; i++ {
select {
case e, ok := <-ch:
if !ok {
return entries
}
entries = append(entries, e)
case <-time.After(1 * time.Second):
require.FailNow(t, "timed out waiting for log entry")
}
}
return entries
}
func TestLogMerger_EmptyStreams(t *testing.T) {
t.Parallel()
merger := NewLogMerger(nil, LogMergerOptions{})
output := merger.Stream()
_, ok := <-output
assert.False(t, ok, "output channel should be closed for empty streams")
}
func TestLogMerger_SingleStream(t *testing.T) {
t.Parallel()
ch := make(chan api.ServiceLogEntry, 10)
merger := NewLogMerger([]<-chan api.ServiceLogEntry{ch}, LogMergerOptions{})
output := merger.Stream()
t1 := time.Now()
t2 := t1.Add(time.Second)
ch <- testEntry(api.LogStreamStdout, t1, "first")
ch <- testEntry(api.LogStreamStdout, t2, "second")
results := collectEntries(t, output, 2)
require.Len(t, results, 2)
assert.Equal(t, "first", string(results[0].Message))
assert.Equal(t, "second", string(results[1].Message))
close(ch)
results = collectEntries(t, output, 0)
assert.Len(t, results, 0, "no entries expected after close")
}
func TestLogMerger_PreservesData(t *testing.T) {
t.Parallel()
ch := make(chan api.ServiceLogEntry, 10)
merger := NewLogMerger([]<-chan api.ServiceLogEntry{ch}, LogMergerOptions{})
output := merger.Stream()
metadata := api.ServiceLogEntryMetadata{
ServiceID: "svc-123",
ServiceName: "my-service",
MachineID: "machine-456",
MachineName: "machine-1",
}
e := api.ServiceLogEntry{
Metadata: metadata,
ContainerLogEntry: api.ContainerLogEntry{
Stream: api.LogStreamStdout,
Timestamp: time.Now(),
Message: []byte("test"),
},
}
ch <- e
close(ch)
results := collectEntries(t, output, 0)
require.Len(t, results, 1)
assert.Equal(t, e, results[0])
}
func TestLogMerger_BasicMerge(t *testing.T) {
t.Parallel()
ch1 := make(chan api.ServiceLogEntry, 10)
ch2 := make(chan api.ServiceLogEntry, 10)
merger := NewLogMerger([]<-chan api.ServiceLogEntry{ch1, ch2}, LogMergerOptions{})
output := merger.Stream()
t1 := time.Now()
t2 := t1.Add(time.Second)
t3 := t1.Add(2 * time.Second)
t4 := t1.Add(3 * time.Second)
// Send interleaved entries from both streams.
ch1 <- testEntry(api.LogStreamStdout, t1, "ch1-first")
ch2 <- testEntry(api.LogStreamStdout, t2, "ch2-first")
ch1 <- testEntry(api.LogStreamStdout, t3, "ch1-second")
ch2 <- testEntry(api.LogStreamStdout, t4, "ch2-second")
// Send heartbeats to advance watermark past all entries.
t5 := t1.Add(4 * time.Second)
ch1 <- testEntry(api.LogStreamHeartbeat, t5, "")
ch2 <- testEntry(api.LogStreamHeartbeat, t5, "")
results := collectEntries(t, output, 4)
require.Len(t, results, 4)
assert.Equal(t, "ch1-first", string(results[0].Message))
assert.Equal(t, "ch2-first", string(results[1].Message))
assert.Equal(t, "ch1-second", string(results[2].Message))
assert.Equal(t, "ch2-second", string(results[3].Message))
close(ch1)
close(ch2)
results = collectEntries(t, output, 0)
assert.Len(t, results, 1, "one debounced heartbeat expected after close")
assert.Equal(t, api.LogStreamHeartbeat, results[0].Stream)
assert.Equal(t, t5, results[0].Timestamp)
}
func TestLogMerger_HeartbeatAdvancesWatermark(t *testing.T) {
t.Parallel()
ch1 := make(chan api.ServiceLogEntry, 10)
ch2 := make(chan api.ServiceLogEntry, 10)
merger := NewLogMerger([]<-chan api.ServiceLogEntry{ch1, ch2}, LogMergerOptions{})
output := merger.Stream()
t1 := time.Now()
t2 := t1.Add(time.Second)
t3 := t1.Add(2 * time.Second)
// Stream 1 sends a log entry.
ch1 <- testEntry(api.LogStreamStdout, t1, "ch1-first")
ch1 <- testEntry(api.LogStreamStdout, t3, "ch1-second")
// Stream 2 is quiet but sends a heartbeat.
ch2 <- testEntry(api.LogStreamHeartbeat, t2, "")
// Now stream 1's entry should be emitted because watermark is t1 (min of t1, t2).
results := collectEntries(t, output, 1)
require.Len(t, results, 1)
assert.Equal(t, "ch1-first", string(results[0].Message))
close(ch1)
close(ch2)
results = collectEntries(t, output, 0)
require.NotEmpty(t, results)
// Depending on timing, we may get a heartbeat before the second log entry.
require.LessOrEqual(t, len(results), 2, "one or two entries expected after close")
if len(results) == 2 {
assert.Equal(t, api.LogStreamHeartbeat, results[0].Stream)
assert.Equal(t, t2, results[0].Timestamp)
results = results[1:]
}
assert.Equal(t, "ch1-second", string(results[0].Message))
}
func TestLogMerger_ErrorForwarding(t *testing.T) {
t.Parallel()
ch1 := make(chan api.ServiceLogEntry, 10)
ch2 := make(chan api.ServiceLogEntry, 10)
merger := NewLogMerger([]<-chan api.ServiceLogEntry{ch1, ch2}, LogMergerOptions{})
output := merger.Stream()
t1 := time.Now()
ch1 <- testEntry(api.LogStreamStdout, t1, "ch1-first")
// Send an error entry.
ch1 <- api.ServiceLogEntry{
ContainerLogEntry: api.ContainerLogEntry{
Err: assert.AnError,
},
}
results := collectEntries(t, output, 1)
require.Len(t, results, 1, "expected error to emitted out of order")
assert.Equal(t, assert.AnError, results[0].Err)
close(ch1)
close(ch2)
results = collectEntries(t, output, 0)
assert.Len(t, results, 1)
assert.Equal(t, "ch1-first", string(results[0].Message))
}
func TestLogMerger_OutOfOrderSingleStream(t *testing.T) {
t.Parallel()
ch := make(chan api.ServiceLogEntry, 10)
merger := NewLogMerger([]<-chan api.ServiceLogEntry{ch}, LogMergerOptions{})
output := merger.Stream()
t1 := time.Now()
t2 := t1.Add(time.Second)
t3 := t1.Add(2 * time.Second)
t4 := t1.Add(3 * time.Second)
// Send entries out of order within the stream.
ch <- testEntry(api.LogStreamStdout, t3, "third")
ch <- testEntry(api.LogStreamStdout, t2, "second")
ch <- testEntry(api.LogStreamStdout, t4, "forth")
ch <- testEntry(api.LogStreamStdout, t1, "first")
results := collectEntries(t, output, 4)
require.Len(t, results, 4)
// Emitted in the same order as sent since no buffering/reordering is done within a single stream.
assert.Equal(t, "third", string(results[0].Message))
assert.Equal(t, "second", string(results[1].Message))
assert.Equal(t, "forth", string(results[2].Message))
assert.Equal(t, "first", string(results[3].Message))
close(ch)
results = collectEntries(t, output, 0)
assert.Len(t, results, 0, "no entries expected after close")
}
// Test that entries from streams with different rates are merged correctly in chronological order.
func TestLogMerger_UnevenStreams(t *testing.T) {
t.Parallel()
numFastEntries := logMergerMaxInFlightPerStream
// Use buffered channels to allow sends without blocking on receiver.
ch1 := make(chan api.ServiceLogEntry, numFastEntries+5)
ch2 := make(chan api.ServiceLogEntry, 10)
merger := NewLogMerger([]<-chan api.ServiceLogEntry{ch1, ch2}, LogMergerOptions{})
output := merger.Stream()
baseTime := time.Now()
// Stream 1 sends many entries quickly (0ms - 104ms).
for i := 0; i < numFastEntries; i++ {
ch1 <- testEntry(api.LogStreamStdout, baseTime.Add(time.Duration(i)*time.Millisecond), "fast")
}
// Stream 2 sends entries that interleave with fast entries.
ch2 <- testEntry(api.LogStreamStdout, baseTime.Add(5*time.Millisecond), "slow")
ch2 <- testEntry(api.LogStreamStdout, baseTime.Add(50*time.Millisecond), "slow")
// Send heartbeats to advance watermark past all entries so they get emitted.
finalTime := baseTime.Add(time.Second)
ch1 <- testEntry(api.LogStreamHeartbeat, finalTime, "")
ch2 <- testEntry(api.LogStreamHeartbeat, finalTime.Add(10*time.Millisecond), "")
// Read exactly numFastEntries + 2 log entries.
expectedCount := numFastEntries + 2
results := collectEntries(t, output, expectedCount)
require.Len(t, results, expectedCount)
// Count entries from each stream.
fastCount := 0
slowCount := 0
for _, entry := range results {
switch string(entry.Message) {
case "fast":
fastCount++
case "slow":
slowCount++
}
}
assert.Equal(t, numFastEntries, fastCount)
assert.Equal(t, 2, slowCount)
// Verify entries are in chronological order.
for i := 1; i < len(results); i++ {
assert.False(t, results[i].Timestamp.Before(results[i-1].Timestamp),
"entry %d (ts=%v) should not be before entry %d (ts=%v)",
i, results[i].Timestamp, i-1, results[i-1].Timestamp)
}
close(ch1)
close(ch2)
results = collectEntries(t, output, 0)
require.Len(t, results, 1, "one debounced heartbeat expected after close")
assert.Equal(t, api.LogStreamHeartbeat, results[0].Stream)
}
func TestLogMerger_StalledStreamExcludedFromWatermark(t *testing.T) {
t.Parallel()
ch1 := make(chan api.ServiceLogEntry, 10)
ch2 := make(chan api.ServiceLogEntry, 10)
stallTimeout := 100 * time.Millisecond
merger := NewLogMerger([]<-chan api.ServiceLogEntry{ch1, ch2}, LogMergerOptions{
StallTimeout: stallTimeout,
StallCheckInterval: 20 * time.Millisecond,
})
output := merger.Stream()
//// Collect all output entries in a separate goroutine.
//var results []api.ServiceLogEntry
//doneCollecting := make(chan struct{})
//go func() {
// for entry := range output {
// results = append(results, entry)
// }
// close(doneCollecting)
//}()
t1 := time.Now()
t2 := t1.Add(time.Second)
t3 := t1.Add(2 * time.Second)
t4 := t1.Add(3 * time.Second)
e1 := testEntry(api.LogStreamStdout, t1, "ch1-entry")
e1.Metadata = api.ServiceLogEntryMetadata{
ServiceID: "ch1-svc1",
ServiceName: "ch1-svcName",
ContainerID: "ch1-ctr1",
MachineID: "ch1-machine1",
MachineName: "ch1-machineName",
}
ch1 <- e1
ch2 <- testEntry(api.LogStreamStdout, t2, "ch2-first")
// Watermark is t1 now so ch1's entry could be collected.
results := collectEntries(t, output, 1)
require.Len(t, results, 1)
assert.Equal(t, "ch1-entry", string(results[0].Message))
// Keep pushing to stream 2 so that it does not stall. But it can't emit anything because watermark is stuck at t1.
time.Sleep(stallTimeout / 2)
ch2 <- testEntry(api.LogStreamStdout, t3, "ch2-second")
// 1/2 + 2/3 = 7/6 > 1, so stream 1 should be considered stalled now.
time.Sleep(stallTimeout / 3 * 2)
ch2 <- testEntry(api.LogStreamStdout, t4, "ch2-third")
results = collectEntries(t, output, 1)
require.ErrorIs(t, results[0].Err, api.ErrLogStreamStalled)
assert.Equal(t, e1.Metadata, results[0].Metadata, "stalled entry should have stream metadata")
// Now that stream 1 is marked as stalled and ignored, all in-flight entries from stream 2 are emitted.
results = collectEntries(t, output, 3)
require.Len(t, results, 3)
assert.Equal(t, "ch2-first", string(results[0].Message))
assert.Equal(t, "ch2-second", string(results[1].Message))
assert.Equal(t, "ch2-third", string(results[2].Message))
close(ch1)
close(ch2)
results = collectEntries(t, output, 0)
assert.Len(t, results, 0)
}
+159
View File
@@ -0,0 +1,159 @@
package client
import (
"context"
"errors"
"fmt"
"io"
"github.com/docker/docker/pkg/stringid"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/pkg/api"
)
// ServiceLogs streams log entries from all service containers in chronological order based on timestamps.
// Keep in mind that perfect ordering of log events across multiple machines can't be guaranteed due to the
// imperfection of physical clocks or potential clock skew between machines.
// It uses a low watermark algorithm to ensure proper ordering across multiple machines.
// Heartbeat entries from the server advance the watermark to enable timely emission of buffered logs.
func (cli *Client) ServiceLogs(
ctx context.Context, serviceNameOrID string, opts api.ServiceLogsOptions,
) (api.Service, <-chan api.ServiceLogEntry, error) {
svc, err := cli.InspectService(ctx, serviceNameOrID)
if err != nil {
return svc, nil, fmt.Errorf("inspect service: %w", err)
}
if len(svc.Containers) == 0 {
return svc, nil, fmt.Errorf("no containers found for service: %s", serviceNameOrID)
}
machines, err := cli.ListMachines(ctx, &api.MachineFilter{
NamesOrIDs: opts.Machines,
})
if err != nil {
return svc, nil, fmt.Errorf("list machines: %w", err)
}
ctrStreams := make([]<-chan api.ServiceLogEntry, 0, len(svc.Containers))
for _, ctr := range svc.Containers {
// Skip containers not running on the specified machines.
m := machines.FindByNameOrID(ctr.MachineID)
if len(opts.Machines) > 0 && m == nil {
continue
}
// Machine name for ServiceLogEntry metadata and friendlier error message.
machineName := ctr.MachineID
if m != nil {
machineName = m.Machine.Name
}
stream, err := cli.ContainerLogs(ctx, ctr.MachineID, ctr.Container.ID, opts)
if err != nil {
return svc, nil, fmt.Errorf("stream logs from service container '%s' on machine '%s': %w",
stringid.TruncateID(ctr.Container.ID), machineName, err)
}
// Enrich log entries from the container with service metadata.
metadata := api.ServiceLogEntryMetadata{
ServiceID: svc.ID,
ServiceName: svc.Name,
ContainerID: ctr.Container.ID,
MachineID: ctr.MachineID,
MachineName: machineName,
}
enrichedStream := logsStreamWithServiceMetadata(stream, metadata)
ctrStreams = append(ctrStreams, enrichedStream)
}
if len(ctrStreams) == 0 {
return svc, nil, errors.New("no service containers found on the specified machine(s)")
}
// Use the log merger to combine streams from all containers in chronological order.
merger := NewLogMerger(ctrStreams, DefaultLogMergerOptions)
mergedStream := merger.Stream()
return svc, mergedStream, nil
}
// ContainerLogs streams log entries from a single container on a specified machine.
func (cli *Client) ContainerLogs(
ctx context.Context, machineNameOrID string, containerID string, opts api.ServiceLogsOptions,
) (<-chan api.ContainerLogEntry, error) {
proxyCtx, _, err := cli.ProxyMachinesContext(ctx, []string{machineNameOrID})
if err != nil {
return nil, fmt.Errorf("create request context to proxy to machine '%s': %w", machineNameOrID, err)
}
req := &pb.ContainerLogsRequest{
ContainerId: containerID,
Follow: opts.Follow,
Tail: int32(opts.Tail),
Since: opts.Since,
Until: opts.Until,
}
if !opts.Follow && opts.Tail == 0 {
// If not following and tail is 0, set tail to -1 to return all logs.
// Otherwise, no logs will be returned at all.
req.Tail = -1
}
stream, err := cli.Docker.GRPCClient.ContainerLogs(proxyCtx, req)
if err != nil {
return nil, err
}
ch := make(chan api.ContainerLogEntry)
go func() {
defer close(ch)
for {
pbEntry, err := stream.Recv()
if err == io.EOF {
return
}
if err != nil {
ch <- api.ContainerLogEntry{
Err: err,
}
return
}
entry := api.ContainerLogEntry{
Stream: api.LogStreamTypeFromProto(pbEntry.Stream),
Message: pbEntry.Message,
Timestamp: pbEntry.Timestamp.AsTime(),
}
select {
case ch <- entry:
case <-ctx.Done():
return
}
}
}()
return ch, nil
}
// logsStreamWithServiceMetadata wraps a container logs stream and enriches each log entry with service metadata.
func logsStreamWithServiceMetadata(
stream <-chan api.ContainerLogEntry, metadata api.ServiceLogEntryMetadata,
) <-chan api.ServiceLogEntry {
out := make(chan api.ServiceLogEntry)
go func() {
for entry := range stream {
out <- api.ServiceLogEntry{
Metadata: metadata,
ContainerLogEntry: entry,
}
}
close(out)
}()
return out
}
+31 -28
View File
@@ -2,7 +2,8 @@ package client
import (
"context"
"slices"
"fmt"
"strings"
"github.com/psviderski/uncloud/internal/machine/api/pb"
"github.com/psviderski/uncloud/pkg/api"
@@ -32,17 +33,39 @@ func (cli *Client) ListMachines(ctx context.Context, filter *api.MachineFilter)
if err != nil {
return nil, err
}
machines := api.MachineMembersList(resp.Machines)
machines := resp.Machines
if filter == nil {
return machines, nil
}
if filter != nil {
var matchedMachines api.MachineMembersList
for _, m := range machines {
if MachineMatchesFilter(m, filter) {
matchedMachines = append(matchedMachines, m)
// Apply the filter.
if len(filter.NamesOrIDs) > 0 {
var matched api.MachineMembersList
var notFound []string
for _, nameOrID := range filter.NamesOrIDs {
if m := machines.FindByNameOrID(nameOrID); m != nil {
matched = append(matched, m)
} else {
notFound = append(notFound, nameOrID)
}
}
machines = matchedMachines
machines = matched
if len(notFound) > 0 {
return nil, fmt.Errorf("machines not found: %s", strings.Join(notFound, ", "))
}
}
if filter.Available {
var available api.MachineMembersList
for _, m := range machines {
if m.State != pb.MachineMember_DOWN {
available = append(available, m)
}
}
machines = available
}
return machines, nil
@@ -76,23 +99,3 @@ func (cli *Client) RenameMachine(ctx context.Context, nameOrID, newName string)
return cli.UpdateMachine(ctx, req)
}
func MachineMatchesFilter(machine *pb.MachineMember, filter *api.MachineFilter) bool {
if filter == nil {
return true
}
if filter.Available && machine.State == pb.MachineMember_DOWN {
return false
}
if len(filter.NamesOrIDs) > 0 {
if !slices.ContainsFunc(filter.NamesOrIDs, func(nameOrID string) bool {
return machine.Machine.Id == nameOrID || machine.Machine.Name == nameOrID
}) {
return false
}
}
return true
}
+74
View File
@@ -273,6 +273,80 @@ func (cli *Client) RemoveService(ctx context.Context, id string) error {
return err
}
// StopService stops all containers on all machines that belong to the specified service.
// The id parameter can be either a service ID or name.
func (cli *Client) StopService(ctx context.Context, id string, opts container.StopOptions) error {
svc, err := cli.InspectService(ctx, id)
if err != nil {
return err
}
wg := sync.WaitGroup{}
errCh := make(chan error)
// Stop all containers on all machines that belong to the service.
for _, mc := range svc.Containers {
wg.Add(1)
go func() {
defer wg.Done()
err := cli.StopContainer(ctx, svc.ID, mc.Container.ID, opts)
if err != nil {
errCh <- fmt.Errorf("stop container '%s': %w", mc.Container.ID, err)
}
}()
}
go func() {
wg.Wait()
close(errCh)
}()
err = nil
for e := range errCh {
err = errors.Join(err, e)
}
return err
}
// StartService starts all containers on all machines that belong to the specified service.
// The id parameter can be either a service ID or name.
func (cli *Client) StartService(ctx context.Context, id string) error {
svc, err := cli.InspectService(ctx, id)
if err != nil {
return err
}
wg := sync.WaitGroup{}
errCh := make(chan error)
// Start all containers on all machines that belong to the service.
for _, mc := range svc.Containers {
wg.Add(1)
go func() {
defer wg.Done()
err := cli.StartContainer(ctx, svc.ID, mc.Container.ID)
if err != nil {
errCh <- fmt.Errorf("start container '%s': %w", mc.Container.ID, err)
}
}()
}
go func() {
wg.Wait()
close(errCh)
}()
err = nil
for e := range errCh {
err = errors.Join(err, e)
}
return err
}
// ListServices returns a list of all services and their containers.
func (cli *Client) ListServices(ctx context.Context) ([]api.Service, error) {
machines, err := cli.ListMachines(ctx, nil)
+1 -1
View File
@@ -55,7 +55,7 @@ func (cli *Client) ListVolumes(ctx context.Context, filter *api.VolumeFilter) ([
proxyMachines = filter.Machines
}
listCtx, machines, err := api.ProxyMachinesContext(ctx, cli, proxyMachines)
listCtx, machines, err := cli.ProxyMachinesContext(ctx, proxyMachines)
if err != nil {
return nil, fmt.Errorf("create request context to broadcast to all machines: %w", err)
}
+27 -7
View File
@@ -2,6 +2,10 @@
set -euo pipefail
# If set to 'true', only install the packages and dependencies, without running, reloading, or
# restarting services or systemd.
INSTALL_ONLY=${INSTALL_ONLY:-false}
INSTALL_BIN_DIR=${INSTALL_BIN_DIR:-/usr/local/bin}
INSTALL_SYSTEMD_DIR=${INSTALL_SYSTEMD_DIR:-/etc/systemd/system}
@@ -57,7 +61,7 @@ verify_system() {
Your system architecture ($arch) is not supported."
fi
if [[ ! -d /run/systemd/system ]]; then
if [[ ! -d /run/systemd/system && "${INSTALL_ONLY}" != "true" ]]; then
error "Cannot find systemd to use as a service manager for the Uncloud machine daemon. \
Uncloud supports only systemd-based Linux systems for now."
fi
@@ -66,9 +70,14 @@ Uncloud supports only systemd-based Linux systems for now."
install_docker() {
if command_exists dockerd; then
log "✓ Docker is already installed."
DOCKER_ALREADY_INSTALLED=true
if [[ "${INSTALL_ONLY}" == "true" ]]; then
return
fi
docker version
DOCKER_ALREADY_INSTALLED=true
# Check if the installed Docker configured to use the containerd image store.
local driver_status
driver_status=$(docker info -f '{{ .DriverStatus }}' 2>/dev/null)
@@ -93,7 +102,9 @@ install_docker() {
log "⏳ Configuring Docker daemon (${DOCKER_DAEMON_CONFIG_FILE}) to optimise it for Uncloud..."
echo "${DOCKER_DAEMON_CONFIG}" > "${DOCKER_DAEMON_CONFIG_FILE}"
systemctl restart docker
if [[ "${INSTALL_ONLY}" != "true" ]]; then
systemctl restart docker
fi
log "✓ Docker installed and configured successfully."
}
@@ -215,8 +226,11 @@ WantedBy=multi-user.target
EOF
log "✓ Systemd unit file created: ${uncloud_service_path}"
# Reload systemd to recognize the new or updated unit file.
systemctl daemon-reload
if [[ "${INSTALL_ONLY}" != "true" ]]; then
# Reload systemd to recognize the new or updated unit file.
systemctl daemon-reload
fi
systemctl enable uncloud.service
}
@@ -284,11 +298,17 @@ RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
EOF
log "✓ Systemd unit file created: ${corrosion_service_path}"
# Reload systemd to recognize the new unit file
systemctl daemon-reload
if [[ "${INSTALL_ONLY}" != "true" ]]; then
# Reload systemd to recognize the new unit file
systemctl daemon-reload
fi
}
start_uncloud() {
if [[ "${INSTALL_ONLY}" == "true" ]]; then
return
fi
log "⏳ Starting Uncloud machine daemon (uncloud.service)..."
systemctl restart uncloud.service
log "✓ Uncloud machine daemon started."
+2 -6
View File
@@ -20,12 +20,8 @@ print_manual_install() {
}
fetch_latest_version() {
api_url="https://api.github.com/repos/${GITHUB_REPO}/releases/latest"
auth_header=""
if [ -n "$GITHUB_TOKEN" ]; then
auth_header="Authorization: Bearer $GITHUB_TOKEN"
fi
VERSION=$(curl -fsSL -H "$auth_header" "$api_url" | grep -o '"tag_name": "[^"]*' | cut -d'"' -f4)
latest_url="https://github.com/${GITHUB_REPO}/releases/latest"
VERSION=$(curl -fsSLI -o /dev/null -w '%{url_effective}' "$latest_url" | grep -o 'tag/[^/]*$' | cut -d'/' -f2)
if [ -z "$VERSION" ]; then
echo "Failed to fetch the latest version from GitHub."
print_manual_install
+25
View File
@@ -533,4 +533,29 @@ func TestComposeDeployment(t *testing.T) {
assert.ElementsMatch(t, serviceMachines.ToSlice(), []string{c.Machines[0].ID, c.Machines[2].ID},
"Service containers should be on machines 1 and 3 from comma-separated list")
})
// Catches regression: https://github.com/psviderski/uncloud/issues/176
t.Run("plan new deployment with volumes and recreate strategy", func(t *testing.T) {
t.Parallel()
project, err := compose.LoadProjectFromContent(ctx, `
services:
redis:
image: redis:7-alpine
volumes:
- redis-data:/data
volumes:
redis-data:
`)
require.NoError(t, err)
deployment, err := compose.NewDeploymentWithStrategy(ctx, cli, project,
&deploy.RollingStrategy{ForceRecreate: true})
require.NoError(t, err)
plan, err := deployment.Plan(ctx)
require.NoError(t, err)
assert.Len(t, plan.Operations, 2, "Expected 1 volume creation and 1 service to deploy")
})
}
+65 -112
View File
@@ -2,14 +2,11 @@ package e2e
import (
"context"
"fmt"
"regexp"
"strings"
"testing"
"time"
"github.com/docker/docker/api/types/container"
dockerclient "github.com/docker/docker/client"
"github.com/psviderski/uncloud/internal/ucind"
"github.com/psviderski/uncloud/pkg/api"
"github.com/stretchr/testify/assert"
@@ -68,94 +65,52 @@ func TestInternalDNS(t *testing.T) {
return true
}, 30*time.Second, 1*time.Second, "Service should be deployed and running on all machines")
// Get Docker client to access ucind machine containers (shared across all DNS tests)
dockerCli, err := dockerclient.NewClientWithOpts(dockerclient.FromEnv, dockerclient.WithAPIVersionNegotiation())
// Deploy a single "network-multitool" service to be used for all DNS queries
queryServiceName := "dns-query-service"
t.Cleanup(func() {
err := cli.RemoveService(ctx, queryServiceName)
if err != nil && !strings.Contains(err.Error(), "not found") {
require.NoError(t, err)
}
})
querySvcSpec := api.ServiceSpec{
Name: queryServiceName,
Mode: api.ServiceModeReplicated,
Replicas: 1,
Placement: api.Placement{
Machines: []string{c.Machines[0].Name},
},
Container: api.ContainerSpec{
Image: "wbitt/network-multitool",
Command: []string{"sleep", "infinity"},
},
}
_, err = cli.RunService(ctx, querySvcSpec)
require.NoError(t, err)
// Helper function to create DNS query service spec
createDNSQuerySpec := func(name, dnsQuery, outputFile string) api.ServiceSpec {
return api.ServiceSpec{
Name: name,
Mode: api.ServiceModeReplicated,
Container: api.ContainerSpec{
Image: "wbitt/network-multitool",
Command: []string{"sh", "-c", fmt.Sprintf("nslookup %s > %s 2>&1 && echo 'DNS query completed' && sleep infinity",
dnsQuery, outputFile)},
VolumeMounts: []api.VolumeMount{
{
VolumeName: "host-tmp",
ContainerPath: "/tmp",
},
},
},
Volumes: []api.VolumeSpec{
{
Name: "host-tmp",
Type: "bind",
BindOptions: &api.BindOptions{
HostPath: "/tmp",
},
},
},
Replicas: 1,
// Wait for the query service to be deployed
var querySvc api.Service
require.Eventually(t, func() bool {
querySvc, err = cli.InspectService(ctx, queryServiceName)
if err != nil {
return false
}
}
// Helper function to run DNS query and get results
runDNSQuery := func(t *testing.T, serviceName, dnsQuery, outputFile string) string {
spec := createDNSQuerySpec(serviceName, dnsQuery, outputFile)
t.Cleanup(func() {
cli.RemoveService(ctx, spec.Name)
})
// Run the DNS query service
resp, err := cli.RunService(ctx, spec)
require.NoError(t, err)
// Wait for the DNS query to complete
var querySvc api.Service
require.Eventually(t, func() bool {
querySvc, err = cli.InspectService(ctx, resp.ID)
if err != nil || len(querySvc.Containers) == 0 {
return false
}
return querySvc.Containers[0].Container.State.Status == "running"
}, 30*time.Second, 1*time.Second, "DNS query container should be running")
// Give some time for DNS query to complete and write to file
time.Sleep(1 * time.Second)
// Find which ucind machine the DNS query container is running on
ctr := querySvc.Containers[0]
machineID := ctr.MachineID
// Get the ucind machine container name for this machine
var ucindMachineName string
for _, machine := range c.Machines {
if machine.ID == machineID {
ucindMachineName = machine.ContainerName
break
}
// Should have 1 container and it should be running
if len(querySvc.Containers) != 1 {
return false
}
require.NotEmpty(t, ucindMachineName, "Should find ucind machine container name")
return querySvc.Containers[0].Container.State.Status == "running"
}, 30*time.Second, 1*time.Second, "Query service should be deployed and running")
queryContainer := querySvc.Containers[0]
// Read the DNS results from the ucind machine's filesystem
exec, err := dockerCli.ContainerExecCreate(ctx, ucindMachineName, container.ExecOptions{
Cmd: []string{"cat", outputFile},
AttachStdout: true,
AttachStderr: true,
})
// Run nslookup for given query
runNslookup := func(t *testing.T, dnsQuery string) string {
dnsOutput, err := execInContainerAndReadOutput(
t, ctx, cli, queryServiceName, queryContainer.Container.ID,
[]string{"nslookup", dnsQuery},
)
require.NoError(t, err)
resp2, err := dockerCli.ContainerExecAttach(ctx, exec.ID, container.ExecAttachOptions{})
require.NoError(t, err)
defer resp2.Close()
// Read the DNS output
buf := make([]byte, 4096)
n, _ := resp2.Reader.Read(buf)
return string(buf[:n])
return dnsOutput
}
// Helper function to verify DNS output doesn't contain errors
@@ -166,7 +121,7 @@ func TestInternalDNS(t *testing.T) {
}
t.Run("service name resolves to all container IPs", func(t *testing.T) {
dnsOutput := runDNSQuery(t, "dns-test-service-name", serviceName+".internal", "/tmp/dns_result.txt")
dnsOutput := runNslookup(t, serviceName+".internal")
t.Logf("DNS query output:\n%s", dnsOutput)
// Verify that all service container IPs are in the DNS response
@@ -181,15 +136,14 @@ func TestInternalDNS(t *testing.T) {
t.Run("machine-specific service DNS lookups", func(t *testing.T) {
// Test the new <machine-id>.m.<service-name>.internal DNS feature
for i, targetContainer := range svc.Containers {
for _, targetContainer := range svc.Containers {
targetMachineID := targetContainer.MachineID
targetContainerIP := targetContainer.Container.UncloudNetworkIP().String()
// Construct the machine-specific DNS name
machineSpecificDNS := targetMachineID + ".m." + serviceName + ".internal"
outputFile := fmt.Sprintf("/tmp/dns_result_machine_%d.txt", i)
dnsOutput := runDNSQuery(t, fmt.Sprintf("dns-test-machine-%d", i), machineSpecificDNS, outputFile)
dnsOutput := runNslookup(t, machineSpecificDNS)
t.Logf("Machine-specific DNS query output for %s:\n%s", machineSpecificDNS, dnsOutput)
// Verify that the specific container IP is returned
@@ -213,7 +167,7 @@ func TestInternalDNS(t *testing.T) {
t.Run("service ID DNS lookup", func(t *testing.T) {
// Test that service ID also resolves (existing functionality)
dnsOutput := runDNSQuery(t, "dns-test-service-id", svc.ID+".internal", "/tmp/dns_result_service_id.txt")
dnsOutput := runNslookup(t, svc.ID+".internal")
t.Logf("Service ID DNS query output:\n%s", dnsOutput)
// Verify that all service container IPs are in the DNS response
@@ -227,36 +181,35 @@ func TestInternalDNS(t *testing.T) {
})
t.Run("nearest mode prioritizes local subnet IPs", func(t *testing.T) {
// Test the "nearest" mode which should sort local subnet IPs first
dnsOutput := runDNSQuery(t, "dns-test-nearest", "nearest."+serviceName+".internal", "/tmp/dns_result_nearest.txt")
t.Logf("Nearest mode DNS query output:\n%s", dnsOutput)
// Find which machine ran the query
querySvc, err := cli.InspectService(ctx, "dns-test-nearest")
require.NoError(t, err)
require.NotEmpty(t, querySvc.Containers, "Query service should have a container")
queryMachineID := querySvc.Containers[0].MachineID
// Find the local container IP (on the same machine as the query)
// Find the service container on the same machine as the query container.
var localIP string
for _, ctr := range svc.Containers {
if ctr.MachineID == queryMachineID {
if ctr.MachineID == queryContainer.MachineID {
localIP = ctr.Container.UncloudNetworkIP().String()
break
}
}
require.NotEmpty(t, localIP, "Should find local container IP on query machine %s", queryMachineID)
require.NotEmpty(t, localIP, "Should find local container IP on query machine %s", queryContainer.MachineID)
// Extract the first IP address from the DNS output using regex
// Pattern matches: "Name: nearest.test-dns-service.internal" followed by "Address: X.X.X.X"
// We will extract the first IP address from the DNS output using a regex.
// Pattern matches "Name: nearest.test-dns-service.internal" followed by "Address: X.X.X.X".
re := regexp.MustCompile(`(?m)Name:\s+[\w\.\-]+\s+Address:\s+([\d\.]+)`)
matches := re.FindStringSubmatch(dnsOutput)
require.Len(t, matches, 2, "Should find Name's Address in DNS output")
firstIP := matches[1]
assert.Equal(t, localIP, firstIP,
"Nearest mode should return local subnet IP first (query machine: %s, local IP: %s, first DNS result: %s)",
queryMachineID, localIP, firstIP)
assertNoDNSErrors(t, dnsOutput)
// Test the "nearest" mode which should sort local subnet IPs first.
// The default behavior randomizes the order, so run it a few times
// to reduce the chance we're just getting lucky with the order.
for range 5 {
dnsOutput := runNslookup(t, "nearest."+serviceName+".internal")
t.Logf("Nearest mode DNS query output:\n%s", dnsOutput)
matches := re.FindStringSubmatch(dnsOutput)
require.Len(t, matches, 2, "Should find Name's Address in DNS output")
firstIP := matches[1]
assert.Equal(t, localIP, firstIP,
"Nearest mode should return local subnet IP first (query machine: %s, local IP: %s, first DNS result: %s)",
queryContainer.MachineID, localIP, firstIP)
assertNoDNSErrors(t, dnsOutput)
}
})
}
+3 -7
View File
@@ -29,10 +29,7 @@ func deployTestService(t *testing.T, ctx context.Context, cli *client.Client, na
}
deployment := cli.NewDeployment(spec, nil)
err := deployment.Validate(ctx)
require.NoError(t, err)
_, err = deployment.Run(ctx)
_, err := deployment.Run(ctx)
require.NoError(t, err)
// Wait for all replicas to be running
@@ -41,9 +38,8 @@ func deployTestService(t *testing.T, ctx context.Context, cli *client.Client, na
if err != nil {
return false
}
if uint(len(service.Containers)) != replicas {
return false
}
assertServiceMatchesSpec(t, service, spec)
for _, ctr := range service.Containers {
if ctr.Container.State.Status != "running" {
return false
+112
View File
@@ -3,6 +3,7 @@ package e2e
import (
"context"
"errors"
"fmt"
"net/netip"
"slices"
"strings"
@@ -1682,4 +1683,115 @@ func TestServiceLifecycle(t *testing.T) {
assert.Equal(t, api.ServiceModeGlobal, svc.Mode)
assert.Len(t, svc.Containers, 3, "expected 1 container on each machine")
})
t.Run("service logs", func(t *testing.T) {
t.Parallel()
name := "test-service-logs"
t.Cleanup(func() {
err := cli.RemoveService(ctx, name)
if !errors.Is(err, api.ErrNotFound) {
require.NoError(t, err)
}
})
spec := api.ServiceSpec{
Name: name,
Container: api.ContainerSpec{
Image: "busybox:1.37.0-musl",
Command: []string{
"sh",
"-c",
"trap 'exit 0' TERM INT; " +
"echo \"Hello from $HOSTNAME\"; sleep 0.5;" +
"echo \"Hello stderr from $HOSTNAME\" >&2; " +
"while true; do sleep 1; done",
},
},
Replicas: 3,
}
deployment := cli.NewDeployment(spec, nil)
_, err := deployment.Run(ctx)
require.NoError(t, err)
svc, err := cli.InspectService(ctx, name)
require.NoError(t, err)
assertServiceMatchesSpec(t, svc, spec)
machines := serviceMachines(svc)
assert.Len(t, machines.ToSlice(), 3, "should have containers on all 3 machines")
time.Sleep(1 * time.Second) // Wait for containers to produce the logs.
logsSvc, stream, err := cli.ServiceLogs(ctx, name, api.ServiceLogsOptions{})
require.NoError(t, err)
assertServiceMatchesSpec(t, logsSvc, spec)
collectLogs := func(stream <-chan api.ServiceLogEntry) []string {
var entries []string
timeout := time.After(5 * time.Second)
for {
select {
case entry, ok := <-stream:
if !ok {
return entries
}
require.NoError(t, entry.Err)
if entry.Stream == api.LogStreamStdout || entry.Stream == api.LogStreamStderr {
entries = append(entries, string(entry.Message))
}
case <-timeout:
require.FailNow(t, "timed out waiting for logs")
}
}
}
logs := collectLogs(stream)
require.Len(t, logs, 6, "should have 6 log entries (1 stdout and 1 stderr from each of 3 replicas)")
for _, ctr := range svc.Containers {
stdoutMsg := fmt.Sprintf("Hello from %s\n", ctr.Container.Name)
stderrMsg := fmt.Sprintf("Hello stderr from %s\n", ctr.Container.Name)
assert.Contains(t, logs, stdoutMsg)
assert.Contains(t, logs, stderrMsg)
outIdx := slices.Index(logs, stdoutMsg)
errIdx := slices.Index(logs, stderrMsg)
assert.Less(t, outIdx, errIdx, "stdout log should appear before stderr log")
}
// Test filter logs by machine ID.
machineID := c.Machines[0].ID
_, machineStream, err := cli.ServiceLogs(ctx, name, api.ServiceLogsOptions{
Machines: []string{machineID},
})
require.NoError(t, err)
machineLogs := collectLogs(machineStream)
require.Len(t, machineLogs, 2, "should have 2 log entries (1 stdout and 1 stderr) from the filtered machine")
ctrName := ""
for _, ctr := range svc.Containers {
if ctr.MachineID == machineID {
ctrName = ctr.Container.Name
break
}
}
require.NotEmpty(t, ctrName, "should have found container on the target machine")
stdoutMsg := fmt.Sprintf("Hello from %s\n", ctrName)
stderrMsg := fmt.Sprintf("Hello stderr from %s\n", ctrName)
assert.Equal(t, []string{stdoutMsg, stderrMsg}, machineLogs)
// Test non-existent machine filter returns error.
_, _, err = cli.ServiceLogs(ctx, name, api.ServiceLogsOptions{
Machines: []string{"non-existent-machine"},
})
require.Error(t, err)
assert.Contains(t, err.Error(), "machines not found")
})
}
+9 -2
View File
@@ -10,6 +10,13 @@ services:
- linux/amd64
- linux/arm64
user: nobody
x-ports:
- uncloud.run:8000/https
x-caddy: |
uncloud.run {
redir /discord https://discord.gg/eR35KQJhPu
reverse_proxy {{upstreams 8000}} {
import common_proxy
}
log
}
scale: 2
+1 -1
View File
@@ -69,7 +69,7 @@ Talos [KubeSpan](https://www.talos.dev/v1.10/talos-guides/network/kubespan/).
### Managed DNS service (optional)
Uncloud can provide **managed DNS records** like `<service-name>.<cluster-id>.cluster.uncloud.run` for your public
Uncloud can provide **managed DNS records** like `<service-name>.<cluster-id>.uncld.dev` for your public
services through free [Uncloud DNS](https://github.com/psviderski/uncloud-dns) service. You can deploy a service and
instantly access it from anywhere with a proper DNS name and HTTPS without any manual DNS configuration. This makes
self-hosting much more accessible and simplifies the process of adding your own domain later.
@@ -45,7 +45,7 @@ This command will:
- Install the Uncloud daemon on your server
- Create a Docker network for Uncloud-managed containers
- Deploy [Caddy](https://caddyserver.com/) as your reverse proxy listening on host ports 80 and 443
- Reserve a free `xxxxxx.cluster.uncloud.run` subdomain via the Uncloud managed DNS service and point it to your
- Reserve a free `xxxxxx.uncld.dev` subdomain via the Uncloud managed DNS service and point it to your
server's IP
All in about a minute!
@@ -144,7 +144,7 @@ Cluster initialised with machine 'machine-dc3c' and saved as context 'default' i
Current cluster context is now 'default'.
Waiting for the machine to be ready...
Reserved cluster domain: 7za6s7.cluster.uncloud.run
Reserved cluster domain: 7za6s7.uncld.dev
[+] Deploying service caddy 7/2
✔ Container caddy-d7uk on machine-dc3c Started 6.1s
✔ Image caddy:2.10.0 on machine-dc3c Pulled 3.7s
@@ -154,7 +154,7 @@ Updating cluster domain records in Uncloud DNS to point to machines running cadd
✔ Machine machine-dc3c (157.180.72.195) Reachable 0.7s
DNS records updated to use only the internet-reachable machines running caddy service:
*.7za6s7.cluster.uncloud.run A → 157.180.72.195
*.7za6s7.uncld.dev A → 157.180.72.195
```
</details>
@@ -177,7 +177,7 @@ You'll see the progress of the deployment and the public URL where you can acces
✔ Image excalidraw/excalidraw on machine-dc3c Pulled 4.7s
excalidraw endpoints:
• https://excalidraw.7za6s7.cluster.uncloud.run → :80
• https://excalidraw.7za6s7.uncld.dev → :80
```
## Verify your deployment
@@ -209,7 +209,7 @@ uc ls
```
NAME MODE REPLICAS ENDPOINTS
caddy global 1
excalidraw replicated 1 https://excalidraw.7za6s7.cluster.uncloud.run → :80
excalidraw replicated 1 https://excalidraw.7za6s7.uncld.dev → :80
```
You can see `caddy` service listed here. That's your reverse proxy, running as a regular Uncloud service.
@@ -266,15 +266,15 @@ We've successfully converted our deployment created with `uc run` to a Compose f
## Use your own domain
Want to use your own domain, for example, `excalidraw.example.com` instead of `excalidraw.7za6s7.cluster.uncloud.run`?
Want to use your own domain, for example, `excalidraw.example.com` instead of `excalidraw.7za6s7.uncld.dev`?
Add a CNAME record `excalidraw.example.com` in your DNS provider (Cloudflare, Namecheap, etc.) pointing to
`excalidraw.7za6s7.cluster.uncloud.run`. Alternatively, you can add an A record pointing to your server's IP.
`excalidraw.7za6s7.uncld.dev`. Alternatively, you can add an A record pointing to your server's IP.
:::info note
These instructions set up your own domain **in addition to** the Uncloud managed DNS name
`excalidraw.7za6s7.cluster.uncloud.run`.
`excalidraw.7za6s7.uncld.dev`.
If you want to avoid the managed service altogether, add `--no-dns` to your `uc machine init` command, and point an A
DNS record to your server(s)'s IP(s).
@@ -62,7 +62,7 @@ This command looks for a Compose file in your current working directory and:
5. **Deploys service containers** using the built images and latest configuration changes with zero-downtime rolling
updates
See the [uc deploy](../../9-cli-reference/uc_deploy.md) reference for all available options.
See the [`uc deploy` ](../../9-cli-reference/uc_deploy.md) reference for all available options.
Watch `uc deploy` building and deploying a demo app (18 seconds):
@@ -98,9 +98,10 @@ You can pass additional build arguments or override existing ones using the `--b
uc deploy --build-arg BUILD_ENV=dev
```
You can also use advanced features like build caches, secrets, and SSH access.
You can also use advanced features like build caches or SSH access.
See [Compose Build Specification](https://github.com/compose-spec/compose-spec/blob/main/build.md) for all supported
attributes.
attributes. Note that build [secrets](https://github.com/compose-spec/compose-spec/blob/main/build.md#secrets) are not
supported though.
:::info note
@@ -140,8 +141,8 @@ This generates tags like:
`uc deploy` renders the image templates when it loads the Compose file and then uses the resulting names for the build
and deploy stages.
See the [Image tag template](../../8-compose-file-reference/2-image-tag-template.md) reference for all available template
variables and functions.
See the [Image tag template](../../8-compose-file-reference/2-image-tag-template.md) reference for all available
template variables and functions.
### Separate build and deploy steps
@@ -42,6 +42,8 @@ The following table shows the support status for main Compose features:
| `replicas` | ✅ Supported | Number of container replicas |
| `resources` | ⚠️ Limited | CPU, memory limits and device reservations |
| `restart_policy` | ❌ Not supported | Defaults to `unless-stopped` |
| `rollback_config` | ❌ Not supported | See [#151](https://github.com/psviderski/uncloud/issues/151) |
| `update_config` | ❌ Not supported | See [#151](https://github.com/psviderski/uncloud/issues/151) |
| **Volumes** | | |
| Named volumes | ✅ Supported | Docker volumes |
| Bind mounts | ✅ Supported | Host path binding |
@@ -7,7 +7,7 @@ Template syntax for tagging built images.
When building service images as part of [`uc build`](../9-cli-reference/uc_build.md) or
[`uc deploy`](../9-cli-reference/uc_deploy.md) commands, Uncloud automatically generates image tags based on the current
Git repository state. You can customise the image name and tag format for the built images using the
[Go template](https://pkg.go.dev/text/template) syntax environment variables.
[Go template](https://pkg.go.dev/text/template) syntax and environment variables.
## Default template
+7 -4
View File
@@ -6,7 +6,8 @@ A CLI tool for managing Uncloud resources such as machines, services, and volume
```
--connect string Connect to a remote cluster machine without using the Uncloud configuration file. [$UNCLOUD_CONNECT]
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], or tcp://host:port
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], tcp://host:port, or unix:///path/to/uncloud.sock
-c, --context string Name of the cluster context to use (default is the current context). [$UNCLOUD_CONTEXT]
-h, --help help for uc
--uncloud-config string Path to the Uncloud configuration file. [$UNCLOUD_CONFIG] (default "~/.config/uncloud/config.yaml")
```
@@ -22,11 +23,13 @@ A CLI tool for managing Uncloud resources such as machines, services, and volume
* [uc image](uc_image.md) - Manage images on machines in the cluster.
* [uc images](uc_images.md) - List images on machines in the cluster.
* [uc inspect](uc_inspect.md) - Display detailed information on a service.
* [uc logs](uc_logs.md) - View service logs.
* [uc ls](uc_ls.md) - List services.
* [uc machine](uc_machine.md) - Manage machines in an Uncloud cluster.
* [uc machine](uc_machine.md) - Manage machines in the cluster.
* [uc ps](uc_ps.md) - List all service containers in the cluster
* [uc rm](uc_rm.md) - Remove one or more services.
* [uc run](uc_run.md) - Run a service.
* [uc scale](uc_scale.md) - Scale a replicated service by changing the number of replicas.
* [uc service](uc_service.md) - Manage services in an Uncloud cluster.
* [uc volume](uc_volume.md) - Manage volumes in an Uncloud cluster.
* [uc service](uc_service.md) - Manage services in the cluster.
* [uc volume](uc_volume.md) - Manage volumes in the cluster.
+2 -2
View File
@@ -41,7 +41,6 @@ uc build [FLAGS] [SERVICE...] [flags]
--build-arg stringArray Set a build-time variable for services. Used in Dockerfiles that declare the variable with ARG.
Can be specified multiple times. Format: --build-arg VAR=VALUE
--check Check the build configuration for services without building them.
-c, --context string Name of the cluster context. (default is the current context)
--deps Also build services declared as dependencies of the selected services.
-f, --file strings One or more Compose files to build. (default compose.yaml)
-h, --help help for build
@@ -59,7 +58,8 @@ uc build [FLAGS] [SERVICE...] [flags]
```
--connect string Connect to a remote cluster machine without using the Uncloud configuration file. [$UNCLOUD_CONNECT]
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], or tcp://host:port
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], tcp://host:port, or unix:///path/to/uncloud.sock
-c, --context string Name of the cluster context to use (default is the current context). [$UNCLOUD_CONTEXT]
--uncloud-config string Path to the Uncloud configuration file. [$UNCLOUD_CONFIG] (default "~/.config/uncloud/config.yaml")
```
+2 -1
View File
@@ -12,7 +12,8 @@ Manage Caddy reverse proxy service.
```
--connect string Connect to a remote cluster machine without using the Uncloud configuration file. [$UNCLOUD_CONNECT]
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], or tcp://host:port
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], tcp://host:port, or unix:///path/to/uncloud.sock
-c, --context string Name of the cluster context to use (default is the current context). [$UNCLOUD_CONTEXT]
--uncloud-config string Path to the Uncloud configuration file. [$UNCLOUD_CONFIG] (default "~/.config/uncloud/config.yaml")
```
@@ -13,7 +13,6 @@ uc caddy config [flags]
## Options
```
-c, --context string Name of the cluster context. (default is the current context)
-h, --help help for config
-m, --machine string Name or ID of the machine to get the configuration from. (default is connected machine)
--no-color Disable syntax highlighting for the output.
@@ -23,7 +22,8 @@ uc caddy config [flags]
```
--connect string Connect to a remote cluster machine without using the Uncloud configuration file. [$UNCLOUD_CONNECT]
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], or tcp://host:port
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], tcp://host:port, or unix:///path/to/uncloud.sock
-c, --context string Name of the cluster context to use (default is the current context). [$UNCLOUD_CONTEXT]
--uncloud-config string Path to the Uncloud configuration file. [$UNCLOUD_CONFIG] (default "~/.config/uncloud/config.yaml")
```
@@ -15,7 +15,6 @@ uc caddy deploy [flags]
```
--caddyfile string Path to a custom global Caddy config (Caddyfile) that will be prepended to the auto-generated Caddy config.
-c, --context string Name of the cluster context to deploy to. (default is the current context)
-h, --help help for deploy
--image string Caddy Docker image to deploy. (default caddy:LATEST_VERSION)
-m, --machine strings Machine names or IDs to deploy to. Can be specified multiple times or as a comma-separated list. (default is all machines)
@@ -25,7 +24,8 @@ uc caddy deploy [flags]
```
--connect string Connect to a remote cluster machine without using the Uncloud configuration file. [$UNCLOUD_CONNECT]
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], or tcp://host:port
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], tcp://host:port, or unix:///path/to/uncloud.sock
-c, --context string Name of the cluster context to use (default is the current context). [$UNCLOUD_CONTEXT]
--uncloud-config string Path to the Uncloud configuration file. [$UNCLOUD_CONFIG] (default "~/.config/uncloud/config.yaml")
```
+3 -1
View File
@@ -16,13 +16,15 @@ uc ctx [flags]
```
--connect string Connect to a remote cluster machine without using the Uncloud configuration file. [$UNCLOUD_CONNECT]
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], or tcp://host:port
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], tcp://host:port, or unix:///path/to/uncloud.sock
-c, --context string Name of the cluster context to use (default is the current context). [$UNCLOUD_CONTEXT]
--uncloud-config string Path to the Uncloud configuration file. [$UNCLOUD_CONFIG] (default "~/.config/uncloud/config.yaml")
```
## See also
* [uc](uc.md) - A CLI tool for managing Uncloud resources such as machines, services, and volumes.
* [uc ctx connection](uc_ctx_connection.md) - Choose a new default connection for the current context.
* [uc ctx ls](uc_ctx_ls.md) - List available cluster contexts.
* [uc ctx use](uc_ctx_use.md) - Switch to a different cluster context.
@@ -0,0 +1,27 @@
# uc ctx connection
Choose a new default connection for the current context.
```
uc ctx connection [flags]
```
## Options
```
-h, --help help for connection
```
## Options inherited from parent commands
```
--connect string Connect to a remote cluster machine without using the Uncloud configuration file. [$UNCLOUD_CONNECT]
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], tcp://host:port, or unix:///path/to/uncloud.sock
-c, --context string Name of the cluster context to use (default is the current context). [$UNCLOUD_CONTEXT]
--uncloud-config string Path to the Uncloud configuration file. [$UNCLOUD_CONFIG] (default "~/.config/uncloud/config.yaml")
```
## See also
* [uc ctx](uc_ctx.md) - Switch between different cluster contexts. Contains subcommands to manage contexts.
+2 -1
View File
@@ -16,7 +16,8 @@ uc ctx ls [flags]
```
--connect string Connect to a remote cluster machine without using the Uncloud configuration file. [$UNCLOUD_CONNECT]
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], or tcp://host:port
Format: [ssh://]user@host[:port], ssh+cli://user@host[:port], tcp://host:port, or unix:///path/to/uncloud.sock
-c, --context string Name of the cluster context to use (default is the current context). [$UNCLOUD_CONTEXT]
--uncloud-config string Path to the Uncloud configuration file. [$UNCLOUD_CONFIG] (default "~/.config/uncloud/config.yaml")
```

Some files were not shown because too many files have changed in this diff Show More