From 68d2bd56563756803af3ad4ae27e40b70fd9a67a Mon Sep 17 00:00:00 2001 From: Pasha Sviderski Date: Fri, 6 Nov 2026 19:15:10 +1000 Subject: [PATCH] docs(caddy): update managing Caddy documentation for v0.22.0 features and upgrade instructions --- .../3-concepts/2-ingress/3-managing-caddy.md | 279 ++++++++---------- .../2-ingress/4-cluster-storage-for-caddy.md | 141 +++++---- 2 files changed, 209 insertions(+), 211 deletions(-) diff --git a/website/docs/3-concepts/2-ingress/3-managing-caddy.md b/website/docs/3-concepts/2-ingress/3-managing-caddy.md index 8659f7a2..9ab6f586 100644 --- a/website/docs/3-concepts/2-ingress/3-managing-caddy.md +++ b/website/docs/3-concepts/2-ingress/3-managing-caddy.md @@ -1,11 +1,22 @@ # Managing Caddy -Caddy is automatically deployed as a global service `caddy` when you initialise a cluster with `uc machine init`. By -default, it runs on every machine to handle incoming HTTP/HTTPS traffic and route it to your services. +Uncloud automatically deploys Caddy as a global service named `caddy` when you initialise a cluster with +`uc machine init`. By default, it runs on every machine and routes incoming HTTP/HTTPS traffic to your services. -Since [v0.21.0](https://github.com/psviderski/uncloud/releases/tag/v0.21.0), Uncloud provides native -[cluster storage for Caddy](4-cluster-storage-for-caddy.md) (opt-in) to share TLS certificates and coordinate -certificate issuance across machines. +Since [v0.22.0](https://github.com/psviderski/uncloud/releases/tag/v0.22.0), the default Caddy deployment uses the +Uncloud Caddy image +[`ghcr.io/unlabs-dev/caddy-uncloud`](https://github.com/unlabs-dev/caddy-uncloud/pkgs/container/caddy-uncloud) with +[cluster storage](4-cluster-storage-for-caddy.md) to share TLS certificates across machines. This image is a Caddy build +that includes the [Uncloud storage module](https://github.com/unlabs-dev/caddy-uncloud) and a few other popular modules +such as [caddy-l4](https://github.com/mholt/caddy-l4) and [cloudflare](https://github.com/caddy-dns/cloudflare). + +:::warning + +If you want to upgrade or redeploy Caddy on an existing cluster created with Uncloud v0.21.0 or earlier, follow +[Upgrading from earlier versions](4-cluster-storage-for-caddy.md#upgrading-from-earlier-versions) before redeploying +Caddy. + +::: ## Checking status @@ -16,55 +27,90 @@ uc inspect caddy ``` ``` -ID: b5b269d5dc5ed4fdae6542894f94de82 -Name: caddy -Mode: global +Service ID: b5b269d5dc5ed4fdae6542894f94de82 +Name: caddy +Mode: global -CONTAINER ID IMAGE CREATED STATUS MACHINE -fb8f390e634d caddy:2.10.0 3 weeks ago Up 3 weeks prod-ap1 -0182f5d7bd9f caddy:2.10.0 3 months ago Up 3 weeks prod-us1 +CONTAINER ID IMAGE CREATED STATUS IP ADDRESS MACHINE +fb8f390e634d ghcr.io/unlabs-dev/caddy-uncloud:0.1.4 3 weeks ago Up 3 weeks 10.210.1.2 prod-ap1 +0182f5d7bd9f ghcr.io/unlabs-dev/caddy-uncloud:0.1.4 3 weeks ago Up 3 weeks 10.210.0.2 prod-us1 +``` + +## Verifying config + +View the complete generated Caddyfile that the `caddy` service uses. It includes global configuration, sites for +published service ports, custom service configs, and comments explaining any skipped invalid configs. It also warns if +the last config load failed. Use `--machine NAME` to check a specific machine. + +```shell +uc caddy config +``` + +See [Verifying Caddy config](2-publishing-services.md#verifying-caddy-config) for an example output and details on how +it's generated. + +## Listing certificates + +List the issued certificates in cluster storage: + +```shell +uc caddy cert ls +``` + +``` +ID NAME ISSUER EXPIRES +c111e23615f7 dns.uncloud.run Let's Encrypt 2026-12-31 (2 months) +b5dcd2a03e1b nginx.2t5ex2.uncld.dev Let's Encrypt 2026-12-31 (2 months) +70707fd2fea2 test-staging.2t5ex2.uncld.dev Let's Encrypt (staging) 2026-12-31 (2 months) +c5f9301e1c06 uncloud.run Let's Encrypt 2026-12-31 (2 months) ``` ## Deploying or updating Caddy -### Using CLI - -Update to the latest stable version using the [caddy](https://hub.docker.com/_/caddy) image from Docker Hub: +Deploy Caddy or upgrade it to the latest version of the Uncloud Caddy image: ```shell uc caddy deploy ``` -Deploy a specific version or custom image: +The command shows a deployment plan and asks for confirmation, then performs a rolling update. + +You can set a specific image, global Caddyfile, or limit deployment to specific machines: ```shell -uc caddy deploy --image caddybuilds/caddy-cloudflare:2.11.4 +# Deploy a specific version or custom image and keep it for later deployments. +uc caddy deploy --image ghcr.io/unlabs-dev/caddy-uncloud:0.1.4 + +# Deploy with a custom global Caddy config and only on selected machines. +uc caddy deploy --caddyfile global.Caddyfile --machine machine1,machine2 ``` -Deploy only to a specific machine or a subset of machines (comma-separated list): +The command remembers these values for later deployments. Pass an empty value to reset a flag to its default: ```shell -uc caddy deploy --machine machine1 -uc caddy deploy --machine machine2,machine3,machine4 +uc caddy deploy --image "" --caddyfile "" --machine "" ``` -Deploy with custom global configuration: +See [`uc caddy deploy`](../../9-cli-reference/uc_caddy_deploy.md) for all flags and examples. -```shell -uc caddy deploy --caddyfile global.Caddyfile -``` +## Custom global configuration -Example global configuration: +A custom Caddyfile replaces the default global config. Uncloud combines this config with the sites generated from +[published ports](2-publishing-services.md) and +[custom service configs](2-publishing-services.md#custom-caddy-configuration). Include `storage uncloud` to keep using +[cluster storage](4-cluster-storage-for-caddy.md): -```caddyfile title=global.Caddyfile +```caddyfile title="global.Caddyfile" # Global options. { - debug + storage uncloud + # Uncomment for troubleshooting: + # debug } # A snippet that can be reused in custom Caddy configs for services (x-caddy). (my_snippet) { - ... + # ... } # Expose an internal service that is not managed by Uncloud. @@ -73,10 +119,29 @@ internal.example.com { } ``` -### Using Compose +Deploy your custom global Caddy config: -You can manage the Caddy deployment with a Compose file for more control. For example, to deploy a custom global Caddy -config that uses the DNS challenge with Cloudflare to obtain a wildcard TLS certificate for `*.example.com`: +```shell +uc caddy deploy --caddyfile global.Caddyfile +``` + +## Custom image + +A custom image must include the [Uncloud storage module](https://github.com/unlabs-dev/caddy-uncloud) to use cluster +storage. It must also include `curl` for the health check and start Caddy with `/etc/caddy/Caddyfile`. + +The default Uncloud Caddy image +[`ghcr.io/unlabs-dev/caddy-uncloud`](https://github.com/unlabs-dev/caddy-uncloud/pkgs/container/caddy-uncloud) satisfies +these requirements. See the [release notest](https://github.com/unlabs-dev/caddy-uncloud/releases/latest) for the Caddy +version and the list of included modules. + +If you need other Caddy modules or want to choose a compatible Caddy version, you can +[build and deploy your own image](https://github.com/unlabs-dev/caddy-uncloud#building-and-deploying-a-custom-image). + +## Managing Caddy with Compose + +Use Compose when you need more control, such as environment variables for a DNS provider. This example uses the +Cloudflare module included in the default image to obtain a wildcard certificate for `*.example.com`: @@ -84,13 +149,11 @@ config that uses the DNS challenge with Cloudflare to obtain a wildcard TLS cert ```yaml services: caddy: - image: caddybuilds/caddy-cloudflare:2.11.4 + image: ghcr.io/unlabs-dev/caddy-uncloud:0.1.4 environment: # unix// is not a typo. Caddy uses network/address format, not a unix:// URL. CADDY_ADMIN: unix//run/caddy/admin.sock - env_file: - # Contains CLOUDFLARE_API_TOKEN=xxxxx - - .env.secrets + CLOUDFLARE_API_TOKEN: secret://cloudflare_api_token healthcheck: test: curl -fsS -o /dev/null --unix-socket /run/caddy/admin.sock http://localhost/config/ interval: 30s @@ -102,22 +165,26 @@ services: - /var/lib/uncloud/caddy:/etc/caddy:ro # Persists TLS certificates and other assets when using Caddy's default local storage. - /var/lib/uncloud/caddy:/data - - /run/uncloud/caddy:/run/caddy - # Required by the Uncloud cluster storage module (https://github.com/unlabs-dev/caddy-uncloud). - # Remove this mount if cluster storage is not used for Caddy. + # Uncloud API socket is required by the cluster storage module (https://github.com/unlabs-dev/caddy-uncloud). # Mount the directory, not the socket file, so Caddy sees a replacement socket after the daemon restarts. - /run/uncloud/api:/run/uncloud/api:ro + - /run/uncloud/caddy:/run/caddy x-ports: - 80:80@host - 443:443@host - 443:443/udp@host - x-caddy: Caddyfile + x-caddy: ./Caddyfile deploy: mode: global # Optional: deploy only to specific machines. # x-machines: # - machine1 # - machine2 + +secrets: + cloudflare_api_token: + # Read the secret value from 1Password by running this command. + x-command: op read "op://prod/caddy/cloudflare_api_token" ``` @@ -126,144 +193,34 @@ services: ```caddyfile # Global options. { - debug + storage uncloud } -# A snippet that can be reused in custom Caddy configs for services (x-caddy). -(my_snippet) { - ... -} - -# Obtain a wildcard TLS certificate for all subdomains of example.name using DNS challenge with Cloudflare. -# It will be used for services that publish ports with hostnames under example.name. +# Obtain a wildcard TLS certificate for all subdomains of example.com using DNS challenge with Cloudflare. +# It will be used for services that publish ports with hostnames under example.com. *.example.com { - tls { - dns cloudflare {env.CLOUDFLARE_API_TOKEN} - } - respond "No host matched" 404 -} - -# Expose an internal service that is not managed by Uncloud. -internal.example.com { - reverse_proxy 192.168.1.100 + tls { + dns cloudflare {env.CLOUDFLARE_API_TOKEN} + } + respond "No host matched" 404 } ``` -:::info note - -The specified `environment`, `volumes`, and `x-ports` properties are essential for Caddy to function correctly in the -Uncloud cluster. Do not change the source paths of the volume mounts as the Uncloud daemon relies on them to communicate -with Caddy and update its configuration. - -The image must include `curl` for the healthcheck and start Caddy with `/etc/caddy/Caddyfile`. Images based on the -official [Caddy image](https://hub.docker.com/_/caddy) do both by default. - -Keep the `/data` mount while using Caddy's default local storage so TLS certificates survive container updates. You can -remove this mount when using [cluster storage for Caddy](4-cluster-storage-for-caddy.md). - -::: - Deploy or update the `caddy` service from the Compose file: ```shell uc deploy ``` -## Verifying config +:::info note -View the complete generated Caddyfile served by the `caddy` service. This is useful for debugging and verifying custom -global and service-specific Caddy configs. +Keep the `environment`, `volumes`, and `x-ports` properties as shown, including the source paths of the volume mounts. +The Uncloud daemon and the storage module rely on them. -```shell -uc caddy config -``` +::: -Example output: - -```caddyfile -# Caddyfile autogenerated by Uncloud (DO NOT EDIT): 2025-12-22T10:30:12Z -# Automatically updated on service or health status changes. -# Docs: https://uncloud.run/docs/concepts/ingress/overview - -# User-defined global config from service 'caddy'. -# Global options. -{ - debug -} - -# A snippet that can be reused in custom Caddy configs for services (x-caddy). -(my_snippet) { - ... -} - -# Obtain a wildcard TLS certificate for all subdomains of example.name using DNS challenge with Cloudflare. -# It will be used for services that publish ports with hostnames under example.name. -*.example.com { - tls { - dns cloudflare {env.CLOUDFLARE_API_TOKEN} - } - respond "No host matched" 404 -} - -# Expose an internal service that is not managed by Uncloud. -internal.example.com { - reverse_proxy 192.168.1.100 -} - -# Health check endpoint to verify Caddy reachability on this machine. -http:// { - handle /.uncloud-verify { - respond "a369b9388812f9557feef6a0f5b46f2e" 200 - } - log -} - -(common_proxy) { - # Retry failed requests up to lb_retries times against other available upstreams. - lb_retries 3 - # Upstreams are marked unhealthy for fail_duration after a failed request (passive health checking). - fail_duration 30s -} - -# Sites generated from service ports. - -https://app.example.com { - reverse_proxy 10.210.1.3:8000 10.210.2.5:8000 { - import common_proxy - } - log -} - -https://api.example.com { - reverse_proxy 10.210.2.2:9000 10.210.1.7:9000 10.210.2.3:9000 { - import common_proxy - } - log -} - -# User-defined config for service 'web'. -www.example.com { - redir https://example.com{uri} permanent -} - -example.com { - reverse_proxy 10.210.0.3:8000 { - import common_proxy - } - log -} - -# Skipped invalid user-defined configs: -# - service 'duplicate-hostname': validation failed: adapting config using caddyfile adapter: ambiguous site definition: example.com -# - service 'invalid': validation failed: adapting config using caddyfile adapter: Caddyfile:61: unrecognized directive: invalid_directive -``` - -The generated config combines: - -- Global Caddy configuration (`x-caddy` from the `caddy` service). -- Auto-generated configs from published service ports (`x-ports`). -- Custom Caddy configs from services (`x-caddy`). -- Skipped invalid configs with error messages as comments. +Continue using `uc deploy` for a Compose-managed Caddy service. Switching to `uc caddy deploy` replaces its +configuration, including environment variables and custom mounts. diff --git a/website/docs/3-concepts/2-ingress/4-cluster-storage-for-caddy.md b/website/docs/3-concepts/2-ingress/4-cluster-storage-for-caddy.md index e306ed08..a98a3167 100644 --- a/website/docs/3-concepts/2-ingress/4-cluster-storage-for-caddy.md +++ b/website/docs/3-concepts/2-ingress/4-cluster-storage-for-caddy.md @@ -1,73 +1,114 @@ # Cluster storage for Caddy +Starting with Uncloud [v0.22.0](https://github.com/psviderski/uncloud/releases/tag/v0.22.0), Caddy uses cluster storage +by default. + The [Uncloud storage module](https://github.com/unlabs-dev/caddy-uncloud) lets Caddy instances share TLS certificates, -private keys, and ACME challenge tokens through Uncloud's cluster store. It uses distributed locks to coordinate -certificate issuance. Once one instance obtains a certificate, the others can use it too. +private keys, and ACME challenge tokens through Uncloud's cluster store. -## Why use cluster storage? +This matters when you run Caddy on multiple machines and distribute traffic between them using DNS or a load balancer. +In this case, an ACME challenge can reach a different Caddy instance from the one requesting the certificate. Cluster +storage lets any Caddy instance read the challenge token from the shared cluster store and complete the ACME challenge. +It uses distributed locks to coordinate certificate issuance. Once one instance obtains a certificate, the others can +use it too. -By default, each Caddy instance stores its certificates locally. When DNS points to multiple machines or a load balancer -distributes traffic between them, an ACME challenge can reach a different instance from the one requesting the -certificate. That instance may not have the challenge token, which can delay or prevent certificate issuance. +## Upgrading from earlier versions -Cluster storage is optional. The default Caddy image does not include the module, so you need to deploy an image that -includes it and configure Caddy to use it. +Starting with Uncloud v0.22.0, `uc caddy deploy` deploys the Uncloud Caddy image ([ +`ghcr.io/unlabs-dev/caddy-uncloud`](https://github.com/unlabs-dev/caddy-uncloud/pkgs/container/caddy-uncloud)) with +cluster storage enabled by default. -## Enabling cluster storage +Earlier versions deployed the official [`caddy`](https://hub.docker.com/_/caddy) image with local certificate storage. +Upgrading Uncloud does not change the running Caddy service. Redeploy it to switch to the new default image and cluster +storage. -:::warning Certificate migration +Before you deploy, upgrade the [`uc` CLI](../../2-getting-started/1-install-cli.md) and the daemon on all machines to +v0.22.0 or newer. Follow the [upgrade instructions](https://github.com/psviderski/uncloud/releases/latest) for daemon +upgrades, including any required migrations from older releases. Check current versions with `uc version` and `uc +machine ls`. -Caddy doesn't migrate existing certificates automatically when you change its storage backend. **All certificates will -need to be reissued unless you migrate your existing storage first.** Use the experimental -[`caddy storage export` and `caddy storage import`](https://caddyserver.com/docs/command-line#caddy-storage) -commands with the old and new configs to transfer the storage contents. +The first `uc caddy deploy` after the upgrade doesn't know the values you set with earlier versions of `uc`. It replaces +your image, global config, and machine list with the defaults unless you pass them as flags. -::: +If you already use `storage uncloud`, your certificates stay where they are, so you only need to run `uc caddy deploy`. -:::info Requirements +### Quick upgrade -Cluster storage requires Uncloud **v0.21.0 or newer** for both the `uc` CLI and the daemon on every cluster machine. -Check your CLI version with `uc version` and daemon versions with `uc machine ls`. Upgrade older versions before -enabling cluster storage. +Caddy does not copy certificates when you change storage backends. Unless you migrate them first, Caddy will reissue +them. HTTPS may fail for those domains until that finishes, and a large number of certificates can hit +[Let's Encrypt rate limits](https://letsencrypt.org/docs/rate-limits/). -::: - -Create a global Caddyfile, or add `storage uncloud` to the global options in your existing one: - -```caddyfile title="global.Caddyfile" -{ - storage uncloud - - # Uncomment to enable debug logs useful for troubleshooting storage operations: - # debug -} -``` - -Deploy Caddy with the pre-built module image and your global config: +Run the command below if you don't mind reissuing all certificates: ```shell -uc caddy deploy --image ghcr.io/unlabs-dev/caddy-uncloud:0.1.3 --caddyfile global.Caddyfile +uc caddy deploy ``` -See the [module README](https://github.com/unlabs-dev/caddy-uncloud#usage) for custom image builds, Compose deployment, -and additional storage options. +### Migrating certificates -## Verifying storage +Follow these steps to keep your existing certificates and avoid HTTPS failures: -Check that `storage uncloud` appears in the Caddy config: +1. Deploy the new image but stay on local storage. Use your existing global config if you have one, or an empty file: -```shell -uc caddy config -``` + ```shell + touch local.Caddyfile + uc caddy deploy --caddyfile local.Caddyfile + ``` -List issued certificates in cluster storage with [`uc caddy cert ls`](../../9-cli-reference/uc_caddy_cert_ls.md): +2. Find the ID of the Caddy container on each machine: -```shell -uc caddy cert ls + ```shell + uc inspect caddy + ``` -ID NAME ISSUER EXPIRES -c111e23615f7 dns.uncloud.run Let's Encrypt 2026-12-31 (2 months) -b5dcd2a03e1b nginx.2t5ex2.uncld.dev Let's Encrypt 2026-12-31 (2 months) -70707fd2fea2 test-staging.2t5ex2.uncld.dev Let's Encrypt (staging) 2026-12-31 (2 months) -c5f9301e1c06 uncloud.run Let's Encrypt 2026-12-31 (2 months) -``` +3. Copy the certificates to cluster storage by running this command for each container ID. Each machine stores its own + certificates, so don't skip any of them. These + [`caddy storage`](https://caddyserver.com/docs/command-line#caddy-storage) commands are experimental. + + ```shell + uc exec -T --container CONTAINER_ID caddy sh -c ' + printf "{\n\tstorage uncloud\n}\n" > /tmp/Caddyfile.cluster + caddy storage export --config /etc/caddy/Caddyfile --output - | + caddy storage import --config /tmp/Caddyfile.cluster --input - + rm /tmp/Caddyfile.cluster + ' + ``` + +4. Check that all your certificates are in cluster storage: + + ```shell + uc caddy cert ls + ``` + +5. Switch Caddy to cluster storage. If you have a custom global config, add `storage uncloud` to it and pass it with + `--caddyfile global.Caddyfile` instead. The command warns that all certificates will be reissued. You can ignore this + because you've already copied them. + + ```shell + uc caddy deploy --caddyfile "" + ``` + +6. Check that the Caddy config includes `storage uncloud` and doesn't show any config load warnings: + + ```shell + uc caddy config + ``` + +### Upgrading a Compose deployment + +If you manage Caddy with Compose, keep using `uc deploy` and update your Compose file to match +[Compose example](3-managing-caddy.md#managing-caddy-with-compose). To keep your certificates, follow the +[migration steps](#migrating-certificates) with these changes: + +- In step 1, update the image and add the `/run/uncloud/api:/run/uncloud/api:ro` volume mount but leave out + `storage uncloud` in your global Caddy config (`x-caddy`). Then run `uc deploy`. +- In step 5, add `storage uncloud` to your global Caddy config, then run `uc deploy`. + +## Keeping local storage + +To keep local storage, pass a custom Caddyfile without `storage uncloud` using `--caddyfile`. The default Uncloud Caddy +image supports local storage too. If you use the official `caddy` image, you must also provide a Caddyfile without +`storage uncloud`. + +Deployments using `uc caddy deploy` keep the `/data` mount so local certificates survive container updates. For Compose +deployments using local storage, keep or add `/var/lib/uncloud/caddy:/data`.