docs(caddy): update managing Caddy documentation for v0.22.0 features and upgrade instructions

This commit is contained in:
Pasha Sviderski
2026-11-06 19:15:10 +10:00
parent 99c928f490
commit 68d2bd5656
2 changed files with 209 additions and 211 deletions
@@ -1,11 +1,22 @@
# Managing Caddy
Caddy is automatically deployed as a global service `caddy` when you initialise a cluster with `uc machine init`. By
default, it runs on every machine to handle incoming HTTP/HTTPS traffic and route it to your services.
Uncloud automatically deploys Caddy as a global service named `caddy` when you initialise a cluster with
`uc machine init`. By default, it runs on every machine and routes incoming HTTP/HTTPS traffic to your services.
Since [v0.21.0](https://github.com/psviderski/uncloud/releases/tag/v0.21.0), Uncloud provides native
[cluster storage for Caddy](4-cluster-storage-for-caddy.md) (opt-in) to share TLS certificates and coordinate
certificate issuance across machines.
Since [v0.22.0](https://github.com/psviderski/uncloud/releases/tag/v0.22.0), the default Caddy deployment uses the
Uncloud Caddy image
[`ghcr.io/unlabs-dev/caddy-uncloud`](https://github.com/unlabs-dev/caddy-uncloud/pkgs/container/caddy-uncloud) with
[cluster storage](4-cluster-storage-for-caddy.md) to share TLS certificates across machines. This image is a Caddy build
that includes the [Uncloud storage module](https://github.com/unlabs-dev/caddy-uncloud) and a few other popular modules
such as [caddy-l4](https://github.com/mholt/caddy-l4) and [cloudflare](https://github.com/caddy-dns/cloudflare).
:::warning
If you want to upgrade or redeploy Caddy on an existing cluster created with Uncloud v0.21.0 or earlier, follow
[Upgrading from earlier versions](4-cluster-storage-for-caddy.md#upgrading-from-earlier-versions) before redeploying
Caddy.
:::
## Checking status
@@ -16,55 +27,90 @@ uc inspect caddy
```
```
ID: b5b269d5dc5ed4fdae6542894f94de82
Name: caddy
Mode: global
Service ID: b5b269d5dc5ed4fdae6542894f94de82
Name: caddy
Mode: global
CONTAINER ID IMAGE CREATED STATUS MACHINE
fb8f390e634d caddy:2.10.0 3 weeks ago Up 3 weeks prod-ap1
0182f5d7bd9f caddy:2.10.0 3 months ago Up 3 weeks prod-us1
CONTAINER ID IMAGE CREATED STATUS IP ADDRESS MACHINE
fb8f390e634d ghcr.io/unlabs-dev/caddy-uncloud:0.1.4 3 weeks ago Up 3 weeks 10.210.1.2 prod-ap1
0182f5d7bd9f ghcr.io/unlabs-dev/caddy-uncloud:0.1.4 3 weeks ago Up 3 weeks 10.210.0.2 prod-us1
```
## Verifying config
View the complete generated Caddyfile that the `caddy` service uses. It includes global configuration, sites for
published service ports, custom service configs, and comments explaining any skipped invalid configs. It also warns if
the last config load failed. Use `--machine NAME` to check a specific machine.
```shell
uc caddy config
```
See [Verifying Caddy config](2-publishing-services.md#verifying-caddy-config) for an example output and details on how
it's generated.
## Listing certificates
List the issued certificates in cluster storage:
```shell
uc caddy cert ls
```
```
ID NAME ISSUER EXPIRES
c111e23615f7 dns.uncloud.run Let's Encrypt 2026-12-31 (2 months)
b5dcd2a03e1b nginx.2t5ex2.uncld.dev Let's Encrypt 2026-12-31 (2 months)
70707fd2fea2 test-staging.2t5ex2.uncld.dev Let's Encrypt (staging) 2026-12-31 (2 months)
c5f9301e1c06 uncloud.run Let's Encrypt 2026-12-31 (2 months)
```
## Deploying or updating Caddy
### Using CLI
Update to the latest stable version using the [caddy](https://hub.docker.com/_/caddy) image from Docker Hub:
Deploy Caddy or upgrade it to the latest version of the Uncloud Caddy image:
```shell
uc caddy deploy
```
Deploy a specific version or custom image:
The command shows a deployment plan and asks for confirmation, then performs a rolling update.
You can set a specific image, global Caddyfile, or limit deployment to specific machines:
```shell
uc caddy deploy --image caddybuilds/caddy-cloudflare:2.11.4
# Deploy a specific version or custom image and keep it for later deployments.
uc caddy deploy --image ghcr.io/unlabs-dev/caddy-uncloud:0.1.4
# Deploy with a custom global Caddy config and only on selected machines.
uc caddy deploy --caddyfile global.Caddyfile --machine machine1,machine2
```
Deploy only to a specific machine or a subset of machines (comma-separated list):
The command remembers these values for later deployments. Pass an empty value to reset a flag to its default:
```shell
uc caddy deploy --machine machine1
uc caddy deploy --machine machine2,machine3,machine4
uc caddy deploy --image "" --caddyfile "" --machine ""
```
Deploy with custom global configuration:
See [`uc caddy deploy`](../../9-cli-reference/uc_caddy_deploy.md) for all flags and examples.
```shell
uc caddy deploy --caddyfile global.Caddyfile
```
## Custom global configuration
Example global configuration:
A custom Caddyfile replaces the default global config. Uncloud combines this config with the sites generated from
[published ports](2-publishing-services.md) and
[custom service configs](2-publishing-services.md#custom-caddy-configuration). Include `storage uncloud` to keep using
[cluster storage](4-cluster-storage-for-caddy.md):
```caddyfile title=global.Caddyfile
```caddyfile title="global.Caddyfile"
# Global options.
{
debug
storage uncloud
# Uncomment for troubleshooting:
# debug
}
# A snippet that can be reused in custom Caddy configs for services (x-caddy).
(my_snippet) {
...
# ...
}
# Expose an internal service that is not managed by Uncloud.
@@ -73,10 +119,29 @@ internal.example.com {
}
```
### Using Compose
Deploy your custom global Caddy config:
You can manage the Caddy deployment with a Compose file for more control. For example, to deploy a custom global Caddy
config that uses the DNS challenge with Cloudflare to obtain a wildcard TLS certificate for `*.example.com`:
```shell
uc caddy deploy --caddyfile global.Caddyfile
```
## Custom image
A custom image must include the [Uncloud storage module](https://github.com/unlabs-dev/caddy-uncloud) to use cluster
storage. It must also include `curl` for the health check and start Caddy with `/etc/caddy/Caddyfile`.
The default Uncloud Caddy image
[`ghcr.io/unlabs-dev/caddy-uncloud`](https://github.com/unlabs-dev/caddy-uncloud/pkgs/container/caddy-uncloud) satisfies
these requirements. See the [release notest](https://github.com/unlabs-dev/caddy-uncloud/releases/latest) for the Caddy
version and the list of included modules.
If you need other Caddy modules or want to choose a compatible Caddy version, you can
[build and deploy your own image](https://github.com/unlabs-dev/caddy-uncloud#building-and-deploying-a-custom-image).
## Managing Caddy with Compose
Use Compose when you need more control, such as environment variables for a DNS provider. This example uses the
Cloudflare module included in the default image to obtain a wildcard certificate for `*.example.com`:
<Tabs>
<TabItem value="compose.yaml">
@@ -84,13 +149,11 @@ config that uses the DNS challenge with Cloudflare to obtain a wildcard TLS cert
```yaml
services:
caddy:
image: caddybuilds/caddy-cloudflare:2.11.4
image: ghcr.io/unlabs-dev/caddy-uncloud:0.1.4
environment:
# unix// is not a typo. Caddy uses network/address format, not a unix:// URL.
CADDY_ADMIN: unix//run/caddy/admin.sock
env_file:
# Contains CLOUDFLARE_API_TOKEN=xxxxx
- .env.secrets
CLOUDFLARE_API_TOKEN: secret://cloudflare_api_token
healthcheck:
test: curl -fsS -o /dev/null --unix-socket /run/caddy/admin.sock http://localhost/config/
interval: 30s
@@ -102,22 +165,26 @@ services:
- /var/lib/uncloud/caddy:/etc/caddy:ro
# Persists TLS certificates and other assets when using Caddy's default local storage.
- /var/lib/uncloud/caddy:/data
- /run/uncloud/caddy:/run/caddy
# Required by the Uncloud cluster storage module (https://github.com/unlabs-dev/caddy-uncloud).
# Remove this mount if cluster storage is not used for Caddy.
# Uncloud API socket is required by the cluster storage module (https://github.com/unlabs-dev/caddy-uncloud).
# Mount the directory, not the socket file, so Caddy sees a replacement socket after the daemon restarts.
- /run/uncloud/api:/run/uncloud/api:ro
- /run/uncloud/caddy:/run/caddy
x-ports:
- 80:80@host
- 443:443@host
- 443:443/udp@host
x-caddy: Caddyfile
x-caddy: ./Caddyfile
deploy:
mode: global
# Optional: deploy only to specific machines.
# x-machines:
# - machine1
# - machine2
secrets:
cloudflare_api_token:
# Read the secret value from 1Password by running this command.
x-command: op read "op://prod/caddy/cloudflare_api_token"
```
</TabItem>
@@ -126,144 +193,34 @@ services:
```caddyfile
# Global options.
{
debug
storage uncloud
}
# A snippet that can be reused in custom Caddy configs for services (x-caddy).
(my_snippet) {
...
}
# Obtain a wildcard TLS certificate for all subdomains of example.name using DNS challenge with Cloudflare.
# It will be used for services that publish ports with hostnames under example.name.
# Obtain a wildcard TLS certificate for all subdomains of example.com using DNS challenge with Cloudflare.
# It will be used for services that publish ports with hostnames under example.com.
*.example.com {
tls {
dns cloudflare {env.CLOUDFLARE_API_TOKEN}
}
respond "No host matched" 404
}
# Expose an internal service that is not managed by Uncloud.
internal.example.com {
reverse_proxy 192.168.1.100
tls {
dns cloudflare {env.CLOUDFLARE_API_TOKEN}
}
respond "No host matched" 404
}
```
</TabItem>
</Tabs>
:::info note
The specified `environment`, `volumes`, and `x-ports` properties are essential for Caddy to function correctly in the
Uncloud cluster. Do not change the source paths of the volume mounts as the Uncloud daemon relies on them to communicate
with Caddy and update its configuration.
The image must include `curl` for the healthcheck and start Caddy with `/etc/caddy/Caddyfile`. Images based on the
official [Caddy image](https://hub.docker.com/_/caddy) do both by default.
Keep the `/data` mount while using Caddy's default local storage so TLS certificates survive container updates. You can
remove this mount when using [cluster storage for Caddy](4-cluster-storage-for-caddy.md).
:::
Deploy or update the `caddy` service from the Compose file:
```shell
uc deploy
```
## Verifying config
:::info note
View the complete generated Caddyfile served by the `caddy` service. This is useful for debugging and verifying custom
global and service-specific Caddy configs.
Keep the `environment`, `volumes`, and `x-ports` properties as shown, including the source paths of the volume mounts.
The Uncloud daemon and the storage module rely on them.
```shell
uc caddy config
```
:::
Example output:
```caddyfile
# Caddyfile autogenerated by Uncloud (DO NOT EDIT): 2025-12-22T10:30:12Z
# Automatically updated on service or health status changes.
# Docs: https://uncloud.run/docs/concepts/ingress/overview
# User-defined global config from service 'caddy'.
# Global options.
{
debug
}
# A snippet that can be reused in custom Caddy configs for services (x-caddy).
(my_snippet) {
...
}
# Obtain a wildcard TLS certificate for all subdomains of example.name using DNS challenge with Cloudflare.
# It will be used for services that publish ports with hostnames under example.name.
*.example.com {
tls {
dns cloudflare {env.CLOUDFLARE_API_TOKEN}
}
respond "No host matched" 404
}
# Expose an internal service that is not managed by Uncloud.
internal.example.com {
reverse_proxy 192.168.1.100
}
# Health check endpoint to verify Caddy reachability on this machine.
http:// {
handle /.uncloud-verify {
respond "a369b9388812f9557feef6a0f5b46f2e" 200
}
log
}
(common_proxy) {
# Retry failed requests up to lb_retries times against other available upstreams.
lb_retries 3
# Upstreams are marked unhealthy for fail_duration after a failed request (passive health checking).
fail_duration 30s
}
# Sites generated from service ports.
https://app.example.com {
reverse_proxy 10.210.1.3:8000 10.210.2.5:8000 {
import common_proxy
}
log
}
https://api.example.com {
reverse_proxy 10.210.2.2:9000 10.210.1.7:9000 10.210.2.3:9000 {
import common_proxy
}
log
}
# User-defined config for service 'web'.
www.example.com {
redir https://example.com{uri} permanent
}
example.com {
reverse_proxy 10.210.0.3:8000 {
import common_proxy
}
log
}
# Skipped invalid user-defined configs:
# - service 'duplicate-hostname': validation failed: adapting config using caddyfile adapter: ambiguous site definition: example.com
# - service 'invalid': validation failed: adapting config using caddyfile adapter: Caddyfile:61: unrecognized directive: invalid_directive
```
The generated config combines:
- Global Caddy configuration (`x-caddy` from the `caddy` service).
- Auto-generated configs from published service ports (`x-ports`).
- Custom Caddy configs from services (`x-caddy`).
- Skipped invalid configs with error messages as comments.
Continue using `uc deploy` for a Compose-managed Caddy service. Switching to `uc caddy deploy` replaces its
configuration, including environment variables and custom mounts.
@@ -1,73 +1,114 @@
# Cluster storage for Caddy
Starting with Uncloud [v0.22.0](https://github.com/psviderski/uncloud/releases/tag/v0.22.0), Caddy uses cluster storage
by default.
The [Uncloud storage module](https://github.com/unlabs-dev/caddy-uncloud) lets Caddy instances share TLS certificates,
private keys, and ACME challenge tokens through Uncloud's cluster store. It uses distributed locks to coordinate
certificate issuance. Once one instance obtains a certificate, the others can use it too.
private keys, and ACME challenge tokens through Uncloud's cluster store.
## Why use cluster storage?
This matters when you run Caddy on multiple machines and distribute traffic between them using DNS or a load balancer.
In this case, an ACME challenge can reach a different Caddy instance from the one requesting the certificate. Cluster
storage lets any Caddy instance read the challenge token from the shared cluster store and complete the ACME challenge.
It uses distributed locks to coordinate certificate issuance. Once one instance obtains a certificate, the others can
use it too.
By default, each Caddy instance stores its certificates locally. When DNS points to multiple machines or a load balancer
distributes traffic between them, an ACME challenge can reach a different instance from the one requesting the
certificate. That instance may not have the challenge token, which can delay or prevent certificate issuance.
## Upgrading from earlier versions
Cluster storage is optional. The default Caddy image does not include the module, so you need to deploy an image that
includes it and configure Caddy to use it.
Starting with Uncloud v0.22.0, `uc caddy deploy` deploys the Uncloud Caddy image ([
`ghcr.io/unlabs-dev/caddy-uncloud`](https://github.com/unlabs-dev/caddy-uncloud/pkgs/container/caddy-uncloud)) with
cluster storage enabled by default.
## Enabling cluster storage
Earlier versions deployed the official [`caddy`](https://hub.docker.com/_/caddy) image with local certificate storage.
Upgrading Uncloud does not change the running Caddy service. Redeploy it to switch to the new default image and cluster
storage.
:::warning Certificate migration
Before you deploy, upgrade the [`uc` CLI](../../2-getting-started/1-install-cli.md) and the daemon on all machines to
v0.22.0 or newer. Follow the [upgrade instructions](https://github.com/psviderski/uncloud/releases/latest) for daemon
upgrades, including any required migrations from older releases. Check current versions with `uc version` and `uc
machine ls`.
Caddy doesn't migrate existing certificates automatically when you change its storage backend. **All certificates will
need to be reissued unless you migrate your existing storage first.** Use the experimental
[`caddy storage export` and `caddy storage import`](https://caddyserver.com/docs/command-line#caddy-storage)
commands with the old and new configs to transfer the storage contents.
The first `uc caddy deploy` after the upgrade doesn't know the values you set with earlier versions of `uc`. It replaces
your image, global config, and machine list with the defaults unless you pass them as flags.
:::
If you already use `storage uncloud`, your certificates stay where they are, so you only need to run `uc caddy deploy`.
:::info Requirements
### Quick upgrade
Cluster storage requires Uncloud **v0.21.0 or newer** for both the `uc` CLI and the daemon on every cluster machine.
Check your CLI version with `uc version` and daemon versions with `uc machine ls`. Upgrade older versions before
enabling cluster storage.
Caddy does not copy certificates when you change storage backends. Unless you migrate them first, Caddy will reissue
them. HTTPS may fail for those domains until that finishes, and a large number of certificates can hit
[Let's Encrypt rate limits](https://letsencrypt.org/docs/rate-limits/).
:::
Create a global Caddyfile, or add `storage uncloud` to the global options in your existing one:
```caddyfile title="global.Caddyfile"
{
storage uncloud
# Uncomment to enable debug logs useful for troubleshooting storage operations:
# debug
}
```
Deploy Caddy with the pre-built module image and your global config:
Run the command below if you don't mind reissuing all certificates:
```shell
uc caddy deploy --image ghcr.io/unlabs-dev/caddy-uncloud:0.1.3 --caddyfile global.Caddyfile
uc caddy deploy
```
See the [module README](https://github.com/unlabs-dev/caddy-uncloud#usage) for custom image builds, Compose deployment,
and additional storage options.
### Migrating certificates
## Verifying storage
Follow these steps to keep your existing certificates and avoid HTTPS failures:
Check that `storage uncloud` appears in the Caddy config:
1. Deploy the new image but stay on local storage. Use your existing global config if you have one, or an empty file:
```shell
uc caddy config
```
```shell
touch local.Caddyfile
uc caddy deploy --caddyfile local.Caddyfile
```
List issued certificates in cluster storage with [`uc caddy cert ls`](../../9-cli-reference/uc_caddy_cert_ls.md):
2. Find the ID of the Caddy container on each machine:
```shell
uc caddy cert ls
```shell
uc inspect caddy
```
ID NAME ISSUER EXPIRES
c111e23615f7 dns.uncloud.run Let's Encrypt 2026-12-31 (2 months)
b5dcd2a03e1b nginx.2t5ex2.uncld.dev Let's Encrypt 2026-12-31 (2 months)
70707fd2fea2 test-staging.2t5ex2.uncld.dev Let's Encrypt (staging) 2026-12-31 (2 months)
c5f9301e1c06 uncloud.run Let's Encrypt 2026-12-31 (2 months)
```
3. Copy the certificates to cluster storage by running this command for each container ID. Each machine stores its own
certificates, so don't skip any of them. These
[`caddy storage`](https://caddyserver.com/docs/command-line#caddy-storage) commands are experimental.
```shell
uc exec -T --container CONTAINER_ID caddy sh -c '
printf "{\n\tstorage uncloud\n}\n" > /tmp/Caddyfile.cluster
caddy storage export --config /etc/caddy/Caddyfile --output - |
caddy storage import --config /tmp/Caddyfile.cluster --input -
rm /tmp/Caddyfile.cluster
'
```
4. Check that all your certificates are in cluster storage:
```shell
uc caddy cert ls
```
5. Switch Caddy to cluster storage. If you have a custom global config, add `storage uncloud` to it and pass it with
`--caddyfile global.Caddyfile` instead. The command warns that all certificates will be reissued. You can ignore this
because you've already copied them.
```shell
uc caddy deploy --caddyfile ""
```
6. Check that the Caddy config includes `storage uncloud` and doesn't show any config load warnings:
```shell
uc caddy config
```
### Upgrading a Compose deployment
If you manage Caddy with Compose, keep using `uc deploy` and update your Compose file to match
[Compose example](3-managing-caddy.md#managing-caddy-with-compose). To keep your certificates, follow the
[migration steps](#migrating-certificates) with these changes:
- In step 1, update the image and add the `/run/uncloud/api:/run/uncloud/api:ro` volume mount but leave out
`storage uncloud` in your global Caddy config (`x-caddy`). Then run `uc deploy`.
- In step 5, add `storage uncloud` to your global Caddy config, then run `uc deploy`.
## Keeping local storage
To keep local storage, pass a custom Caddyfile without `storage uncloud` using `--caddyfile`. The default Uncloud Caddy
image supports local storage too. If you use the official `caddy` image, you must also provide a Caddyfile without
`storage uncloud`.
Deployments using `uc caddy deploy` keep the `/data` mount so local certificates survive container updates. For Compose
deployments using local storage, keep or add `/var/lib/uncloud/caddy:/data`.