---
id: 12022
title: 'Day-0 Caddyfile bare :443 prevents tls internal cert issuance'
state: CLOSED
labels:
  - bug
  - documentation
  - ai
assignees:
  - neo-gpt
createdAt: '2026-05-26T13:09:12Z'
updatedAt: '2026-05-26T15:47:16Z'
githubUrl: 'https://github.com/neomjs/neo/issues/12022'
author: neo-opus-ada
commentsCount: 0
parentIssue: null
subIssues: []
subIssuesCompleted: 0
subIssuesTotal: 0
blockedBy: []
blocking: []
closedAt: '2026-05-26T15:47:16Z'
---
# Day-0 Caddyfile bare :443 prevents tls internal cert issuance

## Context

Sixth friction signal from the first-real-world cloud-deployment exercise (siblings #12014, #12015, #12016, #12017, #12019). The canonical reverse-proxy reference shipped via #10803 + the Day-0 Tutorial's "copy-paste-runnable as-is" claim about `tls internal` are empirically broken: a fresh boot of `ai/deploy --profile ingress` will accept TLS connections on `:443` but never serve a self-signed cert, so every browser / curl hits `tlsv1 alert internal error` on handshake.

Surfaced while wiring an external `ai/deploy`-canonical-mirror compose for OIDC validation. The same shape would fail on a fresh neo checkout the moment anyone follows the Day-0 path.

## The Problem

[ai/deploy/Caddyfile:27](https://github.com/neomjs/neo/blob/dev/ai/deploy/Caddyfile) declares the public server block as bare `:443`:

```caddyfile
:443 {
    tls internal

    route {
        ...
    }
}
```

The inline comment + the Day-0 Tutorial promise:

> Copy-paste-runnable as-is: `tls internal` issues a self-signed certificate, so the stack stands up with zero operator configuration.

This is incorrect. `tls internal` requires Caddy to know **which hostname(s)** to issue the cert against. Bare `:443` gives Caddy no SNI / hostname to bind a cert to. Caddy starts cleanly, listens on `:443`, opens its admin endpoint, installs the root CA — but **never issues a leaf certificate for any identifier**. Every TLS handshake fails with `tlsv1 alert internal error`.

Empirical evidence — Caddy logs on a stack booted with this exact `:443 { tls internal }` block:

```
{"level":"info","msg":"using config from file","file":"/etc/caddy/Caddyfile"}
{"level":"info","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
{"level":"info","msg":"enabling HTTP/3 listener","addr":":443"}
{"level":"info","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
{"level":"warn","logger":"pki.ca.local","msg":"installing root certificate ...","path":"storage:pki/authorities/local/root.crt"}
{"level":"info","msg":"certificate installed properly in linux trusts"}
# NO "obtaining certificate" / "certificate obtained" line for ANY identifier
```

Resulting client behavior:

```
$ curl -k -v https://localhost/oauth2/sign_in
* Connected to localhost (127.0.0.1) port 443
* TLS handshake, Client hello (1):
* LibreSSL/3.3.6: error:1404B438:SSL routines:ST_CONNECT:tlsv1 alert internal error
```

Compare with the same Caddyfile after a one-line change to `localhost:443 { tls internal }`:

```
{"level":"info","logger":"http","msg":"enabling automatic TLS certificate management","domains":["localhost"]}
{"level":"info","logger":"tls.obtain","msg":"obtaining certificate","identifier":"localhost"}
{"level":"info","logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"localhost","issuer":"local"}
```

Client now succeeds:

```
$ curl -k -s -o /dev/null -w '%{http_code}\n' https://localhost/oauth2/sign_in
200
```

## The Architectural Reality

Affected:
- `ai/deploy/Caddyfile:27` — the `:443 {` opener.
- `learn/agentos/cloud-deployment/Day0Tutorial.md` — the "copy-paste-runnable as-is" claim is empirically false for the ingress profile until the Caddyfile is fixed.
- `ai/mcp/deploy/proxy/Caddyfile:14` — the older reference Caddyfile shipped by #10803 with the same shape. Same defect class if anyone uses that one too.

Caddy's TLS issuance requires an SNI hostname per [Caddy automatic HTTPS docs](https://caddyserver.com/docs/automatic-https). With `:443` alone, Caddy has no identifier to bind a cert to and silently skips issuance even when `tls internal` is set.

## The Fix

Replace `:443 {` with an explicit hostname. Two reasonable shapes:

1. **Single canonical hostname** (simplest; matches the Day-0 "copy-paste-runnable" intent):
   ```caddyfile
   localhost:443 {
       tls internal
       ...
   }
   ```

2. **Env-driven hostname** (most flexible for operators with non-localhost deployments):
   ```caddyfile
   {$NEO_DEPLOY_HOSTNAME:localhost}:443 {
       tls internal
       ...
   }
   ```
   The operator overrides `NEO_DEPLOY_HOSTNAME` for production; default `localhost` keeps the copy-paste-runnable promise intact.

Either works. Option 1 is the smallest fix; Option 2 is the smallest fix that *also* documents the production-hostname extension point. Implementer's call.

Day-0 Tutorial should either be updated to reflect the chosen shape (mentioning the hostname override) or the existing "copy-paste-runnable" line stays accurate if Option 2 is chosen.

## Contract Ledger

| Target Surface | Source of Authority | Proposed Behavior | Fallback | Docs | Evidence |
|---|---|---|---|---|---|
| `ai/deploy/Caddyfile:27` server block | Caddy automatic-HTTPS docs; `ai/deploy/docker-compose.yml` ingress profile contract | Use `localhost:443` (Option 1) or `{$NEO_DEPLOY_HOSTNAME:localhost}:443` (Option 2) so `tls internal` has a hostname to issue against | None — bare `:443 { tls internal }` produces a server that 100% fails TLS handshake | `ai/deploy/Caddyfile` inline comment + Day-0 Tutorial cross-ref | Caddy logs from a fresh ingress-profile boot show no certificate issuance for any identifier; curl/browser get `tlsv1 alert internal error` |
| `ai/mcp/deploy/proxy/Caddyfile:14` | Same | Same fix shape (sibling reference Caddyfile from #10803) | Same | Inline comment | Same failure mode by construction |
| Day-0 Tutorial copy-paste claim | This ticket; Caddy automatic-HTTPS docs | Either update claim to reference the hostname override, or keep the claim once Option 2 is in place | None — the current claim is false | Day-0 Tutorial body | Empirically verified by external first-deployment exercise |

## Decision Record impact

`aligned-with ADR 0014` — completes the cloud-deployment ingress contract under existing authority. No ADR successor risk.

## Acceptance Criteria

- [ ] `ai/deploy/Caddyfile` opens with an explicit hostname (either `localhost:443` or `{$NEO_DEPLOY_HOSTNAME:localhost}:443`).
- [ ] `docker compose -f ai/deploy/docker-compose.yml --profile ingress up -d --wait` followed by `curl -k https://localhost/` returns a non-handshake-failure HTTP response on a fresh checkout (regression guard).
- [ ] Caddy logs from that boot include a `certificate obtained successfully ... issuer: local` line for the chosen hostname.
- [ ] `ai/mcp/deploy/proxy/Caddyfile` (sibling reference) carries the same fix or is explicitly retired/superseded.
- [ ] Day-0 Tutorial's "copy-paste-runnable" claim either stays accurate (Option 2 path) or is updated to note the hostname requirement (Option 1 path).

## Out of Scope

- Real TLS-cert provisioning (Let's Encrypt etc.). Stay on `tls internal` for the reference path.
- Production deployment scripts / Ansible / etc. — operator territory.
- IPv6 listener tuning, HTTP/3 buffer-size warnings observed in logs (unrelated to this issue).
- The `auto_https off` global-block question (separate latent gotcha; not in canonical neo but a known external-deployment trap — could be a doc note in DeploymentCookbook).

## Avoided Traps

- Filing as `enhancement` — rejected; the canonical reference is currently *unbuildable end-to-end* for the TLS step. That's a `bug`.
- Recommending an absolute production hostname as default — rejected; would break the "fresh checkout boots cleanly without operator config" promise. `localhost` default + env override (Option 2) preserves both.
- Splitting per-file (one ticket for `ai/deploy/Caddyfile`, one for `ai/mcp/deploy/proxy/Caddyfile`, one for Day-0 doc) — rejected; they share root cause and a single PR can address all three.

## Related

- #10803 — originating ticket for the reverse-proxy reference (closed); shipped the defective `:443 { tls internal }` shape.
- #12014, #12015, #12016, #12017, #12019 — sibling friction tickets from the same external-deployment exercise.
- ADR 0014 — cloud-deployment topology authority.
- [Caddy automatic-HTTPS docs](https://caddyserver.com/docs/automatic-https) — explains the SNI / hostname requirement for `tls internal`.

## Handoff Retrieval Hints

- Empirical: boot `ai/deploy --profile cloud --profile ingress`, then `curl -k -v https://localhost/oauth2/sign_in` → `tlsv1 alert internal error`. After hostname fix: 200.
- Caddy log discriminator: `grep "obtaining certificate" /var/log/caddy.log` returns zero lines pre-fix; at least one post-fix.
- Semantic: "Caddy tls internal no hostname", "Day-0 Tutorial tls internal cert issuance failure", "bare :443 ingress profile TLS handshake".
- File anchors: `ai/deploy/Caddyfile:27`, `ai/mcp/deploy/proxy/Caddyfile:14`, `learn/agentos/cloud-deployment/Day0Tutorial.md` (the `copy-paste-runnable` line).

## Timeline

- 2026-05-26T13:09:15Z @neo-opus-ada added the `bug` label
- 2026-05-26T13:09:15Z @neo-opus-ada added the `documentation` label
- 2026-05-26T13:09:15Z @neo-opus-ada added the `ai` label
- 2026-05-26T13:16:35Z @neo-gpt cross-referenced by #12023
- 2026-05-26T13:17:15Z @neo-gpt assigned to @neo-gpt
- 2026-05-26T13:30:53Z @neo-gpt cross-referenced by PR #12024
- 2026-05-26T13:34:37Z @neo-opus-ada cross-referenced by #12025
- 2026-05-26T14:06:49Z @neo-gpt cross-referenced by PR #12020
- 2026-05-26T14:06:49Z @neo-gpt cross-referenced by PR #12021
- 2026-05-26T15:40:34Z @neo-opus-ada cross-referenced by #12026
- 2026-05-26T15:47:16Z @tobiu referenced in commit `05a165a` - "fix(deploy): restore ingress TLS and integration fixtures (#12022, #12023) (#12024)

* fix(deploy): restore ingress TLS and integration fixtures (#12022, #12023)

* fix(deploy): align orchestrator healthcheck state path (#12025, #12023)

---------

Co-authored-by: tobiu <tobiasuhlig78@gmail.com>"
- 2026-05-26T15:47:16Z @tobiu closed this issue
- 2026-05-26T16:11:23Z @neo-opus-ada cross-referenced by #12028
- 2026-05-26T16:12:27Z @neo-opus-ada cross-referenced by #12029

