* httpcaddyfile: new tls_automate_names global option
Provisioning a certificate for a name that is not served requires giving
it a site block of its own:
*.example.com {
}
foo.example.com {
respond "Real site"
}
That asks the tls app for the wildcard, but it also adds a route to the
http app, so every name pointed at the server that has no site block of
its own -- bar.example.com here -- gets an empty but valid response
rather than no match at all. Wanting a certificate and wanting to serve
a name are separate things, and the Caddyfile had no way to say only the
first.
Name them in the new global option instead:
{
tls_automate_names *.example.com
}
foo.example.com {
respond "Real site"
}
The names are added to the automate certificate loader and to an
automation policy built from the global options, so a name listed here
is managed exactly as it would be from a site block, with the same
issuers; the only difference in the adapted config is that no route is
added for it. Repeating the option appends rather than replaces, so a
long list can be split over several lines.
Names that cannot get a public certificate are given the internal
issuer, the same treatment a site block gives them. Names that already
appear in the automate list are skipped, but a name that also has a site
block is left listed here as well: a site block for http:// only is not
managed by auto-HTTPS, so dropping the name because a block exists could
silently leave it without a certificate.
The option needs no http app at all, so a config consisting only of
global options now adapts to a tls app on its own -- enough to keep
certificates renewed for a mail or XMPP server, or for names served by a
layer 4 app.
Closes#7122
* httpcaddyfile: reuse an existing policy for an automated name
A name given to tls_automate_names may already have an automation policy
from its own site block. Adding a second policy for the same subject is
not just redundant: the adapter rejects overlapping subjects, so
{
email nobody@example.com
tls_automate_names foo.example.com
}
foo.example.com {
tls {
ca https://acme.example.test/directory
}
}
failed to adapt at all, with "hostname appears in more than one
automation policy, making certificate management ambiguous".
Keep such a name in the automate loader but leave its policy alone. The
site block's policy is the more specific of the two, and the loader entry
is still wanted, since a site block served only over HTTP is not managed
by auto-HTTPS.
* Pin that tls_automate_names overrides auto_https off
The names given to the option are managed whether or not auto-HTTPS is
disabled: unlike the hostless-key block above it, that code is not gated
on auto_https, because naming a subject explicitly is a stronger signal
than the general switch. Nothing enforced it, so add the adaptation
fixture @steadytao asked for.
Provision previously only checked for a single opening brace at the
start of a destination, so values like {result}suffix, {result, and
{result}} passed validation. strings.Trim(dest, "{}") then silently
stored a wrong placeholder name (e.g. result}suffix), making the
intended value unavailable.
Require the destination to be exactly one placeholder: a single
opening brace at the start, a single closing brace at the end, and a
non-empty name.
Fixes#8073
when a request exceeds the request_body max_size limit, the request_body handler wraps the http.MaxBytesError into a caddyhttp.HandlerError carrying status 413, but the {http.request.body} and {http.request.body_base64} placeholders ran io.Copy with the error ignored, so they silently returned the truncated prefix as though it were the complete body. docs promise a 413 for reads past max_size, and silently truncating is a bad failure mode in templates and vars_regexp where the body value drives decisions. see #7691 and the narrow follow-up prescribed when #7692 was closed
reading the body now returns a dedicated RequestBodyLimitError marker instead of a generic HandlerError, and only when the read failure is actually the max_size limit. the consumers (template placeholder function, vars and vars_regexp matchers) recognize exactly that marker and wrap it in a status carrying handler error so the oversized request fails with 413, while every unrelated error value keeps its old behavior: templates still render it as text and the vars matchers still match on its error text. the surfaced error is stripped of its generated id and stack trace so a placeholder stringified into a response body cannot leak the call stack
negative regressions prove an unrelated HandlerError in templates, vars and vars_regexp does not start controlling request handling, plus integration tests for the template and vars_regexp 413 paths
ai assisted (GLM agent) under Abdel's direction and local verification
Signed-off-by: Abdel <hktitof@gmail.com>
* Move websocket header normalization later
A regression since #6621, these headers can be rewritten by cloneHeaders
and by .Add from user and transport ops. Moving this closer to where
it's round-tripped feels like the nicest solution.
I've put the harness I used to reproduce the issue at
https://codeberg.org/nfreya/caddy-websocket-headers-repro
* Integration test for normalized websocket headers
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
A request whose body has no length of its own — Transfer-Encoding: chunked, or
HTTP/2 and HTTP/3 requests sent without the header — cannot be forwarded over
FastCGI until it has been buffered in full, because CGI/1.1 requires
CONTENT_LENGTH and php-fpm hangs when it is absent or wrong and the body is not
empty. When request_buffers is too small to hold the whole body, the length
stays unknown and the request is refused with 411.
The refusal said none of that. RoundTrip passes r.ContentLength to Post, which
is -1 for such a request, so the operator got a bare "411 Length Required" with
either no error at all or strconv's "invalid syntax" on a value they never
wrote. On #7386 that sent two people looking for the fault in their backend.
The 411 now carries the reason and the remedy, and the unknown-length case is
told apart from a genuinely malformed value. ParseUint becomes ParseInt plus an
explicit negative check, which is strictly tighter: values above MaxInt64 used
to be accepted and are unreachable anyway, since CONTENT_LENGTH is always
FormatInt of an int64 by the time Do sees it.
No status code changes. Every request that was refused before is still refused;
raising request_buffers is still what makes a large chunked body work.
Tests: the reachable path through Post, the unusable CONTENT_LENGTH values Do
itself guards against, an integration test driving a chunked body through a
fastcgi reverse_proxy over a unix socket at each side of the buffer boundary
(including request_buffers -1, which succeeds at any size and is the remedy),
and a boundary test recording that a body exactly the size of the buffer counts
as partial.
That last one is deliberately not changed here. Telling "exactly the limit"
apart from "more to come" costs a read that a paused stream may never answer,
and bufferedBody also backs response_buffers: measured against a body that
delivers exactly the limit and stops, the current code hands the buffered
prefix on immediately, while peeking one byte first delivers nothing at all.
Co-authored-by: Aditya <205600203+Rohilalala@users.noreply.github.com>
Co-authored-by: Zen Dodd <mail@steadytao.com>
* caddyhttp: shield specific hostnames from a covering wildcard's client auth (@sillyzir)
A connection policy for a wildcard hostname (e.g. *.example.com with
client_auth) is first-match by SNI, so it also applied client
authentication to more specific hostnames served by their own site
blocks (public.example.com) — sites that never asked for mTLS.
Two cases produce the shielding empty policy that fixes this:
- site blocks whose TLS config yields a connection policy with no
settings (previously discarded as having no effect);
- site blocks with no TLS connection policy at all — the reported
case — for which an empty policy is now synthesized.
Either way the empty policy is hoisted directly above the first
client-auth-bearing policy whose wildcard SNI covers the hostname, so
first-match shields it from the client-auth requirement.
Deliberately scoped to client authentication: other wildcard policy
settings, such as certificate selection, are ones a covered hostname
generally WANTS to inherit (see tls_automation_wildcard_shadowing);
sni matchers that fail to decode emit an adapt warning instead of
being silently skipped.
Fixes#7860
* caddyhttp: shield preserves the wildcard policy's other settings (@sillyzir)
The hoisted shield was an empty policy, and connection policies are
first-match: it lifted the client-auth requirement but also suppressed
every other setting the covering wildcard policy carried (certificate
selection, protocol bounds, ALPN). Hoist a copy of the covering policy
with only client_authentication removed instead, so the shielded
hostname keeps inheriting the rest.
The new adapt test gives the wildcard policy protocols and alpn in
addition to client_auth and asserts the shield carries both while
dropping only client authentication; the existing test (client_auth
only) is unchanged, which is exactly why it could not catch this.
* caddyhttp: shield each covering wildcard policy separately (@sillyzir)
The hoisting loop stopped at the first client-auth policy covering any of a
site block's hostnames and gave the shield that block's whole SNI matcher, so
hostnames covered by a *different* wildcard matched it too. Because connection
policies are first-match, those hostnames then took the wrong policy's
settings and lost their own client-authentication requirement entirely.
Map each hostname to the first covering policy individually and hoist one
shield per covering policy, matching only the hostnames it covers. The
existing policies' wildcard SNI names are decoded once up front, and the
shields are inserted back to front so an insertion cannot shift the index of a
covering policy still to be shielded.
* chore: fumpt (@steadytao)
* chore: fix master lint (@steadytao)
---------
Co-authored-by: SillyZir <269283839+SillyZir@users.noreply.github.com>
Co-authored-by: Zen Dodd <mail@steadytao.com>
Caddy's underscore header filter (GHSA-f59h-q822-g45g) only checked
for `_`. PHP folds `.` to `_` when registering $_SERVER keys the same
way CGI/FastCGI folds `-` to `_`, so a dotted alias (e.g. Remote.User)
survived the filter and collided with the legitimate hyphenated
header once it reached a PHP/FastCGI backend, bypassing forward_auth
copy_headers the same way the underscore alias did.
Extends the filter to drop `.` symmetrically, adds an
`expected_dot_headers` allowlist mirroring `expected_underscore_headers`,
and handles header names containing both separators (only an exact
allowlist entry is honored there, since a prefix glob's free-form
suffix can't be vetted for an embedded second separator).
Root cause identified by @iliaal in the FrankenPHP advisory
GHSA-49wc-4hcv-v58q.
Same obsolete two-argument check as the reverse_proxy one removed in
#7869: it referenced the long-removed inline status-code-replacement
syntax and pointed users at replace_status even when no status
replacement was involved. The generic check now rejects all
excess-argument cases with an accurate message.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* caddyhttp: add url_pattern request matcher
Match requests against a URLPattern (https://urlpattern.spec.whatwg.org/),
supporting named groups, wildcards and regexp components beyond the path
matcher. Relative patterns match any origin; absolute patterns or base_url
scope to scheme and host.
Exposes a url_pattern CEL function and publishes captured groups as
{http.url_pattern.<component>.<group>} placeholders.
* caddyhttp: add Caddyfile adapt test for url_pattern matcher
Multiple `hide` subdirectives in a file_server Caddyfile block silently overwrote each other, so only the last one took effect. Append to the list instead, so repeated entries accumulate and imported snippets can compose with site-specific hides.
* test: add failing tests for intercept replace_status (#7805)
Add integration tests that verify replace_status actually substitutes
the HTTP status code sent to the client. Currently these tests fail
because replace_status is silently a no-op due to value-receiver
boxing and shouldBuffer returning false.
Tests added:
- TestInterceptReplaceStatusWithMatcher: 500 -> 200 with @err matcher
- TestInterceptReplaceStatusWithoutMatcher: 403 -> 200 unconditionally
- TestInterceptReplaceStatusNotMatched: 200 passes through unchanged
* fix: make intercept replace_status actually substitute the status code
Fix#7805: replace_status was silently a no-op because:
1. shouldBuffer returned false when a replacement status was set,
causing the original status to be streamed directly to the wire
2. The value-receiver WriteHeader method operated on a stale copy
The fix:
- shouldBuffer now returns true when replace_status matches, so
the response is buffered instead of streamed
- After next.ServeHTTP returns, if routes are nil (replace_status
only), write the substituted status and buffered body to the client
The interceptedResponseHandler.WriteHeader substitution branch is no
longer needed for this path since substitution happens post-ServeHTTP.
* refactor: remove dead WriteHeader method and resolved TODO
The value-receiver WriteHeader on interceptedResponseHandler was
unreachable dead code — the substitution is now handled post-ServeHTTP
via buffering. Remove it along with the TODO comment that noted
status code replacement was unfinished.
* style: apply nit suggestions from dunglas code review
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* reverseproxy: validate on weighted_round_robin policy
Validate that weighted_round_robin has a non-zero total weight.
This prevents configurations such as:
weighted_round_robin 0 0
from being accepted and causing a divide-by-zero panic during request handling.
* test: validation test on zero weight upstreams.
* test: provision called instead of totalweight setting
* reverseproxy: validate on negative upstream weights
* test: regression test on weighted_round_robin selection policy
* fix: error on duplicate named_routes
Fixes issue #7798
Validate named route names before inserting them into the
named route map.
This prevents later definitions from overwriting existing
named routes and returns an error when a route name is
defined more than once.
* test: add test for duplicate named_routes
* reverseproxy: replace placeholders specified for sni while using http3
* add test for placeholder
* reverseproxy: replace placeholders specified for sni while using http3
* add test for placeholder
* reverseproxy: test HTTP/3 SNI host placeholder
---------
Co-authored-by: Zen Dodd <mail@steadytao.com>
* feat: drop headers with underscore in their names
* feat: Caddyfile binding and tests for underscore-in-header drop
Add the `allow_underscore_in_headers` global server option, refine the
doc comment, and cover the filter end-to-end: server-level unit tests
(drop, opt-out, debug log, RFC-7230 space rejection), a fastcgi unit
test for the trimmed header name replacer, and forward_auth integration
tests for both the default-drop and opt-out paths.
* remove allow_underscore_in_headers option for now
* fix(caddyfile): {block} in snippet
Resolve issue #7557
So, here is the situation:
- Pull request #7206 included some changes to the doImport's function of
Caddyfile's parser. What it does is that if there is no token within a
block that follows the import, and the import contains `{block}`, then
the `{block}` token is discarded.
- After this pull request:
- Issue #7518 noticed that in cases that `{block}` was not imported,
a runtime error was raised due to the assumption that tokens were
always added to `tokensCopy` on every iteration of `importedTokens`.
This was fixed by pull request #7543.
- Issue #7557 notices that {block} can be ignored when imported from a
certain file. There, it's again an issue with how the import works.
When `import snippets` is called, this import instruction doesn't
contains any nested blocks. And when the argument replacer that is
the `importedTokens` loop is called and finds `{block}`, it uses the
block from the file's import (which in this case is nothing),
`{block}` is erased, and unavailable when the import directive is
called for the imported snippet.
The changed in this commit addresses the second issue by checking before
replacing `{block}` if we're currently in a snippet definition, and
appending the `{block}` token to `tokensCopy` if we are.
With this changes, when importing those snippets, the `{block}` token
will be available to be replaced by the nested blocks in `tokensToAdd`
if needed, or erased if there are no nested blocks and `tokensToAdd` is empty.
Tests added in pull requests #7206 and #7543 passes with this new
implementation, confirming that unused `{block}` are accepted if nothing
is passed to `import`, as well as the other usual tests.
A new test was also added based on issue #7557 reporting, and also passes.
Signed-off-by: prettysunflower <me@prettysunflower.moe>
* caddyfile: add imported snippet block placeholder coverage
---------
Signed-off-by: prettysunflower <me@prettysunflower.moe>
Co-authored-by: Zen Dodd <mail@steadytao.com>
* feat: add system and combined CA pool modules
* fix: combining pools using `CertificateProvider`
* fix: lint issue
* chore: caddyfiletests
* doing it for first time, so not sure if its right.
* fix: use `x509` native addCert
* chore: explicit err handling
* Apply suggestion from @mohammed90
---------
Co-authored-by: Mohammed Al Sahaf <mohammed@caffeinatedwonders.com>
* add 'root' key to Helper.State for access in frankenphp's `php_server` directive
* clone state before passing it to child directives, but keep sharing it among sibling directives
* propagate named route state from children to parent
* use BlockState to set "root" instead
* gofmt -w .
* go fmt ./...
* here we go
When using copy_headers in a forward_auth block, client-supplied headers with
the same names were not being removed before being forwarded to the backend.
This happens because PR #6608 added a MatchNot guard that skips the Set
operation when the auth service does not return a given header. That guard
prevents setting headers to empty strings, which is the correct behavior,
but it also means a client can send X-User-Id: admin in their request and
if the auth service validates the token without returning X-User-Id, Caddy
skips the Set and the client value passes through unchanged to the backend.
The fix adds an unconditional delete route for each copy_headers entry,
placed just before the existing conditional set route. The delete always runs
regardless of what the auth service returns. The conditional set still only
runs when the auth service provides that header.
The end result is:
- Client-supplied headers are always removed
- When the auth service returns the header, the backend gets that value
- When the auth service does not return the header, the backend sees nothing
Existing behavior is unchanged for any deployment where the auth service
returns all of the configured copy_headers entries.
Fixes GHSA-7r4p-vjf4-gxv4