* caddyhttp: presize buffers in hot paths to reduce allocations
Right-size three per-request allocations that previously grew from a
nil/empty backing array, causing repeated reallocation:
- CleanPath: pre-Grow the strings.Builder used when slash collapsing is
disabled, instead of letting it reallocate as runes are appended
- headers: presize the rewritten value slice in HeaderOps.ApplyTo's Set
loop instead of appending from nil
- reverseproxy: presize the weights slice in WeightedRoundRobinSelection
.Select
This commit also adds benchmarks exercising each path. Measured with benchstat
with count=20 and GOMAXPROCS=1, with p=0.000:
sec/op B/op allocs/op
CleanPathNoCollapse 1013.4n -> 828.6n (-18.23%) 120->64 (-46.67%) 4->1 (-75.00%)
HeaderOpsApplyToSet 1262.0n -> 955.8n (-24.27%) 255->152 (-40.39%) 9->6 (-33.33%)
WeightedRoundRobinSelect 215.8n -> 200.3n ( -7.20%) 112->96 (-14.29%) 2->2 (N/A)
* caddyhttp: presize strings.Builder in query rewrite and path matcher
Two builders whose output size is bounded by a known input length were
growing from empty, causing repeated reallocation. Seed each with a
capacity hint:
- rewrite.buildQueryString: pre-Grow to len(qs) (runs per request when
rewrite query manipulation is configured)
- MatchPath.matchPatternWithEscapeSequence: pre-Grow to len(escapedPath)
(reached only when a path pattern contains % escape sequences)
Adds a benchmark for buildQueryString, measured with benchstat:
BuildQueryString:
- sec/op 965.2n -> 743.3n (-22.99%)
- B/op 248->80 (-67.74%)
- allocs/op 5->1 (-80.00%)
The builder previously reallocated four times per call. The matcher path
is cold and not separately benchmarked.
* chore: add benchmark headers
---------
Signed-off-by: Julien Voisin <jvoisin@users.noreply.github.com>
(cherry picked from commit c8492563ee)
* httpcaddyfile: reject conflicting client_auth when merging TLS policies
* httpcaddyfile: use host and host:port site addresses in client auth conflict test
* httpcaddyfile: enhance client auth consolidation test to handle conflicting configurations
* httpcaddyfile: remove unnecessary blank line in conflict error handling
(cherry picked from commit c1c0ee5dad)
* caddyhttp: lower-case path patterns with placeholders at request time
Patterns with placeholders were lower-cased at provisioning, which
changed case-sensitive placeholder names such as {env.FOO}, while the
replaced values were never lower-cased and so could not match the
lower-cased request path. Leave such patterns untouched at provisioning
and lower-case the expanded pattern at request time.
Fixes#6387
* caddyhttp: restore the encoded separator path matcher tests
(cherry picked from commit 2147da70f7)
HTTPBasicAuth.correctPassword built its hash cache key with
append(account.password, plaintextPassword...). account.password is
shared by all requests and can have spare capacity, so short passwords
were written into the stored hash's backing array, causing a data race
between concurrent requests and potentially mixed-up cache keys.
Build the key in a fresh buffer instead, and add regression tests.
(cherry picked from commit f89626aa55)
* requestbody: reject oversized Content-Length before the next handler
When max_size is set and the request declares a Content-Length above it, respond 413 right away instead of letting the next handler start and fail mid-read. Bodies of unknown length are still limited by MaxBytesReader. Both paths return the same error, so handle_errors sees no difference.
* caddytest: send unknown-length bodies in max_size placeholder tests
request_body now answers 413 for a declared Content-Length over max_size before the placeholder reads the body, so these tests no longer covered the read path and the direct respond one failed.
* requestbody: keep the body limited for error routes
Error routes get the same request, so wrap the body before the early 413 to stop them from reading past max_size.
(cherry picked from commit 6cdffe9502)
{args[-1]} and {args.-1} parsed as -1 and indexed the args slice
directly, panicking with index out of range. Treat negative indices as
out of bounds, like indices past the end.
(cherry picked from commit 482f867395)
* admin: authorize remote requests matching any permission entry
enforceAccessControls rejected a request as soon as one permission entry
did not match its method or path, so only the first entry was usable.
Allow the request if any entry permits both, and deny only after all fail.
* admin: check every access entry that holds the client key
A key listed in several access control entries was only checked
against the first one; deny only after all of them are tried.
(cherry picked from commit c168dbc4ca)
With fewer than two weights, WeightedRoundRobinSelection returned the
first upstream without checking that it is available.
(cherry picked from commit aeea18be86)
Use the chosen file info, not a zero size, to tell whether a candidate
was chosen, so smallest_size keeps an empty file and largest_size can
choose one.
(cherry picked from commit cce035d7ad)
When a handle_response status code did not evaluate to an integer, the
handler returned a 500 without closing the upstream response body,
leaking the connection.
(cherry picked from commit dccce45660)
* caddyhttp: add HTTP/1 body reuse regression tests
* ci: test and release with Go 1.27
---------
Co-authored-by: steadytao <mail@steadytao.com>
(cherry picked from commit 053be179d8)
* httpcaddyfile: Don't merge automation policies if it would shadow another
Signed-off-by: Bartlomiej Wisniowski <56162793+bwisn@users.noreply.github.com>
* httpcaddyfile: add reverse case for shadowing merge, use force_automate to match the real life usage and explain why we're not checking the inverse
Signed-off-by: Bartlomiej Wisniowski <56162793+bwisn@users.noreply.github.com>
* httpcaddyfile: Check overlap in both directions when merging policies
Signed-off-by: Bartlomiej Wisniowski <56162793+bwisn@users.noreply.github.com>
---------
Signed-off-by: Bartlomiej Wisniowski <56162793+bwisn@users.noreply.github.com>
(cherry picked from commit eb04ed52be)
* caddyhttp: let handler read deadlines cap the idle read timeout
IdleTimeoutReader sets the read deadline before every body read, so a
deadline set by a handler with http.ResponseController.SetReadDeadline()
was replaced before it could take effect. Since #7913, handlers can't
bound a stalled request body more tightly than read_body.
A handler's read deadline is now routed to the IdleTimeoutReader, which
applies it right away and keeps it as a ceiling for every idle reset and
for the deadline put back between HTTP/2 reads. A handler can make the
idle timeout stricter but can't loosen it, and a zero deadline removes
the ceiling, as #8145 does for write deadlines.
* caddyhttp: keep the hard read deadline and the drain deadline with handler deadlines
SetReadDeadline installed the handler's deadline as is, so a later or zero
deadline lifted read_timeout. It now installs the earlier of the two.
HandlerDone cleared a deadline set by the handler before arming the HTTP/1
drain deadline, leaving the post-handler drain unbounded.
* caddyhttp: keep the idle deadline of a read in flight on handler deadlines
SetReadDeadline replaced the idle deadline armed for a blocked read with the
handler's ceiling, so a zero or later deadline lifted the idle timeout for
that read. It now installs the earlier of the two until the read returns.
* caddyhttp: keep the expired drain deadline of a timed-out body on handler deadlines
After a body read timed out on HTTP/1, a zero or later deadline set by the
handler replaced the expired one kept to bound net/http's post-handler drain.
(cherry picked from commit a28f061462)
* perf: bypass header key normalization for canonical literals
http.Header's Get, Set, Add and Del run textproto.CanonicalMIMEHeaderKey
on every call. When the key is a canonical literal the result is the key
itself, so the scan is pure overhead. Access the map directly instead.
Only hot, allocation-free paths are converted, and only where the key is
a canonical literal. Call sites taking a key from config, a template or
the request itself still go through the normalizing methods, since they
need it for correctness.
allHeaderValues and lastHeaderValue dropped their http.CanonicalHeaderKey
call for the same reason: all callers pass canonical literals.
benchstat, Apple M5 Pro, go1.27.1:
PrepareRequest-15 659.4n ± 2% -> 457.7n ± 2% -30.58% (p=0.000 n=12)
AcceptedEncodings-15 330.8n ± 5% -> 309.4n ± 7% -6.44% (p=0.019 n=10)
ResponseWriterWriteHeader/event-stream-15 1007.5n ± 3% -> 732.5n ± 1% -27.30%
ResponseWriterWriteHeader/informational-15 60.0n ± 3% -> 48.0n ± 2% -20.01%
ResponseWriterWriteHeader/not-modified-15 214.3n ± 6% -> 175.3n ± 4% -18.20%
ResponseWriterWriteHeader/connect-15 60.2n ± 3% -> 51.9n ± 4% -13.85%
ResponseWriterWriteHeader/html-15 206.5n ± 4% -> 197.9n ± 2% -4.19%
geomean -14.38%
Allocations are unchanged.
* tests: verify http.Header map keys are canonical
Indexing http.Header directly skips the canonicalization Get/Set/Add/Del
perform, which also puts the key outside what CI checks. canonicalheader
only inspects method calls. staticcheck's SA1008 covers reads, but stops
descending as soon as an assignment writes to a header, so it misses the
write, the delete builtin and the read-modify-write append. A key that is
not canonical in those positions is a silent lookup miss, not a compile
error.
Walk the loaded packages and verify every string constant used as a key in
an http.Header map operation matches http.CanonicalHeaderKey. A helper that
takes the key from its caller would hide the literal again, so a parameter
named with a canonical prefix opts its arguments into the same check.
The WebSocket headers Caddy deliberately writes with RFC 6455 casing are
allowed explicitly; they are only ever written out, never looked up against
a parsed header map.
Covers 64 operations, runs in under a second, and promotes golang.org/x/tools
from an indirect to a direct dependency at the version already in the module
graph.
* headercheck: resolve aliases of http.Header
* headercheck: check http.Header composite literal keys
* headercheck: note overlap with canonicalheader
* headercheck: drop parameter-name check
Helpers now take the indexed values, so every key sits in an index expression the analyzer and the linter already check.
* headercheck: scope RFC 6455 exceptions to their call sites
A key-based allow-list also accepted reads such as h["Sec-WebSocket-Key"], which miss because parsed headers are canonical. Honor //nolint:canonicalheader on the line instead, which the linter will use too.
* encode: read every Accept-Encoding field line
AcceptedEncodings read only the first field line, so a client that split
the list across several lines had the rest ignored. RFC 9110 section 5.3
treats repeated list-field lines as one comma-joined list. Iterate over
the items of every line instead of joining them, and skip empty items,
which used to yield an empty encoding. file_server's precompressed
selection uses the same function and is fixed with it.
Fixes https://github.com/caddyserver/caddy/issues/8272
Co-authored-by: Kağan Küçük <kaankck@gmail.com>
* encode: read every Cache-Control and Content-Encoding field line
Only the first field line was read, but RFC 9110 section 5.3 makes
multiple lines of a list-based field equivalent to one comma-joined
value:
- Cache-Control: no-transform on a later line was ignored, so the
request or response was compressed anyway.
- Content-Encoding: an empty first line followed by a real coding let
the handler encode the body a second time.
* encode: parse Cache-Control no-transform per RFC 9111
The substring match missed No-Transform, though RFC 9111 section 5.2 compares directive names case-insensitively, and fired on text that is not the directive: longer tokens such as x-no-transform-ext, quoted arguments such as community="no-transform", and semicolon-separated values, which are not valid Cache-Control syntax. Split the field on commas outside quoted strings and match only a well-formed no-transform element.
* encode: keep accepted encoding preferences on the stack
AcceptedEncodings counted commas to size the preference slice, and the variable capacity put it on the heap. A constant capacity covers typical Accept-Encoding headers and lets the compiler keep the slice on the stack; append still moves it to the heap for longer ones. BenchmarkAcceptedEncodings: 151.8ns -> 120.5ns (-21%), 192 -> 64 B/op, 2 -> 1 allocs/op (n=8).
* encode: build the encoded ETag without fmt
fmt.Sprintf boxed both arguments and parsed the format on every encoded response with a strong ETag. Concatenation: 47ns -> 16.5ns, 56 -> 24 B, 3 -> 1 allocs.
* reverseproxy: access response Server and Via headers directly
finalizeResponse runs on every proxied response; skip the key normalization as prepareRequest already does on the request side.
* encode: access headers directly in Write, WriteHeader and Close
These run once per response; convert the remaining literal-key Get, Set and Add calls like the rest of the handler.
* encode: match Cache-Control no-transform with httpguts
httpguts.HeaderValuesContainsToken already compares list elements case-insensitively across field lines, so the hand-written list parser is unnecessary. A no-transform inside a comma-separated quoted argument now also disables encoding, which only errs toward not transforming.
* encode: inline the no-transform check
isEncodeAllowed only wrapped one httpguts call. Its unit test now drives the response directive through ServeHTTP instead.
---------
Co-authored-by: Kağan Küçük <kaankck@gmail.com>
(cherry picked from commit 857d5731f3)
* fastcgi: keep the php_fastcgi directory redirect on the same origin
* fastcgi: build the directory redirect from the escaped original segment
(cherry picked from commit 32bba15459)
* caddytls: keep ACME issuer network proxy across re-provisioning
LoadModule clears NetworkProxyRaw, so when the HTTP app provisions an
ACME issuer a second time (fillInACMEIssuer), the rebuilt template has
no HTTPProxy. Since #7510 that template is used via SetConfig, and ACME
requests bypass the configured proxy.
Keep the loaded proxy func on the issuer, like the DNS solver.
Fixes#8283
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 2366825f2380383dacd8adeb1508e34c3e25634d)
* caddytls: test that ACME requests use the proxy after RebuildCertMagic
Replace the provision-twice check with a regression for the path the
HTTP app takes: Provision, re-provision in fillInACMEIssuer, then
RebuildCertMagic. The test issues against a local proxy and asserts the
ACME request goes through it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: k4lleanka <257361691+k4lleanka@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit fead9c3f24)
* caddytls: validate curve and protocol names, compare protocol IDs
* caddytls: Add final newline to tests and drop duplicate protocol comment
(cherry picked from commit 71c5812cfe)
* fastcgi: Stop response bodies at the declared Content-Length
* fastcgi: Use a cancellable test context in proxy tests
(cherry picked from commit b058e3f3f6)
* reverseproxy: strip hop-by-hop headers when copying response headers
copy_response_headers copied the upstream header map before
finalizeResponse could strip hop-by-hop fields. Clone and strip
first, so include and exclude cannot forward them, and leave the
upstream response unchanged for later handlers in the route.
Co-authored-by: Grok 4.7 <noreply@x.ai>
(cherry picked from commit 258863f6f6)
* replacer: cache file lookups and cap them
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
* replacer: cache repeated file placeholder lookups
Memoize file placeholder results per replacement call, using an 8 MiB LRU for contents. Preserve empty and failed-read results without changing the output for unknown placeholders.
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
---------
Signed-off-by: Mohammed Al Sahaf <msaa1990@gmail.com>
(cherry picked from commit 4e8de214bc)
* reverseproxy: close backend body and hijacked socket on upgrade setup failures
When an upstream response indicates an HTTP upgrade (101 Switching Protocols),
handleUpgradeResponse takes ownership of res.Body and subsequently hijacks
the client connection.
However, if any setup check fails prior to starting handleUpgradeTunnel
(such as mismatched upgrade headers, non-writable body assertion failure,
hijack unsupported, or handshake buffer flush errors), the function returned
early without closing res.Body or the hijacked client socket.
Because calling Hijack() permanently disables net/http's automatic connection
cleanup, and raw TCP sockets are not closed by the garbage collector, these
early failure exits leaked file descriptors into CLOSE_WAIT/ESTABLISHED.
Fix this by:
1. Adding a deferred cleanup guard that closes res.Body and any hijacked
front connection unless the tunnel successfully takes ownership.
2. Wrapping connections with onceCloseConn to ensure idempotent closure
while preserving optional CloseWrite (TCP half-close) support.
3. Adding TestUpgradeResponseCleanupOnFailure to verify cleanup.
* reverseproxy: test post-hijack cleanup and remove once-close wrappers
Cover handshake flush and buffered backend-write failures after successful hijacking, checking that both connections close. Keep setup-failure cleanup while using the original connections directly.
Code and tests were AI-assisted using an OpenAI coding assistant in Pi.
(cherry picked from commit 62bc606433)
* publish service status reported before the SCM handler attaches
* notify: serialize Windows service status updates
(cherry picked from commit 8e237f1955)
* caddyhttp: add CEL expression support to the tls matcher
MatchTLS was the only core HTTP matcher without a CELLibrary() implementation,
so it could not be used inside an expression matcher. Every sibling matcher
(host, path, path_regexp, method, query, header, header_regexp, protocol,
remote_ip, client_ip, url_pattern, vars, vars_regexp, and the file-server's
file matcher) already has this.
Adds CELLibrary() to MatchTLS, exposing tls_handshake_complete('true') /
tls_handshake_complete('false') in CEL expressions. Mirrors MatchProtocol's
existing string-argument-macro pattern: CELMatcherImpl with cel.StringType,
parsing the argument with strconv.ParseBool and constructing a
MatchTLS{HandshakeComplete: &b} that reuses the matcher's own
MatchWithError. Adds the corresponding CELLibraryProducer interface guard.
Adds 6 cases to celmatcher_test.go's existing table-driven
TestMatchExpressionMatch/BenchmarkMatchExpressionMatch: match on
'true'/'false' against a complete/incomplete handshake, and three
invocation-error cases (no args, wrong arg type, non-boolean string).
Required adding a tlsHandshakeComplete *bool field to the shared test table
and wiring it into req.TLS.HandshakeComplete in both the test and benchmark
request-construction helpers.
Fixes#8087
* address review: use a native bool arg, rename to match the matcher's own name
francislavoie's review on #8088:
- "A boolean as a string argument is very strange." CELMatcherImpl only
supported string/list(string)/map argument types, so the original
implementation parsed a string literal with strconv.ParseBool to work
around that. Instead, extends CELMatcherImpl itself with a cel.BoolType
case (celMatcherBoolMacroExpander + isCELBoolExpr, mirroring the existing
string equivalents), so a matcher can accept a real CEL bool literal.
This is shared infrastructure, so any other matcher needing a bool
argument in the future gets it for free.
- "I'm not sure why tls_handshake_complete was chosen when the matcher is
named tls." Renamed the CEL function to tls(...), matching every other
matcher's naming convention (the function name mirrors the matcher's own
directive/module name: host(), path(), method(), protocol(), etc.).
Usage is now 'expression tls(true)' / 'expression tls(false)' instead of
'expression tls_handshake_complete(\'true\')'.
Updated the 6 existing test cases accordingly; still covers match/no-match
on both true/false and the two invocation-error cases (no args, wrong arg
type -- now a string instead of a bool, confirming the new type check).
* address review: cover tls() against a plain non-TLS request
steadytao asked for a test proving tls(false) doesn't treat the absence
of TLS as an incomplete handshake. MatchWithError already handles this
correctly (r.TLS == nil returns false regardless of HandshakeComplete),
this just adds the missing coverage for both tls(true) and tls(false)
against a plain http:// request.
Confirmed these tests catch a real regression: removing the r.TLS == nil
guard causes a nil pointer dereference panic, not just a wrong result.
* address review: redesign tls() CEL matcher to preserve all three Caddyfile states
The Caddyfile tls matcher has three states: bare `tls` (any TLS connection,
regardless of handshake completion), `tls early_data` (handshake not yet
complete), and the implicit "handshake complete" case. The previous
tls(true)/tls(false) design could only express the latter two, with no way
to express "any TLS, don't care about handshake state" from CEL.
Redesigns the macro to:
tls() -- any TLS connection (mirrors bare `tls`)
tls('complete') -- handshake complete
tls('early_data') -- handshake not yet complete (mirrors `tls early_data`)
tls() is implemented as sugar for tls('any'): a zero-arg macro that expands
to the same 2-arg call the one-arg string macro produces, just with the
literal injected by the macro itself. Both macros share one function
registration and one matcher factory, so there's a single source of truth
for the string-to-MatchTLS mapping.
This replaces the native-bool design from the prior commit, so also removes
the generic bool-argument support (celMatcherBoolMacroExpander,
isCELBoolExpr, and the CELMatcherImpl case for cel.BoolType) added for it,
since nothing else in the codebase used it and it would otherwise ship as
unreviewed, unused surface area.
Rewrites the existing tls(...) test cases for the new string-based calls,
and adds coverage for the new tls() form: matches regardless of handshake
state (both complete and incomplete), and still correctly fails to match a
plain non-TLS request. Also covers the new invocation-error cases: a
boolean argument is no longer valid, and an unrecognized string (anything
other than 'complete'/'early_data') is now a Provision-time error instead
of silently doing the wrong thing.
Verified: full caddyhttp test suite passes, and confirmed each new/changed
test fails when its corresponding implementation piece is reverted
(the zero-arg macro, and the complete/early_data boolean mapping).
* address review: document and test tls('any') as a supported literal form
steadytao pointed out that the zero-arg tls() macro expands internally to
the same tls_request_string(req, "any") call that the one-arg form uses,
which means tls('any') was already silently reachable as a literal string
even though it was never documented or tested as part of the public API.
Rather than special-casing the macro to bypass the shared string factory
(which would need a second function/overload just for the zero-arg case),
document and test tls('any') as a real, supported third spelling alongside
tls() -- they're already fully equivalent, so there's nothing to hide.
(cherry picked from commit b1694e7a4c)
When a forward proxy is configured (network_proxy, forward_proxy_url or
HTTP_PROXY), net/http wraps dial errors in a *net.OpError with Op
"proxyconnect". tryAgain used a type assertion to detect DialError, so it
missed the wrapped error and treated it as a failure after the request
was sent. Non-GET requests were then not retried, even though no
connection to the proxy was ever established.
Use errors.As so a DialError is recognized anywhere in the error chain.
Errors that do not contain a DialError are unaffected, so requests that
may have reached the proxy or upstream are still not retried by default.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit f355b8687d)
