When a prompt is longer than the KV pool, the server should answer with a 400. Cloudflare's GLM-5.3 answered 200, and the response was computed from cache pages that still held another user's data.

Cloudflare published its incident report on October 26. The exposure ran from August 27 to September 9, and 155 responses were affected, all to prompts longer than 681,280 tokens. Rather than retell the report, this post asks what you should verify if you run the same architecture.

Two numbers that disagreed

Two problems stacked.

Item Value Source
Deployed context window 1,048,576 tokens Cloudflare report
Prefill KV pool 681,280 tokens Cloudflare report
Prefill KV pool after the fix about 2.04M tokens Cloudflare report

The first is configuration. The model advertised a window of roughly 1.05M tokens while the pool was smaller. Cloudflare says it knew about the gap before launch. Its tests returned success codes with no error frames, so it assumed SGLang handled the difference gracefully. A success code had been read as proof of a clean rejection.

The second is SGLang's rejection path. According to the report, when a prompt exceeded the pool, Req.set_finish_with_abort recorded an error and replaced the prompt with a one-token placeholder. In disaggregated mode, though, the admission queues checked finished() rather than rejection status, so a rejected request still looked live. Prefill admitted it and transferred KV to decode. Decode reserved pages for the full prompt length, but only the first page had been written by the current request. The remaining pages still held earlier users' data, decode attended over them, and the client got a 200. A user had already filed the same bug as SGLang issue #30233 on July 6, per the report.

The pattern in a toy model

The code below is not SGLang. It is a deliberately small model of the failure pattern, which I ran myself. It does not reproduce the real engine. The pattern is: a rejected request still gets pages allocated, writes only some of them, and reads all of them.

PAGE = 4  # tokens per page (toy value)

class Pool:
    def __init__(self, pages):
        self.free = list(range(pages))
        self.mem = {p: None for p in self.free}   # page -> last writer
    def alloc(self, n):
        got, self.free = self.free[:n], self.free[n:]
        return got
    def release(self, pages):
        self.free += pages

def serve(pool, user, prompt_tokens, pool_tokens, check_rejected_at_admission):
    need = -(-prompt_tokens // PAGE)
    rejected = prompt_tokens > pool_tokens
    if rejected and check_rejected_at_admission:
        return "HTTP 400", []
    pages = pool.alloc(need)
    written = pages[:1] if rejected else pages      # rejected request writes page 0 only
    for p in written:
        pool.mem[p] = user
    seen = {pool.mem[p] for p in pages}
    pool.release(pages)
    return "HTTP 200", sorted(s for s in seen if s and s != user)

Alice is served a 24-token request. Then a second user sends 40 tokens against a 32-token pool limit, using the same pool.

reject-at-admission=False: HTTP 200, other users' data visible to mallory: ['alice']
reject-at-admission=True: HTTP 400, other users' data visible to mallory: []

Rejecting at admission ends the request before any page is touched. Cloudflare's patch goes the same way: a rejected request returns its original error before bootstrap, allocation, or KV transfer. What the toy model cannot tell you is just as clear. It says nothing about what actually leaked, the Mooncake transfer path, or the race conditions around page release.

Checks for SGLang operators

These follow Cloudflare's recommendations, reordered as a checklist.

  1. Read the KV pool size from startup logs and compare it with the context length. Do it on every prefill and decode node, and again after any change to weights, quantization, parallelism, or memory settings. The preflight in the toy script is one line; with this incident's values it prints FAIL pool 681280 < context 1048576.
  2. Upgrade to v0.5.22 or later. Per the report's table, #38935 in v0.5.21 rejects over-length requests at admission and #36988 in v0.5.20 retires aborted prefill requests. Cloudflare recommends v0.5.22+. A follow-up, #39729, was merged but not yet released.
  3. Run deferred KV release on both roles and bound Mooncake transfer batches.
  4. Turn on corruption detection and confirm from logs that it actually runs. The flags are --disaggregation-enable-kv-checksum and --kv-canary, where the model architecture supports them.
  5. Stop the load balancer from aborting before the first token. Mid-transfer aborts open exactly the kind of path this bug used.

Cloudflare itself raised the pool to about twice the context length and added pre-deployment checks for pool size versus context length, deferred release, transfer bounds, and required patches.

What the incident does and does not tell you

What the report states: only GLM-5.3 was affected. After excluding Cloudflare's own testing and the researcher's, the affected requests matched the timing and volume of ordinary long-context use, and the company found no evidence of deliberate exploitation. The model went private at 14:00 UTC on September 9 and returned at 19:40 UTC.

What to keep in mind: 155 is Cloudflare's own finding. The report does not detail what the responses contained or whose data it was, and I found no independent verification of the count.

The more transferable lesson: from the outside this looked like success. HTTP 200 does not mean healthy. Without tests that probe boundary conditions on purpose, at the context limit and just past it, a defect like this never reaches an error log.

If you only consume a managed inference service, there is little to act on. But if you send sensitive documents at very long context lengths, ask the provider how they isolate cache between requests and how they test limit conditions.

Takeaway

This was a serving-configuration and queue-handling problem, not a model or prompt problem. The safeguard was making sure a "rejection" really takes the rejection path. Comparing pool size with context length is a one-line check in a deployment pipeline, and admission-time rejection comes with the engine upgrade.

Related reading

Sources