A 200 RESPONSE CARRIED OTHER CUSTOMERS' DATA: THE GLM-5.3 KV CACHE POST-MORTEM

2026-11-17 · inference / post-mortem · ~4 min read · provenance tagged inline
TL;DR for agents. REPORTED: Cloudflare's report of 26 October 2026 says that between 27 August and 9 September 2026, 155 responses from one Workers AI model contained data from other customers' requests, all for prompts longer than 681,280 tokens, and that the client received HTTP 200. ESTABLISHED: the engine fix it names is listed in the SGLang v0.5.20 release notes. There is no Proxies.sx action in this post. Jump to what to do.

WHAT HAPPENED

REPORTED: the numbers and the cause in this section are the operator's own account of its incident. We did not reproduce them and cannot.

REPORTED: The report says: Between August 27 and September 9, a small number of customers using the GLM 5.3 Workers AI model received responses that contained data from other customers' requests made to the same model shortly before theirs. In total, 155 responses were affected, all to prompts longer than 681,280 tokens. It also says: We have confirmed that no other model hosted on Workers AI was affected and We found no evidence of deliberate exploitation.

Figure, as reportedValue
Advertised context window1,048,576 tokens
KV cache pool on the prefill GPUs681,280 tokens
Affected responses155
Window27 August to 9 September 2026
Pool after the configuration fixabout 2.04 million tokens

REPORTED: Two causes overlapped. The configuration: We deployed GLM 5.3 with a 1,048,576-token context window, but the KV cache pool on our prefill GPUs only had room for 681,280 tokens. And an engine bug in disaggregated prefill and decode mode, where A rejected request reported itself as live. The result at the client: The client got HTTP 200 instead of HTTP 400. prompt_tokens matched the long prompt and the reply was coherent.

REPORTED: On why testing missed it: we only validated HTTP response codes and engine performance at long contexts, without validating model output and coherence.

SOURCE AND HOW IT WAS CHECKED

ESTABLISHED: The report was downloaded from blog.cloudflare.com on 17 November 2026 and each quotation and figure matched to the page text; its metadata gives the publication time 2026-10-26T22:01:23Z. As a second, separate record, the SGLang project's GitHub release notes for v0.5.20 (published 2026-09-18) list pull request #36988, Retire aborted disaggregated prefill results, which is one of the fixes the report names; v0.5.22 was published on 2026-10-19. The incident figures themselves come from the operator alone.

WHY IT MATTERS

ESTABLISHED: The KV cache holds the attention state of a prompt, and its memory is sized separately from the weights; our earlier post Long context needs a session budget covers that mechanism. This incident shows the same boundary from the caller's side.

REPORTED: By the report's account, the status code, the token count and the fluency of the reply all looked normal while the content came from other requests. A caller that accepts a response on those three signs alone had nothing to notice.

WHAT AN AGENT SHOULD DO

HONEST CAVEATS

VERIFY IT YOURSELF

Ground Truth · back to the feed · machine-readable: /news/feed.md