🔍 Read the full analysis: AI In The Wrong Session: Cloudflare’s KV Cache Bug Explained on ThorstenMeyerAI.com
Prime made for students and young adults
- Fast, free delivery for dorm and study essentials
- Prime Video and Amazon Music included
- Member-only deals
TL;DR
Cloudflare’s October 26 incident report says its hosted GLM-5.3 model returned 155 responses containing text from other customers’ requests between August 27 and September 9, 2026. A mismatch in context-window settings and a bug in the SGLang inference engine combined to expose stale GPU memory; Cloudflare says it found no evidence of deliberate exploitation.
Cloudflare says its hosted GLM-5.3 AI service returned 155 responses containing text from other customers’ requests during a 14-day period, after a configuration mismatch and a bug in the open-source SGLang inference engine combined to expose data in GPU memory. The company’s incident report, published October 26, says the issue affected prompts longer than 681,280 tokens; Cloudflare took the model offline after confirming the incident and restored a fixed version that evening.
The issue affected Cloudflare’s Workers AI endpoint for GLM-5.3, a text-generation model developed by Z.ai. Security researcher Gadi Evron reported on September 8 through Cloudflare’s HackerOne program that very long prompts sometimes produced text from other users’ sessions. Cloudflare’s investigation counted 155 affected responses from August 27 through September 9, excluding its own tests and the researcher’s requests.
Cloudflare said the affected requests were consistent in timing and volume with ordinary long-context use, rather than repeated attempts to exploit the flaw. It said it found no evidence of deliberate exploitation. The company took the model offline 38 minutes after declaring an incident and restored a fixed version later that evening. The report does not explain how affected customers were notified.
The model’s advertised context window was 1,048,576 tokens, while the memory pool available to process prompts held 681,280. Cloudflare expected the inference engine to reject prompts that exceeded that pool. Instead, a bug in SGLang’s handling of rejected requests in Cloudflare’s split prefill-and-decode setup allowed processing to continue with incomplete cache data. Cloudflare says the output could contain text still held in memory from other requests.
Infographic 01 · Cross-request KV cache exposure
How a long prompt read someone else's data.
A memory pool smaller than the advertised context window, plus an engine bug that let rejected requests through, made one hosted model attend over other customers' leftover pages.
The mismatch
tokens; Cloudflare knew before launch and expected an error
Only prompts longer than 681,280 tokens could reach the faulty path.
The eight pages
Cloudflare’s own example of one over-long request
written by this requeststill holding earlier users’ prompts and replies
- Decode reserves 8 pages for the long prompt.
- Prefill rejects it and swaps in a one-token placeholder…
- …but the rejected request still passes the handoff checks.
- Prefill writes one page; decode reads all eight.
- The client gets HTTP 200 and a coherent reply, partly built on someone else’s data.
Why pages hold old data
speed trade-off
- GPU memory is a shared pool of fixed-size pages
- Pages are not zeroed between requests; the next one overwrites
- Bookkeeping is the only wall between customers
The fix
9 Sep, 19:40 UTC
- Pool resized so every valid prompt fits
- Engine retires rejected requests before any memory transfer
- Upstream: SGLang v0.5.20 to v0.5.22; operators should run v0.5.22+
Still open
not in the report
- Whether and how affected customers were notified
- How many customers’ requests contributed the leaked data
- What kind of data it was
Sources: Cloudflare incident report (26 Oct 2026); SGLang/LMSYS (26 Oct 2026). Figures and the no-exploitation finding are Cloudflare’s own.
Powered by Thorsten Meyer AI
The incident shows how a failure in inference infrastructure can create a privacy exposure even when the AI model itself is not at fault. Language models use a key-value cache, or KV cache, to retain intermediate calculations while processing a prompt. In Cloudflare’s setup, that cache was held in GPU memory and allocated in pages to requests.
Cloudflare says those pages are not zeroed between requests; the next request is expected to overwrite them. That practice can support performance, but it makes accurate allocation and request-state checks important for keeping one customer’s data separate from another’s. In this incident, an undersized pool and a software bug meant some pages reserved for a long prompt were not overwritten before the decode system read them.
For customers using hosted AI services, the report is a reminder that privacy depends on more than model behavior. Configuration, queue handling and memory management also affect whether prompts and generated text remain isolated. Cloudflare’s account identifies a specific endpoint and prompt-length condition, but the confirmed count of affected responses does not by itself establish how many distinct customers or people were involved.
Popular right now
Breville BES870XL Barista Express Espresso Machine, Brushed Stainless Steel
Ninja CREAMi 7-in-1 Ice Cream Maker, NC301
Shark Navigator Robot Vacuum with 30-Day Self-Emptying Base, White (AV2110S)
As an affiliate, we earn on qualifying purchases.
The Cache and Configuration Mismatch
When a model processes text, it converts tokens into intermediate values called keys and values. The KV cache stores those calculations so the system does not need to repeat them for every generated token. For large prompts, the cache can take substantial GPU memory, so inference engines commonly divide memory into fixed-size pages and allocate them as requests arrive.
Cloudflare said GLM-5.3 ran in a disaggregated setup: one group of GPUs handled prompt processing, or prefill, and another generated responses, or decode. Cloudflare advertised a context limit of 1,048,576 tokens, even though its prefill memory pool held 681,280. The company said it knew about the mismatch before launch and expected oversized requests to fail. Its long-context checks tested HTTP response codes and engine performance, but did not validate output content and coherence.
According to Cloudflare, SGLang marked an oversized prompt as rejected and replaced it with a one-token placeholder. In the split setup, the queues checked whether a request had finished, but not whether it had been rejected. Decode had reserved cache pages for the full prompt, while prefill wrote only the placeholder. The remaining pages could still contain earlier users’ prompts or partial generations. Cloudflare also said the underlying SGLang issue had been reported on July 6, 2026, as issue #30233; it noticed that report during its investigation.
“Pages are not currently zeroed between requests; the next request that gets a page simply overwrites it.”
— Cloudflare’s incident report
Customer Impact and Notification Remain Unclear
Cloudflare has reported 155 affected responses, but its published account does not state how many distinct customers received them, what kinds of text appeared, or whether the data included personal or otherwise sensitive information. It also does not describe how affected customers were notified. Those details are not established by the incident report.
The company said it found no evidence of deliberate exploitation, but that is a statement about the evidence it reviewed, not proof that no one attempted to trigger the behavior. The report also does not provide a full account of the corrective changes or say whether the SGLang issue was fixed upstream. The precise scope of any exposure beyond the identified responses remains unclear.
Follow-Up on Fixes and Customer Notices
Cloudflare says it restored a fixed version of GLM-5.3 on the evening of September 9 after taking the service offline. The incident report is the company’s public account of the cause, timeline and response. Further clarity would depend on Cloudflare providing details about customer notification, the number of distinct affected accounts and any additional safeguards or testing introduced after the incident.
The SGLang issue cited by Cloudflare may also receive further attention from the project’s maintainers. The available source material does not confirm a separate upstream fix or set out a timetable for additional disclosures. For now, the confirmed account remains limited to Cloudflare’s investigation and its reported 155 responses.
Key Questions
What happened in Cloudflare’s KV cache incident?
Cloudflare says its hosted GLM-5.3 service returned 155 responses containing text from other customers’ requests between August 27 and September 9, 2026. A configuration mismatch and an SGLang bug combined to let the decode system read stale cache pages.
Which customers or requests were affected?
Cloudflare said the affected responses followed prompts longer than 681,280 tokens. Its report does not state how many distinct customers were affected or provide details about the content in each response.
Was the GLM-5.3 model itself responsible?
The reported cause was in Cloudflare’s serving configuration and the SGLang inference engine, not a flaw identified in the GLM-5.3 model itself.
Did someone deliberately exploit the bug?
Cloudflare said it found no evidence of deliberate exploitation and assessed the requests as consistent with ordinary long-context use. That is the company’s finding based on its investigation.
What remains unknown about the incident?
The report does not say how many distinct customers were affected, what sensitive information may have appeared, or how customers were notified. It also does not confirm whether the cited SGLang issue received an upstream fix.
Source: ThorstenMeyerAI.com
