Cerebras Disaggregated Inference Delivers 5x Throughput

Cerebras has launched a production disaggregated inference system that separates prompt processing from token generation, achieving five times the throughput per megawatt compared to its previous CS-3-only configurations while maintaining generation speed. Officially released on November 18, 2026, the architecture routes prefill workloads to AMD Instinct GPUs and autoregressive decode to Cerebras wafer-scale systems, addressing power-constrained datacenter environments where agentic applications demand highly interactive inference.
Architecture Splits Prefill and Decode Stages
The disaggregated approach divides inference requests into two distinct stages. Prompt processing, known as prefill, runs on GPU accelerators that handle the compute-intensive work of ingesting input and producing intermediate model state. Response generation, or decode, then moves to Cerebras CS-3 systems optimized for the autoregressive token-by-token generation process. This separation leverages each accelerator type for its architectural strengths rather than forcing a single chip to handle both workloads.
Cerebras systems bring 350 times the SRAM capacity and 5,400 times the memory bandwidth of a typical GB300 GPU, keeping model weights and key-value caches on-chip across tens of gigabytes of SRAM. That on-chip memory advantage eliminates the repeated DRAM round-trips that bottleneck decode performance on traditional GPU architectures, enabling the CS-3, CS-4, and CS-5 to maintain ultrafast generation speeds even under heavy concurrent request loads.
Production Deployment Shows 5x Efficiency Gain
In a measured production deployment on the Cerebras Inference Cloud running a shared GPT-OSS-120B endpoint, the disaggregated configuration delivered five times the throughput per megawatt compared to the CS-3-only baseline. The company reports no loss in token generation speed, meaning users experience the same ultrafast response latency while the infrastructure serves five times more requests within the same power envelope. This measured result came from a November 14, 2026 snapshot of the production system.
The efficiency improvement addresses datacenter power constraints that have emerged as a primary bottleneck for inference scaling in 2026. Goldman Sachs research projects AI agent usage will drive substantial growth in inference demand, while datacenter capacity remains power-limited. Cerebras positions the disaggregated architecture as a path to serve significantly more interactive workloads, such as coding assistants and conversational agents, without proportional increases in megawatt draw.
Partner Deployment Options and SDK Access
Cerebras offers two integration paths for infrastructure partners adopting the disaggregated system. The end-to-end stack includes a prefill engine, KV Portal middleware for state transfer, and the Cerebras decode engine, optimized for partners prioritizing fast deployment timelines. The decode-only SDK provides lower-level control, allowing partners to integrate their own inference frameworks and prefill engines with the Cerebras decode engine through a provided mock replica and reference implementation.
Early-access partners already deploying the solution include Amazon Web Services, Fireworks AI, Gimlet Labs, and General Compute. Cerebras plans to expand disaggregated inference deployments across hundreds of megawatts in both its own cloud and partner datacenters by the end of 2027, alongside its recently announced CS-4 system and scaled manufacturing capacity targeting 700 megawatts of datacenter footprint.
What This Means
The launch signals a shift in inference architecture philosophy, treating prefill and decode as separable workloads rather than bundling them on homogeneous accelerator clusters. For inference providers, the 5x throughput-per-megawatt gain translates directly to revenue potential in power-constrained environments. The disaggregated model may become a template for other chip vendors seeking to optimize datacenter efficiency as agentic workloads scale through 2027 and beyond, particularly in applications where low-latency interactive generation drives user engagement and commercial viability.
on Emergent today





