Cerebras’s Growth Shows Why AI Inference Is Splitting Into Two Hardware Markets

The immediate signal

The earnings matter because the workload is becoming real

Cerebras reported second-quarter revenue of about $180.1 million, up 74% from a year earlier, according to Reuters and The Wall Street Journal. The company also reports a separate “core revenue” measure; the Journal put that at $209.9 million for the quarter. Cerebras raised its full-year core-revenue outlook to $880 million–$890 million and its core gross-margin outlook to 41%–43%. Its remaining performance obligations were reported at $25.4 billion.

Those figures should be handled with some care. Core revenue is a company-defined non-GAAP measure, and the quarter still produced a large GAAP net loss. The stock fell sharply after the results despite the higher guidance. Investors are therefore not treating rapid demand growth as proof that the economics are already settled.

The operational mix is more revealing than the share-price reaction. Reuters reported that Cerebras’s core cloud-and-services revenue rose to $127.7 million, almost four times the year-earlier level, as the OpenAI deployment ramped. That is important because wafer-scale processors are no longer being judged only as unusual pieces of semiconductor engineering. They are being contracted as production inference capacity.

OpenAI announced in January that it would add 750 megawatts of Cerebras low-latency compute to its platform in multiple tranches through 2028. Cerebras said in its first-quarter financial release, filed with the SEC, that the multi-year agreement was valued at more than $20 billion. The exact utilization and economics of that capacity are not public, but the scale changes the question. The issue is no longer whether wafer-scale inference can work in a demonstration. It is whether its architectural advantages can survive procurement, deployment, utilization, power costs and real customer traffic.

That makes Cerebras a useful lens on a larger infrastructure change.

One AI answer contains two very different computing jobs

A large language model does not process a prompt and generate an answer in one uniform operation. In serving terminology, the request usually has two major phases: prefill and decode.

During prefill, the model processes the input tokens—the user’s prompt, retrieved documents, system instructions and conversation history—and creates the key-value cache, or KV cache, that records information needed for subsequent attention calculations. Most of that input can be processed in parallel. Long prompts therefore create a large block of mathematical work that tends to reward dense compute throughput.

Decode is different. The model generates the response token by token. Each new token depends on the state created by all the preceding tokens, so generation is inherently more sequential. The accelerator repeatedly needs access to model weights and the growing KV cache. For many practical serving regimes, especially at low batch sizes or strict latency targets, moving this data fast enough becomes more important than having unused arithmetic capacity.

NVIDIA’s own Dynamo documentation describes the distinction directly: prefill is compute-intensive, while decode has stronger memory and KV-capacity pressures. NVIDIA now supports separate prefill and decode worker pools so operators can give the two phases different parallelism, memory and scaling configurations.

This matters because the traditional approach asks one accelerator architecture to be good at both jobs. That is convenient. It simplifies scheduling, avoids transferring the KV cache between machines and lets operators move capacity between prompt processing and token generation. But convenience can impose a hardware compromise tax: the same expensive device alternates between phases whose bottlenecks are not identical.

The emerging response is disaggregated inference—run prefill and decode on separate workers, then transfer the KV cache between them. AIUpdateWatch has already examined how model routing turns model choice into a runtime decision. Disaggregated inference pushes a similar idea one layer lower: the system can choose different compute resources for different phases of the same model request.

Cerebras’s wager is fundamentally about where model memory lives

Cerebras attracts attention because its Wafer-Scale Engine is physically enormous, but its more interesting inference characteristic is memory placement.

The WSE-3 is a 46,225 mm² processor built from an entire silicon wafer. Cerebras specifies 4 trillion transistors, 900,000 AI-oriented cores and 44 GB of on-chip SRAM. The company reports 21 petabytes per second of aggregate memory bandwidth. These are vendor specifications, not independent application benchmarks, but they explain the architecture’s intended advantage.

Conventional AI accelerators rely heavily on high-bandwidth memory, or HBM, positioned next to the compute dies in an advanced package. HBM is dramatically faster than ordinary server memory, yet accessing it still means moving data off the compute die. Cerebras puts a much larger pool of SRAM directly on the wafer, close to hundreds of thousands of processing elements. For decode, where weights must be accessed repeatedly while tokens are generated sequentially, reducing the distance between memory and compute can translate into very high per-user token rates.

The 44 GB figure also reveals a limitation that marketing language can obscure. A 70-billion-parameter model stored at 16-bit precision needs far more than 44 GB for weights alone. Cerebras therefore does not place every large model on one wafer. Its own inference documentation says models larger than one wafer’s capacity are split at layer boundaries across multiple CS-3 systems; it gives 70B models as an example that can use as few as four systems.

Wafer scale therefore reduces one kind of data movement; it does not abolish distributed-system engineering. Larger models still need multiple systems, networking, scheduling and software capable of keeping the pipeline fed. A buyer comparing architectures should not reduce the decision to “SRAM good, HBM bad.” The relevant question is how the complete system behaves under the model size, batch size, context length, concurrency and latency target that actually matter.

Disaggregation removes one hardware compromise and creates another

The strongest evidence that this is a broader trend does not come from Cerebras claiming superiority over GPUs. It comes from several companies choosing to divide the serving path deliberately.

AWS and Cerebras announced an architecture in which AWS Trainium handles prefill and Cerebras CS-3 systems handle decode, connected through Amazon’s Elastic Fabric Adapter. Amazon’s explanation is unusually clear: prefill is highly parallel and computationally intensive, while decode is sequential and memory-bandwidth intensive. The partnership assigns each phase to the hardware the companies believe fits it better.

AMD and Cerebras announced a similar collaboration in July. AMD’s Helios rack-scale platform is intended to process prompts and large context windows, while Cerebras handles the decode/token-generation phase. The companies claim the combined design can improve tokens per second per watt by as much as 5×, but that figure is a vendor projection and should not be treated as independently established performance.

NVIDIA, meanwhile, does not need Cerebras to embrace the architectural split. Dynamo supports independent prefill and decode pools across GPU backends. Its documentation says disaggregation can improve resource allocation, scalability and latency isolation when the two phases create different pressures.

It also documents the cost of doing so. The KV cache has to move from the prefill engine to the decode engine. That transfer consumes bandwidth and introduces another potential latency path. Separate pools can fragment capacity: a prefill worker may be idle while decode is overloaded, or vice versa. At low concurrency, with short prompts or without a fast interconnect, NVIDIA says an aggregated deployment may be simpler and faster.

This is the engineering trade-off that disappears in headline comparisons. Disaggregation can remove the compromise of asking one processor to optimize two different phases, but it replaces that compromise with coordination overhead. The winning design depends on whether specialization saves more time, energy or hardware than KV movement and capacity fragmentation consume.

HBM scarcity makes memory architecture an economic variable

Memory bandwidth is not only a performance specification. It is now a supply-chain and capital-allocation problem.

Micron said in late 2025 that price and volume agreements had already been completed for its entire calendar-2026 HBM supply. In June 2026, the company argued that reasoning-heavy and agent-based inference was intensifying demand for memory capacity and bandwidth across the stack. Its HBM4 product is designed to deliver more than 2.8 TB/s from a 36 GB stack.

This does not mean HBM-based accelerators are running out of memory tomorrow. NVIDIA, AMD and their memory partners are expanding capacity, bandwidth and package designs aggressively. AIUpdateWatch’s earlier explainer on N3, CoWoS and HBM supply constraints shows why the complete accelerator pipeline matters more than any single component.

But tight, expensive HBM changes the relative economics of alternatives. Cerebras CEO Andrew Feldman told Reuters that keeping memory on the wafer reduces the company’s exposure to rising HBM costs. That is a management claim about Cerebras’s cost structure, and the company has not published enough detail to calculate the full cost advantage independently. Wafers, packaging, networking, power, cooling, yield management and idle capacity still cost money.

The broader point survives without accepting the strongest vendor claim. If inference performance increasingly depends on moving model state rather than performing more arithmetic, then memory architecture becomes part of competitive strategy. A chip vendor can no longer discuss compute throughput as though memory were a secondary attachment.

OpenAI validates demand, not universal hardware superiority

OpenAI’s 750 MW commitment is the most powerful commercial evidence in Cerebras’s favor. OpenAI says its compute strategy is to maintain a resilient portfolio and match different systems to different workloads. That wording is important. It does not say Cerebras will replace GPUs, Trainium or other accelerators throughout OpenAI’s infrastructure. It says Cerebras adds a dedicated low-latency option.

This is a much more plausible market structure than a single “winner takes all” accelerator story. Training remains enormously compute-intensive. Prefill rewards parallel throughput. Decode can reward memory bandwidth and latency. Image and video generation have different kernels again. Batch jobs can prioritize cost per token while interactive coding or agent loops may prioritize response time.

For Cerebras, the same OpenAI contract that validates demand also creates execution risk. A very large backlog is valuable only if Cerebras can install capacity, obtain power, manufacture systems, support the required models and convert contracted obligations into profitable revenue. The jump in cloud-and-services revenue shows that conversion has begun; it does not establish the mature margin profile of a 750 MW deployment.

An independent 2026 accelerator study reinforces the workload-fit argument. The preprint The xPU-athalon compared several accelerator families and found that the optimal platform changed with batch size, sequence length and model size. It also reported higher idle power for some specialized accelerators, including Cerebras, than for the NVIDIA and AMD GPU systems tested. That does not invalidate Cerebras’s latency advantage; it shows why utilization belongs in any serious economic comparison.

A fast processor that sits idle can be expensive. A slower processor that serves enormous batches efficiently can be economically superior for another workload.

The winning metric may be the cost of a useful low-latency answer

AI infrastructure is usually compared through familiar quantities: peak FLOPS, accelerator price, tokens per second or dollars per million API tokens. None is sufficient by itself for the workloads now driving inference growth.

Consider an agent that performs 30 sequential model calls while debugging software. If each call waits on token generation, shaving hundreds of milliseconds or several seconds from each step can reduce the total task time enough to change how the product feels. Faster generation may also let the agent perform more verification or tool use within the same user-visible latency budget.

Now consider an overnight document-classification job processing millions of records. The user does not care whether one stream generates at extraordinary speed. Throughput, utilization and total cost dominate. The architecture that wins the interactive agent may lose the batch workload.

That suggests a more useful economic metric: the cost of meeting a workload’s service-level objective. How many dollars, watts and machines are required to provide the required model quality at the required time-to-first-token, output rate, concurrency and reliability?

This is also why published API token prices cannot tell the whole infrastructure story. An API provider can hide a sophisticated mix of accelerators behind one model endpoint. OpenAI’s Cerebras partnership makes that abstraction explicit. Users may request the same model family while the provider decides which hardware path meets the latency and capacity requirement.

The likely outcome is not that wafer-scale chips “beat GPUs.” It is that inference infrastructure becomes more segmented. General-purpose GPU fleets remain valuable because they are flexible, mature and programmable across many workloads. Specialized decode engines become attractive where latency and memory movement dominate. Prefill engines can be sized around context and throughput. Routers and schedulers decide how requests move between them.

In other words, the hardware market may start to resemble the model market: less a single hierarchy and more a portfolio of systems chosen for a job.

What would establish that this is a durable infrastructure shift?

Cerebras’s quarter supplies commercial momentum. AWS, AMD and NVIDIA supply architectural evidence. The remaining question is whether disaggregated, heterogeneous serving produces better economics outside carefully selected demonstrations.

Four pieces of evidence would make the case stronger.

Production benchmarks with equivalent service levels. Comparisons should hold model quality, quantization, context length, concurrency, time-to-first-token and output latency constant before comparing cost or power.
Real utilization data. Specialized hardware must stay busy enough for its latency advantage to outweigh idle capacity and fixed infrastructure costs.
Measured KV-transfer overhead. The benefit of separating prefill and decode should include the network and synchronization cost of moving state between them.
Customer economics beyond one anchor contract. OpenAI proves that one extremely large buyer values Cerebras capacity. Broader adoption would show whether the same trade-off works across clouds, enterprises and smaller inference fleets.

Cerebras’s Q2 results are therefore best read as evidence of a market opening, not the end of a hardware contest. The company is benefiting from an inference workload that increasingly values memory bandwidth and low latency. Its partners are simultaneously demonstrating that those qualities do not have to live on the same processor that handles every other phase.

The deeper change is architectural: AI inference is becoming divisible. Once operators are willing to split a request into phases and assign each phase separately, the accelerator market stops being a contest to build one chip that is best at everything. It becomes a systems problem—matching compute, memory, networking and workload economics closely enough that specialization is worth the coordination cost.

Sources