AMD Helios stopped being a slide on Thursday. Lisa Su put the rack on stage at Moscone for Advancing AI 2026 and confirmed it is in production with customers including Microsoft, shipping later this year. That alone is a normal hardware story. What made me sit up was a line from Andrew Dieckman, the corporate VP running AMD’s data center GPU group, delivered in a pre-briefing rather than from the keynote stage: “I have almost zero conversations with our customers about CUDA at this point in time. It’s a non-event.”

He went further and said CUDA is “not really the moat that some people still think it is,” because customers are programming at higher levels of abstraction against different serving frameworks, and because AI coding agents have become good enough at porting and optimizing for the AMD platform that the switching cost collapsed. He dated that last effect specifically: a 2026 phenomenon.

I have to own something here. Back in June I wrote that NVIDIA’s GPU monopoly keeps getting stronger, and that CUDA was the reason, not the silicon and not the bandwidth. Dieckman is arguing the exact opposite, from the company with the largest financial interest in that argument being true, at an event built to sell the alternative. Discount it accordingly. But the abstraction point is real, and I underweighted it. Almost nobody deploying inference in 2026 is writing kernels. They are running vLLM or SGLang, and the layer that used to lock you in is now three layers below where the work happens.

The hardware backing the claim is substantial. A Helios rack pairs 72 Instinct MI455X GPUs with 18 sixth-gen EPYC “Venice” CPUs, Pensando networking, and the ROCm stack. Each MI455X carries up to 432 GB of HBM4 at 19.6 TB/s, which works out to roughly 31 TB of HBM4 across the rack. AMD quotes 2.9 exaFLOPS of FP4 and 1.4 exaFLOPS of FP8, 260 TB/s of UALink scale-up bandwidth, and up to 43 TB/s of Ethernet scale-out. The whole thing is a double-wide OCP Open Rack Wide design with backside quick-disconnect liquid cooling, which is a boring detail that datacenter operations people will care about more than the flops. AMD’s headline efficiency claim is 30 percent more tokens per dollar than the leading competing solution. Independent rack-level benchmarks do not exist yet, so treat that number as a vendor claim until someone outside AMD runs the workload.

The memory capacity is the part I keep circling back to, because it is consistent with everything else AMD has done in the last year. Buying MEXT was a bet that the binding constraint on inference is memory, not raw compute, and 432 GB per accelerator is that same bet expressed in HBM4. If you believe FLOPs stopped being the differentiator, a rack built around capacity and bandwidth is the coherent answer, and CUDA lock-in matters a lot less than whether the model fits.

Which brings me to the announcement I think actually matters, and it is not the rack. AMD is partnering with Cerebras on disaggregated inference. The reasoning is that prefill and decode are different computational jobs that have been sharing hardware out of convenience rather than fit: chewing through a long prompt is a throughput problem, and generating the response token by token is a latency problem. Helios handles the first, Cerebras’ wafer-scale part handles the second, and the workload splits across both. Helios racks go into Cerebras data centers later this year.

That is a strange thing for a chip company to do. AMD is conceding, in public and in a signed partnership, that its own flagship is the wrong shape for half of inference. I have written before about Cerebras and the wafer-scale approach, and about Jim Keller’s argument that boring DRAM beats exotic memory at the system level, and the thread running through all of it is the same one I traced when chiplets started pulling the AI hardware moat apart. Monolithic advantages keep decomposing into interfaces, and once something is an interface, somebody else can implement it.

I am leaving the Anthropic side of this alone. The multi-gigawatt commitment and the equity stake attached to it are a different kind of story, about how AI compute gets financed rather than how it gets built, and it deserves more room than a paragraph here.

Dieckman’s quote will get passed around as the headline, and it is the weakest thing AMD said all day. A vendor telling you its competitor’s advantage has evaporated is the least interesting sentence at any launch event. The Cerebras deal is the one carrying weight, because you do not split your flagship workload with a startup’s wafer unless you have concluded that no single chip wins inference outright. If AMD is right about that, the thing NVIDIA has to defend is not CUDA. It is the assumption that one architecture should be doing both jobs at all.

Sources