The headline claim is straightforward: running the GLM-5.2 model on AMD's MI355X accelerators, wafer.ai measured 2,626 tokens per second per node, and pegged the cost at more than half of what a comparable Nvidia Blackwell deployment would run. For anyone budgeting inference at scale, that cost-per-token gap is the number that matters.
Why this is worth your attention: the practical bottleneck in serving large models today is rarely raw peak FLOPS — it's throughput per dollar and per watt across a full node, plus whether the software stack actually delivers those numbers in production. AMD has historically lagged not on silicon but on the maturity of its runtime and kernels. A credible third-party throughput figure on a current model suggests that gap is closing, and it gives buyers leverage in a market where Nvidia supply and pricing have dominated.
A few caveats before you reallocate any budget. Vendor-adjacent benchmarks tend to be tuned for the best case: batch sizes, sequence lengths, quantization, and tensor-parallel configuration all swing throughput dramatically, and "per node" comparisons hinge on how each node is specced. The reported cost advantage also depends on assumed hardware pricing and utilization, which vary a lot by contract and region. Treat 2x as a directional signal, not a guaranteed line item.
What you can do with it: if you're running inference workloads and locked into a single vendor, this is a reason to run your own head-to-head. Test with your actual model, your real prompt and output-length distribution, and your target latency SLA — then compute tokens per second per dollar, not just per node. Verify software support for your serving framework (vLLM, SGLang, or your in-house stack) on ROCm before committing, since kernel coverage and quantization support are where AMD deployments most often stumble.
The broader takeaway is competitive pressure. Every credible AMD inference result narrows Nvidia's pricing power and expands your negotiating options. Even if you never buy an MI355X, benchmarks like this are useful ammunition when you're negotiating capacity or evaluating cloud instances.