AMD introduced Helion on October 12, a rack-scale blueprint for agentic AI. Not a new GPU but a rack drawing, which sounds dull until you read what changed:

Helion = open rack blueprint for agentic AI (OCP-based)
Two shapes: CPU-only (hyperscale) / CPU+GPU (enterprise)
Key part: token router (sends model calls to the right model)
Status: partner builds in progress, targeting Nov 2026

Why 32,768 CPU cores headline an AI launch

The CPU-only rack:

CPU: 6th Gen EPYC (Zen 6 Venice, up to 256 cores each)
Per rack: up to 128 CPUs → 32,768 cores
Role: agent execution only, inference on separate infrastructure

Strange until you list an agent's workday:

An agent's day:
  model inference (GPU) — thinking
  tool execution (CPU) — builds, file reads, tests, search
  job management (CPU) — sandboxes, sessions, network, DB

When working exceeds thinking, CPUs bottleneck before GPUs. Personal-agent systems already slice per-user sandboxes with CPU shares backed by inference servers. Hyperscalers adding dedicated CPU capacity for agents follows the same logic.

The mixed rack is the enterprise answer:

EPYC CPU blades + up to 16 Instinct MI350P GPUs (74 PFLOPS FP4 combined)
+ general-purpose compute (DB, storage, technical computing)
Networking: up to 102 Tbps per rack
Cooling: liquid and air, 21-inch ORv3 44OU rack

Thinking, working, and housekeeping in one rack, designed to sit on existing IT investments.

The token router may be the real story

The quietest important part routes each agent model call between frontier APIs and in-rack open models:

Easy call → in-rack open model (cheaper, faster, data stays)
Hard call → frontier model outside (pricier, more capable)

Every call is a cost, latency, and governance decision, so the router is a handle on a leaking faucet. As agents multiply, token volume outgrows user counts and the handle matters more.

Still, details are missing: routing policy, latency overhead, fit with existing AI gateways. Hyperscalers prefer their own control planes. November partner systems should show the substance.

"Open" is spelled out:

ORv3, DC-MHS, DC-SCM, HPRv2, OCP NIC 3.0, Caliptra
→ AMD to contribute the spec to OCP; OEMs and ODMs sell systems

Named partners include Cisco, Dell, HPE, Lenovo, and Supermicro. If Helios was the training and inference rack, Helion is the first rack-scale attempt to treat agent execution as its own tier.

CodeBridge mini lab: split your agent bill

Do not watch GPUs alone; split CPU alongside:

1. Pick 10 agent jobs
2. Per job, record separately:

   [ ] GPU time and token cost (inference)
   [ ] CPU time and cores (tools, sandboxes)
   [ ] network and storage (sessions, DB, artifacts)
   [ ] total cost and success
3. Compute CPU:GPU cost per success

Tool-heavy agents skew more CPU than expected. Three levers follow:

- route easy calls to smaller or local models
- tune caching and retry policy (fewer calls)
- decide whether to split execution from inference (scale strategy)

The first two need no router. Our model routing guide and cost-per-success guide are exactly that practice.

Conclusion: judge racks by work, not by models

Are our agents spending more on working than on thinking?

Rack evaluation used to mean GPU counts; now CPU, networking, and routing join in. Real CPU:GPU ratios should emerge from November builds. Teams that split per-task cost now will move first.

Further reading

References

Go deeper with a course

Designing retrieval quality, token cost, and latency together connects directly to this article's bill-splitting. Understand the structure before buying infra.