The Compute-to-I/O Area Ratio

If you've ever looked at a die shot of a modern AI accelerator or networking ASIC and felt a little betrayed, you're not alone. The marketing slide says "petaflops." The die photo says otherwise: a modest, tightly-packed block of MAC (multiply-accumulate) arrays sitting in the middle of a much larger sea of DDR PHYs, SerDes lanes, and analog I/O. A chip with, say, 33 mm² dedicated to its actual compute — the MAC array that does the multiply-and-add work behind every matrix multiplication — can easily spend two, three, even four times that area on the plumbing that gets data in and out.

This isn't a design failure. It's one of the most important and least appreciated facts in modern chip architecture: compute scales with Moore's Law. I/O does not. Understanding why explains almost everything about how AI chips, networking ASICs, and CPUs are floorplanned today — and why the "boring" parts of a chip are often the ones that actually decide its performance.

1. The Three Tenants on the Die

Before getting into the tradeoffs, it's worth being precise about what each of these blocks actually does, because they are solving completely different physical problems.

MAC arrays — the compute

A MAC (multiply-accumulate) unit computes a = a + (b × c) — the fundamental operation behind convolution, matrix multiplication, and essentially every neural network workload. MAC arrays are dense, regular, digital logic: rows and columns of identical cells that shrink almost linearly with each new process node. A MAC array built at 5nm is dramatically smaller — and faster, and more power-efficient — than the same array at 16nm. This is the part of the chip that benefits the most from Moore's Law, and it's the part semiconductor roadmaps are usually bragging about (TOPS, FLOPS, TOPS/W, TOPS/mm²).

DDR PHYs — the memory doorway

A DDR PHY (physical layer) is the analog/mixed-signal circuitry that translates the digital world inside the chip into the electrical signaling needed to talk to off-chip DRAM (DDR4, DDR5, LPDDR5X, etc.) over a physical bus. It has to manage:

• Precise timing and clock/data alignment (DLLs, PLLs)
• Signal integrity across a noisy, capacitive off-chip bus
• Voltage reference and impedance calibration
• ESD-protected, high-drive-strength I/O pads

None of this is digital logic that "just scales." A DDR PHY still has to drive real electrical signals down real copper traces on a PCB, at voltages and currents dictated by JEDEC standards — not by what's convenient for a 3nm transistor.

32 Gb/s SerDes — the chip-to-chip highway

SerDes (Serializer/Deserializer) blocks convert wide, slow parallel data into narrow, extremely fast serial streams (and back again) so chips can talk to each other — or to switches, NICs, or other accelerators — over a handful of high-speed differential pairs instead of hundreds of parallel wires. A 32 Gb/s SerDes lane is used in things like PCIe Gen5, chip-to-chip die interconnects, and Ethernet-class links. Each lane needs its own clock and data recovery (CDR) circuit, equalizer, transmitter driver, and receiver front end — all analog-heavy, all sensitive to layout, noise, and die-edge placement.

2. Why the "Boring" Blocks Take More Area Than the "Exciting" One

This is the core insight, and it comes down to a mismatch in how digital logic and analog/mixed-signal circuits respond to process scaling.

Digital logic scales beautifully. Every new process node (28nm → 16nm → 7nm → 5nm → 3nm) shrinks transistor gate pitch and interconnect pitch, so a MAC array — pure digital logic — gets smaller, faster, and lower-power almost automatically. This is why compute density (TOPS/mm²) has improved so dramatically generation over generation.

Analog and I/O circuits scale terribly. A DDR PHY or SerDes transceiver has hard, physics-based constraints that don't care what process node you're on:

• Electrical signaling requirements are fixed by external standards. JEDEC sets DDR voltage swings and timing budgets; PCIe/Ethernet standards set SerDes signaling requirements. You can't just "shrink" a differential driver below what's needed to reliably swing a signal across a PCB trace or backplane.
• Die-edge (I/O pad) constraints don't shrink with the node. SerDes and DDR PHYs are physically tied to the die perimeter, and the achievable bandwidth density per millimeter of die edge is a much harder wall to push against than transistor density.
• Passive components (capacitors, inductors, termination resistors) barely shrink at all across nodes, unlike active transistors.
• Noise, jitter, and power integrity margins actually get harder to hit at smaller nodes because supply voltages drop and transistor variability increases — so PHY designers often can't be as aggressive as logic designers.

The result: every process node shift makes your MAC array smaller and your PHY/SerDes area shrink far more slowly, if at all. Over enough generations, the "compute" fraction of the die keeps shrinking as a percentage, while I/O becomes a larger and larger share — unless architects actively fight this with different strategies (see Section 4).

3. Reading the 33 mm² Example

Consider a chip where the MAC array occupies roughly 33 mm² of the total die area, with the remainder allocated primarily to DDR PHYs and multiple lanes of 32 Gb/s SerDes. A few things this kind of split typically tells you:

• This chip is bandwidth-hungry, not just compute-hungry. If the design team is willing to spend more silicon on interfaces than on the MAC array itself, it's because the workload (large language models, recommendation systems, graph analytics, whatever it is) needs to move huge volumes of weights and activations in and out of the chip, or needs to talk to sibling chips/accelerators at high bandwidth. Compute sitting idle waiting on data is far more common — and far more wasteful — than people assume.
• This is a "memory-bound" or "bandwidth-bound" design point, not a pure FLOPS-maximization design point. Chips optimized purely for peak TOPS/mm² (with minimal I/O) look very different — and are often useless in practice because they can't be fed fast enough.
• The number of DDR channels and SerDes lanes, more than raw area, is what determines usable bandwidth. A wider (but not necessarily larger) I/O area, spread efficiently around the die edge, often matters more than the area itself — because bandwidth is fundamentally limited by die-edge length and pin/lane count, not core area.
• This ratio is a deliberate architectural choice, not a byproduct. Every mm² spent on PHYs and SerDes is a mm² not spent on more MACs, more SRAM, or more cache. Floorplanning at this stage is genuinely a zero-sum budget negotiation between the compute architects and the I/O/systems architects.

4. How Architects Fight Back Against the I/O Tax

Because this imbalance is so costly, the industry has developed several strategies to reclaim area (or at least stop losing more of it) for compute:

a) Move to HBM instead of DDR

High Bandwidth Memory (HBM) uses a wide, short, low-speed-per-pin interface stacked directly on or next to the compute die via silicon interposer, rather than a narrow, fast, long-reach DDR bus. HBM PHYs are physically large too, but they deliver dramatically more bandwidth per mm² and per watt than DDR, which is why virtually every modern high-end AI accelerator (GPUs, TPUs, and custom ASICs alike) has largely moved to HBM for its primary memory, reserving DDR or LPDDR for host-side or lower-bandwidth uses.

b) Chiplets and disaggregation

Rather than cramming compute, DDR PHYs, and SerDes onto a single monolithic die, many modern designs split them onto separate chiplets: a compute die on the leading-edge process node (where logic scaling pays off most), and separate I/O chiplets on an older, cheaper, analog-friendly process node (where PHYs and SerDes don't get worse by using an older node, and are much cheaper to fabricate). This is exactly the "core die surrounded by I/O dies" pattern used in modern switch ASICs and multi-chip AI accelerators, connected via ultra-short-reach die-to-die SerDes.

c) Maximize die-edge bandwidth density

Since I/O is fundamentally constrained by the die's perimeter (not its area), a major axis of competition among PHY/SerDes IP vendors is bandwidth density per millimeter of die edge — how many Gb/s you can push through each linear millimeter of edge real estate. Better bandwidth density lets architects reclaim core area for compute without sacrificing bandwidth.

d) UCIe and standardized die-to-die interconnect

The rise of UCIe (Universal Chiplet Interconnect Express) is a direct response to this problem — a standardized, high-density, low-power die-to-die interface meant to make chiplet disaggregation (compute die + I/O die + memory die) practical across vendors, so no single die has to carry the full burden of both compute and long-reach analog I/O.

5. The Bigger Picture

The tension between 33 mm² of MACs and everything else isn't a niche detail — it's arguably the central design tension of the AI-accelerator era. Nvidia, AMD, Google, Amazon, Tesla, Groq, Cerebras, and every other serious AI silicon player is fighting the same battle: compute logic keeps getting cheaper and denser, but the pipes needed to feed that compute (DRAM bandwidth, chip-to-chip bandwidth) are improving far more slowly. This is often summarized as the "memory wall" or, more broadly, the fact that we've entered a bandwidth-bound era of computing rather than a compute-bound one.

That's also why so much of the public narrative about AI chips ("how many TFLOPS?") misses what actually differentiates real-world performance. Two chips can have identical MAC arrays and wildly different real-world throughput, purely because one of them was given enough DDR channels, HBM stacks, or SerDes lanes to actually keep that compute fed — and the other wasn't.

A Few Things Worth Adding to This Picture

A couple of details are easy to miss when just looking at a floorplan or die shot:

1. SRAM often gets left out of this comparison entirely, but on many modern accelerators, on-chip SRAM (used for weight caching, activation buffers, and scratchpad memory) competes just as fiercely for area as the I/O blocks do — sometimes exceeding the size of the MAC array itself, precisely because it reduces how often you need to hit DDR/HBM in the first place.
2. Power, not just area, is a second currency being spent here. SerDes and DDR PHYs are often disproportionately power-hungry per mm² compared to digital logic, especially at 32 Gb/s+ signaling rates, so the "area tax" comes with a matching "power tax" that further constrains what a chip can do.
3. Yield economics favor smaller, denser compute dies. Since defect density scales with area, keeping the leading-edge compute die as small as possible (and pushing analog I/O onto separate, often larger-node chiplets) also improves manufacturing yield and reduces cost — a second, independent reason chiplet disaggregation has become so popular.
4. The "rest of the die" is rarely just DDR + SerDes. In many real designs, this area budget also has to be shared with power delivery network (PDN) overhead, clock distribution, PLLs, thermal sensors, and test/debug logic — so the DDR-PHY-vs-SerDes split is often a simplification of an even more crowded floorplan.