A systolic array is just a grid of multiply-accumulate cells passing data to their neighbors — but which operand each cell holds fixed, and which ones stream through, is a real design decision with a name, a tradeoff, and a chip built around each choice. This is the taxonomy nobody explains when they show you the grid diagram.
A systolic array is a grid of processing elements (PEs), each doing one multiply-accumulate per cycle, wired so that operands flow from PE to PE like a heartbeat pumping blood (hence the name — coined by H.T. Kung in 1978, from the Greek systole, the heart's contraction). Instead of a single MAC unit fetching operands from memory every cycle, a systolic array loads data once at the edges and lets it ripple through hundreds or thousands of PEs, reusing it along the way. If you want the full mechanics — cycle-by-cycle dataflow, wavefronts, a 4×4 worked example, even an interactive lab you can step through — that's covered in depth elsewhere on this site. This page picks up where those leave off: once you have the grid, which operand do you actually choose to hold fixed inside each cell?
Every systolic array computes a matrix multiply, which means three operands are in play at every PE: a weight, an input activation, and a running partial sum. Memory access is the single biggest energy cost in any accelerator — an off-chip DRAM fetch can cost 100× to 1000× more energy than the MAC operation itself. So the entire point of a systolic array is to minimize how often each operand has to be re-fetched, by picking one of the three to park inside a PE and reuse across many cycles, while the other two stream past it. That single design choice is called the dataflow, and it is arguably a bigger factor in a chip's energy efficiency than its raw MAC count.
| Property | Weight Stationary | Output Stationary | Input Stationary |
|---|---|---|---|
| Weight reads from memory | 1 per weight (best) | Repeated per output | Repeated per input reuse |
| Partial-sum traffic | Streams through array | Zero — stays in PE (best) | Streams through array |
| Activation reuse | Re-read every pass | Re-read every pass | 1 per activation (best) |
| Ideal workload shape | Large batch, weight-heavy | Wide accumulators, few reuses | Few input channels, many filters |
| Real chip | Google TPU (MXU) | Many DSP-style accelerators | Some early-CNN edge accelerators |
Notice the pattern: every dataflow optimizes exactly one operand's traffic to zero (or near it) and pays for it with the other two. There is no free lunch here — a chip architect picks the dataflow that matches their dominant workload's reuse pattern, not the one that sounds theoretically best in isolation.
The 128×128 Matrix Multiply Unit loads a full weight matrix once, then streams activation batches through it for as long as that weight matrix is needed — exactly the access pattern of large-batch transformer inference.
Doesn't pick just one operand. Row-stationary keeps a 1D convolution row fixed and reuses weights, inputs, and partial sums simultaneously across the PE array — a more complex but more globally energy-optimal scheme, published specifically to beat pure WS/OS/IS designs on real CNN workloads.
Common in accelerators doing high-precision accumulation (e.g. FP32 partial sums from INT8 inputs) — keeping that wide accumulator local avoids constantly writing and re-reading a bit-width-inflated intermediate value.
A first convolutional layer might have 3 input channels (RGB) but 64 output filters — parking that one small input and streaming 64 different weight kernels past it avoids re-fetching the input 64 times.
TPUs are built for exactly one dominant access pattern: run the same trained model (fixed weights) against a large batch of different inputs, over and over. Once a weight matrix is loaded into the MXU's 128×128 grid, it can stay resident for every activation vector in that batch — potentially thousands of reuses per weight load. That amortization is what makes weight-stationary the obvious choice when your workload is "one model, many inputs," which describes the overwhelming majority of both training minibatches and inference serving.
The cost is that activations and partial sums now have to move through the array every single cycle — but activations are typically far smaller than the full weight matrix, so this trade is favorable at TPU's target scale.
Because avoiding partial-sum movement only pays off when partial sums are expensive to move in the first place — wide accumulators (32-bit sums from 8-bit multiplies, for instance) genuinely cost more per byte transferred than a compact weight or activation. But if your weights are the operand with the least reuse (small batches, constantly-changing model), output-stationary forces you to re-stream those weights every cycle instead, which can be worse overall. The right dataflow is a function of which operand has the least natural reuse in your specific workload — there's no dataflow that's unconditionally best.
The MIT team behind Eyeriss (Chen, Emer, Sze) observed that picking just one operand to hold stationary always leaves the other two paying full price — and for real CNN workloads, all three operand types have some exploitable reuse simultaneously (weights reuse across spatial positions, inputs reuse across output channels, and partial sums reuse across the reduction). Row-stationary is a scheduling scheme that keeps a single 1D convolutional row resident in each PE's local register file and reuses all three operand types at once through careful spatial mapping across the PE array, rather than sacrificing two of them to maximize one. Their published results showed meaningfully lower total energy on real CNN benchmarks than pure WS, OS, or IS implementations of the same array size.
Both, in practice. At minimum, the dataflow determines which operand gets a persistent register inside each PE (versus which ones only need pass-through wiring) — a weight-stationary PE needs a weight register that holds its value across many cycles, while a purely pass-through operand just needs a pipeline flop. Real designs also differ in how partial sums are accumulated (a running adder chain vs. a final reduction tree) and in the control logic needed to load/drain the stationary operand at the start and end of each computation phase. So while the dataflow starts as a scheduling decision, it ends up baked directly into which registers exist in silicon.