Every day of this course so far has quietly leaned on one fact without unpacking it: CXL traffic doesn't travel as a free-form stream of bytes. It travels as flits — fixed-size chunks with strict internal structure. Day 2 mentioned the 68-byte flit in passing; Day 8 mentioned the jump to 256 bytes. Day 10 is where we actually open the flit up, because the flit format is where CXL's bandwidth, latency, and reliability numbers are all being decided, in silicon, one packet at a time. This closes Phase 2 — the spec-by-spec walk through CXL's version history.
A flit (short for flow control unit) is the fixed-size packet that CXL breaks every message into before it crosses the physical link. Whatever protocol generated the traffic — CXL.io carrying a PCIe-style transaction, CXL.cache carrying a D2H request from Day 3, or CXL.mem carrying an M2S/S2M memory access — it all eventually gets packed into flits for the trip across the wire.
Why fixed size instead of variable-length packets like a network frame? Because CXL is latency-sensitive in a way most networks aren't. A memory access has to complete in tens to low-hundreds of nanoseconds (Day 2's latency numbers), and fixed-size units are dramatically easier for hardware to pipeline, decode, and forward at that speed than variable-length ones. The cost of that simplicity is that every flit carries some fixed overhead — header fields, error-detection bytes, and (at higher speeds) error-correction bytes — whether or not the payload inside is full.
CXL 1.1 and CXL 2.0 (Days 6–7) use a 68-byte flit, sized for the PCIe 5.0 physical layer running at up to 32 GT/s. At these speeds, the physical layer uses relatively simple NRZ (non-return-to-zero) signaling — one bit per symbol — which has a comfortably low raw bit error rate. That means the 68-byte flit can get away with lightweight protection: a CRC (cyclic redundancy check) to detect errors, with retransmission as the fallback when an error is caught. No forward error correction is needed at this signaling rate; catch-and-retry is cheap enough.
Why this matters: the 68-byte flit is small enough that its per-flit overhead (header + CRC) is a meaningful fraction of the total — but that's an acceptable trade at 32 GT/s, because the alternative (a bigger flit) would mean waiting longer to fill it, adding latency for no benefit at a speed where errors are already rare.
CXL 3.0 (Day 8) moves to the PCIe 6.0 physical layer at 64 GT/s — double the previous generation's rate. Getting there required a signaling change with real consequences: PCIe 6.0 uses PAM4 (4-level pulse amplitude modulation) instead of NRZ. PAM4 packs 2 bits per symbol instead of 1, which is how the data rate doubles without doubling the raw symbol rate — but it does this by making each symbol represent four possible voltage levels instead of two, which are inherently closer together and therefore much easier to misread as noise. PAM4 has a substantially higher raw bit error rate than NRZ at comparable rates.
A CRC-and-retry scheme that worked fine for NRZ's low error rate becomes far too costly under PAM4: if errors are frequent enough, constantly discarding and retransmitting whole flits kills throughput and adds unpredictable latency spikes. The fix is to correct as many errors as possible within the flit itself, before ever needing a retry — which is exactly what forward error correction is for.
CXL 3.0 introduces the 256-byte flit. Bigger for a reason: it needs room for a much stronger error-correction payload. Per published PCIe 6.0 flit-mode details, the 256-byte flit breaks down roughly as follows:
| Component | Size | Purpose |
|---|---|---|
| Payload (TLPs + DLPs) | 236 bytes + 6 bytes | The actual protocol data (transaction-layer and data-link-layer packets) |
| CRC | 8 bytes | Reed-Solomon code over GF(28), protects the full 236+6 byte block |
| FEC | 2 bytes × 3 interleaved ways | Single-symbol-correcting forward error correction across the entire 250-byte block |
Flit-mode byte breakdown per PCIe 6.0 / CXL 3.0 specification materials.
The FEC layer is what actually absorbs PAM4's higher raw error rate: a 3-way interleaved, single-symbol-correcting Reed-Solomon FEC can fix a proportion of bit errors without needing a retransmission at all. The CRC still sits on top as a safety net — if an error is too large for FEC to correct, the CRC catches it and the flit gets retried, exactly as before. Header bytes, CRC, and FEC are all added by the Flex Bus layer (Day 2) as part of its error correction, detection, and retry responsibilities.
CXL 3.0 doesn't stop at one 256-byte format — it defines two, because bandwidth and latency pull in opposite directions and different traffic wants different answers:
The latency-optimized format is described in CXL materials as having a zero-latency adder over CXL 2.0 — meaning that despite carrying nearly four times the payload of the old 68-byte flit, a well-designed latency-optimized 256B flit doesn't have to cost any more end-to-end latency than the format it replaced. That's only possible because the sub-block CRC structure lets the receiver start acting on the earliest-arriving sub-block immediately, rather than blocking on the whole 256 bytes.
The trade-off in one sentence: Standard 256B maximizes protection efficiency per byte; Latency-Optimized 256B sacrifices a little of that efficiency (smaller CRC blocks are less efficient than one large one) in exchange for letting the receiver start working sooner. Which one a given link uses depends on whether that traffic is latency-critical (CXL.cache/CXL.mem coherence traffic, where every nanosecond matters) or more bandwidth-oriented (bulk CXL.io transfers, where total throughput matters more than any single flit's latency).
Recall from Day 9 that CXL 4.0 doubles the PHY rate again, to 128 GT/s on PCIe 7.0. You might expect another ground-up flit redesign — but CXL 4.0 instead keeps the standard 256-byte flit and simply adds its own additional 256-byte latency-optimized variant on top, tuned for the new PHY's error characteristics. The lesson here mirrors Day 9's version-ladder takeaway: CXL's architects prefer to extend a working primitive rather than replace it. The 256-byte flit, once it existed, turned out to be a stable enough foundation that two full PHY-speed doublings (3.0→4.0, 64→128 GT/s) didn't require reinventing it — only retuning the error-correction math for the new signaling environment.
There's a question the last nine days have skipped past: CXL.io, CXL.cache, and CXL.mem are three separate protocols with three separate jobs (Day 3) — so how do they all get onto the same physical link at the same time? The answer is a dedicated hardware block called the ARB/MUX (Arbitration and Multiplexer), sitting directly above the physical layer.
CXL.cache and CXL.mem actually share a common link/transaction layer beneath them (separate from CXL.io's own PCIe-style layer), and the ARB/MUX's job is to decide, flit by flit, whose turn it is to use the link: CXL.io traffic, or the combined CXL.cache/CXL.mem traffic. It does this with weighted round-robin arbitration, where the weights are set by the host — meaning the host can bias the link toward more coherence traffic or more I/O traffic depending on the workload, rather than splitting bandwidth blindly 50/50.
This is also where a more granular view of the flit's internal layout comes in. Beneath the commonly-cited "68-byte flit" figure, CXL's flit is more precisely built from four 16-byte slots plus a 2-byte CRC — a 528-bit (66-byte) structure that the flit header maps onto CXL.cache or CXL.mem formats slot by slot. Each slot can be independently dedicated to a different protocol's data, which is the actual mechanism that lets a single flit carry, say, a CXL.cache request in one slot and CXL.mem completion data in another. The flit header is what tells the receiving transaction layer how to route each slot back to the right protocol.
Why this matters: without ARB/MUX and per-slot routing, CXL would need three separate physical links — one per sub-protocol — to keep CXL.io, CXL.cache, and CXL.mem from colliding. Instead, one Flex Bus link (Day 2) carries all three, and the ARB/MUX plus slotted flit structure is the specific mechanism that makes that sharing possible without one protocol starving another.
None of this flit traffic can begin the instant a device is plugged in. Every CXL link first goes through LTSSM (Link Training and Status State Machine) — the same link-training concept PCIe uses, extended for CXL's needs. During LTSSM, the two link partners exchange Training Sequences (TS) and Modified Training Sequences (mTS) to negotiate the basics: link width, signaling speed, and which CXL protocols each side actually supports (recall from Day 1 that Alternate Protocol Negotiation is what determines whether a link even attempts CXL mode over the shared PCIe electrical layer). Only after LTSSM completes successfully does the link reach L0, the active state where CXL flits actually start flowing.
This matters for understanding real-world CXL latency: LTSSM only happens once at link bring-up (and after certain power-state transitions), not per-transaction — so it doesn't show up in the per-access latency numbers from Day 2. But it's a reminder that everything this course discusses about flits, sub-protocols, and coherence assumes a link that has already successfully trained and negotiated CXL mode. A link stuck failing LTSSM never gets to send a single CXL flit, regardless of how correct its higher-layer logic is.
To make the flit format concrete, walk through what happens, roughly, when one CXL.mem read flows across a link:
Every one of Day 2's headline latency figures (CXL-attached memory at roughly 170–250ns versus local DRAM's 80–140ns) has flit assembly, transmission, FEC/CRC checking, and disassembly baked into it, repeated at least twice for a round trip. The Standard vs Latency-Optimized flit choice from earlier on this page is precisely about minimizing step 3 and step 4's contribution to that total, for the traffic that can least afford to wait.
This is worth stating plainly, because it ties the entire course together: every latency number from Day 2, every bandwidth calculation, every "why is CXL memory 100–150ns slower than local DRAM" discussion — all of it ultimately bottoms out in flit-level decisions. A flit has to be assembled, transmitted, error-checked (and possibly corrected), and disassembled on the other end, and every one of those steps takes real time measured in nanoseconds. The flit format is not an obscure implementation detail sitting below the interesting parts of CXL — it is the physical-layer trade-off that produces the interesting parts of CXL. Sub-protocols, device types, coherence models, and fabric topology all describe what CXL moves and who owns it; the flit format describes the actual mechanics of how a bit gets from one side of the link to the other reliably, and at what latency cost.