What CXL means, how CXL memory expanders work, CXL Type 3 devices, CXL.mem protocol, and real products from Samsung, Micron, SK Hynix.
CXL (Compute Express Link) is an open industry interconnect standard built on the PCIe Gen5 physical layer. It enables cache-coherent, high-bandwidth communication between CPUs and attached devices — including accelerators, memory expanders, and smart I/O devices. CXL is maintained by the CXL Consortium (200+ member companies including Intel, AMD, ARM, NVIDIA, Samsung, Micron, SK Hynix, Qualcomm).
CXL defines three sub-protocols that can be used independently or in combination:
PCIe-compatible I/O — discovery, configuration, DMA, interrupts. All CXL devices use this.
Cache coherency — accelerator caches snoop host CPU cache. Used by Type 1 and Type 2.
Memory semantics — CPU load/store access to device memory. Used by Type 2 and Type 3.
| Type | Sub-protocols | Description | Examples |
|---|---|---|---|
| Type 1 | CXL.io + CXL.cache | Coherent accelerator — no device memory. Uses host memory via coherent cache | Smart NICs, FPGAs (no DRAM), CXL switches |
| Type 2 | CXL.io + CXL.cache + CXL.mem | Accelerator with device memory — GPU/AI chip. CPU and device share coherent view of both memories | AI ASICs, GPUs with HBM, FPGAs with DRAM |
| Type 3 | CXL.io + CXL.mem | Memory expander — no caching capability. Presents DRAM to CPU as additional memory tier | Samsung CMM-D, Micron CXL Module, SK Hynix AiMX |
A CXL memory expander (CXL Type 3 device) is a PCIe add-in card or E3.S enterprise module containing DDR5 DRAM connected to the host CPU via the CXL.mem protocol. The CPU treats this memory as an additional NUMA node — accessible via standard load/store instructions, no software change needed for many workloads.
| Vendor | Product | Capacity | Interface | Form Factor |
|---|---|---|---|---|
| Samsung | CXL DRAM CMM-D | 128 GB, 256 GB, 512 GB | PCIe Gen5 x8 | E3.S EDSFF |
| Micron | CXL Memory Module | 128 GB | PCIe Gen5 x8 | E3.S EDSFF |
| SK Hynix | AiMX CXL DRAM | 96 GB | PCIe Gen5 x8 | AIC (Add-In Card) |
| Innodisk | CXL Memory Module | 64 GB | PCIe Gen5 x8 | AIC |
| Spec | Value | Notes |
|---|---|---|
| Interface | PCIe Gen5 x8 | ~32 GB/s peak bandwidth |
| Protocol | CXL 2.0 / CXL 3.1 | CXL 3.1 adds fabric support |
| DRAM type | DDR5-4800 / DDR5-6400 | Standard DDR5 inside module |
| Latency | ~150–300 ns access latency | vs ~60–80 ns for local DDR5 |
| Max capacity | 512 GB per module | Samsung CMM-D (2024) |
| Error correction | ECC + patrol scrub + PPR | hPPR and sPPR supported |
| OS support | Linux 5.14+ (DAXCTL, NDCTL) | numactl for NUMA binding |
Why CXL memory for AI servers? A 2-socket AI server with 8 DDR5 channels per CPU maxes out at ~3 TB of local DRAM (using 192GB RDIMMs). With CXL memory expanders, the same server can reach 6–8 TB+ by adding CXL modules in PCIe Gen5 slots — without board redesign. Critical for large model inference and in-memory databases.
| Tier | Technology | Bandwidth | Latency | Capacity | Use Case |
|---|---|---|---|---|---|
| Tier 0 | Local DDR5 DIMM | ~50 GB/s/ch | ~70 ns | Up to ~3 TB/socket | Working set, latency-critical data |
| Tier 1 | CXL Memory Expander | ~32 GB/s/module | ~150–300 ns | 512 GB/module | Memory expansion, large model weights |
| Tier 2 | Optane PMem / CXL SSD | ~8–12 GB/s | ~350–1000 ns | TB range | Warm data, persistent memory tier |
| Tier 3 | NVMe SSD | ~7 GB/s | ~100 µs | Tens of TB | Cold data, storage |
CXL 3.1 (2023) extends the memory expansion capabilities:
CXL = Compute Express Link. Open standard built on PCIe Gen5 for cache-coherent CPU-to-device communication. Managed by the CXL Consortium (200+ members). Enables memory expansion (Type 3), accelerator coherency (Type 1), and combined accelerator+memory (Type 2).
A CXL memory expander (Type 3 device) is a PCIe card containing DDR5 DRAM that connects to a CPU via PCIe Gen5. The CPU sees it as an additional memory NUMA node — accessible via load/store without software changes. Up to 512 GB per module (Samsung). ~32 GB/s bandwidth via PCIe Gen5 x8.
A CXL memory module is the physical package of a CXL memory expander — DRAM chips in an E3.S (EDSFF) or AIC form factor. Examples: Samsung CMM-D (512GB), Micron CXL Module (128GB), SK Hynix AiMX (96GB). All use PCIe Gen5 x8 and CXL.mem protocol.
DDR5 connects directly to the CPU memory controller via a dedicated DDR5 bus (lower latency ~70ns, higher bandwidth ~50GB/s per channel). CXL memory connects via PCIe Gen5 (higher latency ~150-300ns, ~32GB/s per module) but offers much larger capacity expansion. DDR5 = performance tier, CXL = capacity expansion tier.