Day 11 introduced the shape of Linux memory tiering — CXL memory as a CPU-less NUMA node, DAMON migrating hot and cold pages. Day 13 goes underneath that summary and asks the harder question: how does a CXL device go from a piece of silicon on a PCIe-style link to a NUMA node the kernel can safely hand out to applications at all? The answer is a specific, layered discovery-and-programming pipeline — and it's worth understanding, because every pooling and tiering benefit from Days 11–12 depends on this pipeline working correctly.
The Linux CXL driver isn't one monolithic module — it's deliberately split so that different concerns stay separated:
Core objects — ports, decoders, and regions — are shared across these subsystem drivers and the higher-level drivers built on top of them (a port-driver, a region-driver, and nvdimm object-drivers, since CXL persistent memory reuses much of the existing Linux NVDIMM subsystem from Day 6). This layering means the PCI-discovery logic and the ACPI-firmware logic don't need to know about each other's internals — they both feed into the same core topology model.
Before Linux can talk to a single CXL device, platform firmware has to tell it where CXL memory windows exist in the system's address space. This comes from two ACPI structures:
| ACPI Structure | What It Provides |
|---|---|
| CEDT (CXL Early Discovery Table) | Describes the platform's CXL Host Bridges and Fixed Memory Window Structures (CFMWS) |
| CFMWS (CXL Fixed Memory Window Structure) | Each entry describes one fixed memory window — a range of host physical address space reserved for CXL memory |
During cxl_acpi_probe, the kernel creates one root decoder per CFMWS entry it finds in the CEDT. That root decoder's memory range becomes a memory region, and the hosts it's reachable from are presented in sysfs via a target_list parameter — meaning this ACPI data is what first tells the kernel "CXL memory could exist somewhere in this address range," well before any specific device has been identified.
ACPI's CEDT/CFMWS describe the platform's side of the story — where memory windows are reserved. The device's own CDAT (Coherent Device Attribute Table) describes the other half: what the device itself reports about its own memory. CDAT parsing (handled by the core module) gives the kernel the device's memory ranges along with the latency and bandwidth of each range — real, hardware-reported numbers, not an assumption.
Why this two-table split matters: CEDT/CFMWS answer "where in the address map should I look for CXL memory," while CDAT answers "now that I've found a device there, how fast is it really?" Day 11's memory tiering framework needs exactly this second answer — the kernel places a CXL node into its tier based on the device's own reported abstract distance, derived from CDAT's real latency/bandwidth figures, not a fixed assumption baked into the kernel.
Knowing where memory should be and how fast a device claims to be still isn't enough to actually route traffic. The kernel has to program HDM (Host-managed Device Memory) decoders — both host-side and device-side — so that a contiguous Host Physical Address (HPA) range actually routes through the CXL fabric to the correct physical device. For interleaved regions (memory striped across multiple devices for bandwidth, similar in spirit to RAID striping), the decoders also have to agree on the correct interleave granularity so that consecutive address ranges land on the correct device in the correct order.
This is the step where the abstractions from earlier in this course become concrete register programming: Day 4's HDM concept, Day 7's Logical Devices, and Day 8's GFAM addressing all ultimately resolve down to an HDM decoder configuration that the kernel must get exactly right, on both ends of the link, before a single byte can be safely read or written.
Once HDM decoders are correctly programmed, the resulting CXL region is exposed as a hot-pluggable NUMA node. It typically first appears as a dax device (Direct Access) — a mode that gives applications direct, byte-addressable access to the memory without going through the page cache. From there, an administrator has a choice:
This is precisely the point where all the ACPI/CDAT/HDM machinery above becomes invisible to an application: whether it's a Day 11 DAMON-managed tiered NUMA node or a Day 12 GPU-reachable memory pool, the application (or in Day 12's case, the accelerator's runtime) just sees memory it can allocate from — the entire discovery-and-decode pipeline above is what made that simplicity possible.
A parallel set of userspace tools exists to configure, validate, and debug every layer above:
cxl # CXL-specific CLI: topology, regions, decoders
ndctl # nvdimm/persistent-memory subsystem CXL builds on
daxctl # convert dax devices to System RAM, or configure DAX mode
numactl # allocate/bind memory relative to NUMA nodes, including CXL tiers
lspci # inspect PCIe/CXL device configuration space
setpci # low-level PCI configuration space read/write, used in bring-up debugTogether, these tools validate the entire boot-to-usable-memory path this page just walked through: power-on → DRAM training → DVSEC and HDM reporting → decoder programming → CDAT delivery → ACPI table handoff. When a CXL memory expander doesn't show up as expected, this is the exact sequence an engineer steps through with these tools to find which stage failed — is the CEDT/CFMWS entry missing, did CDAT parsing fail to return sane latency numbers, or did HDM decoder programming leave the HPA range unrouted?