From PyTorch model to running on silicon. Quantize with vai_q_pytorch, compile for the Deep Learning Processing Unit, and run ResNet-50 on a Xilinx Kria board — no RTL required.
Days 1–10 built the accelerator from scratch — RTL and HLS. That's the deep-engineering path: maximum control, maximum effort. But there's a second path that ships products in days, not months: the DPU + Vitis AI flow. You take a trained model, quantize it, compile it, and run it on a pre-built accelerator — writing only Python and a little C++.
The DPU (Deep Learning Processing Unit) is AMD/Xilinx's configurable CNN accelerator IP. It's essentially a parameterized version of everything you built in Days 3–9 — a systolic-style MAC array, on-chip buffers, and a tiny instruction processor — wrapped as a drop-in block. You instantiate it once; the same hardware runs any compiled model.
| DPU Config | Peak ops/clock | Typical Use | DSP Usage |
|---|---|---|---|
| B512 | 512 | Tiny edge / low cost | ~98 |
| B1024 | 1024 | Balanced edge | ~194 |
| B2304 | 2304 | Mid-range Kria | ~438 |
| B4096 | 4096 | High-perf (KV260) | ~710 |
Four steps take a trained model to running silicon.
The quantizer runs calibration images through the model to learn each layer's scale factors, then exports an INT8 model. This is the Day 2 theory, fully automated.
import torch
from pytorch_nndct.apis import torch_quantizer
import torchvision
# 1. Load your trained FP32 model
model = torchvision.models.resnet50(weights='IMAGENET1K_V1').eval()
# 2. Create the quantizer in CALIBRATION mode
dummy = torch.randn(1, 3, 224, 224)
quantizer = torch_quantizer(
quant_mode='calib', # 'calib' first, then 'test'
module=model,
input_args=(dummy,),
output_dir='quantized')
quant_model = quantizer.quant_model
# 3. Feed ~100-1000 calibration images (no labels needed)
for images in calib_loader: # representative data
quant_model(images)
# 4. Export quantization config
quantizer.export_quant_config()
# 5. Re-run in 'test' mode to evaluate INT8 accuracy + export model
# quant_mode='test' → quantizer.export_xmodel(deploy_check=True)
# Produces ResNet50_int.xmodel for the compiler.ResNet-50 typically loses only ~0.5–1% top-1 accuracy going FP32→INT8 with PTQ. If that's too much, run a few epochs of QAT (quantization-aware training) — vai_q_pytorch supports it — to recover almost all of it. This is exactly the trade-off from Day 2, now measured on a real network.
The compiler maps the INT8 graph to the exact DPU instance you instantiated (its arch + frequency are described in an arch.json fingerprint).
# Compile the quantized .xmodel for a specific DPU (e.g. KV260 B4096)
vai_c_xir \
--xmodel quantized/ResNet50_int.xmodel \
--arch /opt/vitis_ai/arch/DPUCZDX8G/KV260/arch.json \
--net_name resnet50 \
--output_dir compiled
# Output: compiled/resnet50.xmodel ← runs on the board's DPU
#
# The 'arch.json' is the DPU fingerprint. If the board's DPU config
# doesn't match what you compiled for, VART refuses to load — recompile.VART (Vitis AI Runtime) is the API you call from your application. Load the model, push input tensors to the DPU, read results back. Here's the core loop in Python.
import vart, xir, numpy as np
# 1. Load compiled model + create a DPU runner
graph = xir.Graph.deserialize("compiled/resnet50.xmodel")
subgraph= [s for s in graph.get_root_subgraph().toposort_child_subgraph()
if s.has_attr("device") and s.get_attr("device")=="DPU"][0]
runner = vart.Runner.create_runner(subgraph, "run")
# 2. Query input/output tensor shapes
in_t = runner.get_input_tensors()[0]
out_t = runner.get_output_tensors()[0]
in_shape, out_shape = tuple(in_t.dims), tuple(out_t.dims)
# 3. Prepare INT8 input (preprocess + quantize to the input scale)
input_data = [np.empty(in_shape, dtype=np.int8)]
output_data = [np.empty(out_shape, dtype=np.int8)]
input_data[0][0] = preprocess_and_quantize(image) # your func
# 4. Run on the DPU (async submit + wait)
job_id = runner.execute_async(input_data, output_data)
runner.wait(job_id)
# 5. Post-process: argmax of logits = predicted class
pred = int(np.argmax(output_data[0][0]))
print("Predicted class:", pred)The DPU only executes the network graph. Image decode, resize, normalize, and the final softmax/argmax run on the ARM cores. On a busy pipeline these can become the bottleneck — which is why production apps pre-process on multiple threads or offload resize to the FPGA fabric alongside the DPU.
| Scenario | Best Choice | Why |
|---|---|---|
| Standard CNN (ResNet, YOLO, MobileNet) | DPU + Vitis AI | Supported out-of-box, days to deploy |
| Fast time-to-market | DPU + Vitis AI | Software-only, pre-verified IP |
| Custom/novel layer not in DPU | Custom HLS/RTL | DPU can't run unsupported ops |
| Absolute best perf/watt, fixed model | Custom HLS/RTL | Hand-tuned dataflow beats general DPU |
| Ultra-low latency (<1ms) | Custom dataflow | DPU's instruction overhead adds latency |
| Hybrid (CNN + custom pre/post) | DPU + HLS kernels | DPU for the net, HLS for the rest |
Everything you learned building from scratch — quantization (Day 2), GEMM & systolic arrays (Days 3–4), convolution (Day 5), memory (Day 6), and pipelining (Day 9) — is exactly what's inside the DPU. The DPU isn't magic; it's a polished, configurable version of your own accelerator. Understanding the internals is what lets you tune the DPU config and debug performance.
Next — Day 12: Power Optimization for Edge AI — clock gating, partial reconfiguration, voltage/frequency scaling, and getting a real accelerator under a 5W budget.