PyTorch profiling isn't just for experts: Here's how to read your first trace

Most PyTorch users know they should profile their models. Few actually do it. The barrier isn’t a lack of tools — it’s that the output looks like a rainbow exploded on a timeline and every tutorial assumes you already speak fluent CUDA. This series from the PyTorch team takes a different approach: it starts from zero and chases down every “wait, why?” moment in the trace.

The opening post walks through a script so simple it’s almost embarrassing — a single matrix multiplication followed by an addition, the computational equivalent of “hello world.” But that simplicity is the point. When you wrap that tiny function in torch.profiler.profile() and export the Chrome trace, you don’t get one event. You get a cascade: a Python call on the CPU, a CUDA kernel launch, and then the actual GPU kernel execution. The profiler hands you two artifacts — a statistical table that answers “what’s eating my time?” and a temporal trace that shows you when things actually happen, including the gaps where the CPU and GPU are awkwardly waiting on each other.

The real value here is learning to read those gaps. The CPU lane and GPU lane don’t line up by accident. When they don’t overlap, you’re leaving performance on the table. The post walks through what changes when you apply torch.compile — some things get fused, some latencies shrink, and some gaps simply vanish because the compiler eliminated the round-trip. But it’s careful not to oversell: compilation isn’t magic, and the trace will show you exactly what it can and can’t fix.

If you’ve ever stared at a wall of colored rectangles in chrome://tracing and closed the tab in defeat, this is the guide that meets you where you are. No prerequisites beyond knowing what a tensor is. Just a genuine, question-driven walk through the guts of PyTorch execution.