Library of human understanding
Thousands of processors in parallel, unleashing massive throughput.
A graphics processing unit (GPU) evolved from the need to render millions of pixels on a screen. Unlike a CPU, which executes a few instructions very quickly, a GPU executes the same instruction on thousands of data elements in parallel. This design was perfect for graphics—transform, shade, and rasterize millions of pixels at once—and decades later proved equally powerful for machine learning and scientific simulation.
A GPU contains hundreds or thousands of cores organized into groups called warps or wavefronts. All cores in a group execute the same instruction on different data, a pattern called single instruction, multiple data (SIMD). The GPU trades CPU flexibility for raw parallel throughput: it can apply the same operation to massive arrays of numbers far faster than a CPU, but only if your problem fits that pattern.
In the 1990s, GPUs were specialized chips on a graphics card, hardwired to transform 3D vertices, project them onto 2D screen coordinates, and rasterize triangles into pixels. Graphics programmers wrote vertex and pixel shaders—programs that ran on the GPU to control how geometry was transformed and how colors were computed.
Around 2006, GPUs became programmable enough to solve non-graphics problems. NVIDIA released CUDA, allowing programmers to write C-like code that would run on thousands of GPU cores simultaneously. A single GPU could outperform a CPU 10× or 100× on problems like matrix multiplication, if the parallelism fit the data. This sparked GPU adoption in machine learning, physics simulation, and scientific computing.
A modern GPU has 5,000 to 100,000 logical processors, far more than a CPU's 8 to 128 cores. They are organized hierarchically: thousands of threads in thread blocks, thread blocks in grids. All threads in a warp (32 or 64 threads) execute the same instruction on different data. If threads in a warp take different control paths (one goes to an if-branch, another to an else), the hardware executes both paths, disabling unused threads—a penalty called divergence.
GPUs tolerate stalls better than CPUs. While some threads wait for memory, others compute, so the GPU stays productive if there are enough threads. This means a GPU can hide latency with parallelism: use 10,000 threads instead of 1,000, and waits disappear in the noise. The tradeoff is memory: each thread needs its own registers and local memory, and a GPU's memory hierarchy is different from a CPU's.
Modern GPUs include specialized circuits called tensor cores that multiply two 4×4 matrices and accumulate the result in a single instruction. Because matrix multiplication is the core operation in deep learning, tensor cores give 10× or 20× speedup on neural networks compared to general SIMD code.
This specialization hints at the future: as problem domains become clear, hardware adapts. GPUs started as rasterizers, became general SIMD processors, and now sport matrix engines. The GPU's flexibility still beats a fixed-function accelerator, but the trend is clear: squeeze the most performance from the parallelism you have.
born 1962 · Computer Science, GPU Architecture
born 1963 · Electronics, Business
“You're either scaling with great talent or you're not going to make it. You need to be surrounded by great people.”
Quotations appear only when we could check them against a named source. Sayings we could not trace are kept out.