CPU

In 1945, John von Neumann described the stored-program computer: a machine with memory holding both data and the instructions that operate on it, a control unit that fetches and decodes instructions, and an arithmetic-logic unit (ALU) that carries out the operations. Before this, programmers rewired machines by hand for each new task. The stored program made the computer universal.

The cycle is simple: fetch the instruction from memory, decode it, execute it, and write back any result. By encoding the instruction as a number in memory, the same machine can run any program. This design is so fundamental that every CPU in the world, from the tiniest embedded processor to the fastest supercomputer, still follows it.

An instruction set is the vocabulary of the machine: the list of operations it can perform. x86 and ARM are the dominant instruction set architectures. x86, created by Intel in 1978, powers most desktop and server computers. ARM, a simpler and more power-efficient design from 1985, dominates mobile phones and embedded systems.

Different architectures optimize for different goals. x86 prioritizes backward compatibility and raw performance. ARM emphasizes energy efficiency. MIPS and PowerPC serve specialized markets. A CPU implements one instruction set by providing circuits to decode each instruction and carry out the operation, so changing the set requires redesigning the chip.

A single-core CPU processes one instruction at a time, but modern CPUs contain multiple cores, each executing its own instruction stream in parallel. A quad-core CPU can run four independent instruction streams, quadrupling throughput for parallel workloads. The challenge is that not all programs parallelize easily: they must be written to divide work among cores.

Within a single core, pipelining overlaps instruction execution. While the ALU is computing the result of one instruction, the control unit is decoding the next, and memory is fetching the one after that. This keeps the hardware busy, but jumps and data dependencies can stall the pipeline. Speculative execution and branch prediction try to guess what comes next, but modern CPUs have learned that some predictions can leak secrets, a vulnerability that required hardware redesign.

Main memory is large but slow; registers are tiny but fast. Caches bridge the gap. An L1 cache (a few dozen kilobytes per core) sits next to the ALU. If data is there, it arrives in one or two cycles. If not, the CPU stalls while waiting for the slower L2 cache, then the even slower L3 cache, then main memory, which can take hundreds of cycles. The job of cache design is to guess which data you will need next and have it ready.

Locality of reference is the key principle: programs tend to reuse the same data and instructions over short time periods. Cache designs exploit temporal locality (hold recently used data) and spatial locality (fetch nearby data too). This is why a program might run 100 times faster by rearranging its data layout: the cache hits more often.