Semiconductors · Vol. V|GPU and accelerators|1999 (GeForce 256); 2012 (AlexNet moment)
GPU and accelerators
1999 (GeForce 256); 2012 (AlexNet moment)
A graphics processing unit was built to draw triangles in parallel. In 2012 a neural network trained on GPUs won a vision contest so thoroughly that the industry turned. The GPU, and later specialized accelerators, are why ‘AI’ became a hardware problem as much as a software one. Parallelism is the feature; pictures were only the first customer.
NVIDIA and peers; researchers using GPUs for neural nets · United States, then global datacenters
How the mechanism is arranged
How it works
- Thousands of simple cores execute the same kind of math on different data.
- High memory bandwidth feeds those cores.
- Software (CUDA and others) maps algorithms onto that grid.
Why it mattered
- Made large neural networks trainable in reasonable time.
- Created a new bottleneck: chips, power, and data-center siting.
- Turned a games accessory into scientific infrastructure.
What it unlocked
For further study
- NVIDIA GeForce 256 launch. ‘GPU’ as a product word.
- Krizhevsky, Sutskever, Hinton, 2012. AlexNet.
Check
Why do neural nets fit GPUs better than a single fast CPU core?