AlexNet and the Deep Learning Breakthrough

Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton submit AlexNet to the ImageNet Large Scale Visual Recognition Challenge, achieving a top-5 error rate of 15.3% – compared to 26.2% for the next best submission. The gap is decisive. The AI research community rapidly redirects attention toward deep convolutional networks.

AlexNet’s success depends on three things that come together in 2012: a large labeled dataset (ImageNet, 1.2 million images), cheap parallel computation (GPUs), and algorithmic improvements. None is new; the combination is.

The practical consequence is that systems trained this way cannot be easily inspected. The learned representations are distributed across millions of parameters with no human-interpretable semantics. Opacity becomes structural at this moment.

Source: Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” Advances in Neural Information Processing Systems 25 (NeurIPS 2012).