Deep learning is a powerhouse in the AI industry that enables AI to learn on its own by leveraging GPUs designed to run machine learning algorithms at scale. However, the invention of deep learning was based on hardware that was not explicitly intended for this type of computing. Nvidia CEO Jensen Huang revealed on Joe Rogan's podcast that the researchers who originally developed deep learning did it all in 2012 on a pair of SLI's 3GB GTX 580s.
Researchers at the University of Toronto have invented deep learning to improve image detection in computer vision. In 2011, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton were researching better ways to build image recognition tools. Back then, there was no such thing as neural networks. Instead, developers used hand-designed algorithms to detect edges, corners, and textures for image recognition.
watch on
The three researchers built AlexNet, an architecture consisting of eight layers with a total of about 60 million parameters. What made this architecture special was its ability to learn on its own using a combination of convolutional and deep neural network layers. This architecture was so impressive that when it first appeared, it outperformed the leading image recognition algorithms (at the time) by more than 70% and quickly gained industry attention.
Jensen Huang revealed that AlexNet developers built the image recognition algorithm on a pair of SLI's GTX 580s. Additionally, the network was optimized to run on both GPUs, and the two GPUs exchange data only when necessary, significantly reducing training time. This makes the GTX 580 the world's first graphics card to run deep learning/machine learning AI networks.
Ironically, this milestone came at a time when Nvidia was making little investment in AI. Most of the graphics R&D was focused on 3D graphics and games, and CUDA. The GTX 580 was designed specifically for gaming and didn't have advanced support for accelerating deep learning networks. It turns out that the inherent parallelism of GPUs is exactly what neural networks need to run fast.
Jensen Huang further revealed that Nvidia has started developing AI hardware by combining AlexNet with use on the GTX 580. Huang said the company invested all of its funding, development and research into deep learning technology in 2012 after realizing that deep learning could be used to solve the world's problems. This gave rise to the original Nvidia DGX, the Volta architecture with first-generation Tensor Cores and DLSS, shipped to Elon Musk in 2016. Without a pair of GTX 580s running AlexNet, Nvidia might not have become the AI giant it is today.
to follow Tom's Hardware on Google Newsor Add us as a preferred sourceget the latest news, analysis, and reviews in your feed.
