Deep learning, an advanced artificial intelligence technique, has become increasingly popular over the last few years thanks to abundant data and increased computing power. It is the key technology behind many of the applications we use every day, such as online language translation, automatic face tagging in social media, smart replying in emails, a new wave of generative models, and more. Deep learning is nothing new, but it has benefited greatly from increased data availability and advances in computing.
ChatGPT is the fastest growing app of all time, an AI-powered chatbot powered by deep learning models trained on billions of words collected from the internet. DALL-E, Midjourney, and Stable Diffusion are AI systems that can generate images from text descriptions, and deep learning systems that model the relationship between images and text descriptions.
Comparing Deep Learning and Machine Learning
Deep learning is a subset of machine learning, a branch of artificial intelligence that configures computers to perform tasks through experience. In contrast to classical rule-based AI systems, machine learning algorithms develop behaviors by processing annotated examples. This process is called “training”.
For example, to create a fraud detection program, you train a machine learning algorithm with a list of bank transactions and their final outcomes (whether correct or fraudulent). A machine learning model examines the examples and creates a statistical representation of features common to legitimate and fraudulent transactions.
The algorithm is then fed data from new bank transactions, which classify the transactions as legitimate or fraudulent based on the patterns gleaned from the training examples. As a rule of thumb, the higher the quality of the data you provide, the better the machine learning algorithms will perform their tasks.
Machine learning is especially useful for solving problems where rules are not well defined and cannot be coded into discrete commands. Different kinds of algorithms are better at different tasks.
Deep learning and neural networks
Traditional machine learning algorithms solve many problems that rule-based programs have struggled with, but they struggle with soft data such as images, videos, sound files, and unstructured text.
For example, according to AI researcher and data scientist Jeremy Howard, in the video above, it takes dozens of experts in the field to create a breast cancer prediction model using classical machine learning approaches. , computer programmers and mathematicians.
Researchers have to perform a lot of feature engineering, the difficult process of programming computers to find known patterns in X-rays and MRI scans. Engineers then use machine learning based on the extracted features. It takes years to create such an AI model.
Deep learning algorithms solve the same problem using: deep neural network, A type of software architecture inspired by the human brain (although neural networks are not biological neurons). Neural networks are layers of variables that adjust themselves to the characteristics of the data they are trained on, allowing them to perform tasks such as classifying images or converting speech to text.
Neural networks are particularly good at independently finding common patterns in unstructured data. For example, after training a deep neural network with images of various objects, it finds a way to extract features from those images. Each layer of the neural network detects specific features such as edges, corners, faces, and eyeballs.
Neural networks have been around (at least conceptually) since the 1950s. However, until recently, the AI community largely ignored it because of the enormous amount of data and computing power it requires. Over the past few years, the availability and affordability of storage, data, and computing resources have pushed neural networks to the forefront of AI innovation.
There are currently various types of deep learning architectures, each suitable for different tasks. Convolutional Neural Networks (CNNs) are particularly good at capturing patterns in images. Recurrent Neural Networks (RNNs) excel at processing continuous data such as speech, text, and musical notes. Graph Neural Networks (GNNs) can learn and predict relationships between graph data such as social networks and online purchases.
A very popular deep learning architecture these days is the transformer used in Large Language Models (LLMs) such as GPT-4 and ChatGPT. Transformers are particularly good at language tasks and can be trained on very large amounts of raw text.
What is deep learning used for?
There are several areas where deep learning is helping computers tackle problems that were previously unsolvable.
computer vision
Computer vision is the science of using software to understand the content of images and videos. This is one area where deep learning has made great strides. Besides breast cancer, deep learning image processing algorithms can detect other types of cancer and help diagnose other diseases.
But this type of deep learning is also embedded in many of the applications you use every day. Apple’s Face ID uses computer vision to recognize faces, and Google Photos also uses various features such as finding objects and scenes, retouching images, and more. Facebook used deep learning to automatically tag people in uploaded photos, but that feature was shut down in 2021.
Deep learning can also help social media companies automatically identify and block problematic content such as violence and nudity. And finally, deep learning plays a very important role in enabling self-driving cars to understand their surroundings.
Speech and speech recognition
Speak commands to your Amazon Echo smart speaker or Google Assistant, and deep learning algorithms convert your voice into text commands. Several online applications also use deep learning to transcribe audio and video files. Gboard, Google’s keyboard app, uses deep learning to provide on-device, real-time transcriptions of what you type as you speak.
Natural language processing and generation
natural language processingnatural language processing (NLP), the science of extracting meaning from unstructured text has been a historical pain point for classical software. It is virtually impossible for computers to define all the nuances and hidden meanings of written language. However, neural networks trained on large amounts of text can perform many NLP tasks accurately.
Google’s translation service saw a sharp increase in performance when the company switched to deep learning. Smart speakers use deep learning NLP to understand different nuances of commands, such as different ways to ask for weather or directions.
Deep learning is also very efficient at generating meaningful text. Natural Language Generation (NLG). Gmail’s Smart Reply and Smart Compose use deep learning to surface relevant responses to your emails and suggestions for completing your writing. A text generation model developed by OpenAI produced long excerpts of consistent text.
Large Language Model (LLM) OpenAI’s ChatGPT and others can perform a wide range of tasks such as summarizing text, answering questions, writing articles, and generating software code. LLM is integrated into a wide range of applications, including corporate messaging and email apps, productivity apps, and search engines.
art generation
One area where deep learning is very useful these days is image generation. Models such as DALL-E and Stable Diffusion can create stunning images from text descriptions. Microsoft already uses his DALL-E in several products, including Designer. Adobe also uses generative models in some applications.
Limitations of deep learning
Although deep learning has many advantages, it also has some drawbacks.
data dependencies
Deep learning algorithms typically require huge amounts of training data to perform their tasks accurately. Unfortunately, we don’t have enough high-quality training data to create deep learning models that can handle many types of problems.
explainability
Neural networks develop their behavior in very complex ways. Even the creators have a hard time understanding how it works. The lack of interpretability makes it very difficult to troubleshoot errors and correct mistakes in deep learning algorithms.
algorithm bias
Deep learning algorithms are only as good as the data they were trained on. The problem is that training data often contains hidden or apparent biases, and algorithms inherit these biases. For example, a facial recognition algorithm trained using primarily Caucasian photos will be less accurate on non-Caucasian people.
lack of generalization
Deep learning algorithms are good at performing focused tasks, but bad at generalizing knowledge. Unlike humans, deep learning models trained to play StarCraft cannot play similar games, such as WarCraft.
Deep learning also struggles with handling data that deviates from the training samples, also known as “edge cases.” This can be dangerous in situations where mistakes can have fatal consequences, such as self-driving cars.
The future of deep learning
In 2019, the pioneers of deep learning were awarded the Turing Award, the equivalent of the Nobel Prize in computer science. However, research on deep learning and neural networks is not over yet. Various efforts have been made to improve deep learning.
Interesting research includes explainable or open-ended deep learning models, neural networks that can develop behaviors with less training data, and Edge AI, deep learning algorithms that can perform tasks without relying on large cloud computing resources. There are models and so on.
And while deep learning is currently the most advanced artificial intelligence technology, it is not the ultimate destination for the AI industry. Advances in deep learning and neural networks may bring about entirely new architectures.
