Research presents large-scale brain-like neural networks for AI

In a new study in nature machine intelligenceBojian Yin and Sander Bohté, researchers at the Dutch National Institute of Mathematics and Computer Sciences (CWI), HBP partners, propose artificial intelligence that can be used on local devices such as smartphones and applications like VR, while preserving privacy. represents an important step towards

They show that by combining brain-like neurons with novel learning methods, fast and energy-efficient spiking neural networks can be trained at scale. Potential applications range from wearable AI to voice recognition to augmented reality.

Modern artificial neural networks, the backbone of the current AI revolution, are only loosely inspired by networks of real biological neurons such as our brain. However, the brain is a much larger network, much more energy efficient, and can respond at lightning speed when triggered by external events. Spiking neural networks more closely mimic the workings of biological neurons. A special type of neural network that Neurons in the nervous system communicate by exchanging electrical pulses, but that communication is minimal.

Implemented in chips called neuromorphic hardware, such spiking neural networks are expected to bring AI programs closer to the user’s device. These local solutions are privacy, robust, and responsive. Applications range from voice recognition in toys and appliances, healthcare monitoring, drone navigation to local surveillance.

Spiking neural networks, like standard artificial neural networks, must be trained to perform such tasks well. However, the way such networks communicate poses serious challenges. “The algorithms required for this require a large amount of computer memory, so only small network models can be trained, mainly for small tasks. ,” said Sander Bohté of CWI’s Machine Learning group. The Human Brain Project addresses hierarchical cognitive processing architectures and learning methods.

Mimicking the learning brain

The learning aspect of these algorithms is a major challenge and is no match for our brain’s ability to learn. Easy to learn. Also, the brain takes far fewer examples to learn something and works more energy efficiently. “We wanted to develop something that approximates how the brain learns,” says Bojian Yin.

Yin explains how this works. If you make a mistake during your driving lesson, you can quickly learn from it. Correct your behavior immediately, not after an hour. “So to speak, it learns as it takes in new information. “learns how information changes and doesn’t” must remember all previous information. This is a big difference from today’s networks, which must accommodate all previous changes. Current learning methods are memory and energy intensive due to the enormous computational power required. ”

6 million neurons

New online learning algorithms can learn directly from data, enabling much larger spiking neural networks. Bohté and Yin, together with researchers at Eindhoven University of Technology and research partner Horst his center, demonstrated this in a system designed to recognize and locate objects. Yin shows a video of a busy street in Amsterdam. Her SPYv4, the underlying spiking neural network, is trained to be able to distinguish between bicycles, pedestrians and cars and pinpoint where they are.

“Previously, we could train neural networks with up to 10,000 neurons. Now we can do the same very easily with networks with more than 6 million neurons,” says Bohté. says Mr. “This allows very capable spiking he training neural networks like SPYv4.”

And where does it lead? With access to such powerful AI solutions based on spiking neural networks, chips are being developed that can run these AI programs at very low power, and eventually, in hearing aids and augmented or virtual reality. It is displayed on many smart devices, such as glasses.