summary: Today's AI can read, speak, and analyze data, but it still has significant limitations. Researchers at NeuroAI have designed a new AI model inspired by the efficiency of the human brain.
This model allows AI neurons to receive feedback and adjust in real time, enhancing learning and memory processes. This innovation could lead to a new generation of AI that is more efficient and accessible, bringing AI and neuroscience closer together.
Key Facts:
- Inspired by the brainThe new AI model is based on how the human brain efficiently processes and coordinates data.
- Real-time adjustmentsAI neurons can receive feedback and adjust on the fly, making them more efficient.
- Potential impactThis breakthrough could give rise to a new generation of AI that learns like humans, enhancing the fields of both AI and neuroscience.
sauce: CSHL
It reads, speaks, collates vast amounts of data, and suggests business decisions. Today's artificial intelligence may seem more human than ever before. But AI still has some significant shortcomings.
“ChatGPT and current AI technologies are all great, but they are still very limited when it comes to interacting with the physical world. Even tasks like solving math problems or writing an essay require billions of training examples before they can get good at them,” explains Kyle Daruwala, a NeuroAI researcher at Cold Spring Harbor Laboratory (CSHL).
Daruwala has been searching for new, unconventional ways to design AI that can overcome these computational obstacles—and he may have found it.
Data movement is key: Currently, most of the energy consumption in modern computing comes from data movement, and in artificial neural networks consisting of billions of connections, data must travel very long distances.
So to find a solution, Daruwala turned to take inspiration from one of the most computationally powerful and energy-efficient machines in existence: the human brain.
Daruwala designed a new way for AI algorithms to move and process data more efficiently, based on the way the human brain takes in new information. The design allows individual AI “neurons” to receive feedback and adjust on the fly, rather than waiting for the entire circuit to update at the same time. This way, data doesn't have to travel as far and is processed in real time.
“In our brains, connections are constantly shifting and adjusting,” Daruwalla says. “It's not like we can just pause everything, adjust it, and come back to ourselves.”
A new machine learning model provides evidence for an as yet unproven theory that correlates working memory with learning and academic achievement. Working memory is a cognitive system that enables you to focus on a task while recalling stored knowledge and experiences.
“Neuroscience has theorized about how working memory circuits facilitate learning, but nothing is as concrete as our rules that actually link the two,” says Dr.
“And that was one of the neat things we stumbled upon here: the theory led to the rule that to regulate each synapse individually, you need working memory alongside it,” Daruwalla says.
Daruwalla's design could usher in a new generation of AI that learns like humans. Not only would it make AI more efficient and accessible, it would mark something of a full-circle moment for neural AI. Long before ChatGPT uttered its first digital syllables, neuroscience has been providing AI with valuable data. Soon, AI may return the favor.
About this AI research news
author: Sarah Jarnieri
sauce: CSHL
contact: Sara Jarnieri – CSHL
image: Image courtesy of Neuroscience News
Original Research: Open access.
“Information bottleneck-based Hebbian learning rules naturally link working memory and synaptic updating” by Kyle Daruwalla et al. The forefront of computational neuroscience
Abstract
Hebbian learning rule based on information bottleneck naturally links working memory and synaptic updating
Deep neural feedforward networks are effective models for a variety of problems, but training and deploying such networks comes at a significant energy cost. Spiking Neural Networks (SNNs), modeled after biologically realistic neurons, offer a potential solution when properly deployed on neuromorphic computing hardware.
Still, many applications require training SNNs. off-linePerforming network training directly on neuromorphic hardware is an ongoing research challenge, with the main hurdle being that backpropagation, which enables training such artificial deep networks, is biologically impossible.
Neuroscientists are unsure how the brain propagates the exact error signal backwards through networks of neurons, and while recent advances have solved some parts of this problem, such as the weight transportation problem, a complete solution remains elusive.
In contrast, a new learning rule based on information bottleneck (IB) avoids the need to train each layer of the network separately and propagate errors between layers: instead, propagation is done implicitly due to the feedforward connections of the layers.
These rules take the form of three-factor Hebbian updates, where a global error signal coordinates local synaptic updates within each layer.Unfortunately, the global signal for a given layer requires multiple samples to be processed simultaneously, and the brain can only see one sample at a time.
We propose a new three-factor update rule for the global signal that accurately captures information across samples via an auxiliary memory network. The auxiliary network is A priori Independently of the dataset used in the primary network.
We demonstrate comparable performance to baselines on an image classification task.Interestingly, unlike schemes like backpropagation, which have no link between learning and memory, our rule shows a direct connection between working memory and synaptic updating.To the best of our knowledge, this is the first rule to explicitly show this link.
We explore these implications in initial experiments examining the impact of memory capacity on learning performance. Going forward, this work suggests an alternative view of learning, in which each layer balances memory-informed compression with task performance.
This view naturally involves several important aspects of neural computation, such as memory, efficiency, and locality.
