Recent studies have shown that shallow brain-inspired feedforward networks can efficiently learn non-trivial classification tasks with reduced computational complexity compared to deep learning architectures. Shallow architectures were shown to be able to achieve the same classification success rate as deep learning architectures, but with less complexity. Efficient learning in shallow architectures is connected to efficient dendritic tree learning that incorporates findings from previous experimental studies on subdendritic adaptation and anisotropic properties of neurons. This finding suggests the possibility of developing unique hardware for fast and efficient shallow learning while reducing energy consumption. (Representation of a deep learning neural network tree.)
Deep learning seems to be the key magic ingredient for realizing many artificial intelligence tasks. However, these tasks can be efficiently accomplished using a simpler, shallower architecture.
According to a study published in , shallow feedforward networks can efficiently learn important classification tasks while reducing computational complexity compared to deep learning architectures.[{” attribute=””>Scientific Reports. This finding may direct the development of unique, energy-efficient hardware for shallow learning.
The earliest artificial neural network, the Perceptron, was introduced approximately 65 years ago and consisted of just one layer. However, to address solutions for more complex classification tasks, more advanced neural network architectures consisting of numerous feedforward (consecutive) layers were later introduced. This is the essential component of the current implementation of deep learning algorithms. It improves the performance of analytical and physical tasks without human intervention, and lies behind everyday automation products such as the emerging technologies for self-driving cars and autonomous chatbots.
Scheme of Deep Machine Learning consisting of many layers (left) vs. Shallow Brain Learning consisting of a few layers with enlarged width (right). Credit: Prof. Ido Kanter, Bar-Ilan University
The key question driving new research published today (April 20) in the journal Scientific Reports is whether efficient learning of non-trivial classification tasks can be achieved using brain-inspired shallow feedforward networks, while potentially requiring less computational complexity. “A positive answer questions the need for deep learning architectures, and might direct the development of unique hardware for the efficient and fast implementation of shallow learning,” said Prof. Ido Kanter, of Bar-Ilan’s Department of Physics and Gonda (Goldschmied) Multidisciplinary Brain Research Center, who led the research. “Additionally, it would demonstrate how brain-inspired shallow learning has advanced computational capability with reduced complexity and energy consumption.”
“We’ve shown that efficient learning on an artificial shallow architecture can achieve the same classification success rates that previously were achieved by deep learning architectures consisting of many layers and filters, but with less computational complexity,” said Yarden Tzach, a PhD student and contributor to this work. “However, the efficient realization of shallow architectures requires a shift in the properties of advanced GPU technology, and future dedicated hardware developments,” he added.
Efficient learning in brain-inspired shallow architectures is closely related to efficient dendritic tree learning based on previous experimental work by Professor Cantor. This learning combines subdendritic adaptation using neuronal cultures with other anisotropic properties of neurons such as different spike waveforms. , the refractory period and maximum transmissibility (see the video above on dendrite learning.)
For years, brain dynamics and[{” attribute=””>machine learning development were researched independently, however recently brain dynamics has been revealed as a source for new types of efficient artificial intelligence.
Reference: “Efficient shallow learning as an alternative to deep learning” 20 April 2023, Scientific Reports.
DOI: 10.1038/s41598-023-32559-8
