Newswise — Neuromorphic embodied intelligence is an emerging field that combines neuroscience and robotics to create systems that can perceive, learn, and adapt through continuous interaction with their environments instead of utilizing cloud-based data. This technology aims to transform healthcare, security, and next-generation AI applications.
We are pleased to invite you to Neuromorphic Embodied Intelligence: From Brain-Inspired Computing to Adaptive Intelligent Systems, featuring researchers who will present their research and answer questions:
Professor Gert Cauwenberghs
IEEE Fellow
American Institute for Medical and Biological Engineering (AIMBE) Fellow
Professor of Bioengineering, University of California San Diego
Co-Director, Institute for Neural Computation, UC San Diego
Professor Gert Cauwenberghs is an internationally recognized pioneer in neuromorphic engineering and brain-inspired computing. Over the past 35 years, he has pioneered the development of silicon integrated circuits that emulate principles of biological computation, enabling highly energy-efficient and noise-resilient intelligent systems. His research has contributed to advances in neuromorphic systems, event-driven computing, human-computer interaction, and wearable health technologies.
Professor Sizhen Bian
Professor, Northwestern Polytechnical University, China
Professor Sizhen Bian received his M.Sc. and Ph.D. degrees from RPTU Kaiserslautern-Landau, where he conducted his doctoral research at the German Research Center for Artificial Intelligence (DFKI), followed by postdoctoral research at ETH Zurich. His research focuses on bio-inspired sensing, neuromorphic perception, embodied intelligence, wearable computing, and edge AI. His work aims to develop adaptive perceptual systems integrating sensing, neuromorphic computation, and embodied intelligence to enable efficient machine perception inspired by biological organisms.
Event Information
Date: July 20, 2026
Time:
- 21:30–22:30 CST (Beijing Time, UTC+8)
- 09:30–10:30 ET (U.S. Eastern Time)
- 06:30–07:30 PT (U.S. Pacific Time)
- 15:30–16:30 CET (Central European Time)
Format: Online Webinar
Jointly organized by the Journal of Frontiers of Computer Science, iGroup Asia Pacific, and Newswise, this webinar will bring together leading experts to explore how advances in neuromorphic computing, spiking neural networks, bio-inspired sensing, and embodied intelligence are shaping the future of intelligent systems.
TRANSCRIPT:
Thom Canalichio: OK, good morning, everyone. Welcome to today’s webinar, Neuromorphic Embodied Intelligence From Brain-Inspired Computing to Adaptive Intelligent Systems. This webinar is jointly organized by Frontiers of Computer Science – the journal, iGroup and Newswise. Today, we are bringing together two distinguished researchers who will share complementary perspectives on the field of artificial intelligence. Professor Gert Cowenbergs from the University of California, San Diego. He’s an IEEE fellow and a pioneer in neuromorphic engineering and brain-inspired computing. Our second speaker is Professor Sizhen Bian from Northwestern Polytechnical University.
Sizhen Bian, NWPU: Hello, everyone.
TC: He focuses on bio-inspired sensing, neuromorphic perception, and embodied intelligence. Welcome Professor Gert Kallenbergs for the first presentation. Go ahead, Dr. Kallenbergs.
Gert Cauwenberghs: Thank you. So, well, I’m very excited to be here this morning or evening. So this presentation will highlight advances in neuromorphic computer memory arrays for massively parallel AI at extreme scale and efficiency. And we all know the advances that we have seen in AI, artificial intelligence over the years, and it has been amazing. Here we see another order of magnitude improvement over a decade and it’s all fantastic. But consider that those advances in AI have come at tremendous costs in the energy consumption and the demand of computation being performed. So that’s just one order of magnitude improvement in the error rates, have implicated about eight or nine orders of magnitude increase in the amount of computation, floating point operations being performed, or equivalently, the power being consumed to the extent that it just to train one model of ImageNet, it now takes about the entire energy budget of New York City in just one month. So it’s totally crazy and, of course, not sustainable in the future. So clearly, a very important aspect of neuromorphic computing, or AI in general, is can we somehow reduce the footprint, the energy footprint, and do it much more efficiently in energy? And then it’s important to consider the, I call here the energy efficiency spectrum. So the horizontal axis here, depending whether you like the glass half full or half empty, operation per second per watt. How much bang you get for the buck, so the throughput versus Power Or equivalently, 1 over that, so a glass half empty it’s the joules per operation. How much energy do you have for each operation? And so, clearly, this is a very vast scale, and so on the limit here, Landauer’s limit tells you for each bit of computation or storage, you require about the thermal energy, K is the Boltzmann constant, T is temperature for each bit, and that’s roughly 10 to the minus 21 joules for each operation. On the other end of the extreme is what is practically achieved still today or maybe yesterday, with CPUs or DDR memory. And I should mention it really the computer for AI is mostly limited by the memory access as opposed to the actual computer itself. So here we’re talking a few picojoules per bit for each memory access in DRAM that amounts for say eight bit of memory for each multiply cumulus, say for a synapse, about half a nanojoule per mac. So clearly orders magnitude worse, right? And the brain is somewhere in the middle, actually closer to the Landauer limit. A femtojoule percentile operation, which is actually very simply arrived at neurons spike about 10 hertz, 10 times per second on average. We have 10 to the 15 synapses, the connections between a neurons and our brain. They’re all active when a neuron gets fired. So that’s 10 to the 16 synaptic operations per second. And our brain consumes about 10 watts of metabolic power, roughly speaking. So the ratio that gives us this is 10 to the 15 operations per second per watt, or 10 to the minus 15 femtojoule kinetics per unit. So clearly, this is great. And how can we get closer to that? Well, all the advances we’ve had over the last 10 years or 15 years have not really pushed us that much further, whether it’s APGs or GPUs, because still they access mostly sequential memory off-chip, fancy high-banded memory is still mostly off-chip memory, and so clearly, So that only gives us a factor 10 improvement. Clearly, we need to do better than that, and so the great advances have been in what’s called compute in memory. Why don’t you bring the compute closer to the memory? In fact, inside of the memory, and that’s how you can really get great advances of massively parallel operations, where the vastly parallel memory itself, will also come with vastly parallel compute all tied together, and I’ll show some highlights how those advances of coding is actually very close to human brains’ performance within Factor 10, with Mamistraf for RM-based computer memory, and I’ll show even where we can beat this further by using adiabatic energy recycling techniques, or you can even recycle some of the fundamental electrostatic energy in the compute. So very exciting. So I’ll dive into it. So neuromorphic engineering taking spirits from the brain of not only compute as the brain is doing in terms of the algorithms, but also adhere to the fundamental physical principles of how neuron synapses physically are organized as a architecture, and also the fundamental principles of, say, the thermodynamics of the compute, in order to achieve those physical realizations of those neural circuits in silicon that they not only perform well in terms of what they compute, but also that are very energy efficient and very resilient to errors, natural errors in in the implementation and the predictable environment in which those things operate through learning and adaptation. So that’s the premise of neuromorphic computing. So neuromorphic stands for neuro and morphic. So you actually morph neural systems into silicon systems, right? And that has been the principle over the last 30, 40 years. Let me just summarize why this is important and how this advances is to great efficiency and resilience, in addition to really great neural computation. Well, for many years, neural systems have been studied by neuroscientists in an analysis framework, basically reducing ever to the least common denominator of complexity and then analyzing that in isolation. Well, engineering is the antidote or the opposite of analysis, right? Of science by synthesizing from the ground up and basically doing systems analysis at each level of scale, where you only take care of that level of scale and then just abstract anything below it, and and so it happens that at each level of scale in hierarchy, from all the way from whole brain systems down to eye channels, and use molecular scale of neural systems, there are isomorphisms, physical isomorphisms, between say what is happening in the biology, biophysics, and then the physics of what is happening in silicon, whole solid state. Even though those media are vastly different, they’re actually quite similar principles at play. And actually, the Boltzmann distribution is at play almost at everywhere here, so we can have those Boltzmann machines, or those Boltzmann energy-based models of say how the brain computes, all the way down into how ion channel operate, where you have ions going to ion channels versus electrons or holes that go through channels of transistors, and so that allows us to build the systems from the ground up, that actually really emulate the physics, the biophysics, using silicon, and that’s the premise of neuromorphic computing. Just to highlight here. So the same exponential distribution of conductance or currents versus membrane voltage or gate voltage versus they all play out whether it’s an ion moving through an ion channel of a through a membrane of a I guess an urine or any electrical active cell, or a equivalently a channel of a transistor in a modern or even ancient MOS, metal oxide semiconductor field effect transistor technology. In fact, the same Boston distribution is given by the energy landscape. The energy barrier that the gate implies for a transport of of ions of this electrons right across a channel plays out also through those channels of biophysical elements in in biology, and so this exponential distribution this exponential distribution has been the founding principle for very efficient analog integrated circuits that operate at extreme efficiency, owing to the six order magnitude of dynamic range that you have in conductance for voltage, and be able to really get massive gain, great efficiency. In fact, that’s the best region of operation of the transistor. Is the subthreshold region of operation for the MOS transistor. That’s also the subthreshold region of operation where many of those cells in biology operate, and so that you can really exploit for doing very efficient computation. Here’s some examples, Silicon retina many years ago. I’m a colleague here, you know, one on now at Stanford, where you map all the functions, the retinotopic functions, on the retina, silicon retina, photo conduction here with those green elements, and then all the circuits around it. You have horizontal cells, bipolar cells, amacrine cells, and then the ganglion cells. Those are the cells that actually spike and then convey the information to the rest of the brain through the optic nerve. So and sure enough, you can build a circuit that emulates spikes, spiking of the optic nerve, and going to the next of the brain that captures spatial-temporal information very succinctly on the focal plane. And so, doing processing on the focal plane allows to do very efficient processing, just like how the human brain is doing it. In fact, very efficient processing on the focal plane is a high mark, for focal plane compression-the same kind of compression that is now emulated, say, with M-PEG, right-but much more efficiently in a spike-based fashion, just with a few transistors on the focal plane. Here’s an example, an actually engineering example: how we have further leveraged that exchange threshold detection CMOS integer. So we’re just adding a few more pixels in addition to the standard APS active pixel sensor that you have in a CMOS integer. Will give you this. Will efficiently encode this temporal difference of information on the focal plane. You can see here clearly on or off events slightly increase or slightly decrease in intensity locally, and that’s enough for you to know what’s going on in the visual scene. This principle of dynamic vision sensors has been highlighted in modern vision sensors, as opposed to just cameras, for as AI front ends, right, for for vision systems, whether it’s vision transformers or others, than today, and allow this very efficient compression for applications, say for surveillance, etc. So this and what is in common between all the systems here, the sensor systems is that they very efficiently encode the sensory world into events. In this case, those events are temporal events or spatial events that encode what is happening in the scene, and those events are then spikes or action potentials. Right, so neurons find action potentials that propagate right from one neuron to the other neuron in the network of two synapses, and so the neuromorphic engineering community has built those those systems not only for doing the sensing, silicon cochlea, silicon retinas, silicon olfaction, etc., but in addition have has built those those processors that are event driven that act on spikes, and they even act based on implementing this added event representation. It’s basically having lookup table to define when a neuron fires. Where does it go in the network in a virtual, virtually wired as opposed to physically wired system, where basically memory encodes the connection strength or connection topologies of the network, Very efficiently, and those are very reconfigurable because you can just change entries in the table that defines how a neuron connects to another neuron by basically changing entries in the table, So you can basically have plastic systems. Plasticity means the ability of the brain to adapt itself over time through activity by basically just rerouting connections by changing entries in the tables. So that’s the principle of reconfigurable synaptic connectivity and plasticity that is being exploited for building very efficient systems with virtual connections, basically implementing the brain at large, almost any connectivity, but basically reconfiguring the connectivity right in those systems. Just highlight a few where we’re going here. So scale is extremely important. We’re dealing here with brains that have 10 to 15 synapses, and so clearly, we need to get two systems that can equate that level of complexity of 10 to 15 synaptic operations per second, 15 watts of power, So, and scaling is actually important, and actually turns out in our advantage. If you look at say machine complexity, what you pay versus cash complexity, what we get, so right there’s a thing of there’s the bug for the bang, on a log scale. So digital systems, AI, classical old school AI, has had this curse of dimensionality, So in which harder problems become even more tremendously harder for the machine, and then scale poorly, less worse than linearly and that’s why we have for many years had the curse of dimensionality. It was getting more difficult once you get really hard, interesting problems. Whereas for analogic rather than logical compute computation, where you do approximate massively parallel pattern-based processing that is analog and massively parallel, as opposed to just purely digital and sequential, you get rather than the course of dimensionality, you get actually I would say the blessing of hyperdimensionality. The higher dimensions you go, the more the better it gets through the central limit or anything of that, Basically, statistical principles. So you can get better performance, Better approximation. approximate reasoning, the more complex things get, and so that is the premise of neuromorphic computing or or this analog based style of computation, And so that have been been pushed forward with the modern AI, this I would say neuromorphic AI that uses deep learning to abstract lower dimension representation of the world, and then neuromorphic engineering that gets us at this really large scale systems. About two principles. If you want to minimize energy per task, it’s simply decomposed into you minimize energy per operation and operations per task, and so those two just beautifully partitioned into energy preparation is minimized by just building these computer memory architectures that do more for less. So that basically just a femtojoule of operation percent compression versus the 10s of or hundreds of picojoules per mole to accumulate in typical digital accelerators and operation potassium is also minimized by this deep learning, basically having better approaches for doing learning. So those two need to come together: those advancing the technology, for neuromorphic compute, as well as also advancing the algorithms and the models, for handling the complexity right, and I’ll just briefly highlight those two complementary perspectives of how deep learning, which is the green, and the neuromorphic compute, which is the blue here, work together, and so it just so happens that multiple disciplines all come together for doing so, whether it’s for getting this systems that can scale to 10 and 15 synapses, At just 15 watts of power, you want to get truly great efficiency in massively parallel systems and 3D integration of memory on top of silicon that can really get you those densities, and then, so we also want to route things in a rhetorical fashion for actually scaling this, and in the on the complementary side of neuroscience, equitable science, cognitive science, we have to then use handle complexity in very efficient manners as well. So here’s some a snapshot of I wouldn’t say it’s latest, but actually still pretty much the state of the art in large scale reconfigurable neuromorphic computing. Those are systems that are event driven, handle spikes events, and that can be wired dynamically, as I mentioned, right through this event based representation. And so here you see that we have many systems where it’s digital or analog. You recognize Spinnaker, Spinnaker Two. Loihi Intel. Loihi is now out. TrueNorth. There’s now NorthPole from IBM, and then there are all the analog systems, right? The FACETS, BrainScaleS, Heidelberg systems, Stanford NeuroGrid, our own work of the integrated five-array transceivers, IFAT at UCSD at In Zurich, there is the Roll Systems, And they all are advancing over digital accelerators. And the key here is that energy efficiency semantic operations per second versus watts or conversion, as I mentioned, joules per centimeter vents. So it’s typically in the order of a few picot, a few 10s of picojoules per synaptic operation. Is that good? Well, that’s already a lot better than than closer to what the brain is doing, but still far away. And I’ll highlight how we can actually do even better than that, right? And so there are two aspects of doing better. First of all, you need to scale, and scaling is not just a matter of building large scale, massively parallel systems, but also you want to make sure that messages can actually come across. This is Amdahl’s Law, actually addressed by the white matter and the gray matter of the brain, So, sure, we want to be able to connect very densely, and that’s the flexibility of the gray matter, A synapse is very densely connected with arbitrary connectivity, but you also need to be expandable. You want to make sure that each neuron, in principle, many neurons in the brain connect very far away. Not many, but quite a few connect over long distances. You have this small world model, where everyone is in the world is connected about six steps away from anyone else, So same is true in the brain. So you have this hierarchical, this structured hierarchical topologies in the brain that are tree-based, Structured that allow you to hop into just a few logarithmic steps, right, to the brain for reaching almost any neuron, And so you need to have both this tree-based topology as well as this arbitrary connectivity, and that’s what this hierarchical additive and routing does. So we have the white matter efficient implemented, this long range connectivity, very implement very richly implemented through this hierarchical adhesive and routing schemes, and then you have this local flexibility implemented, with this dense crossbar arrays of synapses. So here it is shown how this works. We have those systems implemented in silicon that implement neurons, and then we have synapses implemented in memory around it that implement this reconfigurable connectivity. But this is done in a hierarchical fashion, So when on this chip, when neurons fire, they fire extra potentials. They they go out, and then the externally we have this memory access right through this FPGAs here that go to these memory tables in memory that define where that event reaches other neurons in the in the network, and those events are then again transmitted, going into the chip, and then there is that is encoded and then transmitted to other neurons in the network. So we have this arrays of neurons that are addressable, and we have these input events and output events, synaptic input events that define inputs of the neurons, and then neural output events that define where once the neurons spike where they go. It’s important to know internally everything is analog. Those neurons are analog, but externally we have visual an interface where we have spikes in and spikes out that are asynchronous digital events. Right. And this is a hallmark of neuromorphic compute, where you can combine, you can do very efficient communication in digital, while internally you have very efficient compute in analog. It’s all great, except this consumes again, as 10s of picojoules of energy, because you mostly go outside for accessing memory, and that’s the key here. So how do you do better than that? Sure, we bring the compute closer to the memory. In fact, as it happens, modern memory today, this whether it’s phase change memory or RAM, which stands for resistive random access memory, or M, So magnetic MRAM, magnetic RAM, those are conductance-based memory technologies that are non-volatile and extremely efficient and dense. That are now leading advances in memory technologies today. As it so happens, a conductance-based connection is also a synapse. So for every bit in this conductance-based memory that you have now, this whereas PC RAM or RAM, you can actually recycle and reuse this same element rather than just one bit in a memory that is just serially accessed as a synapse in a neuromorphic network, where the word line is basically your action potential, presynaptic action potential that drives this conductance here, this and then and the bit line that’s your dendrite that accumulates information right from from all the synapses, so you can just accumulate and then basically voila, there you have a biophysical synapse. So each memory element, each bit of a memory in a digital modern memory, is actually dubbed as a full biophysical synapse in a massively parallel network, and that’s exciting because now you can build networks, crossbar arrays of those synapses that can be massively parallel. Right. So you have massively parallel events coming in along, say, horizontal lines, and then along the vertical lines, there you get your dendrites that accumulate right information. So at each junction, you have this conductance that is activated, So physically, I complement these 3D structures where you can implement synapses, right, in a very efficient manner, right, where you have massively parallel operations for a matrix now of n inputs and n outputs, you have n squared operations all done in parallel, in in zero time, in zero order time, as opposed to typically in a digital compute, you need to go through several for loops here. You can even do learning. You can adapt synapses locally based on these biophysical learning rules, such as spike time and dependent plasticity (STDP) right for doing learning that is biologically motivated. Basically, only learn when the preston of the action potential arrives before the prosynaptic action potential, and otherwise you depress the synapse. Right? And it’s kind of it’s logical from a causal point of point of view, and so we build systems right that can scale where we have those crossbar arrays of synapses directly right locally implemented on top of arrays of those neurons, those invitation fire array transceivers as I mentioned, and now that can be one core architecture, and then we have this hierarchical arse event routing scheme to allow you to that send messages across the network of cores in a reconfigurable manner, right? Where these routers can be reprogrammed for implementing almost any connectivity. So this allows us to build large-scale systems that have this long-range connectivity, while locally have this very dense gray matter right connectivity. So why then gray matter going together in very efficient matter. So here is an example of how meant that those systems neuroram is the first CMS innovator RM compute a memory chip that combines three hallmarks of where we want to go with these AI accelerators: high versatility and diverse architectures, record efficiency and high inference accuracy comparable to software. Right? Normally, you would think to build a very efficient hardware is going to be very dedicated to one application. No, no longer. You can actually have reconfigurable, right? Fully programmable adaptable architectures. You can do almost anything as microprocessor or GPU, but now with much greater efficiency owing to this computed memory architecture that are implemented. And so, yeah, you can get now down to 1300 joule per operation, which is now factor 10 or less than factor 100 from the efficiency of the brain, owing to this architectures that are actually only transposable, in which you can change the directions of of your connectivity by locally switching activity in of the neuron between rows and columns architecture, you can actually transpose the connectivity, which means you can go, you can backpropagate, you can go back from outputs to inputs in your network, which is extremely important for learning, but also for some other applications. I’ll just this, I’ll just skip the circuits, but we have very efficient circuits for implementing those crossbar arrays and the neurons that implement these connections, this rewritable, transposable architectures, and so this transposing of the direction of the flow, right from inputs to outputs, back from output to inputs, is extremely important not only for learning, but also actually for generative AI, which is now the hallmark of modern AI, right? Where you can just flip the direction of basically going from your visible units to the hidden units. You can go back from hidden units to your visible units in network, and this allows you to generate to dream, right? To come up with new representations of in the pixel space in the pixel space that basically can fill in the blanks or basically can dream or or diffuse I guess information that is missing and that’s the hallmark generative AI is of course the modern foundation of AI, whether it’s Stable Diffusion or modern transformer architectures, and they’re all based on this principle. So you can build this very efficient hardware. I’m just highlighting here. For instance, not only can you discriminate and detect, say, this six, but you can also then, so when you corrupt the image, you can actually reconstruct it, right? And so this reconstruction capability is, as I mentioned, the hallmark of generative AI. You can fit distributions in space as opposed to just discriminate, and this allows you to do really great Bayesian inference and do all the great stuff that we’re seeing now happening in AI. And so we’ll do all this. It’s all mapped through this very efficient layered architectures, where we have this. We can map almost any architecture, whether it’s a deep layered neural network or it’s a LSTM, right? Long short term memory, or a Boltzmann machine, or generative AI, or auto encoder. Those are all mapped by this mapping architectures onto this tiled array of this hierarchical event writing tile array of cores in this architecture for doing almost any architecture, and then we got around the imprecision, the analog imprecision of those networks by basically doing careful modeling of the hardware. But very fundamentally, we were able to actually through a learning process, chip in loop learning, we’re able to fine tune the performance, and recover the imprecision of the analog hardware purely by learning one layer at a time, and then programming it just one layer at a time. Basically, each time then each new layer by retraining that work, we’re able able to recover the imprecision that was in previous layers owing to the the massive parallel distributed representation in the architecture, where it can still take advantage of the the large dimension they have of the architecture to overcome the imprecision of its own environment. I just highlighted just I’ll just wrap up mostly here. But so in addition to this resistive random access memory technologies, whether it’s I mean conductance based synapses, whether it’s RAM or PC RAM, right? These conductance based memories. We also have memories now that can be just purely charge based, and the nice thing about charge as opposed or capacitance as opposed to current, right? Or or conductances, is that you can actually really recover the energy that it takes for doing the compute, owing to these adiabatic energy principles, right? Because the capacitor itself doesn’t consume energy. Resistance conductances consume energy, but not capacitances. And so, even further, we can get rid of noise because you can actually avoid KTMC sample noise in the process of doing this compute. The principle is actually quite simple, right? You have these crossbar arrays of now charge elements, charge coupled elements, or charged injection device elements, just like in a DRAM where you store charge, but rather you store your charge. Actually, use a charge in the computation process where you can move selectively move the charge from your input line to your output line, capacitively coupled, purely right in this process. Right, so this is a non-destructive process. You just move the charge and get it back after right when you bring the voltage back down. This transfer of charge not only is charge conserving, but is even energy conserving if you do it carefully. If you drive the lines very carefully, in fact, it’s very simple principle. Most CMOS circuits consume dynamic power due to the fact that the charge that is needed for charging capacitor, the output right, comes from one supply and is returned to the other supply, two static supplies. So you lose entire voltage, entire potential in the process of one cycle in your compute. If you use a hot clock rather than having DC supplies, but have a hot clock where you basically alternate the voltage right on this hot clock line, you can actually recycle the energy. But because rather than returning it to different supplies, you actually adiabatically drive your lines right with zero voltage across the switch here that is driving these outputs, and there’s a principle of energy recovery logic that has been used for many years. It’s called adiabatic digital compute, and there’s now we’re been researching this for for analog compute operations, where basically we have this this array of charge-based elements that are now driven with these IRL drivers, coupled with an external inductive tank, and then this right, so inductive inductor, and so this tank of this inductor and this capacitive array here of of synapses is at resonance. We recycle right. Basically, we go back between electrostatic and magnetic energy. We recycle almost all the energy. Almost 99% of the energy is recycled, or 98% is recovered, which means we operate in here at less than a femtojoule of energy per multiplied cumulative. In fact, our latest work show we can actually go down to 10, 10 to 100 attojoules per multiplied. That’s a world record. We’re able now to compute right at a fraction of the energy that that any digital or even analog system can do today. Very exciting, and even for communication, we can use the same principle by recycling the energy that this takes right for sending a bit over a wireless link by us again closing a switch only when the voltage across it zero. Very simple principle, and so this way we can actually send data wirelessly at just less than around a femtojoule of operation of bit being communicated wirelessly. So that sounds super exciting. So just the many exciting things. I haven’t said much about the actual algorithms, but I’ll leave that to some other experts in the field. But yeah, AI is really great today. You know, deep learning and important thing to note here is we can even have learning algorithms that exploit the natural stochastic nature of those synapses, right? And so this way can actually do compute in a statistical fashion that is actually extremely efficient. We have put all the resources available to the community. You can go online and have this compute in memory architectures with FPGAs implemented. Implement your Python code for doing deep learning on this system with 40 billion synapses, 160 million neurons over the neuroscience gateway. We call this the higher spike system. Anyone is more than welcome to use it. And I’ll stop there. Sorry, I probably went over time.
TC: Okay, thank you, Dr. Cauwenbergs.
GC: I should mention that I have I’ll share this, but all the references are also included here. I’m happy to share those slides whenever.
TC: Okay, great. Thank you. We’ll invite now Dr. Bian to present about his work.
SB: Wait a moment. Let me share the screen. Can you see the screen?
TC: Yes, we can. Thank you.
SB: Okay, great. So, thank you, everyone, and I’m actually very happy to share some thoughts after years of work in the neuromorphic field. So the title is for me is neuromorphic sensing for embodied intelligence, and this title is recommended by the opening committee, and I like it a lot because it matches my work very well. So the central idea of this talk is simple. So if we want the machines to behave intelligently in the physical world, and we cannot only build better artificial brain, we also have to build a better sensory system. So that is my claim. So my work started from embedded neuromorphic algorithms, especially the concrete system and some applications platforms built on top of it. But over time, the question shifted. So I became less interested in asking only how efficiently we can compute, and more interested in asking what form of information should enter the computing system in the first place. So let’s start with this ghost knifefish. So while most fish rely on the slight sight or smell, knife fish use electro location, so they continuously generate weak pulsing electric fields around their bodies and use receptors of the skin to detect the disruptions caused by the nearby objects. And because of this unique ability, knifefish actually displays several specialized smart behaviors. First, the instant spatial awareness, so they can pinpoint the exact distance, size, and use the material of objects in the murky waters. And second, the omnidirectional movement, so their unique rib-like phase allows them to move forward, backward, and also cover with incredible agility, and third is the complex social communications. So they use those electric chirps to communicate with mates and avoid jamming each other’s electric signal. So there is a simple question: Why is a knife fish intelligent? Is it because it has better brain, or because it senses the world differently? The common intuition that intelligent behavior comes from the powerful brain. That is partly true, right? But it is incomplete. The brain does not receive wrong pixels, raw pressure waves, or the wrong chemical concentrations. So before the signals arrive in the central nervous system, peripheral structures actually have already imposed structure on the world, and they are not generic analog to digital curvatures. So they use frontend computational angles, and they decide what is silent and what can be ignored. This is a biological exploration behind this talk. So first, the intelligence never begins with spray; it begins actually with sizing. And second, the brain does not receive the wrong data, and it receives already structured events. So this slide shows the arc of this talk, right? I will begin with conquing and some conquer-based embedded edge systems because they were my entry point into the neuromorphic engineering at ETH, so those projects looked like hardware projects, but for me, they created a system level question. So what does it mean to build and to build an efficient perception to action pipeline? Then I will move to event cameras and low power eye tracking project, where the sensor itself already performs an event based transformation. From there, I will generalize the question beyond vision. So, if a dynamic visual sensor can be beyond inspired, why not the other sensor in front end? So, I will argue that decoding is not a preprocessing step; it is first layer of intelligence, and finally I will close with a broader vision. So beyond inspired assessing as underestimating front end for embodied neuromorphic intelligence, and here are some images of my journey of the neuromorphic field. So. Kraken was designed around hybrid ideas. So on one side there is event-based perception and spiking neural etching, and on the other side there is frame-based perception at conventional neural network accelerator. So instead of assuming that one paradigm replaces the other. The system asks how both can coexist in a low-power embedded platform. So this matters because embodied systems are messy, actually, right? A draw a variable system may need fast event-based reaction, but it also may need richer frame-based context, so Kraken was not really a chip. It was question about the whole loop. So we built the ColibriUAV. This is platform by integrating the event and frame interfaces and also the processing the accelerators into a complete embedded system. So the scientific lesson for me from this project was not that neuromorphic engineering was that neuromorphic engineering cannot stop at the neural model of the accelerator block, the event sensor interface, and the data movement path, and also the representation, some part of the architecture. So once you see the full path, sensing is no longer outside the computing problem, so it becomes the first architectural decision. And here are some hard numbers about the ColibriUAV. So I will just skip this end-to-end evaluation, and after those projects, I started to reinterpret what I had been building. So the more I looked at complete systems, the more I realized that the first bottleneck is often the information stream, like what is being assessed and what is being transmitted? Does the representation match the physical dynamics of the task? Those questions came before the model architecture, and they also determine whether the energy efficiency of neuromorphic processor can actually be realized. So, if the sensor produces the wrong representation, the processor is forced to spread energy recovery structure that could have been created and thwarted. So, event camera reports look brightness changed synchronously. This that means that the sensor itself has already made a decision about what they what deserves representation, so this is why event camera are so central to neuromorphic sensing. So they show that a sensor can embody an algorithm. It can transform the physical world into a temporarily precise event stream before the processor sees the data, for neuromorphic engineering, this is powerful because it lies the sensor output with spiking computation, and the sensor and the processor speak a more similar language, which can reduce latency and energy while preserving the temporal structure of the physical world, and the retinal eye track project is concrete example of this idea. So it used event camera, a multi-all glasses, out glass, and a spiky neural network regression model deployed a low power neuromorphic processor. The result was an end-to-end pipeline with power around, if I remember, just five milligrams and latency around six milliseconds. So the important part is that the whole chain is neuromorphically split, like the sensor is event-based and the model is spiking. Also, the processor is designed for low-power event processing. So this makes the system more than a compressed neural network, right? It is assessing computation pipeline where the representation is already adapted to the dynamics of the task. So this brings me to one of the most important sentences of this talk. Every sensor deserves its own neuromorphic representation. Event cameras are the most mature example because dynamic visual sensors have a clear biological analogy, but embodied systems do not live our vision live by vision alone. So for each modality, we should also ask what the equivalent of a meaningful event might be, like is it a change, maybe a threshold crossing, a temporal pattern, a frequency component, a special contrast. So the answer will not be identical across sensors. That is exactly the point. The neuromorphic sensor is not a universal format, right? It is design. It is a design of modality specific representations for efficient embodied behavior. So once we generalize event cameras, we arrive at encoding for dynamic vision sensor. Part of the encoding is built into the pixel circuit, but for many other ubiquitous sensors, we need to decide how to convert those signals into spikes or events. This may sound like technical preprocessing, but it is much deeper than that. Equity decides what deformation is preserved and what is compressed, what time precision is kept, and finally, what the downstream spiking neural network can learn. So it also determines the firing rate and therefore the energy cost of the neuromorphic processor, in other words, encoding is a scientific problem. It links the the physics of the sensor to the dynamics of the neural model and finding to and also to the cost trace of the hardware. Here, I want to make a claim that encoding is not preprocessing. It is the first layer of intelligence. In many machine learning pipelines, pre-processing is considered as a necessary step, but uninteresting step. But in neuromorphic system, the encoding is part of the computation. It determines the spiking, spike timing, spike density, temporal precision, robustness to noise, and also the information loss. So actually, the biological sensory systems shows this very clear, clearly, and they do not passively digitalize the vote. They convert the physical stimuli into the neural codes that are already adapted to behavior, so if we want to build efficient embedding embodied systems, we need to take encoding as seriously as we take the model architecture. So a useful way to think about encoding is to look across the biological modalities. Right, the retainer emphasizes spatial and temporal contrast. The cochlea performs a mechanical frequency decomposition before the auditory signals reach higher processing centers, and the vestibular system includes acceleration and orientation in ways that are useful for balance and action, so those systems do not use one universal representation. Their codes are shaped by the physics of the stimulus and the behavioral needs of the organism. So this is a biological argument for sensor-specific neuromorphic encoding. Instead of forcing every signal into the same sample data format, we should ask what neural code makes sense for that modality at task. So to study the coding, we used the inertial sensing as a case study. Unlike the event camera, the IMU sensor does not naturally output spikes, so the question becomes: How should this signal be represented for a spiky neural network? Should we include the values as the spike rates? Should we include the timing with time to first spike? And should we emit spikes only when the signal changes? So each choice creates a different information stream. It changes the fiery rate, the temporal structure, and the robustness to noise, and also the cost of deployment. Actually, so this makes IMU-based activity recognition an ideal less for understanding encoding as a design problem, so the four inclusive schemes that we evaluated can be understood as four different beliefs about time. So, rate including says that information is represented by spike frequency over a window, and it is often accurate, but it can require many spikes. Time to first spike encoding says that information is represented by when the first spike occurs, and this can be very sparse and low latency. But it may also be sensitive to spike errors. And binary encoding maps values into binary channels and create a structured, discrete representation. And the auto modulation says that spikes should occur where the signal changes, making it closer spread to the event-based setting. Those are not just technical variables; they create different meaning for spike, so the spiking neural network receives a different vote depending on the encoding. And on the neuromorphic hardware, spike density and timing as a workload, and more spikes generally means more events to the routine and more neural updates and more energy, so the encoding scheme therefore determines whether the hardware can express its advantage. A highly accurate encoding that produces too many specs may erase the benefits of event-driven computing. A very sparse encoding may be efficient, but also may be too fragile. A practical design rule from this work is to begin by asking what spike should mean. So, if a spike is evidence accumulated over time, red encoding may be natural, right? If a spike should communicate should communicate values as quickly as possible, so the time to first spike may be appropriate. So this question is more fundamental than choosing the right spiky neural network architecture. It links the sensor physics to the downstream computation. and this leads to the idea of peripheral intelligence. So in a questionnaire pipeline, the sensor measures and the processor computes, and finally the model decides. But in a bio-inspired pipeline, computation can be distributed across materials, circuits, encoders, and neuromorphic processors. So a material may respond selectively to physical stimulators. A encoder may preserve timing and express redundancy, and a processor may integrate events into decisions. So this distribution is necessary for systems that must be fast and energy efficient. So, if we push all those intelligence into the central model, we actually pay for the data movement and redundant computation. So, at system level, this means that neuromorphic engineering should be understood as sensor to action nervous system design. The sensors, the encoders, the processor, the memory, the communication path, and the actuator should share a timing model. So, if a sensor produces synchronous events, but the rest of system batches them into a large video, a large video, and so the temporal advantages may be lost, right? If the processor.
I think we lost Professor Bian. Professor Cauwenberg, can you hear Professor Bian?
GC: No, I cannot.
SB: I can hear you.
TC: We lost your audio for a moment, Professor Bian, but please go ahead and continue. We’re hearing you again now.
SB: Hello, can you hear me?
Yes, perfect.
SB: Now you can hear me, right? Am I right?
Yes, yes.
SB: Okay, great. Yeah. So for artificial embodied systems, we need similar disciplines. So represent only what matters, preserve timing weight matters, and the loop with minimal waste. And in the context of this workshop, so Professor Cauwenbergs has presented fundamental contribution to the neuromorphic circuits, and my talk is complementary. So rather than focusing on how to build the artificial nervous system in silicon, I’m asking what information should fit that nervous system. So better brain -like processor needs a better body-like sensor. So to make the message concrete, I would summarize the edge the five principles. So first, start from the physical stimulus and the behavior, not from the model. And second, design the representation before optimizing the neural network. Third, evaluate the for loop, including the sensor readout, encoding, inference and latency. Fourth, treat robustness as a property of the sensory code, not only of the classifier. So fifth, the distributes intelligence into the front end energy latency matter, so those principles are simple. They change how we build system. They encourage us to ask whether a sensor should sample densely, emit events, depth thresholds, equal timing, or exploit material properties. So let me now connect this talk to the research program I’m building next. Next, so if we if we take the idea of bio-inspired sensing seriously, then an encoder should not be a fixed block placed before a neural network. So in most artificial systems, however, this is exactly what happens now. So we choose the coding rules and set thresholds, fix the time window, then train the network, and then hope the same represents good in every situation. But biology doesn’t work like this. The neural coding is modulated by task adaptation. So the same sensory pathway can become more sensitive, more selective, or more sparse depending on what the organ needs to do. So this creates a concrete scientific gap in low load dynamics. See a fixed encoder may generate two field events and lose useful formation. In high dynamic C, encoder may generate an event flood, causing the computational load and energy to rise sharply. The problem is not simply which encoder is best? The deeper question is whether encoding can become a controllable system variable. And now let me return to the sentence from the beginning. So for decades we have invested tremendous effort in building better artificial brain. That effort is valuable and it will continue, but in biology intelligence. never begins with brain alone. It begins with sensing. It begins with the body meeting the environment and transforming the physical interaction into meaningful neural events. So, artificial agents that are fast, efficient, robust, and embodied, then smarter algorithms are not enough. We need sensing. We need sensors that do not merely capture data but shape the information, and we need encoders that do not merely convert formats but increase useful neural representation. So thank you very much for the attention, and I would be very happy to discuss any part of this talk.
TC: Thank you, Dr. Bian. Eric Tu will now moderate the Q and A portion with our professors to talk further about the research. Eric,
Eric: Thank you, Professor Bian and Professor Cauwenberghs for the insightful presentations. Now let’s quickly just move into our discussion session, focusing on still on this topic. I would like to start with a question for Professor Cauwenberghs. So many AI systems today are becoming increasingly powerful, but also increasingly energy intensive. So, from your perspective, how can neuromorphic computing fundamentally change the future of AI?
GC: Excellent question. Of course, this is the core, one of the core questions, right? So, how can we curb the amazing appetite of AI systems? And I wouldn’t re distinguish AI and neuromorphic. I think they’re all in the same, the same, the same family. And I think that’s why I think the future of AI would be has to be neuromorphic. It has to address this extreme energy efficiency. And so I gave a versatile metric: energy per synaptic operation, right? You can call that just energy per mac, whatever it is. But and that’s the key in these computer memory systems, right? We can really push that efficiency from a technology point of view. Of course, there’s also the complement, as I mentioned, right? Having more efficient algorithms, but I think if history is an indication. Typically, when technology advances, when you get more for less, of course, you want to get more, right? So chances are that what we end up having is that as we can go, as we get say 1000 factor improvement in efficiency, rather than bringing the energy down by factor of 1000, most likely all we have is we have 1000 more times more compute for the same energy, right? So we have to because it’s more we have to exercise discipline in the use usage of our energy resources. But we as scientists and engineers will just keep advancing and lower the energy footprint. We definitely can do so. I wish you to do so.
Eric: Yeah. Yeah, thank you, Professor Cauwenberghs My next question is for Professor Bian. So your presentation emphasized that intelligence does not begin only with computation, but also with sensing. Right, embodied intelligence requires continuous interaction with dynamic environments. So, what role do event-driven sensing and spike neural networks play in enabling real-world intelligence?
SB: So, I would say that they play two very different but deeply connected roles. So, actually, events. driven sizing changes what the system receives from the vote, right? And instead, instead of something everything at a fixed rate, it reacts only when something meaningful changes. So this is much closer to how biological accessory system work. Our eyes, our skin and nervous system, they are not passive cameras frame by frame. They are constantly selecting and filtering the the information. And for the spiking neural network, it provides a natural computational model for processing this kind of sparse event-based information. Right, they allow the communication computation to happen when the event arrive, rather than at every clock cycle. So this is important for embodied intelligence. It must perceive, decide, and act continuously, and often under strict constraints of power, latency, and robustness. So for me, the key point is like this: so event-driven sensing and spiking neural networks. They are not only tools for saving energy; they change the structure of perception itself. Right? They allow the intelligence to become more tightly coupled to the dynamics of the environment of the body. Let’s say, yeah, yeah. That’s my opinion for this question, yeah.
Eric: Thank you, thank you, Professor Bian. Now I would like to invite both speakers to discuss on more broader questions. So, Professor Bian, I’ll just let you lead the following talks with Professor Cauwenberghs, if that’s fine with you.
SB: Yep. Sorry, I ask you a second question for from me.
Eric: Yeah, I think um next next session will be you leading the discussion with Professor Cauwenberghs on the topics. Yeah, but if you want me to also bring some questions, I can also I can do as well. Yeah.
SB: Yeah. Maybe I can make this question to Professor Cauwenberghs like so the current large AI models excel at reasoning, right, and with new, but neuromorphic system only emphasizes efficiency at real time adaptation. So, do you see those two paradigms and like as competing approaches or some other rules? It is.
GC: Yeah, no. This is a, of course, a key point you’re raising here, Sizhen, that you already mentioned before, right? So that we are embodied, our systems, right? Our brains are not isolated. We live in a world, right? And so the interface between sensing and compute and action is extremely important, and of course, their reasoning is important because you need to make important decisions to guide your. I mean, after all, we have to survive and thrive. So I don’t think it’s any competition, and so it has been a myth that neuromorphic computing has been purely about lower the energy or doing things with just spikes or modeling how the brain works, is a lot bigger than that. I would say, at a time when AI became big, which was about 15, 10, 10 years ago, that time that was a time. This was not resurgence of AI. What it really was is the resurgence of neuromorphic and and neural approaches. It was in fact neuroscience that started it all, right? AI in old school. Old school AI was about symbolic reasoning, purely that, right, and nothing else. And so then, what came into the mind literally is understanding of say how our brains operate, and of course reasoning is important, but is the reasoning is very different from the reasoning that we have in this symbolic reasoning. Reasoning also involves this perceptual capabilities, right? For instance, how do you play the game of Go? So everything is right. So you have to really look at the board and understand the gameplay, the strategies, right? And so this perceptual reasoning has been augmenting this symbolic reasoning, and I think this is really what made it go, and literally what made it go. And I think we’re getting more this convergence. I would say today’s AI is at least as much neuromorphic and neuroscience than this AI. And we should definitely the synthesis between us two is becoming extremely important. And I think that’s why we’re making such great progress. It’s not by chance that the founders of DeepMind-they’re actually neuroscientists and they fully understood the layer representations in cortex, the reinforcement learning. What is reinforcement learning? It’s really about understanding the basal ganglia, the reward conditioning, reward conditioning in basal ganglia, right? So, so brain science has already been driven latest advances in AI by combining this. This, I mean, the different perspectives that are extremely important. So, yes, absolutely, those things come together, and we have to keep going in that joint direction.
SB: I would also like to share some thoughts from myself, I actually worked on many neuromorphic processors like the Loihi 1,2 and also some neuromorphic edge processors from The Innatera’s group. I joined the development, and also some neuromorphic edge processors from some startups in Swedish, so so my my feeling is that so so like Spiking Neural Network at Eric’s some site actually it works the progress is very good right even there’s SpikeGPT right from the group of I forget the name of the University but the American.
GC: Yeah, very impressive. But the problem is that the problem is that to show to present the power of neuromorphic, you need to you really need a very good processor, right? And my experience when developing those general neuromorphic platforms, all those age specialized neuromorphic processors, is really hard actually, and this is first problem. And second problem is that I think we for this community we really need to find a very very good application. Very very good. Like the eye checker project is very good one, a super good one. And that we use a very small event camera, DVS camera, and the eight neuromorphic processor, and really achieves the solar the pupil location latency and accuracy, so this really needs some very good applications. And so let’s go back to those questions. I think that those two paradigm, the current AI models and neuromorphic those projects, they’re not competing with each other, I think they’re playing a complementary roles in just different places.
GC: Absolutely, yeah.
SB: Great. Maybe any questions from the audience or some questions that Professor.
TC: Thank you, Dr. Bian. We do have a question from the audience. Hang on, just one second while I pull that up here. If you have any other questions, audience, please feel free to chat those here in the Zoom chat. From Tam Lee, she asks: You mentioned the issue of an event flood in high dynamic scenes, how do you envision the feedback mechanism from the processor back to the front end encoder operating in real time to dynamically adjust thresholds without losing salient information?
Another question for you.
SB: So this is ongoing project. This is ongoing project. I’m still trying to. So the background of this project is that I read a lot of papers. Those papers were written many years before about those biology, those perception of human neuros. Actually, those the how our neuros the neural perception perceptive neurons like the ski neuros they actually encode the physical signals in different ways. Sometimes they use red encoding, and sometimes they it will be changed to like the other form of encoding. So spike traits generated from the ski neuros they they’re not fixed. So you could use methods are not fixed, and so I’m thinking about thinking how do we how do we simulate this this this mechanism in math in a mathematical way, and try to build like adaptive encoding approach, and so, and also following this adaptive encoding, we also need to need to build like the encoding resistive spiking neural network. So this spiking neural network need to be able to deal with encoding approaches. The answer for to this question is that actually I don’t know. I’m still trying to link the spiking the adaptive spiking neural network and the adaptive encoding. How to link to these two sides? I’m still trying to figure it out. This is a very good question
GC: from my perspective. So there are hybrid systems today. First, I mentioned dynamic vision sensors. There are hybrids between dynamic vision sensors and this standard APS or active pixel sensors, Where to choose, and so of course, if there’s a burst of events, if everything, let’s say, have an entire new scene, Of course, then it makes sense to have a frame, and rather than just keep sending events, event processing is only efficient when you get sparse events, And so when you get a burst, it may actually be good to combine this. This, I guess, more frame-based as opposed to just event-driven representation. But yeah, those are kind of simple approaches that can, again, combining AI and neuromorphic in ways that can benefit both is a key for making advances in many of these.
SB: And also, I remember that there’s paper, a Nature paper, a Nature Communication paper, just published last year, 2025 So that paper build a encoding mechanics that has two encoding team approach. Like I say I think it was rated encoding at a time to first spike, so they can choose that new morphic processor can choose can choose these two kinds of encoding, but actually still that is not adaptive. So during the real time spike encoding, the encoding is still fixed. Only before the data processing, the spike processing, you can choose you can choose encoding. But it’s still good to see that this direction has some progress. Yeah, and I believe there will be some more interesting work in following years.
TC: Okay, thank you, Dr. Bian and Dr. Cauwenberghs. Just to draw our presentation here to a close. Pardon me, I wanted to share my screen here full screen with this image. In conclusion, for today’s webinar, I’d like to mention that the journal here, the Frontiers of Computer Science, is planning and producing to publish a special edition of their journal on neuromorphic embodied intelligence. The special issue aims to bring together the latest advances and emerging trends in this rapidly developing field. And researchers working in related areas are invited to contribute their latest research to that special edition, including Dr. Cauwenberghs and Dr. Bian, thank you to all the participants for joining today’s webinar. Our speakers, Professor Gert Cauwenberghs and Professor Sizhen Bian. Thank you also to the Frontiers of Computer Science, to iGroup for putting on this event together with us here at Newswise. We hope today’s discussion has provided new insights into the future of neuromorphic intelligence, and that you found this useful. Thank you very much, everyone. That will bring today’s presentation to a close.
