Neural networks made of light could make machine learning more sustainable

Scientists have proposed a new way to implement neural networks in optical systems that could make machine learning more sustainable in the future. Researchers at the Max Planck Institute for the Science of Light have proposed this new method. Natural Physicswhich shows a much simpler method than previous approaches.

Machine learning and artificial intelligence are becoming increasingly prevalent in a variety of applications, from computer vision to text generation, as ChatGPT shows. However, these complex tasks require increasingly complex neural networks, some with billions of parameters.

The rapid growth in size of neural networks has led to exponential increases in energy consumption and training times, putting the technology on an unsustainable path: For example, it is estimated that training GPT-3 consumed over 1,000 MWh of energy, equivalent to the daily electrical energy consumption of a small town.

This trend has created a need for faster, energy-efficient and cost-effective alternatives, spurring the rapid development of the field of neuromorphic computing, whose aim is to replace the neural networks of digital computers with physical neural networks that are designed to perform the required mathematical operations physically, in a faster and more energy-efficient way.

Optics and photonics are particularly promising platforms for neuromorphic computing because they allow for minimal energy consumption. Computations can be performed in parallel at extremely high speeds, limited only by the speed of light. However, so far there are two major challenges. First, high laser powers are required to realize the complex mathematical calculations required. Second, there is no general way to efficiently train such physical neural networks.

In a paper published on 24 November 2010, Clara Wanjula and Florian Marquardt of the Max Planck Institute for the Science of Light propose a new method that overcomes both of these challenges. Natural Physics“Usually, data input is imprinted onto a light field. However, in our new method we propose to imprint the input by varying the light transmittance,” explains laboratory director Marquardt.

In this way, the input signal can be processed in any way. This is true even if the light field itself behaves in the simplest possible way: waves interfere without affecting each other. This approach therefore allows us to avoid complex physical interactions to achieve the required mathematical functions that require high-power light fields.

This makes it very easy to evaluate and train this physical neural network: “It's as simple as sending light into the system and observing the light that passes through. This allows us to evaluate the output of the network, and at the same time measure all the information relevant to its training,” says Wanjula, first author of the study.

In simulations, the authors demonstrate that their approach can perform image classification tasks with the same accuracy as a digital neural network.

In the future, the authors plan to work with experimental groups to explore the implementation of this method. Our proposal significantly eases experimental requirements and can therefore be applied to many physically very different systems. This opens up new possibilities for neuromorphic devices and enables physical training on a wide range of platforms.