Theoretical particle physicists tackle machine learning black boxes

From self-driving cars to facial recognition, modern life is growing with machine learning. This is a type of artificial intelligence (AI) that learns from datasets without explicit programming.

Despite its ubiquity in society, we are only just beginning to understand the mechanisms that drive technology. In a recent study, Zhengkang (Kevin) Zhang, an assistant professor at the Ministry of Physics and Astronomy at the University of Utah, has shown how physicists can play an important role in solving the mystery.

“We used to say machine learning is a black box. We enter a lot of data, and at some point, we talk about it for reasons, and make decisions, like humans. It feels magical because we don't know how it works.” “Now, we use AI in many important sectors of society, so we need to understand what our machine learning models are actually doing. Why does something work, why does something work?”

As a theoretical particle physicist, Chan explains the world around him by understanding how the smallest and most basic elements of matter behave in an infinite world. Over the past few years, he has applied tools from his field to better understand highly complex models of machine learning.

Scale up while reducing costs

Traditional methods of programming a computer use detailed instructions to complete a task. Let's say you need software that can detect irregularities in CT scans. Programmers need to create step-by-step protocols for countless potential scenarios.

Instead, machine learning models train themselves. Human programmers provide relevant data such as text, numbers, photos, transactions, medical images, and more, allowing models to find patterns and make predictions on their own.

Throughout the process, humans can adjust the parameters to obtain more accurate results without knowing how the model uses data input to provide output.

Machine learning is energy intensive and extremely expensive. To maximize profits, the industry trains the model on a small dataset and then scales it to a real scenario with much more data.

“I want to be able to predict how good a model is on a large scale. If I double the size of the model or double the size of the dataset, will the model be twice as good? Will it be four times as good?” Chan said.

Physicist Toolbox

The machine learning model looks simple: Input Data > Computing Black Box -> Outputs that are functions of input.

The black box contains neural networks. Neural networks contain a suite of simple operations that are connected via the web to approximate complex functions. To optimize network performance, programmers traditionally rely on trial and error, fine-tuning, and network retraining, which has earned the cost.

“I'm trained as a physicist, so I want to get a better understanding of what's actually going on to avoid resorting to trial and error,” Zhang said. “What are the characteristics of machine learning models that give us the ability to learn to do what we want to do?”

A new paper published in the journal Machine Learning: Science and TechnologyZhang solved the proposed model scaling method. This explains how the system works on a larger scale. It's not easy. The calculation requires adding an infinite number of terms.

Chan applied the method that physicists use to track hundreds of thousands of terms known as Feynman diagrams. Richard Feynman invented this technique in the 1940s to deal with the hopelessly complex calculations of basic particles in the quantum domain. Instead of writing down algebraic equations, Feynman drew a simple diagram. All lines and vertices in the diagram represent values.

“It's very easy for your brain to grasp, and it's easy to track what terms will enter your calculations,” Zhang said.

Zhang used Feynman Diagrams to solve the models brought in research published since 2022. In that paper, physicists studied the model with certain limitations. Zhang has obtained a new, more accurate scaling law that can solve the model beyond its limitations and governs its behavior.

As society is the first to move towards AI, many researchers are working to ensure that tools are being used safely. Zhang believes physicists can join engineers, computer scientists and others working to use AI responsibly.

“We humans are already building machines that control us. It's a Youtube algorithm that encourages videos that suck each person into their own little horns and influence our behavior,” says Zhang. “It's the danger of how AI tries to change humanity. It's not the robots of human colonization and enslavement. We humans are building machines that we struggle to understand, and our lives are already deeply influenced by these machines.”