Can machine learning be used to predict emotions?

From weather to sports to stock market trends, predictions are a regular part of our lives. Most of these sectors rely on historical data and models that can give a good idea of ​​what to expect in the future.

And as complex as meteorology and economics are, one researcher is trying to bring an arguably even more complex subject into the world of forecasting: human emotion. Specifically, Joshua Curtis, an assistant professor of applied psychology and mental health researcher at Northeastern University, is studying how machine learning models can be used to predict people’s emotions.

Curtis says that predicting someone’s emotions in this way could provide insight into how that person experiences mental health disorders such as anxiety and depression. Armed with that insight, healthcare providers can provide pre-tailored mental health support.

According to Curtis, two models had the lowest error in predicting the four emotions. For satisfaction and cheerfulness, models that based their predictions on past performance seemed to be the most accurate. For sadness and anxiety, Curtis said, it was ensemble models that produced something like a composite of the results of the individual models to make predictions.

Curtis, who heads the Computational Clinical Psychology Laboratory, said that treatments and interventions for these diseases, such as therapy and drug therapy, often take a one-size-fits-all approach. He argues that humans are so diverse that what works for one person may not work for another.

“There may be better ways to personalize mental health and help us understand individuals better,” he says.

The pilot study, which Curtis outlined at the Cognitive Brain Health Institute’s Institute Day, focused on 34 people who had been formally diagnosed with an affective disorder. These participants were asked to report their feelings at that moment on a 7-point scale: contentment, cheerfulness, sadness, and anxiety. Questions were asked five times a day for two weeks.

The responses were then fed into six separate machine learning models to see if the models could identify patterns and predict participants’ future emotions. Individual models ranged from using the average score of an individual’s reported emotion and correlating it with a seven-point scale of emotions to finding other patterns using neural networks that mimic the way the brain processes data. These models were compared to a baseline of average responses across all study participants.

The accuracy of these models was tested by comparing the predicted values ​​of human emotions with the actual recorded values. Curtis preliminarily found that individual models were more accurate than group-level benchmarks when predicting people’s emotions about a day into the future.

Additionally, this study suggests that one type of predictive model may work for one person, an emotional disorder or emotion, but may be inaccurate for another, driving home the need for individualization.

Although the research is still in an early “proof of concept” stage before it can be routinely performed worldwide, Curtis said machine learning’s predictions could inform more personalized interventions if further testing is conducted.

It can be as simple as giving the other person a heads up to prepare them for how they will feel at some point in the future. It could also mean allowing people with knowledge to start certain habits or discourage others based on emotion, “giving you the bandwidth and wiggle room to pre-empt or offset some of the things you think might happen in your future,” Curtis said.

“Better prediction means better care,” says Don Robineau, an assistant professor in Northeastern University’s department of applied psychology who was not involved in the study.

“Human beings are very complex, and while the challenges people face may seem very similar on the outside, there are often very unique, individual-specific factors that cause trouble in that area, such as depression or anxiety,” Robineau said.

Curtis’ research “really embraces that complex reality in a way that we understand how unique we are and that there can be a variety of factors that can cause a particular individual’s distress. And we need models that are truly tailored to individuals to better predict what their mental health is like and how we can best support them.”

The lab is currently working to expand the study to more people, different populations, and longer periods of time. They are also considering how data from smartphones and wearable technology (determining whether someone is active or sitting at home, for example) can add value to someone’s subjective emotional reports.

However, this research is not without its challenges. One is how far into the future these predictions are made, both due to internal factors such as an individual’s personality, thoughts, feelings, and preferences, and external factors that can influence a person’s mental health.

“There is no way to predict whether someone will receive health news from their doctor two months from now. For example, there is no way to know whether someone will experience frustrating news about their job or career three weeks from now,” he said.

“The key is to try to find a good balance. It might be a stretch to say, ‘Let’s develop a perfect model to predict how I’m going to feel on Tuesday a year from now,'” he says. But more realistically, it could be helpful to make these predictions a week or two out, Curtis said.

This is why Curtis compares it to the weather. These predictions are our best guesses, but may not be 100% accurate.

“Even with perfect information, these things are very difficult to predict because, like the butterfly effect, small individual differences in a person’s mood or emotions on one day can lead to more chaotic and unpredictable behavior later on,” Curtis said.

He added: “No matter how good a job we think we’re doing, things can still go wrong. So we need to be very responsible about this.”

Robinow also noted the challenges of predicting emotions, but was excited about the research, saying the potential is “huge” and “game-changing.”

“We have a long way to go, but I think this is a really exciting and promising direction. I think the more we can help people understand that these ideas exist, the more we can try to help continue this work,” he said.