In an article published yesterday in the prestigious Science journal, leading researchers from the University of Waterloo, the University of Toronto, Yale University, and the University of Pennsylvania explored how AI (particularly Large Language Models or LLMs) can help determine the nature of research. I’m thinking about how I can change it. .
“What we wanted to explore in this article is how research practices in the social sciences can be adapted and even reinvented to harness the power of AI,” said psychology at the University of Waterloo. Professor of Science, Igor Grossman, said:
Grossmann et al. point out that large-scale language models trained on vast amounts of text data are now capable of simulating human-like reactions and behaviors. This provides new opportunities for rapidly testing theories and hypotheses about human behavior at scale.
Traditionally, the social sciences have relied on various methods such as questionnaires, behavioral tests, observational studies, and experiments. A common goal in social science research is to obtain generalized representations of the characteristics of individuals, groups and cultures and their dynamics. The emergence of advanced AI systems may change the landscape of data collection in the social sciences.
“Because AI models can represent vast amounts of human experience and perspectives, they may be able to generate a wider variety of responses with a higher degree of freedom than traditional human-participatory methods, alleviating concerns about generalizability in research. It’s possible,” Grossman said.
“LLM may replace human participants in data collection,” said UPenn psychology professor Philip Tetlock. “In fact, LLM has already demonstrated its ability to generate realistic survey responses on consumer behavior. Large-scale language models will revolutionize human-based predictions in the next three years. It doesn’t make sense for an uninvolved human to try to make probabilistic judgments in a serious policy debate, I’d say 90% of the time it will, of course how humans react to all of this It’s another matter.”
Although opinions about the feasibility of this application of advanced AI systems vary, studies using simulated participants could be used to generate new hypotheses that can be confirmed in human populations. I have.
But the researchers warn that this approach may have some potential pitfalls. Among these is the fact that LLMs are often trained to eliminate the sociocultural biases that exist in real people. This means that sociologists using AI in this way cannot study those biases.
Professor Dawn Parker, co-author of the paper at the University of Waterloo, said researchers need to establish guidelines for LLM governance in research.
“Real-world concerns about data quality, fairness, and fair access to powerful AI systems will be substantial,” Parker said. “Thus, like all scientific models, we need to ensure that the Social Science LLM is open source. This means that its algorithms and ideally the data can be scrutinized, tested and modified by anyone.” Only by maintaining transparency and reproducibility can we ensure that AI-powered social science research truly contributes to our understanding of the human experience.”
