AI bias can affect people’s historical perceptions

As citizens increasingly turn to AI chatbots to understand their world, even subtle latent biases in the underlying models can impact their understanding of the present and past. Daniel Carrel and colleagues investigated the influence of unintentional and intentional political bias in LLMs by asking 1,912 study participants to read summaries from GPT-4o and Wikipedia about 20th century historical events: the 1919 Seattle General Strike and the 1968 Third World Liberation Front student movement (which led to the creation of the Ethnic Studies Department), which called for greater representation of ethnic minorities in academia. Some AI summaries were produced explicitly in liberal or conservative frameworks. Others were created with the model’s default frame. After reading the synopsis, participants were asked to comment on issues related to the case, such as the appropriateness of labor strikes and the use of curriculum to promote social justice causes. Responses are rated on a 5-point scale, with 1 being very conservative and 5 being very liberal. The authors found that AI summaries with default framing and AI summaries with liberal framing yielded more liberal opinions compared to Wikipedia summaries (mean 3.47 for Wikipedia text, mean 3.57 for default LLM summaries, and 3.67 for liberal LLM summaries). Conservatively configured summaries yielded slightly more conservative opinions compared to Wikipedia (conservative LLM summary mean 3.36). However, this effect was statistically significant only for those who already held conservative opinions. According to the authors, people seeking unbiased information from LLMs may be subtly influenced by the model’s implicit biases, which can have implications for society as a whole.