This research raises concerns about political exploitation of generative AI and weak safeguards
New Delhi – The joint report, released days before the India AI Impact Summit 2026 (February 16-20), raised serious concerns about the political and social use of artificial intelligence in India, particularly its impact on the Muslim community.
The report, titled “India AI Impact Summit 2026: AI Governance in an Era of Democratic Backwardness,” was published by the Internet Freedom Foundation and the Center for the Study of Organized Hate. The group argues that generative AI tools are being used to spread anti-minority rhetoric, strengthen surveillance systems and influence electoral processes, while transparency and regulation remain weak.
The report claims that creative AI is being used by political actors to deepen social divisions and target minorities, particularly Muslims.
The report cites the example of Assam, where the state unit of the Bharatiya Janata Party shared an AI-generated video on its official social media account. Assam Chief Minister Himanta Biswa Sarma was seen shooting dead two Muslim men. The clip was captioned “No Mercy.”
The report’s authors described the video as “inflammatory content that could pose a serious threat to social harmony.”
“When politicians use AI to depict violence against a particular religious community, it sends a dangerous message. It normalizes hatred and creates fear among the public,” said an executive at the Internet Freedom Foundation.
The report also mentions similar instances in Delhi, Chhattisgarh and Karnataka where AI tools were allegedly used for political messaging.
For many Indian Muslims, these developments are worrying. A community activist in Delhi said, “We are already facing suspicion in many spaces. When technology is used to show violence against us, even if it is fake, it increases insecurity and makes people feel unsafe.”
weak safety device
The report points to gaps in safety measures within common generative AI systems. The report notes that widely used text-to-image conversion tools such as Meta AI, Microsoft Copilot, OpenAI ChatGPT, and Adobe Firefly lack effective controls regarding Indian language and local social context.
Research shows that these tools can respond to prompts in ways that can reinforce stereotypes about certain communities.
“Content moderation systems are often designed with a Western context in mind; they do not fully understand India’s political signals, dog whistles, and coded language. This gap allows harmful content to circulate,” researchers associated with the report said.
The report also criticizes social media platforms and AI companies for weakly enforcing community guidelines, saying harmful content often spreads before it is removed.
Monitoring measures
The report also raises concerns about surveillance measures. This is in reference to a statement by Maharashtra Chief Minister Devendra Fadnavis regarding the development of AI tools in collaboration with the Indian Institute of Technology Bombay.
The tool is reportedly intended to help identify illegal Bangladeshi immigrants and Rohingya refugee suspects through initial screening based on language and accent.
Language experts question the reliability of such systems. “There are deep similarities between Bengali dialects across borders. It is extremely difficult, if not impossible, to accurately determine nationality based on accent alone,” said one academic quoted in the report.
The report warns that such measures could increase the risk of discrimination against Bengali-speaking Muslims in India.
“When technology is used to flag people based on the way they speak, the burden of proving their belonging falls on poor and marginalized citizens, especially on day laborers and rural families,” said a civil rights lawyer.
Facial recognition and enforcement
Another key concern raised in this report is the use of facial recognition technology (FRT) by police across multiple states.
The study says there is little public information about how these systems are sourced, their accuracy, and how errors are handled. The group warns that cases of mistaken identity can have serious consequences, especially when connected to a criminal investigation.
“If facial recognition systems incorrectly match individuals, the mistakes can persist for years. The risk is higher for minorities who already face profiling,” the digital rights group said.
The report claims that there is no clear and effective complaint mechanism for individuals who are falsely reported by AI systems.
Welfare system and Algorithmic exclusion
The report also highlights problems in welfare provision. It claims that flaws in the AI system are excluding eligible beneficiaries from government programs in several states.
The authors say vague algorithms and automated decision-making systems have been introduced without public consultation. If the system flags them as ineligible, citizens will be required to prove their eligibility.
“Many families don’t understand why their rations and pensions are being stopped. They are told the system has rejected them. There is no clear explanation and no easy way to appeal,” said a social worker in Uttar Pradesh.
The report suggests that these systems can disproportionately impact poor Muslims and other marginalized communities that rely heavily on state welfare.
concerns about election process
The study also touches on the electoral process. Questions arise about the lack of transparency in the software used to mark “suspicious” voters.
The report says there is limited clarity about how voters will be flagged, how the data will be verified and what safeguards exist to prevent errors.
“The right to vote is fundamental. If automated systems are used without transparency, citizens may have to go through lengthy legal procedures just to protect their right to vote,” a constitutional expert said.
Community leaders have expressed concern that Muslims, who often face intense scrutiny on citizenship-related matters, could be affected if a flawed system is used to authenticate voters.
democratic safeguards
The report concludes with several recommendations for governments, industry and civil society.
These include transparent policy-making, independent review of algorithms, strong human oversight, clear complaints systems, and alignment with international human rights standards.
A representative from the Center for the Study of Organized Hate said, “Artificial intelligence should serve people, not target them. Governance must be rooted in constitutional values and equal rights.”
As the India AI Impact Summit 2026 approaches in New Delhi, this report adds urgency to the debate on how AI is being used in India.
For many Indian Muslims, the central concern is not the technology itself, but how it is used.
A young student from Mumbai summed up the mood: “We’re not against technology. We just want fairness. We want to know that new tools won’t be used to exclude us.”
The report concludes that aligning AI governance with democratic values and fundamental rights is essential to maintaining trust in a diverse country like India.
