The use of artificial intelligence (AI) is becoming increasingly common in the legal world, with lawyers, law firms, and even courts using it for research, drafting, and administrative tasks. While AI promises increased efficiency, it has also raised concerns that inaccurate or “hallucinatory” output could be introduced into judicial proceedings.
The debate has gained urgency recently after the Supreme Court cautioned against the use of AI-generated materials in courts while staying orders of the National Company Law Tribunal (NCLT) and the National Company Law Appellate Tribunal (NCLAT), which had relied on AI-generated fake judgments.
The Supreme Court also directed the Bar Council of India (BCI) to set up a committee to look into the issue of advocates presenting hallucinatory or fabricated AI-generated materials in court as if they were valid precedents.
The Supreme Court’s intervention sparked a broader debate not about whether AI should be used, but how far its role should be expanded. Against this background, indian express We speak to legal experts about whether AI can be safely integrated into justice and legal work, where the boundaries should be drawn and what role regulators should play in governing the use of AI.
As an aid rather than relying on it blindly
Experts emphasized that the problem is not to exclude AI from the legal field, but simply to use it as an aid. advocate Alia TripathyPartner Cyril Amarchand Mangaldas said the Supreme Court’s decision was a “timely” reminder that AI must remain a supporting tool rather than a replacement for human intelligence.
“Blind reliance on AI-generated output in legal work is problematic on multiple levels. It not only affects the rights of clients, it dilutes the trust and privilege entrusted to it and exposes poor professional ethics,” she said, adding that the Supreme Court’s intervention fundamentally calls for the ethical and responsible use of AI.
Technology acceptance and its problems
Tripathy acknowledges that over-reliance and delegation issues for important legal work have led to a steady increase in the problem of false citations and inaccurate legal positions.
Story continues below this ad
In June, Justice Wasim Sadiq Nargal of the Jammu, Kashmir and Ladakh High Court warned judicial officials against relying on AI-generated legal research without independent verification after inaccurate and untraceable case citations were found in trial court orders.
Meanwhile, courts themselves are steadily adopting this technology.
Mentioning the iconic “Tariq Pe Tariq” dialogue from the movie daminiIn May, the Allahabad High Court had issued a series of directions to the Uttar Pradesh government and police authorities aimed at streamlining the system. One such directive was for police to implement an AI module that converts audio to text to record witness statements.
Meanwhile, the Gujarat High Court introduced a policy in April to regulate the use of AI in judicial and administrative functions, acknowledging “rapid technological advancements” and the rise of tech-savvy judicial officers. This allows judges and court staff to use AI as a controlled administrative and research aid, such as translating documents, checking grammar, and managing cause lists, while requiring human oversight.
Story continues below this ad
regulation and supervision
advocate Mathuvanshi MathavanPartner at Poovayya & Co says complete denial of AI is “unrealistic”. Referring to the Supreme Court’s opinion, she said there is a need to focus on proper regulation and oversight, saying, “While the judgment accepts AI as a useful tool, it insists on human control.”
Mathavan said the BCI should organize interactive sessions and training programs for advocates on the responsible use of AI, as well as specify “clear disclosure requirements” for the use of AI in legal practice, stressing the need for disciplinary action when fabricated, hallucinated, or unverified material is submitted to judicial fora.
The disclosure requirements for AI tools are consistent with the Supreme Court’s 2026 Draft Rules on the Use of Artificial Intelligence in Courts, which require disclosure of the source of generated AI content and verification by specified mechanisms before such content is produced in court.
“The inability to identify the hallucinations and biases inherent in these cases may simply indicate that lawyers need to be trained as ‘AI-enabled lawyers,'” Tripathy said. She warns that prescribing norms may not be enough if the accountability and transparency matrix is not properly upgraded, and suggests referring to the recently published International Bar Association guidance on the use of AI in mediation.
Story continues below this ad
Acceptable use and over-reliance
Advocates talk about drawing the line between acceptable uses of AI in judicial work Akshat PandeManaging Partner of Alpha Partners says relying too much on anything other than your own intelligence is harmful.
He says the use of AI by non-lawyers to perform legal tasks in lieu of seeking professional advice could be more dangerous than the misuse of AI by lawyers. Mandatory disclosure could also impact clients, as AI-generated drafts could be discounted by courts. “It would be better if courts could also use technology to spot incorrect data, drafts, or fake case estimates through AI and reject drafts on that basis.”
Massavan also warns that over-reliance on AI risks turning useful assistance into a substitute for independent thinking. “Human judgment and verification must always remain at the core of legal practice.”
accountability issues
advocate Anshul VermaThe more important question is who is responsible if AI is used, said the SKV law firm partner, adding: “The responsibility lies with the lawyers and judges, regardless of what tools they have with them.”
Story continues below this ad
She says the Supreme Court’s latest intervention provides a clear path for the BCI to define what actual verification should look like and set guidelines to prevent similar incidents and back it up with actual disciplinary results.
But Pande believes it is best for BCI to stay out of this. “They generally have a hard time understanding the modern aspects of the legal profession. With AI, God knows what they’ll end up doing.”
As courts, lawyers, and regulators grapple with the growing influence of AI, the challenge will be to leverage its benefits without compromising the accuracy and accountability that are at the heart of the justice system. Pande sums it up: “Whether law enforcement likes it or not, AI will be used, but it is better to get used to it and build systems to detect its misuse than to ban it.”
