When you ask lawyers for their best advice on using artificial intelligence in your practice, some will argue that artificial intelligence should be avoided at all costs. But despite all the news about lawyers getting arrested for citing hallucinatory cases in court filings, there are ways to use this technology carefully and wisely, lawyers and technologists told the ABA Journal.
“When you give an AI a prompt, it sounds great, but if you get something suspiciously specific, ask a few follow-up questions. You can also ask more general questions like, ‘How confident are you about that?’ Ask them to vet their own answers. where is the hole? What’s the problem? Flag the problem and put the AI to work. ”
—bobby williams jr.Director of iDiscovery Solutions, trains lawyers on proper use of AI
“Methods and strategies must be established so that how authorities are cited, the facts and citations and analysis, all of which are independently verified.”
—Jian Cha, Cha Lo Ethics
“Just like when you’re using Westlaw, you don’t just take a case and just cite it because it comes up in a query. You go back and read the case. You can’t just say, ‘Oh, the AI told me this’ without checking to see if the case actually exists.”
—julie bennett Litigator/Investigator, Indiana Supreme Court Disciplinary Committee, President, National Bar Association
When AI mischaracterizes a real-life incident, it can be worse than a hallucinatory incident, says David S. Kemp, a member of Ropes & Gray’s AI innovation team.
“Instances in which lawyers cite hallucination cases tend to make the news, but cases in which AI mischaracterizes real cases are more egregious and less reported. After all, there is no substitute for actually reading the case cited.”
—David S. Kemp, Ropes & Gray, AI Innovation Team
“The best way to use AI is to make sure we haven’t missed something mid-project. But AI shouldn’t replace our judgment. It’s not foolproof and can lack context.”
—David Majchrzak, partner and general counsel at Rosing Pott & Strohbehn; President of the Association of Responsible Lawyers
“Read all the cases your colleague has cited, and if they cite a hallucination case or an inaccurate citation, call them and ask them to correct your submission. If they don’t, or if the correction doesn’t correct the citation, you’re on your way. as As other attorneys, we are responsible for any inaccuracies. ”
—Dan D. Kohane, member of Hurwitz Fine
“Push back on the AI. Dispute the information the AI provides. The AI may, but not always, admit it’s wrong. Don’t be afraid to change the prompt or use the same prompt multiple times to see if the results change. Since AI is based on predictions, the same prompt can reveal additional information or errors. You can also ask.
—Jennifer Ellis, attorney and legal technology consultant
Josh Kubicki, a visiting lecturer at Indiana University Maurer School of Law, says generic chatbots like ChatGPT and Anthropic’s Claude should be avoided.
“Don’t use general-purpose chatbots for case research. Tools like ChatGPT and Anthropic’s Claude generate text by probability, so they produce authentic-looking citations without knowing whether a case law exists. They mimic the form of a citation, not its content. For research, use tools that are based on real legal databases. Currently, frontiers like ChatGPT Models are better at avoiding spurious precedents. But you shouldn’t rely on them anyway.”
—Josh Kubicki, Visiting Lecturer, Indiana University Maurer School of Law
Click here for an overview of how state bar regulators are addressing the use of AI by lawyers. here.
