ChatGPT's first foray into the legal community sent shockwaves. A New York attorney used the artificial intelligence-powered chatbot to conduct legal "research" for one of his cases. ChatGPT located six judicial opinions for him — all supporting his case. The problem: they were fictitious cases. ChatGPT made them up.
The bigger problem: the attorney doubled down on ChatGPT's findings without any independent verification. When the federal judge questioned the authenticity of the cases, the attorney asked ChatGPT itself to confirm that the cases were real. Based on ChatGPT's assurances, he and another attorney verified authenticity to the court.
Judge P. Kevin Castel of the Southern District of New York was understandably shocked by the submission of non-existent judicial opinions and, more concerning, the doubling down on their authenticity without any independent verification or research.
In show cause papers, the attorney attributed a human-like level of fault to the bot — arguing that he was "beguiled by a new technology" and "did not know it would, in essence, defraud him." He explained that the bot "assured him the cases were real." Exasperated by ChatGPT's "fraud," he explained that "when [he] asked ChatGPT to confirm its false results, it did so unabashedly."
"Indeed, even when [he] directly asked ChatGPT whether the cases it had provided were fake, the program not only said the cases were real, but also assured [him] that the cases were available on Westlaw and LexisNexis." And, although he "did not have time to fully research the risks and benefits of this new technology," he "did not think he needed to doubt this assurance."
Judge Castel issued a 34-page opinion sanctioning both attorneys and their law firm $5,000 each. Judge Castel found that the attorneys acted in "bad faith" and "abandoned their responsibilities" when they "continued to stand by the fake opinions after judicial orders called their existence into question." He explained that, had the attorneys initially come "clean" about their use of ChatGPT, "the record now would look quite different."
Judge Castel cautioned: "Technological advances are commonplace and there is nothing inherently improper about using a reliable artificial intelligence tool for assistance. But existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings." As artificial intelligence continues to revolutionize the legal world, attorneys must remain diligent and use common sense. In the end, anything too good to be true probably is.
Originally published by San Diego Lawyer.
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.
