Second Department takes a strong stand on AI hallucinations

The Appellate Division has imposed $11,000 in sanctions against a lawyer and his firm for submitting an appellate brief containing fake case citations, what the legal world now calls hallucinations. This appears to be the Second Department's most extensive discussion thus far on the practice of using artificial intelligence for legal research without double-checking the ensure the AI-generated research is finding real or imagined cases.

The case is Landberg v. City of New York, issued on June 23. Every generation has its new problems. For lawyers, its AI, which can produce research results quickly and provide case citations. The problem is that AI may either make up a non-existent case or cite a real case with holdings that do not appear in the case. What we are seeing -- from the multitude of rulings like this one -- is that some lawyers are using AI for legal research without reading the cases and determining if they are accurate or even if the cases exist. The courts have had it with AI hallucinations, and decisions are issuing nearly every day condemning the practice. Take a look at this database, updated daily.

In this personal injury case, the brief had a number of fake case citations, and the Appellate Division asked counsel about it at oral argument, offering him a 15-minute recess to figure things out. Counsel declined the invitation and "initially confirmed that the authorities within the brief came from Lexis, Westlaw, or a book, or were 'cited from a previous case,'" though "he later hypothesized that the erroneous citations may have possibly been the product of him 'over-relying' on 'other briefs' or string cites." 

This is humiliating for the lawyer, so I will keep his name out of this discussion. I guess the Second Department wants to make a statement in this case, so it details counsel's response to the order to show cause on sanctions:

[Counsel] stated that in connection with his research, he recalled "utilizing traditional legal research resources, including Westlaw, appellate briefs, publicly available sources, and other secondary materials. However, [he] also recall[ed] utilizing artificial intelligence-assisted research tools but [did] not recall which exact tool [he] used, it was one of the free ones available to the public." [Counsel]  stated that after carefully reviewing the brief, it was his "belief that the non-existent citations identified by the Court originated during the AI-assisted portion of [his] supplemental research which [he] negligently failed to verify before filing [the plaintiff's] brief with this Court." [Counsel] stated that this was a violation of the policy of the law firm, which "had made known its policy that all AI generated citations to facts and the law required personal review and confirmation by the lawyers using artificial intelligence tools."

[Counsel] further stated in his affirmation that when he appeared for oral argument and the Court asked him where the fabricated cases came from, he "should have stated that they were hallucinated by artificial intelligence. [He] was genuinely scared at the time. It felt like [his] career was on the line, and [he] was afraid to even use the words 'AI.'"  [Counsel] went on to "expla[in], not justif[y]" that he was "confused and stunned at the questioning because [he] was prepared to address the nuance of the cases [he] cited for the central legal issue as to whether the location of the accident qualified as a 'tree well' within the meaning of Administrative Code § 7-210. The cases with the wrong citations were cited for the more simple, and [he] believed to be non-questionable basic principles of Administrative Code § 7-210."

[Counsel] stated that he was "deeply embarrassed" for his error and that he "can assure this Court with every fiber in [his] body that going forward, [he] will be extremely rigorous, to verify every citation in any paper [he] ever file[s] going forward."

That's a real mea culpa, but the Court still issues a sanction. It notes that we all know about the dangers of AI research, citing to a New York Times article on the issue. The Court then says that counsel's response to questioning at oral argument was inappropriate:

when confronted about the fabricated cases cited in his brief at oral argument, [Counsel] declined the opportunity to take a 15-minute recess to discern where he had found the cases, and maintained that he had found them either on Westlaw or Lexis or in a book, a previous case, or another brief. As [Counsel] later admitted in his affirmation, he was not being candid at the time of oral argument, as he should have said that the cases were hallucinated by artificial intelligence. [Counsel] "expla[ined]" that his decision to be dishonest with the Court was due to being "scared." This may be true.

However, [Counsel]'s decision not to be honest with the Court during oral argument also demonstrated his professional immaturity, arrogance, and profound lack of respect for the judicial system.

Even after being given the opportunity to respond to the Court's concerns in writing, it appears that [Counsel] was still not completely candid. In his affirmation, [Counsel] recalled using artificial intelligence research tools in connection with his supplemental legal research, which he blamed for the three fictitious cases that were cited for what he believed to be "non-questionable basic principles" of law. However, as outlined above, the brief that [Counsel] submitted did not merely contain fictitious case citations for non-questionable principles of law. The brief also contained fabricated quotations from the Court of Appeals, which said the opposite of what the Court of Appeals has actually said about whether Administrative Code § 7-210 is to be construed liberally or strictly. 

In addition, the brief completely misrepresented what was decided or discussed in two real cases cited therein. Since "fabricated quotations from actual cases" and "misstatements of law that are not representative of what a case actually decided" are well-recognized "pitfalls posed by the use of GenAI in drafting briefs" (Matter of Julien v Arthur, ___ AD3d at ___, 2026 NY Slip Op 03308, *1-2), it appears that GenAI was used to do more than simply conduct supplemental research, as suggested by [Counsel] in his affirmation.  [Counsel] wholly failed to fulfill his obligation to review the work of GenAI to prevent the submission to this Court of a brief containing significant misrepresentations of the law.

Counsel was sanctioned $8,500 and his law firm -- where Counsel is an associate -- was sanctioned $2,500 for the AI hallucinations. You should know that the lawyers on the other side of the appeal were also grilled by the Appellate Division panel on the fake citations, and they asked counsel why they did not flag this issue writing their own briefs. These lawyers were not sanctioned, by the oral argument makes it clear that all lawyers have a duty to tell the court when their adversary is citing hallucinations.