Generative AI in the Courtroom: A Wake-Up Call from the Bench
The “Good” in “the Good the Bad and the Ugly” in our Modern Slavery Update from 19 September 2025.
With generative AI becoming increasingly embedded in our daily lives, and now making inroads into the legal profession, it was only a matter of time before it landed squarely in the courtroom. From correspondence to submissions, and even entire applications, AI-generated content is no longer a novelty. However, as its use grows, so do the concerns.
While AI tools may offer greater accessibility to self-represented litigants, questions around accuracy, relevance, and ethical use have lingered. These concerns were recently addressed head-on by Chief Justice Andrew Bell.
In a recent case, a self-represented litigant relied heavily on generative AI. So much so that when questioned, she admitted to reading directly from a script prepared by AI. The case involved a property dispute before the Supreme Court of NSW, which was appealed to the Court of Appeals, where Chief Justice Bell made no personal criticism of the litigant, but issues a broader warning about the implications of AI in legal proceedings.
What the Bench had to say
The Chief Justice’s remarks were candid and cautionary:
Unintelligible and flawed submissions – Much of the AI generated oral submissions were described as “misconceived, unhelpful and irrelevant” and they failed to engage with the actual issues before the Court.
Fabricated Authorities: Several citations were not only incorrectly referenced but entirely fabricated, a phenomenon known as “hallucination.” This is an ongoing challenge that many legal professionals face and often results in wasted time and increased costs.
Ethical and Professional Risks: the Supreme Court of NSW has issued Practice Note SC Gen 23, which outlines strict limitations on the use of generative AI in legal proceedings. Lawyers are now prohibited from using AI to draft affidavits, witness statements, and character references. Even judges are subject to new guidelines.
A Call for Caution
Chief Justice Bell acknowledged that AI is here to stay but urged both legal professionals and self-represented litigants to proceed with care. For lawyers, the message is clear: professional standards must be upheld, and critical assessments of any AI-generated material must be undertaken before it reaches the courts. For litigants without legal representation, the warning is even more stark: AI is not a substitute for legal advice.