I typically use my blog to talk about Court of Appeal cases, not to tell war stories. That is because I find the cases that I talk about to be interesting, and people find my war stories to be anything but.
But this time, I am going to tell a war story. I am going to do so because I find it both topical and frightening. And hopefully you will find it interesting.
I participated as counsel in an arbitration hearing last November. I was assisted by one of our very capable associates and we involved other lawyers with the specific area expertise that was required for the particular subject matter of the arbitration. Our opponent was a sole practitioner who markets himself as an expert in the use of artificial intelligence. A month after the arbitration was concluded, he published his own blog post bragging about how he, with the use of AI, had been able to take on a firm with far greater resources (he didn’t bother mentioning that he had lost the case, quite miserably).
We had realized during the hearing that he was using AI to create daily transcripts for himself. It never occurred to us that he had been doing considerably more than that.
The case started with a long and rambling Notice of Arbitration from our opponent. It meandered through page after page, eventually getting to the point. I thought nothing more of it beside the fact that opposing counsel was simply the type of person who takes a long time to get to the point.
Now that I look back on it, it is clear that this and all of the other paper he subsequently generated was long, meandering, and sometimes completely meaningless. His document requests went on for pages and pages. He brought several motions prior to the ultimate hearing, papered in much the same style. His cross-examinations of our witnesses were lengthy, repetitive, and ultimately unsuccessful. Following the conclusion of the evidence, the arbitrator requested written closing submissions and his somehow managed to emphasize his weakest points and little else.
Following the release of the Award (in my favour), he chose to submit a list of questions to the arbitrator under section 40 of the Arbitration Act, which entitles a party to inquire of the arbitrator as to certain aspects of an award. His request document, once again, went on for pages and posed almost 100 questions. Several days later he delivered a supplementary list of questions.
The Award is not subject to appeal on any basis, so he has now served a Notice of Application to set the award aside under section 46 of the Arbitration Act, complaining of unfair treatment. Yet again, the Notice of Application goes on. And on. And on. And the listed grounds for the Application look an awful lot like grounds for an appeal.
At the risk of appearing obtuse, I must admit that it never occurred to me until I read his blog post a few days ago that this entire exercise, on his side, seems to me to have involved nothing more than figuring out the right prompts to put into his AI app in order to generate the various documents. I don’t know that he actually authored anything himself, at all. And what’s worse, I am not at all certain that he spent any time reviewing the AI product before using it or submitting it.
We use AI in our office. It’s called Harvey. It really is very good. I use it to edit work that I originate. It helps me with grammar and it helps streamline language. I use it to summarize transcripts. I use it for all kinds of tasks. However I do not use it to create documents from scratch.
What I learned in the course of this arbitration, however, is a valuable and important lesson. We are all aware of all those cases in which lawyers have used AI for legal research and ended up handing the court citations to cases which don’t exist. And we all know that the penalties for doing that are extreme. But there are no penalties for handing the court a bunch of AI-produced garbage.
As I said, this particular lawyer uses the fact that he makes such comprehensive use of AI as a marketing tool. Presumably, his point is that he can compete with larger firms having far greater resources, and that he can do so at a much lower cost. What the unsuspecting public may not appreciate is that, as the saying goes, you get what you pay for.
Worse yet, his indiscriminate use of computer-generated junk, at least in this case, increased our client’s expense considerably. While he was in front of his computer clicking on icons, the result of his efforts was that our client was compelled to fund our review of the material and our preparation of appropriate responses. The fact that almost all of the documents to which we had to respond were off target did not mean that we could simply ignore them.
So the lessons learned from all of this, at least for me, are quite simple. Firstly, do not use AI to create documents from scratch. Just don’t do it. Your AI app wants to make you happy. It will spew out all kinds of nonsense for that purpose.
Secondly, whatever you use AI for, read it, edit it, and read it again. There is a real risk that it will be far too long and at least partially off topic.
Simply having your computer spit out a document and handing it to a judge is a disservice to the court, to opposing counsel, and ultimately to your client. Your bill to the client might be lower but your client’s chances of success will probably be reduced as well.