More on the Shortcomings of AI Markups


Some AI-and-contracts companies say their artificial intelligence will learn the patterns in your stash of signed contacts and use that, together with a menu of your preferences, to create, in an instant, a markup of the other side’s draft.


That sounds great, but in this April 2021 blog post I pointed out some shortcomings. In particular, I said this:


If your AI review is based entirely on your negotiation record, any issues not reflected in the negotiation record will be invisible to it, no matter how relevant they might be. And you won’t have the opportunity to learn about those issues.


In this post, I’ll offer an example of how that shortcoming might play out.


A big part of the “expertise” component of LegalSifter’s AI-and-expertise promise is that we’re acutely aware that a given concept might be expressed several different ways. This week I explored yet another example of that.


The concept in question is that a buyer of goods isn’t allowed to return goods, except perhaps in specified situations. (I’m aware this isn’t a thrilling issue, but for this post it has the benefit of being simple.) How is that concept expressed? So far I’ve come up with five good and not-so-good ways. Here’s an example of each of those ways, in all their unedited glory:


All sales of Product under this Agreement are final.


Distributor shall not have the right to return any Product to Company, except in the case of a defect notified by Distributor in accordance with Section 2.5.


Seller does not accept the return of products for refund, credit, exchange, or any other purpose.


All orders for Third Party Products are non-cancellable and non-returnable to Vendor.


Wholesaler acknowledges and agrees that no right of returns exists in relation to any unsold Product, including expired Products.


Imagine you’re the client of an AI-markups company. You’ve explained to them that when you buy goods, you make sure the contract doesn’t prevent you from returning goods. That’s why no contract in your stash of signed contracts contains such a provision. In expressing this concern, you used language comparable to the first of the above examples.


What your AI-markups company does about this is a function of whether it’s in the technology business or in the technology-and-expertise business. As far as we’re aware, all such companies are in the technology business. So their markups will look for just what they know about.


In this case, their markups will look just for language comparable to that first example and will ignore anything that looks like the other four examples. That’s a serious weakness. (Conceivably their AI-markups might flag, offering no explanation, everything except what you’ve told them is safe. That seems unhelpful; I don’t know whether any company does that.)


The AI-markups company could instead look for all the different ways contracts say returns aren’t allowed, but then they’d be in the technology-and-expertise business—LegalSifter’s business. That exemplifies the approach built into LegalSifter Review, our contract-review product.


They’re welcome to join us in this business. But they should think carefully before adding expertise to the mix. It’s challenging, it doesn’t offer quick wins, and you won’t find the necessary expertise in a squad of generic behind-the-scenes lawyers.


On the other hand, their users would be better served as a result—with the technology-only approach, you’re limited to refighting old battles. That’s why we wouldn’t dream of changing our approach, whatever its challenges.