Expert systems and the legal world(legaltechnology.com)
legaltechnology.com
Expert systems and the legal world
https://legaltechnology.com/2022/02/24/guest-post-expert-systems-are-here-lets-welcome-them-to-the-legal-world/
12 comments
You've described exactly what the company where I was previously working was doing in france for small company owners legalstart.fr
basically you answers a questionnaire (with conditions betweens questions like "ask this question only if A and B are both physical person with revenue < 20000 euros a years")
and at the end these answers are used to generate a template (with things like "put this paragraph as many times as they are people involved" )
the templates and questionnaires were made once (and updated every once in a while when law changed) by a specialist.
basically you answers a questionnaire (with conditions betweens questions like "ask this question only if A and B are both physical person with revenue < 20000 euros a years")
and at the end these answers are used to generate a template (with things like "put this paragraph as many times as they are people involved" )
the templates and questionnaires were made once (and updated every once in a while when law changed) by a specialist.
Instead of using templates to generate a contract based on questionnaire answers, my idea is to use a single unaltered contract that refers to the questionnaire answers provided as a separate document.
One could even imagine those "universal contracts" being open-source, if you could find a business model.
One could even imagine those "universal contracts" being open-source, if you could find a business model.
IANAL (but spend a lot more time than I would prefer writing and reviewing contracts)...but I don't think your proposal would work in the US. I believe your additional 'document' would have to be structured as an addendum, which in itself is a sort of contract and technically free-standing from the main/original document. But in this case the original contract wouldn't be valid, because it wouldn't have any specific terms relevant to an agreement between parties, and AFAIK you can't render an invalid or unenforceable contract valid through addendum.
What you're proposing also bucks 100+ years of legal writing tradition (both in US and Europe), which itself is very poorly received by attorneys and judges.
What you're proposing also bucks 100+ years of legal writing tradition (both in US and Europe), which itself is very poorly received by attorneys and judges.
Lawyer here: It's not unusual to have "master agreements" — either negotiated between two specific parties OR agreed to as industry standards — that are "customized" by agreeing to a schedule, a purchase order, or a work order. Technically, the schedule/PO/WO is the contract, and it incorporates the master agreement by reference.
Example: https://www.investopedia.com/terms/i/isda-master-agreement.a... (master agreement for derivatives trading)
More discussion of two-party master agreements: https://toedtclassnotes.site44.com/Notes-on-Contract-Draftin... (my course materials)
Incorporation by reference: https://toedtclassnotes.site44.com/Notes-on-Contract-Draftin... (ditto)
Example: https://www.investopedia.com/terms/i/isda-master-agreement.a... (master agreement for derivatives trading)
More discussion of two-party master agreements: https://toedtclassnotes.site44.com/Notes-on-Contract-Draftin... (my course materials)
Incorporation by reference: https://toedtclassnotes.site44.com/Notes-on-Contract-Draftin... (ditto)
Yeah; IANAL but I've spent a lot of time writing contracts for a specific vertical that have been reviewed by several lawyers, and I always built my contracts in this same structure. There would be a MSA that governs the overall relationship -- sets a baseline for what things cost and what their conditions are, and then there would be various contracts that would include the MSA by reference for enhanced or fixed-rate services.
Everyone preferred this structure that I dealt with, because the total length of the contracts were dramatically shorter and the structure was much easier to read and comprehend.
Everyone preferred this structure that I dealt with, because the total length of the contracts were dramatically shorter and the structure was much easier to read and comprehend.
What's being proposed here seemed so odd, structurally speaking, that I didn't even consider it analogous to a master agreement + follow-on contract structure. I'm not sure the analogy is totally a fit - a master agreement should itself be a freestanding, valid contract, no? But in this case the 'base' document is incomplete and invalid.
I suppose one could just add the addendum of specific terms at the end of the document and both parties sign off on everything in one go, but I certainly wouldn't want to test it in court.
I suppose one could just add the addendum of specific terms at the end of the document and both parties sign off on everything in one go, but I certainly wouldn't want to test it in court.
IANAL also, but: IIRC contracts typically have an "integration clause" that specifies that the contract constitutes the entirety of the agreement between the two parties. Couldn't the integration clause instead specify that the contract, as combined with the additional document into a single whole, constitute the agreement? See: https://en.wikipedia.org/wiki/Integration_clause
See my comment just above yours, concerning "incorporation by reference" (with a link).
Lawyer here also (though not actively practicing) and I agree with your thoughtful comment about. I’ve negotiated lots of commercial agreements, and it’s surprising that lawyers still spend time redlining basic things like severance clauses, integration clauses, etc. There’s no real benefit from much of the back-and-forth. It’d be great to have trusted, open source “standard legalese” terms. That way the parties could focus on the contract’s meaningful parts and say something like “This Agreement incorporates by reference the Standard Legal Terms at xyzterms.org.” And of course you could have different versions for various states and industries.
> * It’d be great to have trusted, open source “standard legalese” terms.*
Working on it :)
Working on it :)
When I did paralegal work in NY, we often used Blumberg Forms and just assumed that it covered our needs.
https://www.blumberg.com/forms/
https://www.blumberg.com/forms/
Lawyer here.
I've seen B2B contracts like the kind proposed above (the terms as a form attached to a pre-existing contract). They've been a thing for decades; they're just very uncommon and usually aren't used until the businesses already have a pre-existing contractual relationship.
Generally, the integration clause is uncommon in the contracts I've seen; it's used in big-ticket contracts (like M&A) but not so much in day-to-day contracts where friction can kill a transaction.
I've seen B2B contracts like the kind proposed above (the terms as a form attached to a pre-existing contract). They've been a thing for decades; they're just very uncommon and usually aren't used until the businesses already have a pre-existing contractual relationship.
Generally, the integration clause is uncommon in the contracts I've seen; it's used in big-ticket contracts (like M&A) but not so much in day-to-day contracts where friction can kill a transaction.
> 100+ years of legal writing tradition
If tradition keeps us from inventing something better for normal human beings, it's called oppression and/or rent-seeking.
If tradition keeps us from inventing something better for normal human beings, it's called oppression and/or rent-seeking.
The issue here is that jurisprudence heavily depends upon established precedents that hinge on particular logical and semantic structures present in legal texts. The more you deviate from the norm, the more likely you are to end up in court due to disagreements over interpretation. Safer to stick to what has already has a consensus behind it.
Eh, not necessarily. Eg musical notation has a lot of tradition and is a bit weird. But I don't see much oppression or rent-seeking there.
(Or look at English spelling..)
(Or look at English spelling..)
Maybe in the case where one party has more power than the other EG If you want to rent this house, take it or leave it.
Having been involved in Legal blacklining pissing matches before, probably not between two equal parties. I think the legal terms are as much as a part of the negotiation as the price - "If you want to be indemnified against X, we will have to hire a team to do Y and that will cost an extra $Z" "You want to use our name in publicity, that means we pay Q% less" etc etc.
Having been involved in Legal blacklining pissing matches before, probably not between two equal parties. I think the legal terms are as much as a part of the negotiation as the price - "If you want to be indemnified against X, we will have to hire a team to do Y and that will cost an extra $Z" "You want to use our name in publicity, that means we pay Q% less" etc etc.
In the US, what you're suggesting is more or less how the majority contracts are written, but rather than using a separate form, specific parties, objects, etc. are defined and indicated by a generic term used for the remainder of the text. Law firms and corporations generally maintain a library of templates for use.
+1. Many times the real value an attorney brings is working with the client to tailor the template to their situation and needs. Most parties try to mainly use standard terms, e.g., shipping Incoterms, and language that has been litigated in case law (if in the US) so that the parties can better understand and estimate the likely outcome of any dispute. However, each situation is usually slightly different. Even if we ignored any unique aspects of each contract, each template would have to be tweaked to reflect the unique and changing case law (if in the US) of the jurisdiction governing the contract.
Indeed. What I'm suggesting just amounts to taking that approach to its limits, by putting the "definitions" into a separate document and having the contract contain conditionals that depend on that document.
So in other words, you just want to use a standardized legal template?
Yes, you can do that. People have been doing that for decades. Pretty much every major lease or purchase agreement you or your parents have made in the past 4 or 5 decades has used a standardized legal template.
Note that these templates have the variable fields in-line; if you want to do it as a separate document as proposed, you would need to do that as an attachment or exhibit to the original underlying contract. The former is generally required for contracts between businesses and individuals; the latter is uncommon but is used for B2B contracts.
Generally, if there is a significant amount of conditional logic, you would need/want to split that out as a separate (possibly standardized) contract addressing those particular logical conditions. But note that plenty of standardized contracts have conditional blocks that affect the selection of a particular term of the contract (i.e., amount of monthly payment, points paid, choice of extended warranty, etc.).
Yes, you can do that. People have been doing that for decades. Pretty much every major lease or purchase agreement you or your parents have made in the past 4 or 5 decades has used a standardized legal template.
Note that these templates have the variable fields in-line; if you want to do it as a separate document as proposed, you would need to do that as an attachment or exhibit to the original underlying contract. The former is generally required for contracts between businesses and individuals; the latter is uncommon but is used for B2B contracts.
Generally, if there is a significant amount of conditional logic, you would need/want to split that out as a separate (possibly standardized) contract addressing those particular logical conditions. But note that plenty of standardized contracts have conditional blocks that affect the selection of a particular term of the contract (i.e., amount of monthly payment, points paid, choice of extended warranty, etc.).
This is why I left the legal profession. I was working for the in-house legal department for a very, very large athletic shoe company, and they had a team working downstairs on automated contract drafting. The company is big enough that they wanted managers to be able to do things like leases without consulting the lawyers. Some sales person would go to a factory outlet mall outside of Cleveland, negotiate for a lease, and rather than run the lease by legal or get legal to draft it, they'd just use a giant web form to generate the lease. The software would be able to parse terms added by the other party and associate a "risk" that may or may not necessitate the involvement of a human.
Since writing contracts was my favorite part of practicing law, I saw that staring at me for a year and noped out. I'm a software developer now, and so much happier.
Since writing contracts was my favorite part of practicing law, I saw that staring at me for a year and noped out. I'm a software developer now, and so much happier.
When I worked for the Danish and Swedish division of Thompson Reuters WestLaw we had a product named Pacta that lawyers used to generate all sorts of contracts that operated on this principle. I have to assume that other parts of the WestLaw platform internationally, and other services like Lexis-Nexis, must provide similar functionality. It is, after all, not too difficult a thing to do.
When I left (2013) a project was underway to add lots of functionality to it. Not sure what the current status is if any.
When I left (2013) a project was underway to add lots of functionality to it. Not sure what the current status is if any.
Isn't this basically how every lease is already written? Nobody's writing a new contract for each tenant - you just fill in the blanks.
In banking, this is pretty much how everything works.
There are vendors who took this one step further. They define a document "contract" which is populated with the raw business data and other important facts regarding the context of the transaction. You send this XML into their system and it will dynamically construct the required document based upon the inputs and send back the final PDF. Documents produced in this manner usually require far fewer printed/imaged pages, especially when working with business customers and complex retirement accounts that require volumes of potentially inapplicable disclosures.
We have also hand-rolled a lot of this kind of stuff in-house for our customers using HTML templating and PDF conversion. That said, anything that needs to stand up in court comes from one of 2 legal document vendors and we are required to use it pretty much as-is.
There are vendors who took this one step further. They define a document "contract" which is populated with the raw business data and other important facts regarding the context of the transaction. You send this XML into their system and it will dynamically construct the required document based upon the inputs and send back the final PDF. Documents produced in this manner usually require far fewer printed/imaged pages, especially when working with business customers and complex retirement accounts that require volumes of potentially inapplicable disclosures.
We have also hand-rolled a lot of this kind of stuff in-house for our customers using HTML templating and PDF conversion. That said, anything that needs to stand up in court comes from one of 2 legal document vendors and we are required to use it pretty much as-is.
You might enjoy the writings of /dev/lawer
https://writing.kemitchell.com/
He wishes to create such forms, and created PolyForm to try and do so.
https://writing.kemitchell.com/2021/06/23/PolyForm-Commercia...
He wishes to create such forms, and created PolyForm to try and do so.
https://writing.kemitchell.com/2021/06/23/PolyForm-Commercia...
Aren't most residential real estate contracts already done this way? There are standardized forms that are recognized as legal within your state (or maybe within all states). This lets an agent draw up a legally binding contract for a transaction that's worth hundreds of thousands of dollars, without needing lawyers.
[deleted]
Ontario uses exactly this system in it's standard rental agreement contracts.
I'll trust expert systems when I have access to the documented source code of the decision making process after the fact. With modern AI, that's practically impossible because machine learning has replaced manual algorithms in this space ages ago.
Even without AI, companies like these don't like to hand out their source code, so you wouldn't be able to trust them anyway. Programmers are not lawyers, no matter how hard we pretend to be sometimes, and unless every programmer on this project was also a judge I wouldn't trust it to advice or make any judgement. You're not going to find judges who happen to run a startup as a side-gig anyway, so let's just bin the idea.
Automated systems are great at reducing the human factor in a lot of things. For stupid factory work where human minds can be put to much more useful tasks, that's great. For the legal system, no thank you.
Even without AI, companies like these don't like to hand out their source code, so you wouldn't be able to trust them anyway. Programmers are not lawyers, no matter how hard we pretend to be sometimes, and unless every programmer on this project was also a judge I wouldn't trust it to advice or make any judgement. You're not going to find judges who happen to run a startup as a side-gig anyway, so let's just bin the idea.
Automated systems are great at reducing the human factor in a lot of things. For stupid factory work where human minds can be put to much more useful tasks, that's great. For the legal system, no thank you.
> I'll trust expert systems when I have access to the documented
source code of the decision
FWIW, that's precisely one of the things ES are designed to do, to backtrack and introspect, to explain their decision making.
(I did a module on expert systems (blackboard models) back in 1988 when they were a hot topic)
FWIW, that's precisely one of the things ES are designed to do, to backtrack and introspect, to explain their decision making.
(I did a module on expert systems (blackboard models) back in 1988 when they were a hot topic)
That only applies to tree search GOFAI expert systems. If somebody strings together one using for example a BERT model for the language generation and another neural network for the reasoning, then the guardrails are no longer in place.
Then that is no longer an expert system. Expert systems are systems that capture the expertise of people in rule form, they are GOFAI by nature. If it is using neural networks instead it is not an expert system.
If a neural network is doing the reasoning, is that still an "expert system" (as the term is used in AI)?
> with modern AI
Expert systems are classic AI, often with a rule engine underpinning it. It really could not be simpler in context and this is what makes them good. Everything is simple, the knowledge in it (i.e. the codified expert opinion) is what makes it what it is.
Expert systems are classic AI, often with a rule engine underpinning it. It really could not be simpler in context and this is what makes them good. Everything is simple, the knowledge in it (i.e. the codified expert opinion) is what makes it what it is.
uhm, there is source code to the networks, or where else do they come from?
The problem is not the logic, but the shear size of it. That is not at all different from any sufficiently complex piece of software that allows side effects. And the comments can be totally useless or downright misleading anyway.
Also, there may be persons who are professional in both programming and lawyering.
The problem is not the logic, but the shear size of it. That is not at all different from any sufficiently complex piece of software that allows side effects. And the comments can be totally useless or downright misleading anyway.
Also, there may be persons who are professional in both programming and lawyering.
Sounds like a mundane way of saying "let's create more templates so we can direct customer requests to callcenter agents who don't know anything about the enquiries they're dealing with (and probably even try to replace those humans with chat bots / online forms)". Perhaps being a bit cynical here, but I'm not really optimistic about the UX that's going to create for regular customers.
Might be a European perspective, but I wonder how much a "right to speak to a human" would improve Bit UX as a 2nd order effect. Like, after a certain amount of revenue, your 1st level support needs to be able to be able to( or forward you to someone who can) do every possible interaction that is possible on your system, within 5 or 10 minutes median waiting time, with some simplicity constraints on basic things like cancellation (including a ban on upselling, delays etc), written with a "spirit not letter" style like GDPR. Either make a very expensive callcenter or a good UX
I think the problem there is that talking to a human per se doesn't help much, if said human isn't qualified (and that might be for reasons as simple as not speaking/understanding your language sufficiently). I've had interactions with human call center agents for things as simple as rebooking a flight that could have been taken straight out of a Kafka piece.
Yeah, first the human have to understand the problem. Then they must have capability and rights to solve it. Want to solve it might be less important. But the metrics driven environment we are means they sometimes aim is not to fix issue, just have it look right in metrics.
I watched documentary not so long ago, 500 years from now those systems will fire 100% of the soft drink company employees after the stock dropped to almost zero because some idiot decided that water is better for, well, watering plants. So as a society we can get quite some mileage out of them!
Edit: Spelling, I suffer from severe fat finger syndrom which is compounded by typing specific legastenie.
Edit: Spelling, I suffer from severe fat finger syndrom which is compounded by typing specific legastenie.
"documentary" is an interesting way to describe the plot of Idiocracy (2006)...
This is the current reality, except the templates are generated by ML, and they are impossible to debug or explain.
The UX opportunity is how you apply information from the system. You can’t just accept the output as it is.
The UX opportunity is how you apply information from the system. You can’t just accept the output as it is.
Templates generated by ML? Do you have any links to examples/explanations of how that works? I'm incredulous, why would one even think of doing that? The high frequency task is whatever the template is supposed to solve, not creating it. What would you even use to train the model (unless it's an OpenAI/DeepMind-level of general language model that can create a form from your verbal instructions, but talk about cracking nuts with sledgehammers)?
Expert systems tend not to scale very well. Once you get beyond $some_threshold rules, the decision algorithms get slow and the rules start conflicting with each other. What's needed is a meta-rule system that works during the knowledge capture phase to keep the rule set consistent.
Lots of work was done on this problem in the late 80s/early 90s but I don't know what the current SOTA is since expert systems fell out of fashion. I'd expect it to be fairly straightforward to maintain a consistent rule base with modern computers that are 1000x bigger and faster than those of the 80s.
Lots of work was done on this problem in the late 80s/early 90s but I don't know what the current SOTA is since expert systems fell out of fashion. I'd expect it to be fairly straightforward to maintain a consistent rule base with modern computers that are 1000x bigger and faster than those of the 80s.
The Rete algorithm used in OPS5 scales as O(N) where N is the number of rules - which is good.
However, it scales as O(D^2) where D is the number of working memory facts.
However, it scales as O(D^2) where D is the number of working memory facts.
> I'd expect it to be fairly straightforward to maintain a consistent rule base with modern computers that are 1000x bigger and faster than those of the 80s.
Maybe not. Maybe they'll just be used to create a rule base that is 1000x bigger than those of the 80s, and you'll still have trouble maintaining consistency.
Maybe not. Maybe they'll just be used to create a rule base that is 1000x bigger than those of the 80s, and you'll still have trouble maintaining consistency.
I think if the set of rules underpinning an expert system is explicitly written down and available to everyone, and if it is possible to appeal your decision and request a human review, I don't mind this.
That is a reasonable position, but I don't think things are likely to work out this way.
Companies have various more-or-less self-serving reasons to keep their decision-making process hidden. They will persuade policy-makers that unless they are allowed to keep this information 'proprietary', then 'society' (read: themselves) will not be able to reap the benefits of this technology.
The next step is that the cost of having humans resolve problems becomes 'prohibitive', so any meaningful process of redress is removed through the use of one-sided 'agreements.'
Just look to the big social media companies and telcos to see what can happen when decision-making is automated.
Companies have various more-or-less self-serving reasons to keep their decision-making process hidden. They will persuade policy-makers that unless they are allowed to keep this information 'proprietary', then 'society' (read: themselves) will not be able to reap the benefits of this technology.
The next step is that the cost of having humans resolve problems becomes 'prohibitive', so any meaningful process of redress is removed through the use of one-sided 'agreements.'
Just look to the big social media companies and telcos to see what can happen when decision-making is automated.
They do exist already, just not as widespread as I imagine the author is hoping to see. I work for one of the only software vendors in the space. One place where I've seen an expert system excel greatly is in automated billing resolution. Legal bills are often handled by third parties who maintain the billing contracts between a law firm and it's hiring party (who are often just other lawyers). What can and can't be billed for is usually determined by an exhaustive set of rules defined in the contract, and invoices are usually fed into a variety of out-of-date web forms. The task isn't cut-and-dry enough to automate via ML, other than small NLP tasks, so the system relies on a long list of hard-coded conditions managed by a swarm of bots that fail quickly and loudly to call out entry errors.
It sounds messy, but it works very well, and the errors aren't muddied by a black box.
It sounds messy, but it works very well, and the errors aren't muddied by a black box.
Expert systems will be welcomed right up until the point where they start making the legal profession look incompetent. Vide what happened with MYCIN in the medical community if you need a refresher course on how this is going to go down.
Maybe the new generation of young lawyers will be running virtual firms this way and serving a much higher volume of clients for cheaper. There is a huge untapped market of those who need services for black letter issues but can't afford it.
I was disappointed that the article didn’t mention which tools were used. In the 1980s I did a lot of hacking on OPS5, specifically modifying the Rete Network for multiple data worlds, etc. I really enjoyed that but in present time most of my work experience is in deep learning. I am keenly interested in hybrid AI.
Let's not.
[deleted]
Writing an expert system was a high school assignment decades ago.
Why would it work now after 60 years of not working?
Experts haven't changed.
The software is simple.
Creating expert systems are easy in a child's head, that's why it's a good school assignment, they get to see how quickly it gets out of hand. It's the same as the Semantic Web.
Just making a cake is too hard, why is that not online if expert systems are possible?
Substituting butter for vegetable oil, how much, what sort of vegetable oil, does that work for all cakes? Is it 10% worse as a cake. Do some culture actually like it better one way. Do kids care. Moist or dry matter. How do you translate measurements to stop getting silly results for empirical to metric.
This article and webcast doesn't actually say there exists a working expert system. It will stay that way.
Search is what has filled this gap. Maybe someday it will be AI (Outside of the AI in search)
Why would it work now after 60 years of not working?
Experts haven't changed.
The software is simple.
Creating expert systems are easy in a child's head, that's why it's a good school assignment, they get to see how quickly it gets out of hand. It's the same as the Semantic Web.
Just making a cake is too hard, why is that not online if expert systems are possible?
Substituting butter for vegetable oil, how much, what sort of vegetable oil, does that work for all cakes? Is it 10% worse as a cake. Do some culture actually like it better one way. Do kids care. Moist or dry matter. How do you translate measurements to stop getting silly results for empirical to metric.
This article and webcast doesn't actually say there exists a working expert system. It will stay that way.
Search is what has filled this gap. Maybe someday it will be AI (Outside of the AI in search)
soapdog(2)
For instance, instead of using an expert system to generate a contract that says "Alice will pay Bob $2000 a month in rent, plus utilities," you could write a contract that says "The party listed on line 1 of the attached form will pay the party listed on line 2 of the form the amount listed on line 3 of the form; if the checkbox on line 4 of the form is checked, then the first party will also pay for utilities."
Then the attached form could look something like this:
1. Name of tenant: _______
2. Name of landlord: ______
3. Amount of rent: $_____
4. [ ] Check here if tenant pays for utilities
That solution would seem to be much more efficient and transparent than using an expert system; essentially, the "templating" is being done by the judges and/or lawyers reading the contract, not by an expert system writing one.
Edit: of course any such contract might contain a lot of "conditional logic" that would only apply in very specific circumstances and thus would prove irrelevant for most users. But although the contract would be long, you'd only need to write it once, and there would only be one copy to maintain.