August (00:00):

Welcome everyone. Thank you so much for joining us for today's webinar with Evisort. My name is August and I'll be your host this morning. Today we'll be showing Evisort's monthly demo on generative AI presented by Hal Marcus, our VP of Product Marketing at Evisort. Just a little bit of housekeeping before we get started. If you have any questions during the presentation, please type them into the Q&A box in your Zoom control panel and we'll answer them at the end. Now without further ado, we'll turn our time over to Hal. Hal, go ahead and take it away.

Hal Marcus (00:29):

Thanks, Augie. All right, so let's jump in. First off, as Augie just said, my name is Hal Marcus, VP of Product Marketing, so there's something I need to get out of the way right upfront, so let's deal with that first. Yes, my name really is Hal and I'm here to talk to you about AI and not to be afraid of it. If you don't get this reference, by the way, congratulations, you're younger than I am. If I don't deal with this though upfront, then there's always somebody in the Q&A typing and open the pod bay doors and things like that. So want to get this covered before I needed to, and I already see a raised hand, so I think someone's about to ask me to open the pod bay doors or sing Daisy or something like that.

(01:11):

Anyway, thanks for being with us today. It's great to talk to you about this topic. My name is Hal. Again, I've been in the legal technology space for quite some years now. I started as a practicing litigator. I'm a former general counsel and I've been working with AI in legal technology now for over a decade. What's changed recently is basically this: generative AI is really changing the game in terms of people's awareness, their appetite, their interest level, in what's going on with AI. It's been a hot topic for a while, but suddenly this is the year. The advent of generative AI, everywhere we go, these are all from recent events. We've been at Clock and Concero and Hall and all these different events. This is what everyone wants to talk about, tons of questions coming up.

(02:00):

That said, most of the inquiries not necessarily that well-informed about the nuances of what's going on and for all of the interest that it's generating, there's also equal parts, I would say, trepidation and sometimes even misinformation. So it's really great to be able to delve into what is generative AI, what does it mean for the space of contract management, what is Evisort doing about it?

(02:25):

So let's do a quick definitional intro for anyone that's not too well acquainted with it. Generative AI is actually a category of AI. It's a pretty broad class of artificial intelligence that helps automate the creation of content. So this is not just about text and large language models like GPT, which is really what's captured a lot of the attention. It's also about other kinds of generative tools for imagery and for other things, for code. So it's being used in a lot of different ways, and one of the reasons it's grabbing so much attention is that it's not just one thing. It's really spanning a lot of industries. It's spanning a lot of use cases and it can have such an impact in such a short period of time, and it's getting so much better all the time. So for all those reasons, it's grabbing lots of attention and understandably so.

(03:16):

We've got a visual there on the screen of ChatGPT, OpenAI's publicly available chat bot. Now, this was launched in November. I don't have the latest stats on it, but I know as of only two months into its release, it was up to 30 million subscribers and by subscribers, it was free, but these were people that had signed up for accounts and connected to it and were using it and 5 million people were saying that they were using it daily, and that was a few months ago. So in a short period of time, a whole lot has happened here.

(03:49):

Now that said, as I mentioned, trepidation and misinformation and all the rest of it has come along with that. There have been a lot of issues that this has raised, and among them, we've seen companies manning the use of ChatGPT and other publicly available generative AI tools, including Apple recently, just the other day announced that. We've also seen strong adoption of it. Allen & Overy, for example, major law firm, made strong announcements about using this pretty extensively throughout the firm. A judge in India used it in a court proceeding to look things up and then to write some content. I think some issue arose out of that though Samsung had a data, not a breach, I don't want to say, but some coders, some engineers at Samsung put into, I think it was co-pilot, which they were using to generate code. They put in some code that they really shouldn't have, and that became an issue.

(04:48):

I know that OpenAI recently acknowledged that they had a data breach, so some of the prompts that had been put in by users to create new content had been exposed to others. And there's lots of IP litigation happening around the use of people's data, including artistic works to train generative AI without giving consent necessarily. So there'll be some interesting issues going through the courts.

(05:13):

So there's so much swirling around all of this. What's really interesting from our perspective and why we are engaged with this and hopefully why you're here today, is what are the implications for contract management and the legal profession a little more broadly? Because OpenAI's GPT and other large language models have great potential to generate legal language, including contracts, and many of you may be aware, actually ChatGPT passed the bar exam I think in the top 10% recently, and I think it was one of the tougher bars. So this is real. This is happening. It's very significant.

(05:50):

Now, all of that said, ChatGPT and these large language models were not trained specifically for legal use or for contract use in particular. So there's a lot of implications both ways about how you leverage this for contract management. All right, why do we care about this? Because Evisort is AI first contract management. We're an AI company, started in 2016 with contract AI to extract data effectively from large volumes of contracts at scale very efficiently, very quickly, and with a high degree of accuracy. And that's endemic throughout the platform. I joined Evisort earlier this year impressed by the quality of the AI having worked on AI in these kinds of use cases again for quite some years now.

(06:37):

So what we're about is we connect to all of your contracts wherever they are. We apply the AI to analyze them and extract the clauses and terms that are of interest to you. We give you control to do more of that yourself, more on that later. And then we use those tools as well to facilitate the entire management process for pre signature contracts so you can get through negotiations more effectively.

(07:01):

Now as we have been bringing generative AI into Evisort, there are three important points I want to relay upfront before we get into the specifics, and then I show you the live demo. Our use of generative AI compliments but does not replace our proven proprietary contract AI. The reason, again, that I joined the company. That's significant because we're seeing a lot of companies now jump into AI. Sometimes for the first time. Sometimes it's a swap out for whatever they've licensed into their product in the last couple of years and substituting GPT and just saying that's it. Large language models, this is the way to go, that will be the new extraction engine and anything else that we need in our product, and they're building around it very quickly. So great to see all of this energy and attention go to it, but we have industry leading contract AI that was purpose built for this. So that's what we're building around, and the generative AI is adding its aspects to it.

(08:00):

There's a lot of concerns over data privacy as I alluded to before, and being in control of your data, not having the AI do things for you that you don't stay in control of. That's not a factor with how we're applying this, and I'll show you more on that as we go. And then I also want to call out that this is just the beginning, especially what I'm going to show you in the demo today. There's much more innovation on this front to come and it's happening quickly. And the reason I think that's important to call out is as soon as you see the basics of what the generative AI can do, the mind tends to go very quickly toward what if it could also do this or what if I could skip that step or automate this piece of it? And so there's a lot in the works there and I'll give you a sense of what some of that is before we're done.

(08:44):

Okay. As I've mentioned, our use of generative AI is complimentary to all the other AI that we've had in our product from the beginning, and I want to call out what that is so that those distinctions are clear and the use cases are clear. So we have four categories of AI within Evisort. Let's go through them individually. There's our pre-trained AI. This is AI that works right from the start. More than 70 AI models deployed as a standard, hundreds more available that can be deployed. They're trained on 11 million contracts, over a billion data points have been extracted, and we also combine that AI with an advanced and unique OCR plus AI process. So as we scan all of your agreements, we bring them in in a way that ensures that we've got really good clean texts, that we're clearing up errors. We can identify handwriting, we can identify non-Latin characters, we can deal with headers and footers and more.

(09:44):

Because if you don't have good quality texts coming in, then the AI doesn't have very much to work with. So it's very important that there'd be high quality there, and that's definitely a differentiator. So that's the pre-trained AI. Pull in any contract and this is what it will apply.

(10:01):

Then there's the ability for you to adjust the AI, create your own model, search for the things that you want, and track them on the fly. And we enable that in two ways. What we call quick AI or guided AI. Quick AI is just about selecting the clause of interest in a contract, and then the system will find semantically similar clauses across all of your agreements. Semantically similar means that doesn't have to be exactly the same wording. If it's termination for convenience clause you're looking for, it doesn't have to be labeled termination for convenience to be found. Language that suggests that will be enough for us to identify that with the AI as a similar clause. So it's just the fact that you can find more like this so easily. That's why we call it quick AI.

(10:48):

Then there's guided AI, which is if you want to get a bit more sophisticated, do some rounds of iteration, do thumbs up and thumbs down to say, yes, this is what I'm looking for. Nope, that one doesn't quite meet the criteria and refine the models more effectively. We do this on a no code basis. There's no special skills you need. There's no external dependencies. You don't have to hire anyone to do it, though we do have a great partner ecosystem and folks that are trained up with practice areas around this if you want them.

(11:17):

But this is so straightforward to use, you can do it yourself and companies do all the time from their legal teams. It's great to see Microsoft there as a user and a quote, but that might have people thinking that you need Microsoft engineers for this. You don't. That's the legal operations team doing this work, and it gives you the ability then as you see on the left, to look for new things that pop up as they arise. ABA model clauses for human rights. When GDPR, when the standard contractual clauses were introduced, suddenly there was something new. People wanted to look through all their agreements to find. We make it easy to do that. Tracking FCPA compliance on and on. So for all of these special use cases, you can just create your models as you go.

(11:58):

So that gets us to generative AI. So what is that bringing? Generative AI is leveraging the large language models that are all about predicting human language. What word comes next? If you've never used ChatGPT, you kind of have without realizing it in a sense. You've used large language models in that when you're writing a text and it's suggesting the next word you may want to use or you're writing an email and a phrase is popping up for you to accept or reject, that's really the same underlying technology give or take that's being used. It's predictive language. So we get new capabilities from that that are really advantageous and they help you with two features that we're releasing now, automated redlining and clause creator. So let's jump into what those are.

(12:46):

Automated redlining, just as it sounds like, it's about making the redlining process the negotiation of your contract easier and faster. Now, automated tends to imply to people that it just happens and it just does it. And of course, we're not quite there yet, A. And B, you don't necessarily want us to be because you want to be reviewing any of these changes before they go into a contract and go out to the other side in a negotiation. What we're doing is making it easier for you to make those edits and it's more effective as a lot of people that negotiate contracts now to make minimal edits, right? To make surgical edits to a clause rather than just rip and replace it. Because doing that, saying, I don't like your indemnification clause, I'm just going to give you ours, that can then lead to a lot of back and forth.

(13:34):

So if you can make the changes in a more surgical way, that's a much better way to get to signature faster when you can do that. Well, this makes it a little easier to do that, saves you some of that work. You enter guidance on what needs to be changed, and then you have suggested edits that you can copy into your document. And I'll take you live into this shortly, but that's automated redlining.

(13:56):

And then the other is clause creator. So this is about generating a new clause from whole clause. Something new that's not in the contract, you need it, you don't have a clause library set up or you don't have quick access to a clause or it's something very specific to this particular contract and you want to add some language in. Rather than draft that clause from the start on a sheet of paper if you will, a blank sheet of paper. Rather instead, you can get a suggested clause and you'll be really surprised at the quality of what you'll likely see. So that's automated redlining and clause creator. Let's go live into the product. We'll take a look at that now.

(14:40):

So I'm going into Evisort and I'm going to start here in the workflow section. So I'm in my workflow tab because this is where pre signature contracts exist and these are the tickets that have been set up that I'm connected to either as a reviewer or something that I started myself that could have been started by creating a new ticket, starting a new contract from a template with a workflow that's built in, or it could have been from a third party agreement that just came in and I've loaded it into the system. I've just done a simple upload. The AI is automatically applied, extractions are done, clauses are identified, and then I have that agreement to take a look at and start working on.

(15:20):

So I'm going to go into this vendor services agreement, and when this contract pops up, we'll see that it's already identified certain key clauses, and I'll show that on the right here by clicking on the clause tab. And that's interactive with what you see on the left with the actual agreement itself. So for example, if I want to go to the indemnification clause, I can just click right there and it will take me into what it automatically found with the AI to be the indemnification clause. In this case, not the greatest challenge. It is labeled that way, but even if it hadn't been, the AI would've been able to identify that based on the semantic meaning, the implications of the language that's there.

(16:10):

Now, I'm looking at this and I go, you know what? This doesn't really meet all of our requirements. There's a number of things that I'm going to need to do with this. So I could go into edit mode and start editing this document manually making the changes that I think are going to work, or if I have another full clause at the ready, I could swap it out. But again, as I mentioned before, then you've got that back and forth issue and that's not the fastest way to get there.

(16:35):

So a little more effective might be instead, I'm going to click over here on the little magic wand that we have with generative AI. Click on that and it will take me to the automated redlining clause creator options. What I'm going to use right now is automated redlining. So I'm just going to select that clause and I'm going to paste it in here, and then I'm going to give it the instructions of what it is that I want to change.

(17:01):

Now, I do have a few things I know I want to do here. I could just start typing on the fly or I know that these are four things that are going to be important because this is how we approach indemnification in our organization. That's something that I've lived as a former GC where making exceptions on indemnification was a real issue for us and we had to take that very seriously. So here are some of our requirements. I'm going to put those in. All I need to do now is click generate. This is sending a call out to the large language model to give us suggested red lines to the agreement, and there they are.

(17:41):

So couple of things to note here. Sometimes it needs just put new texts like it did here at the end. Sometimes it can just make minor changes throughout. It also identified that in order to make the change that I want, it was going to have to delete this whole section because it goes too far in what it offers. So it's showing me exactly what those changes are. It's easy to spot what's been removed and added.

(18:04):

Something else to note here. I did not have customer or losses capitalized in the instructions that I gave, but the GPT was clever enough to identify that those were defined terms within the agreement, and so it actually capitalized them for me. So it's pretty remarkable what the AI can deliver back. But here's also another note on that. If you try doing this with just ChatGPT and doing things like this, sometimes, well, firstly you won't have the redlining, but if you try to ask it to do various things, sometimes you'll get good results, sometimes less. So a lot of it is a function of the prompt. What is the quality of the instructions that you're giving it? Well, the instructions that I just gave it right now went beyond those four bullets that I pasted in because behind the scenes, Evisort has actually been programmed pretty extensively with a whole lot of other instructions for the GPT to define the scope of what it is that we're looking to do with auto redlining.

(19:05):

So this is really helpful in terms of... Oh, I just saw an interesting question pop up. Did it remove the definition? I think it still has losses in there. I'm sorry you didn't see that. Somebody called out that it recognized losses as a defined term, but did it also remove the definition? In this version, it does look like it removed the quotes. Interesting point. And if that is something that I don't want to see happen, which in this case it would not be, another thing I can do is click on edit mode and go there and make that change myself. So nothing that's happening here locks you down.

(19:45):

By the way, I want to look at one more thing. You may notice there are a couple of other options that appear, and I want to see if those are different in that regard. Nope, they're different in other ways. Comes up with a couple of different ways that it can word things so you have options, but in this case, you're right, losses. It did take out that definition. So if it wasn't defined elsewhere in the agreement, that's something I would want to check. So as always, you need to be in control of this. You need to be keeping your eye on what this is doing. But again, behind the scenes, there's a lot more happening in the prompt that we've hard coded in to help ensure that you're getting some good results.

(20:22):

All right, so that's one way of generating auto redlining. I'm going to move on to clause creator in the interest of time. So let me switch over there. Now, this is again, what happens when you want to just create a new point, a new clause, new provision from poll clause. So let's say here, I realize in this agreement there doesn't happen to be an assignment clause. I'm just going to tell it draft an assignment clause requiring 30 days written notice, make sure my spelling is fairly good, so I'm not giving it a word it doesn't know what to do with. And I'm going to say generate.

(21:06):

And within a few seconds we'll have something a little better than what I was able to create in that period of time. And again, as always, multiple options, different ways of wording it. If I like what I see here, I've got text I can copy right into the agreement. Let me try another one. Sometimes you want to do something that is very, you don't have in the clause library because it's specific to this particular agreement and you're something you realized on the fly that you want. So I'm going to say write a clause granting Hal Marcus full signing authority as an officer of the company. You generate more. I could also have hit reset and it would've cleared that away, but instead I just did generate more. So it gives me a fourth suggested clause, and there's the one that I just requested.

(22:04):

So none of this is meant to imply that, again, you're not in control, you're not looking at all of these changes, you're just leaving into the bot to do all of this automatically. But I think you can see the potential for how we can get more and more automated in these capabilities over time. The more you set up your guardrails and your guidelines for what you want to see happen.

(22:27):

All right, let me go back over to the PowerPoint to raise a couple more points and we'll talk about data privacy and some roadmap elements as well. So, as I just mentioned, you can see the potential with all of this. This is all early stage new technology, something that we're incorporating into the product and it's already in the hands of customers that are giving us great feedback and experimentation around how to use this to really make their workflows a lot more efficient. Also, to identify pitfalls and ways to train it better, ways to get the prompts as tight as they can be. But the more that we can automate this process, the better. And a key part of that is ensuring that you've got a clause library with your preferred language, your fallback language, and the guidance on when to use each.

(23:22):

Now, what we've heard from so many customers, and I've seen it even in previous roles in my career, is that it takes a lot of work to set up a clause library, to define all of those parameters. And as soon as you do, these things tend to fall out of date. In a dynamic organization, it's hard to get all of this into one place.

(23:39):

So we are developing an intelligent clause library where our AI, which is so good at spotting semantically related clauses across all of your agreements, we're going to be able to help you build and maintain your clause library on a much more automated basis. And then you've got the generative AI to adapt content and create new content as needed. So the more we have those instructions built in, the more we're able to automatically prompt the system, automatically identify where the clause needs to be changed to be compliant with what it is that you would normally be asking for it. So like I said at the beginning, a lot of new innovation happening on this front. This is very much just the beginning.

(24:26):

Now, one more thing I want to cover and very important before we take questions that remain for the last number of minutes, and that is around data privacy. As I indicated at the top, there have been a lot of concerns and rightly so around the implications of publicly available chat bots and such for data privacy implications. And that extends to all your contract data and how you use it across the board. It's not just for publicly available systems, but that is often the misperception that because people are familiar with ChatGPT and we're incorporating GPT into our system, people think those two are closely related. Well, we're licensing GPT4 via the Microsoft Azure OpenAI service. That's what we've incorporated into our system. So no data's being shared with public chat bots. We haven't added sub-processors. Anyone doing GDPR compliance knows the importance of that or CCPA, CPRA and all the rest. Data is deleted via clear retention policies published by Microsoft and on and on.

(25:28):

Now, one more point related to this. It's really significant. You see that dotted line there? We've been getting the questions from some folks, if our organization has policies around generative AI and we're not allowed to use it, or if we have other concerns for any reason, can we disable that in your product without sacrificing any other functionality? And the answer is yes. The way we have deployed this, this is its own area, its own modular component, if you will, of the product. So you don't lose out on any of the other AI capabilities that Evisort is known for by all of our customers.

(26:04):

And the way that we've done this, we're being very careful because we're not going to compromise our ISO 27701 certification for data privacy, which is unusual among CLM providers, and we take that very seriously. So any further questions around data privacy and using training and all of that, strongly encourage those questions to come in.

(26:28):

I've got one last very quick note to give you and then we've got a few minutes still for questions. I see a couple that have accumulated and that is this. It's always a great idea, especially with cutting edge technology like this, try before you buy, do a proof of concept. And Evisort is known for this and very happy to do this. Note, we say with your contract data, we don't mean 10 contracts. We mean give us your contract data. Let us set up a proof of concept and ingest a whole lot of your contract data so that you can really live in the system, understand how it works, see firsthand for yourself what it does. Companies can talk about accuracy and reliability all they want, but at the end of the day, until you're using it, you're never really sure. And that's the way that we encourage you to do it. So please do reach out to us. We'd be very happy to work with you on a proof of concept so you can see this for yourself.

(27:20):

Okay, let's look at some of the questions that have come in. I see a question there of does the tool have an approval function and does the clause creator point out alternate versions of the clause which are acceptable with approvals? So yeah, hopefully, you saw in the demo, you will normally get three alternative versions. Sometimes they're not very different. Sometimes they are meaningfully different. Depends on the prompt, depends on a lot of variables. But yes, we'll normally give you a few versions you can look at and yes, you can grab any of them or edit them as you like either before you paste it into the agreement or after. So yeah, you have total control over that.

(28:05):

I see a question of how do you train your models without sharing the models, I think... Oh, I see. How does a customer then train the models without sharing the models? Yeah, so you've got the ability to train your own models and those models then are not shared. We could spend more time on the different options that you have about how data is used to train models and be happy to get into that with you on a deeper level. But suffice to say, yes, you have control over your own models that you train.

(28:39):

If you have integrated with Microsoft Azure services, isn't that a sub-processor? That's something worth delving a little more specifically into in another context in terms of what I mean when I say we don't add any sub-processors is we are delivering this in a way where we're not compromising the tech stack and model that we lay forward for our customers. So that's a little more nuanced point that'd be worth exploring further and be happy to do that with you.

(29:10):

Where an embedded definition is removed from a section, will the program add that definition to the top of the document? So the short answer is no. You are only going to make edits to the text that you've pasted in to make edits to. Those are the only suggested edits that will happen that will be proposed to you automatically. So the fact that in that case it removed the quotes around that definition so it wasn't clear it was a definition, that is something that would be worth making sure that that's caught and thanks for people for pointing that out.

(29:47):

And then I think we are just about out of time. And so yeah, we are out of time. So I leave it at that. We really hope that you reach out to us and raise more questions with us. Again, this is a fast moving area. I'm sure if we do this again in two months, it'll be different visuals, different story, a lot more happening on this front. So definitely stay tuned and we look forward to lots of interesting conversations with all of you about the advent of generative AI for contract management.

‍