Questions about the safety and data privacy of large-scale language models have also come up frequently in recent weeks as the legal industry has debated how generative artificial intelligence can be leveraged.
At the “Generative AI: Uses, Risks, and Impacts” session at the Masters Conference in Washington DC on Tuesday, panelists revealed some of the key elements of the large-scale language models behind generative AI, highlighted some of the risks and limitations of technology.
In recent months, the term generative AI has become a buzzword among many legal experts, but has been inaccurately called synonymous with ChatGPT, GPT models, large-scale language models, among other terms. It happened often.
Simply put, generative AI allows users to enter prompts into a computer program and generate text, audio, video, or image output, said Aaron, Senior Director of Applied Sciences at Relativity. Ahmadia explained.
On the other hand, a large-scale language model, which is an example of a generative AI system, is “a certain amount of text input, [predict] Text is missing or added after it. So we can see how a language model can naturally become a generative model,” Ahmadia said.
Does size matter in LLM?
The large language model itself is most like a “general-purpose engine,” said Tom Shen, machine learning team leader at Bloomberg Industry Group. By itself, its capabilities are limited.
“When you think about LLM, it helps to think of it in two different perspectives: the model itself and the application,” he said. “Usually there’s a thin layer of code or something on top of the language model that tells the language model to do whatever you want or intend to do.”
He added that some of the key differentiators between LLMs are size and data. A small amount of training data allows the model to perform well on certain features. But enhancing tools like his ChatGPT, which serves as a Swiss Army knife, “will probably require a larger and larger model.”
But bigger isn’t always better, especially for the legal industry. In fact, Shen said the legal industry is beginning to look at a “right-sizing” approach to large language models — an approach that distills larger models into smaller models that are cheaper and run more efficiently. I was.
Does LLM store user data?
Another well-known concern in the legal industry concerns the safety and data privacy these models provide. But even here the panelists cleared up some misconceptions.
“These models have additional code, components, or objects around them. So when people talk about large language models, you’re only talking about one thing: type something Then you have a prediction,” Ahmadia said. “They aren’t talking about something that stores all the data or something that updates it every time it gets a new update. It’s really just adding extra code or feeding back extra reinforcement learning to that engine. is needed.”
In fact, he explained, the model has two distinct “modes”: an update or improve mode and a scoring mode. Most models that legal professionals work with can be in scoring mode and do not store any information.
“Many of our law firm clients are very concerned about the use of GPT models because they say that either the prompts or the information you enter trains the model and trains that model. We believe that our data will be used for … that’s not true,” said Jennifer Cody, account technology strategist at Microsoft. “The model is established, but it’s not learning.”
Additionally, Cody pointed out that there are certain services, such as Azure OpenAI, that provide additional guardrails to protect user data when using LLM.
“If you’re using Azure OpenAI, for example, you have a little more control, guardrails and boundaries around using GPT models,” she explained. “So you have some definition of where your data resides, and it will be stored within your tenant within Azure. So that was a big concern for the law firm I work with. “
What LLMs can’t do
Of course, despite the recent excitement about what new capabilities advances in generative AI will bring to lawyers’ legal tech tool stacks, panelists also highlighted some of the technology’s limitations.
For example, legal professionals need to fine-tune prompts to achieve desired results, Shen explained, because large language models are highly dependent on instructions and context.
Currently, the amount of context or input that a model can consider is limited. In fact, models have something called a “context window”, which is the amount of inputs that can be considered simultaneously when making decisions.
“The idea is that LLM can only see a limited amount of input text at a time. Unfortunately, output text is bound to the same context window as input text,” Ahmadia said. “I mean, if you ask him to write a sonnet, and spend all his windows of context explaining what the sonnet needs, you’ll get something a little more like a haiku.”
