The new suite is purpose-built to assess and monitor production deployments of language models at scale
Berkeley, California, May 18, 2023 /PRNewswire/ — Arize AI, the market leader in machine learning observability, today debuted new capabilities for fine-tuning and monitoring large-scale language models (LLMs). This product brings greater control and insight to the teams looking to build with LLM.
As industry retoolers and data scientists begin to apply the underlying models to new use cases, there is a clear need for new LLMOps tools to reliably assess, monitor, and troubleshoot these models.according to Recent research43% of machine learning teams cite “response accuracy and hallucinations” as one of the biggest barriers to deploying LLM in production.
Arize’s LLM observability tool, now available as part of a free product, evaluates LLM responses, identifies areas for rapid engineering improvement, and uses vector similarity searches to identify fine-tuning opportunities. It is the first tool to identify. The new product is built to work with: phoenixan open-source library for LLM evaluation announced on stage at Arize:Observe.
By leveraging Arize, teams can:
- Detecting Problematic Prompts and Responses: By monitoring the model’s prompt/response embedding performance using LLM evaluation scores and cluster analysis, the team can narrow down areas where LLM needs improvement.
- Analyzing Clusters Using LLM Evaluation Metrics and GPT-4: Automatically generate clusters of semantically similar data points and sort them by performance. Arize supports LLM-assisted metrics, task-specific metrics, along with user feedback. Integration with ChatGPT allows teams to analyze clusters for deeper insights.
- Improving LLM response with rapid engineering: Identify prompt/response clusters with low evaluation scores. The workflow suggests how LLM models can enhance prompts to generate better responses and improve acceptance rates.
- Fine-tuning LLM using vector similarity search: Find problematic clusters, such as inaccurate or unhelpful responses, and fine-tune them with better data. Vector similarity searches can lead you to other examples of emerging problems, so you can start enriching your data before the problem is codified.
- Leverage pre-built clusters for prescriptive analysis: Use prebuilt global clusters identified by the Arize algorithm or define your own custom clusters to simplify RCA and make prescriptive improvements to the generative model.
“Despite the power of these models, the risks of implementing LLM in high-risk environments can be immense.” Jason Lopatecki, CEO and co-founder of Arise. “As new applications are built, Arize LLM observability provides the right guardrails to safely innovate on this new technology.”
About Arise AI
Arize AI is a machine learning observability platform that enables ML teams to deliver and maintain more successful AI in production. With Arize’s automated model monitoring and observability platform, ML teams can quickly detect problems when they occur, troubleshoot why something went wrong, and analyze structured data and images and large language models. It can improve overall model performance across both. Arize is a remote-first company, headquartered in Berkeley, California.
Media contact: crystal kirkland, [email protected]
Source Arise AI
