JFrog is deepening the niche it started carving out in MLOps last year, acquiring MLOps partners to better connect AI model development with the DevSecOps pipeline.
In February, JFrog partnered with Israel-based machine learning operations (MLOps) startup Qwak to integrate JFrog's Artifactory and Xray software artifact management products into Qwak's platform for building, training and deploying AI models and apps. Now, JFrog has acquired Qwak for an undisclosed amount and plans to integrate the company's AI model training and servicing workflows across JFrog's DevSecOps product line.
“Fundamentally, we've seen customers increasingly embed machine learning into the typical software supply chain,” said Yoav Landman, CTO and co-founder at JFrog. “We're really seeing a huge boost from GenAI.”
Landman said these companies are finding that malicious code appears in open-source AI models at similar rates to other open-source package repositories, so they are exploring similar ways to scan and filter AI models.
Many projects never make it to production due to siloed pipelines. [AI] It still needs to be part of the DevOps loop.
Yoav LandmanCTO and Co-Founder, JFrog
The Qwak acquisition takes JFrog another step closer to blending management of existing software deliverables with new AI models and applications, he said.
“what [Qwak is] “The biggest benefit of delivery is being able to put your models into production in a very simple way,” he says. “Integrating your models with the applications around them and getting them into production in a very simple way is actually very difficult. A lot of projects never get to production because of siloed pipelines. [AI] It still needs to be part of the DevOps loop.”
GenAI's failure prompts new vendor pitch
Industry surveys show that enterprise AI projects, especially generative AI, have a high failure rate, models tend to produce inaccurate or low-quality results, and the cost of training large language models can be prohibitive. A 2023 Gartner research report found that 52% of enterprise AI projects fail to move into production. More recently, a SolarWinds survey of nearly 700 IT professionals found that only 38% have a high degree of confidence in the quality of the data used to train AI models, and security was identified as the biggest barrier to AI integration.
And JFrog isn't the only one trying to bring MLOps into DevSecOps: Microsoft announced several new AI integrations and guidance documents for app developers at its Build conference in May. In April, AWS and Google took a similar approach with their Amazon Q and Vertex model development services, respectively. Other vendors, from Docker to GitLab to Red Hat, are also developing integrations between AI and DevSecOps tools and infrastructure.
Industry analysts also expect further consolidation between MLOps and DevSecOps in the form of M&A activity.
“A significant number of companies [MLOps] “We're tracking about 40 companies, including Datarobot, Domino and SAS,” says Andy Thurai, an analyst at Constellation Research. “But these companies don't offer a software pipeline management process, so they've had to integrate with one of the CI/CD companies. Going forward, we expect most of the MLOps companies to be displaced by CI/CD or AI platforms.”
Though it's not a top priority, some companies plan to integrate MLOps and DevSecOps tools this year, according to an IDC study on DevOps perceptions, practices, and tools in 2024. The study ranked “integrating MLOps and DevOps” 11th out of a total of 20 potential priorities, with 10% of 311 respondents selecting it as their first or second priority.
“This may seem low, but there was a lot of variability in this question… the top two were [options] “MLOps only garnered 19.9 percent (automation) and 16.1 percent (continuous deployment),” said Katie Norton, an IDC analyst who conducted the as-yet-unpublished study. “This data certainly validates the steps JFrog and others in the DevOps market have taken to integrate MLOps capabilities into traditional DevOps platforms.”
Early Adopters vs. Mainstream AI/ML
Deploying AI/ML apps into production will ultimately require integration with a DevSecOps pipeline, but one early adopter of generative AI app development said that MLOps and DevSecOps are not fully integrated and likely never will be, in part for security reasons.
“MLOps is used for both research and production projects to manage versions, audit logs, data lineage for GDPR, artifacts, and model performance,” says Ian Beaver, principal scientist at Verint Systems, a contact center-as-a-service provider in Melville, N.Y. “However, DevSecOps tools/practices are only applied once the research project is promoted to a production project.”
During the research phase, AI engineers work in isolated lab environments without internet access, where an MLOps framework helps manage experimental models and track experimental results, Beaver said.
“At this stage, we don't have DevSecOps built in because many of these prototype models will never see the light of day and we want to iterate quickly,” he said. “For example, we don't want researchers to spend time fixing Tensorflow vulnerabilities in experimental models when the final version may be built on top of the AWS Bedrock foundation model and we'll never deploy Tensorflow in production.”
Verint already uses several self-hosted DevSecOps tools, though Beever declined to name them due to the company's unique compliance requirements and the need to deploy AI apps globally in more than 80 languages, but he said tools such as JFrog and Qwak could be appealing to companies with smaller AI project scopes.
“I think that in projects where you know from the beginning what models and libraries will be deployed, you can build in DevSecOps tools from the beginning,” he said. “For example, if you have a corporate policy that all GenAI solutions are built with GPT-4o, it simplifies the research phase and allows security to be built in early.”
JFrog and Qwak may also be attractive to companies that don't want to centralize their AI apps with a major cloud vendor, Beever said.
Industry experts say these descriptions would ring true for many mainstream companies that are still struggling to get AI/ML apps into production.
“The separation between DevSecOps and MLOps is artificial and needs to be addressed quickly,” said Torsten Volk, an analyst in TechTarget's enterprise strategies group. “Enterprises and software vendors will eventually figure out how to best synchronize both types of processes. This may take some time, but the payoff will be great, especially for larger organizations.”
IDC analyst Jim Mercer said that even if MLOps and DevSecOps remain at least partially separate processes for IT teams, JFrog's artifact repository could be an attractive control center.
“In the rush to GenAI, people have typically worked in silos, which has contributed to efforts stalling due to poor coordination and differing perceptions of truth,” Mercer said. “There may be parallel paths, but as developers increasingly get involved in model development and integrate GenAI capabilities into their applications, the repository needs to be a common source of truth.”
Beth Pariseau, Senior News Writer at TechTarget Editorial, is an award-winning veteran of IT journalism covering DevOps. Have a tip? Email her Or reach out @Parisault TT.
