At a scripting event not being held in London, DevOps Company Harness has announced the latest AI-driven modules, including AI pipeline builders, AI test automation, autonomous code fixes when a build breaks down, AI AppSec (application security), and even AI-driven Chaos testing.
According to Harness, software teams spend only 30-40% of their time planning and coding, with the rest being spent by testing, protecting, deploying and optimizing their applications. The meaning is that by extending AI support to these postcode processes, significant productivity increases can be achieved.
Harness CEO and co-founder Jyoti Bansal Presents is not registered in London
Given that AI is non-deterministic and vulnerable to issues like hallucinations and rapid injection, is it safe for security to leave critical DevOps processes to technology?
CEO and co-founder Jyoti Bansal said hallucinations are more likely to occur with the commonly used language models (LLMS) than Harness AI agents. Considering tasks like building and creating deployment pipelines, he said, “Our agents split tasks into smaller tasks and have dedicated agents for those smaller tasks.” “These agents also intersect each other's output.”
Another important factor, he said, is the organization's knowledge and knowledge of “what we've done in the past, builds, security testing, code changes, services and dependencies.”
Plus, he assured us. “None of this is done without human input… Our AI is not doing deployment to production. Our AI is creating a deterministic pipeline to do deployment to production.” He said that non-determinism cannot be completely removed, but after AI creates an automation, he said, “You audit it, you review it, its compliance, its governance… We are not creating a deployment pipeline and automation through AI, it is deterministic and iterative because there is no AI when you are running it.
However, the problem of checking humans, sometimes called “looping people,” is difficult. In the keynote speech at the event, Bangsal explained how non-experts who previously failed to create user interface tests can do by explaining what they need in plain English. He also said that using AI-generated code can make it difficult for humans to check every line, as the amount of code can be four times larger than before. “That's why it's necessary for the check and balance process to be much more robust,” Bansal said. “The whole process needs to be significantly improved: all testing, deployment, rollback, governance, compliance, everything around it.”
With a harness platform, considering that all these processes can include AI, does that mean you are asking the AI to check its own output?
“You can check with the AI what the AI has done, but I always recommend that you need two different AIs. Don't trust them to do both.
The related problem is that the very human skills needed to validate code or complex processes can be difficult to find out whether people rely on AI or not.
“I think engineers need skills like AI Taskmaster,” Bansal said. “A good engineer needs to understand what to ask AI and how to do it better and better… they're becoming the best skills.”
Using AI is “an iterative process. Ask AI something, review it, ask it, review it.
This is reminiscent of the conclusions of the latest Google Dora (Devops Research and Assessment) report, and in fact, DORA's research was mentioned in its unscripted keynote.
There is one good news for AI skeptics. The AI on the harness platform is optional. “You can turn everything off of AI and even tear it apart,” Bansal said. “Testing is fine with AI, but you don't want AI for security runtime protection. You can do it for different teams and applications too… There are different degrees of comfort and skepticism and allow it.”
The Harness DevOps platform runs on Kubernetes and delegates workers who run either in the cloud or on-premises (80% cloud according to Bansal) and always run on-premises or in the organization's virtual private cloud. There are small team free plans, essential plans for up to 500 users at $30 per user per month, and enterprise plans usually cost 100k-200k per year. ®
