The threat of AI-generated software code raises questions about the mechanisms and methods currently used to build the next generation of software applications and data services. Many tech industry commentators agree that using AI to generate code is quick, but it is also often wasteful in that it is inefficient and comes with weaknesses ranging from lack of documentation to human intuition. However, there are places where the benefits of code automation can be realized to some extent. There is a consensus that AI code bots can be used to perform testing, debugging, and other system-level tasks, so if we know where to apply these new accelerators, we can find areas where AI-assisted development can be used effectively.
With IT teams under pressure to rapidly produce the highest quality features to the highest standards with limited time and resources, developers are constantly being pushed to do more and do better, which directly contradicts what developers truly desire at heart: the ability to pay attention to detail, ask questions, and the opportunity to perfect their work at their own pace.
“New tools like generative AI and Copilot promise to ease the burden by freeing up as much as 20 to 30 percent of developer time (according to McKinsey), but we're not yet at the stage where we can completely rely on them at face value,” said Peter McKee, head of developer relations at Sonar, a company known for its static code analysis technology.
The reality is that when it comes to AI-driven code, it is up to the developer to properly understand the output and how to produce it. And this reality is further complicated by the fact that it is widely acknowledged that AI is still in the prototype experimentation domain (some call it the hype cycle) when it comes to real-world implementation.
AI code pitfalls
Sonar's own research shows that developers typically spend almost half of their time rewriting bad code, which remains a major challenge to productivity.
“The situation may become even worse as the use of AI tools increases. We know that AI can free up developers' capabilities, allowing them to work on more complex tasks or shift their focus to projects that interest them, but it only works as intended when properly checked,” McKee said.
He noted that Stanford University researchers found that while GitHub's Copilot makes developers more likely to write insecure code, it also makes them more confident in its security. In an update to its Copilot information hub, GitHub publicly acknowledged that Copilot can generate “undesirable patterns,” adding that developers are responsible for ensuring the security and quality of their code, and recommends taking the same precautions as if they weren't writing the code themselves.
Is this really robot reasoning?
“Gen-AI coding assistants are good at suggesting code, but not good at stepping back and reflecting on the code and reasoning about its validity,” McKee advises. “Layering AI tools into the coding process can produce more code faster, but if the quality is not good it can take time and increase developer frustration. The editing process to fix issues in the generative AI code is probably not worth the time it would take a developer to write the code themselves from scratch.”
Of course, we know that AI is like any other technology that moves users higher up the abstraction stack, making the task they're trying to perform less painful. McKee and the Sonar team think of AI as essentially a “better hammer.” [but perhaps not quite a perfect artisan craftsperson’s tool] Better than anything we've done before… and it looks promising. It's clear that AI technology has a lot more room to improve, but for now, developers need to make sure they don't miss out on some key steps to ensure that their AI-generated code is clean.
Using AI the right way
“Right now, generative AI is more of a hindrance than anything — we're not at a point where developers can trust the quality of the output without cross-checking,” McKee suggests. “It's up to the developers to address the underlying issues that are causing bad code to get into production in the first place, so that code generated by an AI generator doesn't just create more code to fix.”
To effectively integrate AI into workflows and avoid disruption, Sonar development leaders say it's essential for development teams to adopt a Clean as You Code approach. In full disclosure, this is exactly the brand tool the company offers. McKee suggests that by adopting this methodology, software developers can be confident that their code is consistent, intentional, adaptive, and responsible. This leads to software that is secure, maintainable, reliable, and accessible.
“and [this] “This methodology allows developers to improve the quality of the entire code base with minimal cost and effort. The focus on quality and security doesn't have to come at the expense of speed. Automation allows developers to clean up as they work, making them more productive and turning the underlying code into a business asset. Developers can rely on AI for volume and need to be able to put in place the right checks to ensure the correctness and security of their code,” said McKee.
Ultimately, the debate seems balanced. That balance comes from the understanding that AI can increase productivity, but can also negatively impact code quality if left unchecked.
Feel the pain of changing code
“A recent study from GitClear underscores this idea, which found that code churn (the percentage of lines of code that are abandoned within two weeks of being created) is on the rise and is expected to double by 2024. AI requires guardrails to be in place when it comes to software development, so developers need to 'trust but verify' when using AI,” McKee concluded.
As GitClear points out, GitHub has published some research on the growth of AI and its impact on software development. There is a need to understand the speed and acceleration here, but with the abundance of generative AI-powered, large language model (LLM)-based stuff, for every gain you get, you have to deal with drawbacks in maintainability, understandability, and functionality. GitHub asks whether the AI code you see is closer to the careful and polished contributions of a senior developer, or closer to the disjointed work of a short-term contractor. This is a good question to ask for all the AI tools we employ today, but it might not be.
