As developers continue to strive for faster time-to-market, the latest tool in their toolbox is generative AI (GenAI). From code generation to documentation to product marketing, this technology is having a positive impact on productivity, even if it's not as much as the current hype suggests.For example, global agricultural and construction companies CNH Industrial achieved a net 5% increase in developer productivity after debugging and security scans, but the benefits don't end there. The company uses generative AI in a variety of ways.
Currently, CNH has approximately 2,000 developers, 10% of whom are actively using GenAI. The company has launched its first customer-facing application that allows dealer technicians to use their mobile phone and app to inquire about how to repair their vehicle, rather than consulting a 500-page manual or PDF file. .
“We were able to go from dealer discussion to prototype in 30 days, from prototype to pilot in 75 days, and from pilot to launch in 60 days, which meant a total of five and a half months from idea to first product. Mark Kermish, global chief digital and innovation officer at CNH, said:
Farmers can now track their fleet of vehicles and the agronomic data generated by those vehicles.
Kamish's team has been experimenting with GitHub Copilot since the summer of 2023. So far, traditional web-based technologies such as C#, Java, HTML, and SQL are being used the most. Specifically, I use it to develop test cases, comment code, and develop repeatable procedures. It has less adoption and effectiveness when it comes to embedded systems that utilize C and C++ code.
The team is also using Microsoft Copilot and testing Google Gemini for things like creating job descriptions, press releases, and employee information.
Queries such as huge data sources
Data orchestration platform provider Astronomer Use generative AI for code generation and enterprise chat. The company's customers expressed interest in prompt-based code generation. This feature has already been successfully used by several customers. Typically, these customers use Python code to create data pipelines and then deploy and run them on the Astronomer platform, but they are looking for help creating these pipelines.
Astronomer has also created a chatbot that can be used to answer engineering questions across the company's public documentation pages, Slack chats, Stack Overflow, and other sources. According to Julian LaNeve, Astronomer's chief technology officer, the chatbot's code has been open sourced, along with the pipeline that feeds the chatbot.
“I think the big difference was making sure you got the answer in the last 20%. It's very easy to get the first 80% using a commercial or open source LLM, but the last 20% It was very difficult to get 20% of that, especially when it comes to keeping it up to date,” LaNeve said. “Fortunately, we are in the data pipeline business, so we were able to do that by simply creating a pipeline.”
It has a reference implementation so others can understand how to build similar enterprise chatbots. This includes checking if the data is up to date, what the pipeline should be, what LLMs to use and how to interact with them, etc. As a way to monitor the quality and cost of responses.
New technology, new benchmarks
Digital services company West Monroe uses Github Copilot for difficult, repetitive tasks and Nigel (built on the ChatGPT engine) for simpler tasks, such as integrating code with well-documented third-party APIs. We are using our own chatbot. However, LLM requires context, such as telling developers to generate integrations when using the .NET framework and specific libraries.
Sean McHale, a former software developer and partner at West Monroe, focuses on mergers and acquisitions. In this space, private equity firms interested in acquisitions want to understand what tools and technologies their target companies are using, including how they use GenAI in their coding.
“We look at the whole R&D thing: R&D benchmarks, SDLC methodologies, and how we use different tools to complement our work,” McHale says. “I think over the past year we've started to see more companies using generative AI, and we're still building benchmarks. From an in-house developer perspective, if all of a sudden there's no generative AI development, it's going to be a stone age. It's like going back to.”
Internally, McHale said that while GenAI is not used for all development, the use of generative AI has increased developer efficiency by 15% to 25%.
Code understanding and code generation
Joe Reeve, Software Engineering Manager at a Digital Analytics Company amplitude Although GenAI can help you understand code, GenAI warns that developers should be careful with code generation as it can introduce coding errors. For example, when I was working on a hackathon project involving collision detection, the generative AI wrote about 20 lines of code based on function names. It would take Reeve about 20 minutes to write the code himself, so generative AI appears to be a huge time saver.
After a few hours, it became clear that the code wasn't properly detecting collisions, so Reeve had to spend three to four hours trying to determine the underlying case. It turns out that the code written by the AI was inserting a less-than symbol where a greater-than symbol should have been inserted.
“That's a really amazing mistake. Even while debugging, I thought there was no way that function could be wrong, and I finally came across this function. [generated] All at once,” Reeve says.
Reeve, one of the first users of Github Copilot, says the technology allows him to think much faster.
“It's definitely a time saver, but it doesn't change your typing speed. Instead, it allows you to consider higher level chunks. [and] It cuts out a lot of detailed thinking,” Reeve says.
Mr. Alexei Didik, Representative [the] The Engineering Excellence Program at EPAM Systems, a global provider of digital engineering, cloud, and AI-enabled transformation services, created systems in Python using generative AI without Python knowledge. (He is actually his Java developer.)
“In the short term, we wouldn't have been able to do it without generative AI. It all depends on the applicability, the system,” Didik says. “[W]If you have a lot of legacy code, the codebase itself is very specific to that company, so the results won't be as magical or immediate, but still. [generative AI] is a very powerful technology. After updating Gemini Pro to the latest version, [introduces] 1 million tokens, that [will] Increase the context available to understand how LLM generates appropriate code. ”
EPAM has been using generative AI since its initial implementation and has seen productivity gains at the individual level, but less so at the team level. In fact, the company recently white paperbased on our client engagements, explains what organizations should do to succeed with generative AI.
“If you're talking to engineers, there are certainly gains in terms of individual productivity and individual performance, but how do you translate that into team performance, product performance, company performance, business performance? It is much more difficult to decide whether to do so or not,” Didik says. “A generative AI can generate elements of code, but he still needs the entire SDLC [in place] Validate the correctness of your solution. It should be validated against a broader set of requirements, for example a more comprehensive set of user stories. ”
Generative AI is rapidly being adopted within and outside of IT, but it's still in its infancy, so organizations' expectations should be set accordingly.
