Two years ago, Sreekhand Balakrishnan had his worst performance in professional history. The 45-year-old has built a successful career as a technology leader and in 2020 was appointed Director of Technology at global travel company Travelopia. But a wave of AI seemed poised to render him worthless.
“Many senior technology leaders felt a tectonic shift beneath them. We had spent decades perfecting the technology, and suddenly there was an AI that could produce in minutes what would take a good engineer hours. The instinctive reaction was fear,” says Balakrishnan.
Balakrishnan initially dismissed AI as a fad, but realized that his fear made him even more vulnerable. “One day, a CFO will come into the office and ask how they can use AI to reduce the number of employees on their team by 50%. But either I don’t know the answer, or I can tell them I’ve found a way to use AI for more than that,” he says.
experiment and learn
Balakrishnan started learning all the AI tools available to him. Rather than being told how AI improves productivity, he wanted to experiment and gain a practical understanding of the technology. Only then will he decide what to do with it.
By settling on an “experiment and learn” approach, some nerves were calmed. People tend to be nervous about tools that might replace them. Care was also taken to install guardrails. Balakrishnan agreed to take two full-time developers from his current team and let them play with the AI for three months.
You can't lead an organization from behind. You need to encourage people to learn with you
Balakrishnan then selected six real-world business initiatives, had the new AI-enhanced team work on the projects alongside traditional human teams, and compared their performance.
Early progress was slow as engineers struggled to figure out how to use AI tools. They used inconsistent prompts and had uneven productivity. But Balakrishnan believes that chaos makes for great classrooms. “That's when we started learning where AI could accelerate value and where human intuition was most important: context, quality, storytelling.”
Balakrishnan explains that the project ran more smoothly by keeping humans informed and treating the AI as a thinking partner rather than an autopilot. The humans' job was to consider the AI's suggestions, iterate and improve the prompts, and understand that the AI is fundamentally a process, not just a tool.
After six weeks, Balakrishnan gave both teams tickets to process and the differences became apparent. “The traditional team said they needed two weeks to make the change, but the AI team asked for just an hour. It was a lightbulb moment.”
How to set up and track metrics
Another early lesson was that small projects may seem easy, but they often fail because they don't have enough impact. Instead, Balakrishnan says it's important to choose priority projects that are truly important to the business. “This means that if it doesn't go live, the CEO should call me and ask me why,” he says.
One of the first projects the AI team worked on was rewriting a module used by businesses to capture leads and assign them to the best sales reps. “We weren't convinced that AI could do it, but we were confident that if it worked, we could immediately bring revenue to the business,” says Balakrishnan.
These early projects helped define the process for implementing and evaluating AI. Everything the AI team built went live and was monitored for a month. A project is considered successful if the existing team is able to learn it, manage it, and fix at least one operational issue. If not, it was a learning opportunity.
Travelopia tracks new projects over a six-week period, collecting metrics such as time from idea to reality and total time to completion. The company also monitors value-added and non-value-added work. “Default [platforms for planning and monitoring projects] Because we didn't have enough information, we started tweaking them and tracking whether our work was mitigating risk, cutting costs, generating revenue, or improving service. Then we add a sub-metric to indicate whether AI was used or not,” Balakrishnan explains.
These metrics track 25 teams, and early results show dramatic reductions in delivery times. A complex module that took six weeks to build can now be completed in three days.
Challenges in AI integration
Significant challenges remain, including the unpredictable costs of AI. Balakrishnan estimates the company pays up to $200 (£153) per developer per month for access to multiple AI models. This equates to an annual investment of $250,000 (£191,330) and should be factored into your ROI calculations.
Don’t be the person who doesn’t know what to say when the CFO comes knocking on the door
In some cases, Travelopia chose to upgrade after realizing that a “pro” model would expose data to a training model or that the AI vendor would own the company's code. Travelopia also pays for private LLMs hosted on AWS when needed to protect commercially sensitive data.
Second, the speed of change is extremely fast. “Every two weeks, Copilot gets better, ChatGPT gets better, Gemini gets better, and every time they get better, some features start working better and other features break,” Balakrishnan said.
He admits that change management is very difficult. He spent four weeks having “heart-to-heart” conversations with employees about the company's intentions and outcomes regarding AI. While some were discouraged and wanted to quit, Balakrishnan explained that the purpose was learning, and that even if the company learned that the team could do more with AI, that would be a positive outcome.
Redeploy resources
After a year of experimentation, has Travelopia been able to save headcount and costs through AI? Balakrishnan says it's not that simple.
It certainly allows developers to work faster. Previously we had a team of 8 people divided into 4 pair programming teams, now we have 8 teams, each person paired with an AI. These teams work much faster, but this means the business needs more quality assurance engineers and more product owners. Balakrishnan has repeatedly considered whether to slow down or increase the budget and hire more people.
His approach so far has been to redeploy employees to new roles and take steps to reduce work rates. “In a normal team, you would have time to take breaks, maybe get enough sleep, and wake up with new ways to solve puzzles. AI works at such a pace that there are no breaks, so I worry that people will burn out.”
Balakrishnan's advice for technology leaders just starting their AI journey is to embrace it. “You can't lead an organization from behind. You have to learn the ecosystem and encourage people to learn together,” he says. “Above all, don’t be the guy who doesn’t know what to say when the CFO comes knocking on your door in two years.”
Two years ago, Sreekhand Balakrishnan had his worst performance in professional history. The 45-year-old has built a successful career as a technology leader and in 2020 was appointed Director of Technology at global travel company Travelopia. But a wave of AI seemed poised to render him worthless.
“Many senior technology leaders felt a tectonic shift beneath them. We had spent decades perfecting the technology, and suddenly there was an AI that could produce in minutes what would take a good engineer hours. The instinctive reaction was fear,” says Balakrishnan.
Balakrishnan initially dismissed AI as a fad, but realized that his fear made him even more vulnerable. “One day, a CFO will come into the office and ask how they can use AI to reduce the number of employees on their team by 50%. But either I don’t know the answer, or I can tell them I’ve found a way to use AI for more than that,” he says.
