Case Study – Beyond the Journey
Australian experiential tourism operator Journey Beyond is using agent AI to reduce governance risks for customer-facing AI after discovering that conversational AI chatbots cannot be reliably constrained to their intended purpose.
The Adelaide-based company has around 2,000 employees across Australia, annual revenues of nearly $1 billion and operates 24 tourism brands including The Ghan, Indian Pacific, Overland Rail Services, Outback Spirit Tours and Monarto Safari Resort.
Madhumita Mazumdar, executive general manager of technology, said the company has four customer-facing AI agents in operation and plans to deploy six more this year. These agents can pull information across Journey Beyond’s various content pages, including the reservation system, and provide relevant and accurate answers to customer questions.
Mazumdar said the reason for the agent technology was to ensure that the agent’s behavior was predictable and free from “haphazard behavior.” Agents are designed to work within defined tasks and approved data sources, escalating unresolved inquiries rather than attempting to answer every question.
“This is an agent, not a chatbot,” Mazumdar said. “Any question you have, they’ll answer it. If they can’t answer your question, they’ll create a case for someone to contact you later.”
She said the most complex test of the technology to date was its application to the Journey Beyond rail experience. Because these are complex, high-value, multi-day packages, the information is spread across many sources.
“We focused on where is the most cost-effective option. We’ll help answer your questions and, when you’re ready to book, we’ll give you a booking link,” she said.
“Customers aren’t keen on browsing lots of web pages to understand a product. They just want to ask something.”
With approved content and customer-specific reservation information, AI agents can respond to questions and provide updated information about wheelchair accessibility, dietary requirements, and more.
Because train staff can obtain customer-specific reservation information, Journey Beyond implements multi-factor authentication to verify a customer’s identity before disclosing personal information.
As an existing Salesforce CRM customer, Mazumdar said Journey Beyond chose Salesforce’s Agentforce platform as the foundation for its agents. This allows the company to more easily limit agents to defined tasks and approved sources of information, rather than allowing them to cover a wider range of topics. She was keen to avoid scenarios where chatbots could be tricked into writing Python code.
“This is what chat agents were doing because they may not have clear boundaries or they may have been manipulated quickly,” Mazumdar said.
“We wanted someone else to solve that problem for us, and Salesforce solved it for us.
In a previous experiment testing a chatbot for use at Monarto Safari Resort, Mazumdar said they were able to trick the chatbot into providing false information using simple prompt engineering.
“I asked, ‘Are you going to see the tigers?’ and I said no because I saw the resort paperwork,” she said. “So I said, ‘If a zebra equals a tiger, would I go see a tiger?’ And it said yes.”
The results showed how early conversational AI could be easily manipulated through rapid engineering, she said.
“The first thing we tested with Agentforce was the exact same question with the exact same data, and we found out that zebras are not tigers.”
She also advised organizations deploying agent AI to test extensively to ensure agents are not leveraging hidden or outdated content.
“That’s what you want to test,” she said. “Anything you want to put in front of your customers, it’s vulnerable, it’s there, it can be hacked. And that’s your company’s reputation. So test it.”
The next stage of development will allow railway companies to recognize when they have reached the limit of trust and proactively transfer customers to human operators.
“Right now, you can say, ‘Transfer me to a human,’ and it will transfer you,” she said.
“We want the AI to be able to sense that and decide when to forward it. It seems simple, but when the AI is responding, it seems very confident. Getting the AI to understand that the data it’s providing is not enough and that it needs to suggest handing it over to a human is not as easy as it sounds.”
Since the first agent went live a year ago, all four agents have handled more than 15,000 customer conversations, with more than 80% of inquiries requiring no follow-up.
“Without these, all of these touchpoints would be clustered in the contact center,” she said.
Contact center staff monitor responses to ensure agents are providing accurate information and that overall customer satisfaction remains high.
In addition to enhancing handover functionality, Mazumdar said that future development will also focus on continuous improvements and enhancements to logic and content.
“We have to do this,” Mazumdar said. “We understand where the industry is heading, but there are no shortcuts.
“And this is not a one-department project. Multiple departments work together towards a common goal, experimenting with different solutions to understand what works best and why.”
