In the rapidly evolving world of artificial intelligence, the user experience (UX) of AI-powered applications is often a point of friction. Mike Christensen, a staff engineer at Ably, gave a compelling presentation at AI Engineer Europe called “Why AI UX is Broken (And It’s Not the Model’s Fault).” Christensen argued that the common perception of AI UX problems caused by flawed models is often a misdiagnosis, and that the real culprit lies in the underlying architecture and implementation of these systems.
AI UX is broken, it’s not a model — from an AI engineer
Visual TL;DR. AI UX is broken, not because of a flaw in the model. It’s not a model failure due to the default architecture. The default architecture has HTTP streaming issues. The HTTP streaming problem is solved by decoupling and persistent sessions. Decoupling and persistent sessions example Ably’s AI Transport. Isolation and persistent sessions enable better AI UX.
AI UX is broken: User experience issues in AI applications are common and frustrating
It’s not a flaw in the model: The problem stems from the infrastructure, not the AI model itself.
Default architecture: Common patterns for using HTTP streaming for AI chat applications
HTTP streaming issue: Single point-to-point connection causes issues with AI event stream
Isolation and persistent sessions: The solution involves isolating components and maintaining persistent connections.
Ably’s AI Transport: A practical implementation of the proposed persistent session architecture
Improving AI UX: Improving the AI user experience by fixing the infrastructure
Visual TL;DR
Default architecture: HTTP streaming
Christensen first outlined the most common architectural pattern for AI chat applications: HTTP streaming. In this model, a client (such as a browser) sends a message to an agent (server-side), and the agent requests a large-scale language model (LLM). LLM generates an event stream that is sent back to the client over a single point-to-point connection. Although this setup is easy to implement, Christensen highlighted its inherent limitations.
According to Christensen, the main problem with this default approach is that the user experience is often brittle and frustrating. If the network connection becomes unstable or the user switches devices, the experience will be interrupted. “The default architecture is fundamentally based on the idea of a single client, single connection to a single agent,” he elaborated. This one-to-one mapping poses significant challenges when dealing with the dynamic nature of user interactions in modern applications.
HTTP streaming issues
Christensen identified three key issues that arise from AI UX’s reliance on HTTP streaming.
Resilient delivery: Streams built based on this model often become unstable when cuts occur. Unlike traditional web applications, where users can refresh the page and pick up where they left off, AI streams are typically lost and the user must resume the conversation or task.
Continuity between surfaces: Users expect a seamless experience when moving between devices or applications. Point-to-point HTTP connections make it difficult to maintain this continuity, resulting in fragmented and disjointed interactions.
Live control: Users need to be able to see what’s happening and control the AI’s output. Limitations of HTTP streaming can make it difficult to provide real-time feedback and allow users to effectively interrupt or manipulate the AI generation process.
He illustrated the vulnerability of this model with an example. If a user loses connection prematurely, the entire interaction is often lost. The agent may continue to generate tokens, but since there is no client to receive them, the work is wasted and the user is left with an incomplete or broken experience. This lack of resilience is a key reason why many AI UX implementations are perceived as “broken.”
Solution: Decoupling and durable sessions
To address these issues, Christensen advocated a more robust architectural approach that separates the agent layer from the client layer. He introduced the concept of “persistent sessions” or “channels” as a more effective solution.
In this model, persistent sessions act as a stateful layer between agents and users. Agents write events to this session, and clients subscribe to it. This separation has several advantages.
Resilient delivery: Events are stored in a persistent session, so even after a network interruption, clients can reconnect and resume the experience from where they left off.
Continuity between surfaces: Clients can subscribe to the same persistent session from different devices, ensuring a consistent and uninterrupted user experience.
Live control: Managing state in a central session allows the system to more effectively handle requests to interrupt or control the AI’s output, giving users greater control.
Christensen showed a diagram showing how this works. Agents publish messages to channels, and clients subscribe to those channels. If the client disconnects, you can simply resubscribe and get the stream from the persistent session without losing any progress.
Ably’s AI Transport: Real-world implementation
To facilitate these more robust AI UX patterns, Ably has developed “AI Transport,” a drop-in session layer for AI applications. This layer is designed to help developers build durable, operable, multi-device experiences without having to rebuild the entire stack.
Ably’s solution leverages the publish/subscribe messaging channel to enable agents and clients to communicate asynchronously and reliably. This approach allows for fine-grained control over the AI’s output, including the ability to cancel responses and receive detailed progress updates. The demo showed how this enables a smoother and more intuitive user experience, even in scenarios with multiple concurrent users or frequent connection changes.
Christensen emphasized that Ably’s channels allow developers to move away from a problematic single point-to-point HTTP streaming model to a more scalable and resilient architecture. This enables features such as real-time progress updates, the ability to cancel in-progress AI tasks, and seamless transitions between devices, all of which are important for a positive AI UX.
The future of AI UX
The core message of Christensen’s presentation is clear. The perceived flaws in AI UX are often the result of outdated or poor architectural choices rather than inherent in the AI model itself. By adopting more resilient and flexible communication patterns, such as those provided by persistent sessions and publish/subscribe channels, developers can significantly improve the quality and responsiveness of their AI applications, ultimately leading to a better user experience.
