These incentives are within a large and rapidly growing industry. The Ada Lovelace Institute estimates that the UK AI companion sector will generate revenue of approximately £1.3 billion in 2024, with annual growth of 32% in each of the next four years. A collaborative analysis of 110 AI companion sites found that UK users visit these platforms between 46 million and 91 million times each month, as people increasingly treat digital characters like friends, therapists and romantic partners.
regulatory gap
The law governing these issues in the UK is the Online Safety Act, regulated by Ofcom. But Nuala Polo, head of UK public policy at the Ada Lovelace Institute, said the law was “not designed to cover AI systems” and left “many gaps” when it comes to companion chatbots. Incidents that are obviously harmful (for example chatbots encouraging self-harm or sharing pornographic content with children) may fall within Ofcom’s remit. However, she said design features that make chatbots “snobbish,” “manipulative,” and “addictive in design” are not illegal.
When we contacted Ofcom about this, a spokesperson said: “We are working rapidly to ensure that AI platforms have appropriate safeguards in place. We expect enforcement action will be taken against services that put children at risk of harm.”
Sam Geisler, a partner at the US law firm Aylstock Witkin Kreis & Oberholz, said companion chatbots should be treated like products and liability laws should focus on foreseeable harm. “The intended use of these chatbots is for intimate and long-lasting conversations. The expected harm is psychological destabilization of the user,” he said. Geisler said basic safety measures should include reality checks, interrupting prompts, limiting sessions and immediate escalation if there are signs of suicidal thoughts. “There’s no way you can chat 24 hours a day,” he said. “That’s not healthy.”
