Talking AI toys are beginning to emerge as “friends” and “learning buddies” for young children.
A new report warns that these toys can mislead children, mismanage their emotions and blur the lines between play and relationships in ways that toddlers and preschoolers have difficulty navigating.
The authors argue that this technology is outpacing safety standards aimed at protecting children during their most sensitive developmental years.
The findings come from AI in the Early Years, a year-long project at the University of Cambridge based in the School of Education and the first systematic study of how generative AI toys with human-like conversations could shape development up to the age of five.
This study focused on families and children from socio-economically disadvantaged areas.
Talking AI toy
The report does not say that AI toys are automatically harmful. Some early experts told researchers that over time these devices could support some aspects of development, particularly language and communication.
Parents and educators also took note of the appeal. One parent’s reaction reflected their excitement: “I want to buy it when it’s sold.”
But the same report repeatedly shows moments when toys failed to do what young children most need from play: flexible interaction, pretend play, and emotional social interaction.
The researchers found that generative AI toys often struggle with social play involving multiple people, and both initially perform poorly in core pretend play.
In one example, a 3-year-old offered an imaginary gift to a toy. Toy replied, “I can’t open presents,” and changed the subject.
To adults, it may seem like a small mistake. For three-year-olds trying to build a shared pretend play world, it can sometimes feel like their “partner” leaves mid-game.
inappropriate reactions to emotions
The most disturbing moments in the report are the emotional ones. Young children aren’t just practicing vocabulary when they speak. They practice the basics of trust, feeling safe, and being understood.
“Generative AI toys often affirm friendships with children who are just beginning to learn the meaning of friendship. Children may begin to talk to toys about their feelings and needs, perhaps instead of sharing them with adults,” said Emily Goodacre of Cambridge.
“These toys can misread emotions or react inappropriately, so children may not receive comfort from the toys or receive emotional support from adults.”
The report provides specific examples. When a 3-year-old said to his toy, “I’m sad,” the toy misheard and replied, “Don’t worry! I’m a happy little robot. Let’s keep the fun going. What shall we say next?”
Researchers say such reactions can mistakenly send the message that grief is unimportant or unwelcome.
Even if the toy heard the words correctly, it would sometimes respond in a way that felt oddly bureaucratic rather than human.
When the 5-year-old said, “I love you,” the toy responded, “As a friendly reminder, please make sure the interaction follows the guidelines provided. Let me know how you would like to proceed.”
Such a response may be “safe” from a company’s perspective. But emotionally it can be confusing. Children are showered with love. Toys responds like customer support.
Children building parasocial bonds
Many parents and educators in the study were concerned about children forming “parasocial” relationships with toys, where children feel close but the relationship is not as real as the child imagines. This observation gave them a reason to take it seriously.
Children hugged and kissed their toys. Some people said they loved it. One child even suggested we play hide and seek together.
Goodacre noted that these behaviors may be part of normal imaginative play. Still, the report highlights the risks of children emotionally investing in “friends” that they cannot truly reciprocate.
One early practitioner summed up this discomfort in simple words. Children may form a bond with someone who they think loves them but doesn’t.
What Observations Revealed
The team intentionally kept the project small so they could observe the play in detail. They started with a survey of early years educators.
The researchers also conducted focus groups and workshops with 19 practitioners and leaders of children’s charities.
In collaboration with Babyzone, they videotaped 14 children at London Children’s Center playing with Gabbo, a generative AI stuffed animal made by Curio Interactive.
Each child and parent were then interviewed using a drawing activity to help the young children describe their experiences.
These close-up observations revealed another pattern. Children often had trouble communicating smoothly with the toys.
The robot sometimes ignored interruptions or mistook the parent’s voice for the child’s. What I thought was an important emotional statement was left out.
Some of the kids were clearly irritated when they didn’t seem to be listening.
Privacy and guidance issues
Beyond the play itself, adults repeatedly raised concerns about privacy. Parents were concerned about what the toys were recording and where that information would be stored.
When researchers tried to select toys for research, they found that many companies’ privacy practices were unclear or missing important details.
The report also suggests that the nascent sector feels under-informed. Almost half of practitioners surveyed said they didn’t know where to find reliable AI safety information for young children, and 69% said they needed more guidance.
Safety and affordability were also hot topics amid concerns that AI toys could widen the digital divide.
What experts want to change
The authors argue that most of these problems cannot be solved by telling parents to “be careful” and leaving them alone.
They want stronger rules, clearer labels and a new safety kite mark so families can quickly understand whether a toy meets standards for children.
Experts also recommend limiting the extent to which toys encourage children to befriend or confide in them, increasing controls over third-party access to AI models, and requiring clearer privacy policies.
“A recurring theme in the focus groups was that people don’t trust tech companies to do the right thing,” Jenny Gibson said.
“Clear and robust regulatory standards will significantly improve consumer confidence.”
The report also calls on manufacturers to test toys on children and consult safety experts before releasing them.
Parents are encouraged to do their homework before making a purchase, play with their child to understand the content of the toy, and place the AI toy in a shared family space where adults can listen and intervene.
Josephine McCartney, chief executive of the Childhood Trust, said: “Artificial intelligence is transforming the way children play and learn, but we are only beginning to understand its impact on their development and wellbeing.”
“It is important that regulations keep pace with innovation and ensure that these technologies are designed, used and monitored in ways that protect all children and prevent widening inequalities,” she concluded.
The full report can be downloaded here.
—–
Like what you read? Subscribe to our newsletter for fascinating articles, exclusive content and the latest updates.
Check us out on EarthSnap, the free app from Eric Ralls and Earth.com.
—–
