Is there any story you had with AI about your breakup? Or a chatbot therapy session where you didn't talk about other people? Humans may have read it – and in some cases they can see your personal information.
Meta, like many tech giants, uses contract workers to improve AI by reading real conversations between users and their chatbots. While rating and reviewing AI responses, a common industry training technique, some contractors look at personal information that can identify individual users, Business Insider learned.
Four contract workers and Alignerr, employed through Outliels' training platform (owned by Scale AI), informed Business Insider that they had routinely encountered chats between users and Meta's AI, including their full name, phone number, email address, gender, hobby and other personal information. One estimated that more than half of the thousands of chats reviewed each week had personal identifiable information. The two contractors said they encountered a chat in which a user sent a chatbot selfie. Users were found all over the world, including the US and India.
According to documents from one of the three outsourcing metaprojects reviewed by Business Insider, some of the personally identifiable information was placed by Meta along with the chat history to help contractors personalize AI responses. In other cases, users disclosed their personal data to the AI chatbot via text within the conversation. This is a warning to the user by Meta's privacy policy.
Two of the four contractors worked on similar projects from other major high-tech clients, and estimated that inclusion of uncompiled personal data would be more common in the META projects they worked on.
The project document was in use last month, but told contractors that the interaction they are evaluating is a real conversation between users and chatbots.
The contractors who spoke with Business Insider explained many conversations they saw as deeply personal. Conversations with Meta AI took the form of treatment-like sessions, private conversations as if talking to friends, or intimate interactions with romantic partners, the contractor said.
Users open up about their fantasies, cheating on chatbots, ranting about people and issues in their lives, and seeking advice. The contractor said users may include contact information, job openings, locations or details about the children in these exchanges.
One contractor who worked on a project run by Alignerr called Omni, which aims to improve retention and engagement at Meta's AI studio, told Business Insider that some users share intimate information with chatbots, including selfies and explicit photos.
META's AI Terms of Use states that it could “review” user interactions with AI, which could be “automated” or enforced by humans.
A Meta spokesperson told Business Insider that both employees and contractors have “stiff policies” regarding who has access to personal data.
“While working with contractors to improve the quality of training data, we have a process and guardrail that intentionally limits the personal information they see and directs how to process such information they may encounter,” the spokesman said.
A Scale AI spokesperson said “contributors are authorized to process personal data only when necessary for the project,” and that they will adhere to Outlier security standards and “instruct them to skip tasks with responses containing PII and such content.”
Related Stories
Business Insider tells the innovative stories you want to know
Business Insider tells the innovative stories you want to know
“The data generated by users will not leave the customer's platform,” the spokesman added. “Many projects are being implemented on customer labeling platforms, which provides additional protection by retaining all user-generated data contained in the system.”
Alignerr did not respond to requests for comment.
“Don't assume that chatbots are inherently private.”
If personal data shared with the chatbot falls into the wrong hands, it “opens the door to manipulation, fraud and other misuse,” so the interests are high,” Miranda Bogen, director of the AI Governance Lab at the Center for Democracy Technology, a nonprofit focused on digital rights and privacy, told Business Insider.
Bogen advises users who are concerned about privacy that “we never assume that chatbots are inherently private, especially since practices vary widely between businesses.”
Bogen added that AI tools like chatbots are “used very differently than other online tools,” adding that “people may not realize that their conversations are still being used to develop products and that humans can access them.”
Many tech companies, including Meta, Openai and Google, have said in their privacy policies that they can use conversations between users and AI to train and tweak models.
“Data that people don't expect to see has been a problem in the industry for a long time,” Bogen added. “This is just the next version. The difference is that the context can sometimes feel more intimate to those engaged in these tools to get support that can feel like a treatment, or to the place where they provide sensitive information about their work.”
Identifying Data
Business Insider was unable to accurately determine how often a contractor encountered personally identifiable information. However, the contractor who spoke with Business Insider said he met regularly while working on a meta AI project.
One contractor who worked on Project Omni said he could complete as many as 5,000 AI training tasks a week. They were told to flag and reject chats containing personally identifiable information. The contractor estimated that he encountered personal data, such as a user's phone number and Instagram username, at “60% to 70%.”
Another project with Outlier called PQPE aims to make conversations between users and Meta's AI more personalized, including gender, location, location, hobbies, and more, by referencing what they know about the user. Each user chat log comes with a list of facts about the users that AI will be referring to. These facts are based on previous conversations between users and AI and the user's “social profile activity,” the project document says. In this project, the contractor was unable to deny chats containing personal information.
A Meta spokesman said the AI Personalization Project “contractors are permitted in the course of the work to access certain personal information.”
In some cases, contractors may encounter personal information needed for tasks such as location data if they help users find a coffee shop nearby, a Meta spokesperson said.
The contractor added that if they combine some of the user's descriptions provided in the training task, someone can “absolutely” find the user's true identity.
A Meta spokesman said that all contractors will complete the assessment to ensure that they meet cybersecurity and privacy risk management standards.
The user data accompanying the sexually explicit chat history was specific enough for business insiders to find Facebook profiles containing matching names, cities, genders and hobbies. It took less than 5 minutes to search for Business Insider.
The contractor assigned to the chat said Business Insider saw it but chose not to explain, but that content had to finish the day's tasks.
“It's rough. I had to put it down overnight,” they said.
Chatbot Privacy
Tech companies like Meta are racing to create personalized AI. This means increasing the amount of personal and sensitive user data.
Meta CEO Mark Zuckerberg recently outlined his vision for “personal superintelligence.”
In another recent incident, users don't seem to notice that they are exposing and chatting with AI. Business Insider reported in June that he unconsciously shared personal information, including medical questions, career advice and relationship issues, in conversations within Meta's AI. Some included identification information such as phone numbers, email addresses, and full names.
In response, Meta introduced new warnings to the Meta AI app.
This app allows users to share chats. This means that Google searches can be indexed and displayed, Business Insider reported last week. The same week, Openai removed the feature that meant that shared ChatGPT conversations were indexed by Google.
Sara Marcucci, founder of the AI + Planetary Justice Alliance, a global group of researchers and activists, said the Business Insider's report on contractors looking at personal information “suggests that data minimization, editing and user control are heterogeneous and not uniformly enforced across the industry.”
Bogen said that automated filters can detect and delete personal identifiable information, but cannot catch everything, allowing human reviewers to intervene.
“Just because there is a process of human marking and editing it doesn't mean that nothing has already happened, but if there is a system in which it exists it indicates that it is perceived as incomplete and incomplete,” Bogen added.
