Generative AI needs more than a light touch

Chatbots such as ChatGPT raise enormous data protection and moral issues that regulators must address.

Italian users cannot access ChatGPT. Chatbots based on artificial intelligence, launched in November 2022, are currently geographically blocked in the country.At the end of last month, an investigation revealed that the Italian Data Protection Authority (DPA, aka Galante) has adopted a landmark preventative order temporarily restricting OpenAI’s local processing of Italian users’ data.

Even the mainstream media and powerful ministers lamented the DPA’s move as reckless. Tech commentators and start-ups accused it of plotting against the country’s international competitiveness. But the story is more complicated, and the increased scrutiny by Spain and her EU data watchdogs offers important lessons.

“Privacy Nightmare”

Across the world, there is widespread concern about the pernicious consequences and “risks to society” posed by generative AI models, and experts and business leaders are calling for updates to endorse research and implement safety protocols. It is now asking for a moratorium. For those fascinated by “digital enchantments,” this cautious approach may seem like a neo-Ludite conspiracy ignited in academia and policy circles, but it is about fundamental values ​​in a democratic society. In fact, technology companies are often given “regulatory latitude not given to other sectors.”

Large Language Models (LLMs) are a “privacy nightmare,” as experts have demonstrated. They are based on processing huge amounts of data scraped from non-public sources. It relies heavily on free underlying infrastructure for personal data, sometimes proprietary or copyrighted. Never mind the sensitive data your users may casually share when interacting with these systems.

Professionals are starting to use generative AI applications as low-cost assistants. The information they enter (draft employment contracts, budget reports for revision, or top-secret data) can be the output of other users’ queries. This kind of privacy nihilism is nasty.

data breach

On March 22nd, OpenAI CEO Sam Altman tweeted that “a bug in an open source library caused serious problems with ChatGPT.” This meant that some users had full access to titles in other users’ conversation histories where chats were stored.The company admitted that it “feels”[ing] Terrible about this.” Similar data breaches were reported for information related to subscriber payments.

Both glitches “seemed to indicate that OpenAI had access to users’ chats,” the BBC reported from San Francisco. In a parallel reality, this informational violation of self-determination does not go unnoticed, causing public outrage and reputational damage. But not for the first time, the alleged corporate “disruptor” appeared to enjoy a giant “jailbreak” card.

The Italian DPA notified ChatGPT of a series of serious breaches. First, the company did not provide information to users and data subjects whose data was collected by OpenAI (as required by Article 13 of the General Data Protection Regulation). Second (and surprisingly), we did not identify a strong legal basis for the large-scale collection and processing of personal data (Article 6 of the GDPR).

Third, it showed a lack of respect Accuracy: Chatbots tended to make up details that turned out to be false. Finally, according to OpenAI’s terms, the service was only intended for users over the age of 13, but the lack of an age verification mechanism could expose children to age- and awareness-inappropriate responses. I have.

explicitly obligated

of reaction I used to be an Altman fan. Nevertheless, Galante It was explicitly mandated by the EU GDPR. Other national authorities may soon follow, using their prerogative to “impose temporary or permanent restrictions, including prohibitions on processing” (Article 58(2)(f), GDPR) Set an example.

OpenAI has been given a few weeks to explain its intentions to get within the European guardrails. However, we have decided to discontinue our service in Italy. The move caused uncertainty for all operators on site. However, after meetings with the company and the DPA, several conditions were identified that must be met by the end of April for the ban to be lifted. If you cannot prove that you are a company, your company may face fines, sanctions, or eventual bans.

OpenAI’s reaction is typical of some tech companies when they believe they can circumvent universal constraints. It selectively withdraws from the market, denounces regulators, and mobilizes users (and others who fall for this pitch) to defend unconstrained service operations. It harkens back to the dawn of the platform age. At the time, food delivery and other gig economy players circumvented the law under the bizarre assumption that innovations were only genuine if retroactive rather than prior permission was sought. Given that similar controversies could soon emerge after the European Data Protection Board launched a dedicated task force to “facilitate cooperation and exchange information on possible enforcement actions”. , it is not yet known what the response will be.

light touch approach

The proposed EU regulation on AI envisages minimum transparency obligations when certain systems are used, especially chatbots and “deep fakes”. A draft will be presented in April 2021 and is under scrutiny by the legislature.

However, the emergence of complex generative AI models indicates the need to understand AI broadly. Risks include mass misinformation, generation of biased and stereotypical content, and large-scale manipulation (which would otherwise be prohibited under the proposed regulations). This calculation should encourage EU co-legislators to reconsider the lite-touch approach, which imposes notification obligations only on low-risk systems.

We believe that, despite our aim to provide a modular and targeted framework, AI technologies are classified within drafted regulations in an ‘abstract’ and ‘situational’ manner, It has been argued that case-specific uses are not considered. This fails to appreciate the versatile, versatile and adaptive nature of AI systems. Aside from developer obligations, this framework does little to accommodate the gradual expansion of the system’s use beyond what it was originally intended and designed for. Co-legislators should consider the EU Council’s general approach and add provisions for situations where AI systems can be used for different purposes.

There is another aspect of this that is often neglected. Sociologists Jenna Burrell and Marion Fourcade write that “Underlying the fetishism of AI is a global digital assembly line of silent, invisible men and women, many of whom are post-colonial and unstable in the global South. working under the circumstances,” he wrote. and, time Investigations revealed that OpenAI relies on the exploitation of workers in Kenya, Uganda, and India.

To reduce toxic and unsafe content, the company outsources the labeling to San Francisco-based company Sama, whose contractors employ “child sexual abuse, bestiality, murder, suicide, torture, self-harm, incest. I had to tag situations like “incest”. as unsuitable material. These labeling, classification, and filtering tasks were paid between $1.32 and $2 per hour, depending on role and seniority.

great concern

ChatGPT and its sisters (DALL E, Synthesia, MusicLM, etc.) raise technical, ethical, social, environmental and political concerns. DPA only addressed the challenge from a data protection perspective and is currently one of the few sets of operational rules that cover the first phase of the AI ​​lifecycle. Non-European technology companies dealing with EU-based data subjects must follow the same rules as European companies.

OpenAI’s initial response lacks moral qualms. Imagine if the car company didn’t provide the mandatory requirements. Seat belt In that car, the country’s transport authorities warn them so. How should we judge a company’s choice to stop selling cars in the country instead of correcting its mistakes?

The norm-breaking ethos of tech companies must be met with less-than-permissive responses to keep obscure pro-innovation rhetoric from being discussed. Digital advancements have the potential to improve the way we live, work, learn and socialize. However, emerging technologies must be managed in such a way as to achieve socio-economic sustainability.

Antonio Aloisi is a Marie Sklodowska-Curie Fellow at the IE School of Law, IE University, Madrid, and Assistant Professor of European and Comparative Labor Law.

Valerio De Stefano is Professor of Law at the Osgood Hall School of York University in Toronto.