Elizabeth Holmes persuaded investors and patients that she had a prototype microsampling device that could perform a wide range of relatively accurate tests using a fraction of the blood volume normally required. She lied; Edison and her miniLab device didn’t work. Worse yet, the company knows it’s not working, but it’s telling patients it’s doing things like telling healthy pregnant women that they’re having a miscarriage, or giving false-positive cancer or HIV tests. You continued to provide inaccurate information about your health status.
But Holmes had until May 30 to appear in prison and was found guilty of deceiving investors. She was not found guilty of deceiving a patient. This is because we have well-established ethical principles regarding disclosure to investors and legal mechanisms for taking action against fraudsters like Holmes. The law is well laid out, though not always fully enforced. For medical devices, things are even more opaque. In facilitating innovation, legal standards give broad leeway to those attempting to develop technology, knowing that even the best of them will sometimes fail.
The current debate around AI regulation shows a similar dynamic.
As lawmakers struggle to figure out what to do, doomsday predictions collide with stifling sales pitches about how AI technology will change everything. Either the future will be a blissful panacea generated by algorithms, and students won’t have to write term papers anymore, or we’ll all be radioactive rubble. (Students don’t even have to write term papers in that case.) The problem is that new technology without ethical and legal controls can do a lot of damage. And, as with Theranos patients, a lot of the time we don’t do that. There is no good way for people to recoup their losses. At the same time, technology is particularly difficult to regulate due to its speed of change, and lax standards increase opportunities for fraud and abuse, whether it’s for a flashy startup like Holmes or a shabby cryptocurrency or NFT scheme.
Holmes literally created an opaque box, arguing that people couldn’t see or report what was inside, so it’s a useful case to consider when developing ethical standards for AI. Even if the technology spells death for healthy patients, doing so would violate her intellectual property, she said. We see many of these same dynamics playing out in the conversations surrounding the development of ethical standards and regulations for artificial intelligence.
Developing ethical standards to underpin AI regulation is a new challenge. But this is a challenge that we have the tools to work with and the lessons learned from failure can be applied to manage other technologies.
Like Theranos devices, artificial intelligence technology is generally a small box understood by designers (at least everyone), but often not subject to outside scrutiny. Algorithmic accountability requires some degree of transparency. When a black box makes decisions that cause harm or have a discriminatory effect, we open it up to ask whether these mistakes are due to occasional blind spots or systematic errors in design. It is necessary to decide whether it is a thing or (as in the case of Holmes). Complete fraud. This transparency is critical to preventing future harm and determining accountability and liability for existing harm.
There is a great deal of urgency when it comes to AI regulation. Big AI companies and researchers alike are urging lawmakers to act quickly, with proposals varying but consistently including transparency requirements. To prevent systemic problems and fraud, not even intellectual property laws should prevent big AI companies from showing how their technology works. Sam Altman’s recent congressional testimony on OpenAI and his ChatGPT included discussions of how the technology works, but that only scratches the surface. While Mr. Altman appears enthusiastic about drafting regulations, he threatens to leave the European Union based on proposed AI regulation proposals to the European Parliament.
In early May, the Biden administration announced the rollout of proposals to address artificial intelligence. Most important was the promise of major AI companies (Alphabet, Microsoft, OpenAI, etc.) to subject their technology to independent testing and opt-in to “public evaluations” to assess potential impact. This assessment, like Altman’s congressional testimony, is not exactly “public.” External experts will be given access to evaluate technology on behalf of the general public. If companies keep to these promises, it is hoped that experts will be able to spot problems before their products are widely implemented and used, thus protecting the public from dangerous consequences. This is an early stage proposal. Because it’s unclear who these experts are and what their powers are, and even if they helped create the rules, companies may not want to follow them. is. Still, it’s a step forward in establishing conditions for more scrutiny of private technology.
The Biden administration’s broader proposal, the Blueprint for an AI Bill of Rights, identifies various areas where AI technology is already known to cause harm. Social media algorithms can promote violent and sexual content, adopt (loosely) ethical principles to deal with those issues, and codify them into law to make them enforceable. . These principles include non-discrimination, safety, the right to be informed about data collected by the system, the right to refuse algorithmic services (and the right to access alternative human services).
Horrible stories claiming these principles are widely circulated. Researchers, including Joy Buolamwini, have extensively documented the problem of racial bias in algorithmic systems. Facial recognition software and self-driving systems, overwhelmingly trained on datasets of white subjects, are unable to recognize or distinguish black subjects. This poses obvious dangers, from someone being falsely identified as a criminal suspect based on faulty facial recognition to being run over by a self-driving car at night where black people can’t see. People shouldn’t be discriminated against (run over by cars) because of a biased algorithm. The Biden administration’s proposal would require designers to engage in pre-deployment testing.
This obligation is important. Many technologies have error and failure rates. For example, COVID-19 tests have false-positive rates and many complex variables, which is why it is important to test technology to assess and disclose those failure rates. There’s a difference between a false-positive antigen test and a machine Holmes sold that simply didn’t work. The designer’s responsibilities and responsibilities should correspond to what the designer has done. If designers followed best practices, they should not be held responsible. If they have gross negligence, they should. This is the principle of engineering and design ethics across everything from medical tests to algorithms to oil wells.
There is a long way to go between proposing a mandate and implementing a legal framework, but in an ideal world, companies would submit their algorithms to independent audits before they hit the market, ensuring that the potential for discrimination would be eliminated during the testing phase. You can deal with sex. As studies like Buolamwini become part of the standard for developing and evaluating these technologies, companies that do not test their algorithms for bias will be negligent. These test standards have legal implications and should help establish when a consumer harmed by a product can claim damages from the company. This was missing in the Theranos fraud case and is still missing in the startup’s standards for medical tests and devices.
Businesses should support clear and well-founded AI standards, such as those outlined in the Biden administration’s proposal. This is because by doing so, we can obtain the grounds for the trust of society. That grounding is not absolute. Knowing that your toothpaste is contaminated is a sideways glance at the toothpaste company, but knowing that they have basic regulatory controls means that the products we use on a regular basis are It helps establish that you are safe. Most of us feel safer because doctors and lawyers have codes of ethics and engineers who build bridges and tunnels have professional standards. AI products are already embedded in our lives, from recommendation algorithms to scheduling systems to voice and image recognition systems. Making sure these things aren’t subject to serious and inappropriate bias is the bare minimum.
Algorithms, like medical tests, give us the information we need to make decisions. We need regulatory oversight for the algorithm for the same reason Holmes’ box needed it. If the information we are getting is generated by a machine that makes systematic errors (or worse, doesn’t work at all), it can and will. endanger those who use them. If you know what the error is, you can prevent or mitigate the error. do harm.
Future Tense is a partnership of Slate, New America, and Arizona State University that investigates emerging technologies, public policy, and society.
