Editor’s Note: Deep Dive – A feature that delves into timely issues from technology to work is TechWire’s Wednesday regular feature.
+++
Chapel Hill – The recent proliferation of large language models such as ChatGPT has raised questions about bias in their results. However, in recent years society has become increasingly dependent on algorithmic systems and the outputs they generate, so this problem is not new. Such a system has an efficient data-driven approach, especially for highly complex or cognitively demanding binary questions (i.e. questions that have one of two possible answers). It has the power to provide decision making.
While the digitization of decision-making may appear to offer a more objective process than human judgment, it actually plays out quite differently. From facial recognition software to hiring and interview prescreening, algorithms have been found to reinforce systemic biases and exacerbate rather than mitigate existing asymmetries and inequalities.
Alternatives to the “Fairness Principle”
Computer scientists have addressed this problem by trying to design various “fairness principles” that can be incorporated into decision-making models. However, consistent application of these principles has not yet been possible, and advocated fairness principles are very often in conflict with each other.
A recent study by Edward M. Bernstein Distinguished Professor of Economics at the University of North Carolina at Chapel Hill, Professor of Finance at UNC Kenan Flagler School of Business, and Faculty Director of Resink, Eric Gissels. Labs offers an alternative approach to implementing fairness in machine learning models. Ghysels and his co-authors, Andrii Babii, Xi Chen, and Rohit Kumara, approach this problem by adding an explicit cost to the model’s classification error and considering different kinds of costs. increase. High-damage costs are weighted more heavily than low-damage costs. As the algorithm is trained, it internalizes a cost that reflects the possible asymmetry in its output. More simply, rather than approaching each decision uniformly, the algorithm is trained in such a way that the decision-making process takes context and asymmetry into account. This is a radical departure from traditional machine learning settings where a generic symmetric cost function is used that ignores gender, ethnicity, and other minorities. This is the way economists think in terms of cost-benefit analysis and utility functions.
This may seem ambiguous in the abstract, but that is why the authors conclude in their analysis of the model in the context of pretrial detention decisions. The pretrial detention decision process involves judges evaluating whether a particular defendant should be held in pretrial detention or released on bail. Both decisions can have financial costs. Holding a defendant inevitably requires housing costs and additional prisoners, and releasing a defendant before trial may cause him to flee or commit further crimes. The challenge is that existing algorithmic models (COMPAS, or Correctional Offender Management Profiling for Alternative Sanctions) have been shown to be racially biased. In this model, he is twice as likely to mispredict recidivism rates for blacks as for whites. Therefore, pretrial detention questions have the necessary qualifications to test Gissel’s asymmetric model.
Both potential decisions can have deeper social and personal repercussions. Several studies have shown that incarceration not only increases the psychological burden on an individual, but also increases the defendant’s likelihood of reoffending. On the other hand, crimes committed by defendants while out on bail can take countless tolls on other members of society. Ghysels and co-authors compare an asymmetric deep learning approach that exploits the costs and benefits of pretrial detention, commonly used in the legal community, with traditional machine learning COMPAS-type models. They found that their model reduced total costs by 10% to 13%. Ultimately, their model could be expected to slightly increase the number of false positive errors (defendants who should not have been in prison but were detained), whereas this It is offset more than offset by reducing the number of defendants who should be admitted. imprisoned in prison). In doing so, the asymmetric model reduces racial bias in pretrial detention.
Below, we provide some follow-up questions to further discuss the implications of Gissels’ study.
How should we think of these results in the context of the explosion of large-scale learning models like ChatGPT?
Eric Gisells: ChatGPT’s situation is even more complicated. Our research is about bias in binary choice problems, or what machine learning researchers call classification problems. Many algorithmic decisions, such as fraud detection, college admissions, loan applications, and hiring, are binary. ChatGPT is a generative AI that provides answers to your queries. Some decisions may be made based on the answers provided, but that relationship is more difficult to formalize. More importantly, the reliability and accuracy of the answers provided by ChatGPT are of primary concern. The problem we’re looking at and the problem we just talked about with ChatGPT have some things in common. In both cases, AI is data-driven, with biases built into the data to produce outputs that replicate or amplify such biases. For classification problems, cost-benefit trade-offs neutralize, or at least mitigate, bias. Filtering ChatGPT’s output is even more difficult.
- How do you think society and regulators should approach algorithmic unsupervised decision-making? What role do you envision for regulation and oversight?
Eric Gisells: Lawmakers and industry leaders are discussing regulation of AI. It’s a classic situation where an industry declares it can self-regulate while some lawmakers advocate a more heavy-handed approach to regulation. There are many sides to these arguments. First, all generative AI models are trained using large amounts of data. Data privacy issues are already at stake at this point. For example, the Italian government used the European Data Protection Law (GDPR) to ban ChatGPT. Italy’s data protection authority, in an investigation into a possible data breach at OpenAI, which allowed users to view the titles of conversations that other users had with the chatbot, asked OpenAI not to process Italian users’ data. ordered to suspend temporarily. The agency argued that there is no legal basis to support the large-scale collection and processing of personal data to train the algorithms on which the platform relies.
Apart from data protection, there are broader social issues. Jeffrey Hinton, one of the innovators of deep learning models widely used in AI, has resigned from his job at Google to speak out about the “dangers” of the technology he helped develop. bottom. Tech industry leaders, including Bill Gates, Elon Musk, and Steve Wozniak, also announced in late March that they would develop an AI system more powerful than OpenAI’s latest large-scale language model, GPT-4. Signed an open letter asking for a six-month suspension. The open letter calls on AI laboratories around the world to suspend development of large-scale AI systems, citing concerns about the “serious risks to society and humanity” posed by the software. We are not sure that the six-month grace period will provide enough time to find a good solution, and there is clearly no clear mechanism to encourage adherence to the open letter’s recommendations. Nonetheless, it is clear that there are enough voices to argue loud and clear about the urgent need to install guardrails.
- What are the benefits of applying your model to pretrial detention systems? What elements of racial justice does this model address?
Eric Gisells: This question brings me to a broader topic. Some judges simply object to the use of any form of algorithm, even in a subordinate advisory role. So, first and foremost, there are educational hurdles to getting judges, and the wider legal community, familiar with his AI decision-making. Of course, this problem is not limited to the legal profession. As a business school educator, I strive to teach AI insights to future generations of business leaders. My goal is to get people used to making decisions based on input from AI. It’s important to understand the limits and benefits of AI and understand the jargon of data scientists. Once that hurdle is over, I think the implementation of the model will include a conversation about the advantages of the setup compared to, say, COMPAS software.
- Are there other areas of society where this technique can be easily applied?
Eric Gisells: Simply put, yes. We used pretrial detention because the data is in the public domain. Many other applications contain their own data. For example, think of applying for a loan, getting a job, or getting into college.
Read the full paper
(C) Kenan Private Enterprise Institute
