Judicious use of advanced analytics The effectiveness and efficiency of the public sector’s most important operations are changing. For example, in the United States, the Internal Revenue Service (IRS) helped identify potentially fraudulent tax returns, and the City of Las Vegas collected and targeted information about public health concerns from social media. Helped create restaurant inspections and assisted NASA jets. A propulsion laboratory creating lightweight designs for space exploration.
There are many other applications that help government agencies streamline processes to improve outcomes and make the most of limited resources. Artificial intelligence (AI) and machine learning (ML) models are helping customs officials identify containers that may contain dangerous goods, and banking regulators identifying emerging risks to the banking system, to name a few. , and help recruitment agencies match job seekers with available jobs. As technology advances, the possibilities increase, but so do the risks.
While any model can be poorly designed or misused, AI and ML models carry additional potential risks due to their algorithmic complexity. These risks include poor performance over time and lack of transparency about how results are produced. But perhaps the most pernicious risk associated with advanced analytics in the public sector is stigma and discrimination, especially against vulnerable segments of the community. Examples are not hard to find. Sophisticated analytical models can result in harsher sentences for people of color, false accusations of wrongdoing against low-income and immigrant families, and lower grades for students in disadvantaged areas. is shown. This may help explain why many public institutions are reluctant to deploy advanced analytics at scale. In fact, according to a recent report, 45% of US government agencies are still experimenting with advanced analytics, and only 12% are using highly sophisticated technology.
However, this low adoption rate does not necessarily mean low risk. This could mean a data scientist working without a system to ensure formal peer review or oversight of the models being used, or a government agency leader knowing what his AI work is. Because it could mean that you may not have. What is done within the organization and the potential risks associated with it.
Part of the solution could be found in better risk management of the model. Here, we present best-practice approaches to developing and monitoring algorithms that can help public sector agencies harness the power of advanced analytics to deliver better public services while reducing stigma and other unfair treatment. Outline.
where is the prejudice
Fair treatment is central to the mission of public institutions, but it can be difficult to maintain when decisions are based on algorithms built on skewed data sets. Bias can be introduced because the training data contains biased human decisions or reflects historical or social inequalities. Alternatively, it may be due to flaws in the sampling of the data, for example, that certain groups of people are underrepresented or overrepresented. Unless models are carefully developed, prejudices and disparities related to race, ethnicity and socioeconomic status are at risk of growing.
A lack of transparency can also exacerbate the problem. AI and ML techniques can make it difficult to track how the underlying data drives a model’s output, making it difficult to identify biases and unfairness. This presents a particular challenge in many public sector applications where transparency is a legal requirement. Even when they don’t, the public expects government agencies to be able to explain the technology used, what is causing the consequences, and the oversight that is in place. This lack of transparency can exacerbate mistrust in technology.
Despite these high risks, the model risk management infrastructure in many public sector institutions is still in its infancy compared to other high-risk sectors. In the financial sector, for example, U.S. regulators have applied risk management standards for at least 20 years, and now place a great deal of emphasis on AI-related risks, with companies making significant investments to manage them. is developing new tools for
Such investments have clearly reduced the bias towards AI. For example, a review of the ML consumer credit model by a bank validation team found that while the model was good at predicting credit risk, the reasons given for granting or denying credit were erratic, resulting in similar It turns out that customers are given different reasons. Denial of credit. Retraining the model using different modeling techniques fixed the issue. Also, in the travel industry, one company’s model management process found that a sophisticated analytical model that provided targeted promotions to customers discriminated against seniors by offering low-value promotions. bottom. The development team was able to fix the issue and put in place procedures to continuously test for bias in production.
Legislation and guidelines are emerging that will undoubtedly help strengthen model risk management in the public sector. Rather than wait for such deployments, public sector leaders can take important steps now to help government agencies effectively address the risks of advanced analytical models.
way forward
Six key actions help reduce the risk of bias, along with other risks associated with advanced analytical models.
1. hold someone accountable. Senior leadership should be held accountable for model risk management. In financial institutions and other regulated industries, responsibility usually rests with the chief risk officer (CRO). Federal agencies lacking robust enterprise risk management structures may have a combined chief information officer (CIO), chief data officer (CDO), or senior-level executives responsible for governance and technology oversight there is. However, it remains the responsibility of senior agency leaders to help translate agency missions and values, such as equity and diversity, into guidance for AI risk management leaders. This may require an in-depth discussion. For example, if your goal is to avoid gender bias in hiring, choose equal numbers of men and women to interview from resumes, guarantee equal chances of success in an interview, Is an algorithm that guarantees ties considered fair? Were men and women recruited?
2. Develop and communicate a clear set of analytical practices and standards. All government agencies should establish a clear set of analytical practices and standards that are codified, widely communicated, and adhered to. These may include clarity of the specific issues the model seeks to address, rigorous peer-review processes, and empirical review of results to detect unintended biases. One good practice to combat bias is to ensure diversity in your analytics team. If there is no one in the room with life experience to alert you to problems, it becomes difficult to spot biases in the training her data and model output. See the sidebar, Key Analytical Practices and Standards, for additional good practices.
3. Build a model risk management infrastructure. A fundamental principle of good model risk management is to continuously challenge the model from the very beginning as it is built and implemented, not as a corrective exercise.
An effective model governance program may include actions such as:
- Articulate what your organization defines as a model within its governance program. This clarifies the governance of the model and provides an opportunity to consider the risk management required for other decision-making processes that do not qualify as models, such as manual decision-making processes.
- Create and maintain an inventory of models used across your agency.
- Develop and maintain standard workflows for models to ensure widespread adoption of best practices in data science and bias recognition/reduction.
- Develop an approach to assess the model’s importance and potential to cause harm in order to establish the necessary level of oversight. Some models may require an audit as shown below.
Four. Consider creating an algorithm review panel. Just as government agencies use acquisition review boards to evaluate acquisitions above certain spending thresholds, algorithm review boards are used to de-risk especially high-risk AI and ML projects. can be mitigated. The panel may include both technical and non-technical leaders, and their role only involves assessing the potential impact of the model on stocks and whether the intended model results are fair. rather than stepping back and considering whether other stakeholders see the actual results as well. In the previous example, even if the number of men and women who were ultimately hired were unbalanced, the AI hiring model that sought to assume an equal number of resumes from men and women would still fail. Will it be seen as impartial by the stakeholders? The committee effectively takes collective responsibility for the algorithm and can relieve one person of the burden.
For some particularly high-risk applications, it may be appropriate to involve external academic or industry experts to audit models for bias. Such audits help uncover hidden problems in the system, increase transparency, and increase public confidence in the government’s use of sophisticated models.
Five. Consider appointing an ombudsman. The Analytical Ombudsman may act as a point of contact and spokesperson for external stakeholders who wish to raise issues. This structure is used by his IRS to allow the taxpayer advocacy office, an independent organization, to process appeals on behalf of taxpayers and to report any problems they may encounter, thereby helping taxpayers ensure that you are treated fairly.
6. Strategize at the corporate level. Advanced analytics are more effective when implemented across government agencies, rather than just a few advocates. By systematically identifying and prioritizing the most impactful use cases, supporting them with funding, and making a focused effort to communicate progress, lessons learned, and professional standards across the agency, effective can improve sexuality. Greater visibility fosters better peer review, promotes the dissemination of best practices across government agencies, and increases momentum and demand for advanced analytics solutions.
Advanced analytics technologies offer opportunities to transform public services. By applying practices that minimize the risk of bias and other mistreatment, leaders can fulfill their mission while increasing public confidence in the government’s use of analytics to improve outcomes for all. We can help institutions adopt an enhanced AI and ML approach. It’s a prize worth getting.
