Building Responsible AI – Opinion News

Kunal Walia and Rositsa Zaimova

Santosh, a small farmer in Uttar Pradesh, is looking forward to the harvest season and has recently turned to artificial intelligence. As part of a pilot run by a global agritech company, an AI-powered agricultural advice device was distributed to his village. The device runs on software that assesses weather, soil and water conditions to predict crop patterns. Having proven useful to farmers in the Midwestern US, the technology has resonated strongly in this village in Uttar Pradesh. But a week before harvest, Santosh found his fields ravaged by pests endemic to the region. He learnt that the software was built on data based on the North American terrain. Santosh now has to rely on informal work. They migrated to the nearest city to eke out a living until the next sowing season.

This scenario illustrates the dangers of positioning AI technologies as transformative without ensuring the accuracy and reliability of the data that underpins them. In March, the European Union approved an Artificial Intelligence Law that could help address these concerns, including in sectors that directly impact people's lives and livelihoods, such as agriculture, healthcare and education. The pioneering law defines four risk levels for AI and imposes strict obligations on “high-risk” systems, including ensuring quality data sets to minimize discriminatory outcomes and bias.

The approval of this law comes at a timely time, given that developing countries are increasingly adopting off-the-shelf AI solutions that are trained on Western or non-diverse datasets. This new regulation, and the catalyst it will give other regions to draft their own laws, provides a backdrop for AI developers and influential institutions to work together to help reverse this trend.

Assessing AI bias

Looking back at Santosh's predicament, he may have been able to avoid disaster if the inaccurate data used by tech companies had been detected earlier, or if the AI ​​advice had been supplemented with locally relevant information. To achieve this, the people and organizations building AI systems need to introduce additional layers of testing to assess bias. Luckily, there are several frameworks to choose from.

One example is the Evaluation of Search Augmentation Generatives (RAGA), devised by researchers from Facebook AI Research, University College London, and New York University. Search Augmentation Generatives refers to a group of large-scale language model (LLM) applications that use external data. It has two components: search (extracting information from external context) and generating a response based on that information.

Another popular framework, Recall-Oriented Learning for Gist Evaluation (ROUGE), uses human output as the metric, which is primarily useful for LLM applications that generate text-based outputs such as summaries and translations.

Equally important is establishing regular feedback loops with human oversight, known as Reinforcement Learning with Human Feedback (RLHF). This is essential as AI systems will improve their ability to mimic human output over time and perform more layered tasks. Such AI bias assessment frameworks should be rigorously applied by all parties involved in the creation of AI tools. Additionally, productizing these frameworks could usher in even higher levels of compliance within the private sector.

But if well-developed solutions already exist, why aren’t they more widely adopted?

Financial support

Significant investment is needed in tools to rigorously stress-test AI models before they are deployed. Although comprehensive data on AI funding is limited, we know that the private sector tends to refrain from investing in areas where there is no clear evidence of a return on investment. Influential institutions such as charities and international development organizations can play a key role in filling the funding gap.

Social impact agencies are increasingly turning to AI innovation: USAID, for example, supports GeoKRISHI, a web application that integrates government, geographic information systems, and crowdsourced data to provide crop suitability assessments at key stages, and has also invested in Dimagi, an open-source platform that combines mobile health and AI. Satellite imagery analysis based on data and machine learning identifies and predicts Zika virus hotspots. Similarly, the World Bank Group is supporting Project Artemis, which builds AI models to predict famine risk at a local level.

This is promising, but support needs to be broadened to include risk and bias assessments. Dalberg's analysis of 10 of the world's leading philanthropies found that only 23% of AI grants are directed at addressing risk and ethical concerns. With more push from social impact organizations and the right incentives for the private sector, AI systems can be more inclusive to the people they serve, and we can ensure that conversations driven by a handful of large institutions in the Global North do not influence courses of action in the Global South.

Kunal Walia and Rositsa Zaimova are partners at Dalberg Advisors and Dalberg Data Insights, respectively.

Disclaimer: The views expressed are personal and do not reflect the official position or policy of FinancialExpress.com. Reproduction of this content without permission is prohibited.