Some technology providers present this as a new frontier for lenders looking to further optimize decision-making. What could be better than delegating the entire process to a sophisticated algorithm?
This marketing pitch is tempting, but requires some research.
If you take a closer look at the products these software companies advertise, you’ll often find that they don’t really equate to what is defined as AI.
The AI moniker is used to make something look more impressive than it actually is.
Perhaps this too, as the technology is not fully backed by regulators. The UK, US and Australia have expressed concerns about the use of AI in making lending decisions.
Their concern is that if acted on prematurely, it may not actually improve decision-making at all.
The U.S. Consumer Financial Protection Agency has warned lenders and intermediaries that they cannot attribute “agency” to AI systems, given the risk of depriving companies of decision-making accountability.
It warns that letting a black-box model make lending decisions does not exempt companies from legal liability.
The law gives all applicants the right to a specific explanation if their credit application is denied, and that right is reduced simply because the company uses a complex algorithm they don’t understand. It is not.
The bottom line, he said, is that complex algorithms must provide specific and precise explanations for rejecting an application.
Reading between the lines suggests that many AI platforms may not do this, increasing the likelihood of liability claims in the future.
Lending that is not backed by rigorous, documented decisions is always unwise.
In the UK, the problem was identified in a recent Bank of England/Financial Conduct Authority report, which suggests that “AI lack of explainability” poses potential reputational and regulatory dangers.
The implicit question that is again raised is whether a company can justify its decision when faced with an unfair marketing claim.
AI doesn’t necessarily make the wrong decision, it could very well be the right one. The key is being able to prove to the customer how the decision was made in the first place.
Comprehensive proof of this decision, as AI is already known to be susceptible to what is known as AI bias, AI model risk, or in everyday parlance the “Law of Unexpected Results” is particularly important.
Model bias can occur during the AI training process and incorporate certain outcomes. Automated model selection tools can exacerbate risks, as can incomplete datasets.
