- Key insights: The use of AI to screen job applicants, common in banks and other companies, has come under fire for silently blacklisting qualified candidates.
- What’s going wrong: Loss of employment transparency can result in legal, reputational, and compliance costs for employers.
- Expert quote: “AI employment must comply with existing fairness and transparency laws.” ―David Seligman, Toward Justice.
Erin Kistler has a degree in computer science from The Ohio State University and spent nearly six years as a program manager at Microsoft, with 19 years of experience in product management, including AI, data management systems, and user experience technologies.
But for the past four years, when she applied for qualified positions at PayPal, Microsoft, Netflix, and other employers, she was automatically rejected.
Processing details
“I’ve applied for hundreds of jobs, but I feel like invisible forces are preventing me from being fairly considered,” Kistler said. “I’m disappointed. I know I’m not the only one who feels this way.”
Each time Kistler received an automatic rejection, she had to submit her application through a link that included “eightfold.ai/careers.” She noticed that all of her prospective employers were working with a company called Eightfold AI, which provides artificial intelligence-based recruitment and recruitment services.
read more:
Kistler is one of two plaintiffs in a lawsuit filed against Eightfold in California state court last week. Eightfold provides AI-based background and credit checks. Three of the company’s financial industry clients, BNY, Morgan Stanley and PayPal, did not respond to requests for comment on the lawsuit.
The case highlights the risks banks face when they rely on AI to do some of the heavy lifting of filtering through vast numbers of job applications.
Indeed, employers were aware that:
According to a new lawsuit filed by nonprofit law firm Towards Justice, Eightfold “utilizes hidden AI technology to collect sensitive information about unsuspecting job applicants and then scores potential employers on a scale of 0 to 5 based on their perceived ‘likelihood of success’ for the job.”
According to the complaint, Eightfold’s technology lurks behind job applications and “for thousands of applicants who don’t even know Eightfold exists, much less that Eightfold collects personal data such as social media profiles, location data, internet and device activity, cookies and other tracking to create profiles of candidate behavior, demeanor, intelligence, aptitude, and other characteristics that applicants never described.”
read more:
The complaint alleges that job applicants were not given a meaningful opportunity to review or challenge Eightfold’s AI-generated reports before a decision was made regarding whether or not they would receive a job.
“Just because this company uses fancy AI technology and is backed by venture capital doesn’t mean it’s above the law. This is not the Wild West,” said David Seligman, executive director of Towards Justice. “AI systems like Eightfold are making life-altering decisions about who gets a job, who gets a home, and who gets health care. And we have a choice to make: Are we going to let them and their investors steal the show from us and take over the market? Or are we going to allow them to follow the law and provide the most basic things: fairness, transparency, and accuracy? That’s what this lawsuit is about.”
A spokesperson for Eightfold said the lawsuit’s allegations are without merit.
“Eightfold’s platform operates on data intentionally shared by candidates or provided by our customers,” he said. He said Eightfold does not covert or collect personal web history or social media to create confidential documents. “We are deeply committed to responsible AI, transparency, and compliance with applicable data protection and employment laws.”
Eightfold uses information applicants choose to submit about their skills, experience and education, as well as data authorized by customers, he said.
But Rachel Dempsey, an attorney with Towards Justice, said Eightfold’s own statements suggest the company is consuming information from all over the internet.
“Our case is based on the allegation that they created this LLM that took all the information from various places online,” Dempsey told American Banker. “One of their big selling points is that they have over a billion pieces of data, all of which feeds into LLM and ultimately leads to these ratings.”
read more:
An Eightfold spokesperson said job seekers will have the opportunity to view the data Eightfold collects and make corrections if necessary.
“This is one of the many differentiators of our platform,” he said. “Eightfold supports a purpose-built experience that allows candidates to view resume information used in our system, correct inaccuracies, and accurately represent their skills.”
But Dempsey said the plaintiffs in the case were not given the power to review the data that led to the automatic denial or correct any inaccuracies.
At the heart of the lawsuit is a simple privacy issue, Dempsey said.
“Going forward, I think fear is really getting stuck in the black box of being on the employment screening company’s blacklist and not knowing why,” she says.
At no point in the application process did Kistler receive a disclosure notice that a consumer report based on her personal data would be obtained for the purpose of evaluating her job application, the complaint said.
She also said she did not receive a summary of her consumer protection rights or information about Eightfold’s name, address, phone number, website address, privacy practices, or other required information under federal and state law.
Kistler claims that during the job application process, she did not have the opportunity to opt out of having her consumer data collected and evaluated for employment purposes. According to the complaint, Eightfold used the consumer report information it collected to score Kistler’s applications in comparison to data from other applicants.
“We have a right to certain control over the information about us that is made public,” Dempsey said. “Part of it is fundamentally an AI issue as well. One of the things that Eightfold does is run a person’s profile through this LLM and make serious predictions about that person. And those predictions may not always be correct and may be informed by discriminatory input. And this LLM I can’t understand what that says about people and how they get to that point.”
According to Towards Justice, under the Fair Credit Reporting Act, the lack of visibility into where a model’s data comes from and the inability to correct it is itself harmful.
Eightfold says it uses what it calls a “match score” to determine whether a candidate is a good fit for a role.
“This is not a universal or portable score that tracks candidates across different companies or unrelated roles, nor is it generated outside the context of the role,” an Eightfold spokesperson said.
Eightfold’s AI model is “supportive, but not rejecting applicants. Importantly, the model is only one part of the system. The match score is determined by how the employer defines the job requirements and criteria, and the AI evaluates alignment against those inputs,” he said.
Eightfold’s models are trained on large and diverse datasets that reflect skills, experience and job-related attributes, a company spokesperson said. “Like all modern AI systems, our models learn patterns from past data, but are explicitly designed not to reproduce past biases. We apply anonymization, rigorous fairness testing, bias reduction techniques, and continuous monitoring to ensure that our results are independent of protected characteristics. Our technology is also used to support human decision-making, not to replace human decision-making. Responsible AI and fairness are fundamental to how Eightfold builds and deploys models.”
The company offers tools such as candidate masking, which hides personally identifying information such as name and gender, a company spokesperson said. “This allows hiring managers to focus on candidate qualities when making hiring decisions.”
The company also undergoes regular internal and external bias testing, including independent third-party bias audits as required by New York City Law Section 144, the spokesperson said. Perform heterogeneous impact analysis to measure whether the results of a system differ significantly for one group from another. We also perform an equalized odds check to verify that the model’s accuracy is consistent across different demographic groups.
Kistler didn’t want to speculate on why she received so many automatic rejection letters, but Dempsey pointed out that the plaintiffs in the case are both women, have STEM backgrounds, and are eligible for the jobs they’re applying for, but they keep getting rejected.
“They’re both passionate about getting a job, they’re passionate about working hard, they’re passionate about sharing and developing their skills, and they’re just being excluded and they don’t know why, and I think that’s really painful,” Ms Dempsey said. “These AI hiring selection tools are a big part of the reason.”
The California Superior Court in Contra Costa will have to find out whether Eightfold violated the law. On the other hand, the frustration people feel when they are denied work by automated systems is real, regardless of whether an AI model is involved.
One industry observer points out that “20-somethings are increasingly wary of things like AI and deepfakes, and are feeling discouraged and dehumanized by the black box of the job application process.”
