Gabriela Riccardi talks to host Scott Norber about how AI algorithms are impacting jobs around the world. season 5 finale of Quartz Obsession Podcast!
🎧 listen now again read the transcript
Introducing our new hiring manager. she’s an algorithm. Over the past decade, the growing AI development industry has started building algorithmic tools that it hopes will enable companies to outperform humans in hiring. Vendors proclaim that AI can make data-driven decisions that are more objective than human judgment, more reliable than human error, and fairer. than human prejudice.
But when we look at real-world use cases, it becomes clear that AI is a capricious recruiter.take this An oft-quoted Amazon anecdotetrained a proprietary tool on 10 years of resumes submitted to the company from 2014 to 2017. But thanks to the male-dominated tech industry, Most of those applications were from men, and the algorithm recognized that fact.
result? The computer learned that male candidates were ideal and became biased against women. In fact, algorithms deducted points for using the word “women,” such as “women’s softball” or “women’s rights” on a resume. The program could not be debugged and Amazon discontinued the project.
Amazon’s failure isn’t the only documented case of the hiring algorithm behaving strangely. AI has been found to make all kinds of arbitrary decisions. prevent job seekers from being hiredAnd in many cases the decisions are also discriminatory.
Still, software companies weren’t deterred from pushing forward with tools that promised to defeat prejudice. (In fact, even Amazon may be trying to build an algorithm) once again.) After all, AI is already in our job search. More than 80% of her employers covertly use AI to make hiring decisions, Charlotte Burroughs, chairman of the U.S. Equal Employment Opportunity Commission, suggested. at the hearing this year.
So pull out your Algorithmic interview desk chair. However, do not be surprised if your application is rejected. Quartz has rounded up some of the weirdest ways AI screens job candidates, from the quirky and bizarre, to the insane, to the wrong, to the blatantly biased. Internal to the algorithm, there are an infinite number of reasons for rejection.
👩 Your name is Ebony not Emily
in a famous place 2004 experimentThe researchers sent 5,000 resumes to more than 1,000 job openings nationwide. The resumes were identical with one difference. The names at the top were clearly white (like Alison and Todd) or black (like Aisha and Tyrone). And as the study revealed, white-looking candidates were 50% more likely to be offered an interview.
In 2016, researchers from Princeton University and the University of Bath, I reproduced this finding with an AI algorithm. They found that GloVe, a word mapping AI, more readily associated Caucasian-sounding names with positive attributes. Again, the AI made him 50% more likely to offer an interview to a candidate with a white-sounding name than a candidate with a black-sounding name.
👨 Or your name isn’t Jared
Employment Attorney Mark J. Girouard told Quartz in 2018 One of his customers was scrutinizing an AI-powered resume screening tool and wanted to know what the algorithm prioritized.
“After auditing the algorithm, a resume review firm found that the algorithm found two factors that best indicated job performance: their name was Jared and they played lacrosse in high school. It was whether or not,” writes reporter Dave Gershgone. Girard’s client did not use this tool. “
📚 I interviewed with the bookshelf behind me.
In 2021, german public broadcasting We tested an AI-powered video interviewing startup. The company claimed that it could “evaluate candidates in all areas,” so to speak. big five character traits. Often used in psychometric recruitment assessments.
Candidates can record video responses to questions posed by the software, and the company’s AI can analyze the candidate’s voice, language, gestures and facial expressions to generate a candidate’s personality profile, the company claimed. The AI then recommends whether or not to proceed to the next round.
Broadcasters had actors and actresses interviewed using software, then interviewed again with minor adjustments to clothing, camera, and background. When a bookshelf was added behind an actor, the AI’s reading of his personality profile changed. It ranked him as fairly unnerving. And while that may sound like a good thing, high neuroticism can be considered a plus in some positions, among other supposed changes. to his character.
🥸 Or wearing glasses or a scarf
The same goes for these, according to the broadcaster. When the actress put on her glasses for a video interview, the AI lowered her conscientiousness score by nearly 10 points. The same actress took off her glasses and put on a scarf, and found she was more open, more honest, and less nervous. When another actress’ video went dark, her four of her five scores dropped.
🇬🇧 You have Leeds, Liverpool and Cardiff accents
Another frequently used feature of video interviewing platforms relies on answer transcription. The video software shows the candidate’s interview questions on camera. Answers are recorded and transcribed for someone (or something) to review.
To see the impact of voice in transcription, watch the BBC Three documentary. computer says no (2022) asked participants from different parts of the UK to recite the popular British football anthem “Three Lions”. (You may know the line I often shout, “I’m coming home!”).
But when volunteers with regional accents read the words for the system, the AI proved unable to interpret their words. Instead, the transcription service treats them as “[Voice Not Recognised]”
🇩🇪 But it would be great if you could only speak German!
In 2021, journalists will At MIT Technology Review tested two AI-powered interviewing platforms, MyInterview and Curious Thing. They set up fake job postings in each tool and applied for the posted roles. The algorithm will likely score candidates’ English proficiency, among other qualifications.
Hilke Schermann used AI software to complete phone and video interviews. On each platform, she interviewed her twice. At first, she answered questions as if she were being interviewed normally. The second time, she answered her question by reading the German Wikipedia entry on psychometrics.
result? Curious Thing ranked her German interview a 6 out of 9 in terms of English proficiency, and MyInterview predicted that the German Sherman would be 73% fit for the role. The transcript system interpreted the German words as English even when the reading made no sense. Here’s what Argo heard:
“I mean, humidity is a big hit for desks. Sociology, does that apply? Nematodes of mined matter adapt.”
🎓 Your degree was obtained from the Seven Sisters School
That’s the case with Amazon’s internal algorithm, which was trained on existing male-dominated resumes and learned its prejudices against women in 2017. AI explicitly downgraded graduates of two women’s colleges, according to people familiar with the matter. told Reuters. perhaps even a presidential candidate AI cannot lock down jobs.
👂 you are hearing impaired
In 2019, a hiring specialist who primarily works with hearing-impaired job seekers had customers apply for jobs on Amazon.Required application form HireVue interviewis an AI-powered video platform that, according to experts, did not offer clear avenues for addressing hearing impairments.
The professional, who is deaf himself, tried to call the company for lodging, but was unsuccessful. Amazon hired the customer only after the professional and applicant showed up on site with her ASL interpreter.
💬 Or it could be a speech impediment or a visual impediment.
They didn’t mention any documented lawsuits, but in 2022 the U.S. Department of Justice and the Equal Employment Opportunity Commission joint instruction On how AI-based tools can be used to discriminate against workers with disabilities.
The Justice Department is “sounding alarm bells” about AI adoption, said Assistant Attorney General Kristen Clarke. “We cannot allow these tools to become high-tech channels for discrimination,” added EEOC Chairman Charlotte Burroughs. For 2023, the agency announced: Additional guidance How AI used in employment can violate civil rights.
One example of the danger the EEOC cited is an algorithm that screens candidates based on their disability. For example, video interview software that assesses an applicant’s speech patterns is “unlikely to score the applicant fairly” if the applicant has a language disorder that causes differences in speech patterns, such as stuttering, the agency said. Stated. A chat-based tool trained to ask questions about employment gaps could reject candidates who had to take time off from their careers for health treatment. Alternatively, gamified tests that require applicants to cross a threshold (e.g., obtain a score of 90% on a memory game) indicate that they are visually impaired but have all the memory skills required for the job. We may not be able to accommodate applicants with
🔥 Bonus: AI Shooting Special
In 2020, a group of British makeup artists at MAC Cosmetics, owned by Estée Lauder, were told the company would be laid off. They were video interviewed and asked to reapply. The interview turned out to be on his HireVue, and the algorithm started scoring his answers.
Instead of doing makeup trials like the standard procedure, the artists had to verbally describe their approach. For example, instead of actually doing my smokey eye makeup, I had to tell the computer how to apply it. I repeat: Artists had to use words instead of brushes to demonstrate their makeup prowess.
All of the artists have been fired, a decision based in part on AI analysis. However, all three said they scored highly on ratings they knew, such as sales figures and performance reviews. “Basically my performance has been brilliant and everything else has exceeded expectations,” said one of the women. told the BBC. “This interview definitely set off alarm bells for me.” According to the artists, no one at the company was able to provide a reason for the dismissal.
“We literally thought we were being videotaped and someone would mark it later. It turns out not,” said another woman. “Nobody watched the video. It was all an algorithm.” The artists reached an out-of-court settlement after Estee sued her loader for firing her.
