What happens to hiring when AI writes job applications?

Applications of AI


in his book About Writing: A Craft MemoirStephen King said that writing is the closest thing to telepathy. Through the magic of the written word, authors can convey facts, fiction, thoughts, emotions, arguments, art, and even messages to millions of readers, sometimes within the same paragraph. When we take the time to write a letter to someone and sign it with “Truly, it's yours,” we imply that the person who wrote the letter is the one who signed the letter. With the advent of generative AI and large-scale language models (LLMs) that can write like humans, true communication between writers and readers is being disrupted. One of the first victims? job market.

When AI enters the chat

Tuck economics professor Anais Gardin and Princeton doctoral student Jesse Silbert were early on studying the impact of AI on the employer-job-seeker matching process. In January 2022, the company began examining data from Freelancer.com. Freelancer.com is a platform where employers can find freelance workers for individual digital tasks such as building websites or coding software. To apply for these jobs, workers write short proposals and bid for the wages they expect to receive. When Galdin and Silbert began analyzing these proposals, they found that more specific, detailed proposals tended to get workers hired, while low-effort proposals often failed. “We already thought this proposal would play an important role on this site,” Silbert says.

Anais Gardin, an expert in international trade, empirical industrial organization, and health economics, teaches managerial economics in the Tuck MBA program.

The world has changed since then. In November of that year, OpenAI released ChatGPT to the public. People quickly marveled at his ability to write in a derivative narrative style, from Shakespeare to a business coach giving advice on side hustles. Freelancer.com joined the fray in April 2023, introducing an AI creation tool on its platform that allows job seekers to create applications with one click. The LLM used job descriptions and user profiles to create suggestions, but employers could not see which suggestions were generated by the AI.

end of signaling

For Galdin and Silbert, this moment was not only an exogenous shock to the system, but also a perfect research opportunity. Traditionally, written job applications and cover letters served as a demonstration of an applicant's skills and abilities. These signals are especially important in a context like Freelancer.com. “Employers receive about 60 applications within minutes, so it's very difficult to tell who's good and who's bad. Any little bit of information beyond the bid is valuable to the employer during the selection process,” Gardin said.

But when combined with AI, anyone can create customized, well-written applications. Therefore, the signal value of writes is reduced, at least in theory. Galdin and Silbert wanted to know what the empirical evidence showed. They present it in a new research paper, “Making Talk Easy: Generative AI and Labor Market Signaling.”

They started by looking at data before and after generative AI, looking at the signals employees sent and the effort they expended in doing so. The co-authors used LLM to quantify how customized and relevant the offers were to job postings, and measured this by looking at click data that showed how much time each employee spent on each application. Before platforms introduced AI, employers were more willing to pay workers to submit customized suggestions. This is thought to be because workers' signals were predictive of their ability to successfully complete a task.

After AI, the situation will change dramatically. As they write in their paper, “employers' willingness to pay workers who send higher signals sharply declines, proposals created with the platform's native AI authoring tools show a negative correlation between effort and signal, and signal no longer predicts job success conditional on employment.” AI appears to have erased a key indicator of a candidate's suitability for a job.

Employers receive around 60 applications within minutes, making it very difficult to know who's good and who's bad. For employers, having even a little bit of information beyond your bid is valuable during selection.
— Anais Gardin, assistant professor of business administration

But Gardin and Silver had to dig deeper. Other factors in the post-AI world may be impacting the job market. Perhaps employers are already outsourcing some of these jobs to AI. Perhaps AI has changed worker behavior regarding job selection. To isolate the impact of AI on the signaling capabilities of job applications, the co-authors built a structural model that captures the market-specific logic. Inside this little lab, they asked what would happen to the market if everyone, or everyone, could write professional job applications, and AI didn't change anything about the market. The result is an alarming dystopian race to the bottom. When simulating a post-AI job market, they write, “workers in the bottom quintile of the ability distribution are hired 14% more often, while workers in the top quintile are hired 19% less often.”

What happens next?

What can employers do today to find the best workers? Perhaps more work is needed. Some jobs, especially high-value jobs, may require a more thorough screening with live talk or even live testing. “Another solution is to allow for a more exploratory type of temporary contracting, where you hire someone to do the first 10% of the work and see how things go before offering them a full-term contract,” Galdin said.

Ironically, AI could help solve the problems it created. Platforms like Freelancer.com can deploy AI tools to streamline employer searches and highlight workers who have completed the most jobs similar to those listed. Another possibility is AI-powered surveys that help employers find the right talent based on job details.

For workers, AI will likely increase the importance of highlighting their work history. No longer will you use eloquence and rhetoric to prove your worth, but the challenge will be to come up with solid currency: “What have you done lately?”





Source link