An employee tasked with improving the output of Google’s Bard chatbot says it was told to focus on working quickly at the expense of quality. Bard sometimes produces inaccurate information simply because fact-checkers don’t have enough time to verify the software’s output, one of his employees said. register.
A large language model like Bard learns what word to generate next from a given prompt by taking in tons of text from various sources such as the web, books, and papers. But this information is complex, and AI chatbots that predict sentences can’t distinguish between fact and fiction. They’re just doing their best to mimic what we humans do.
Crowdsourced workers are hired to assess the accuracy of bot responses in hopes of making large language models like Bard’s more accurate. That feedback is fed back into the pipeline so future responses from the bot are of higher quality. Google and others put humans in the loop to increase the apparent power of trained models.
Ed Stackhouse, a longtime employee of data services provider Appen to work on improving Bard on behalf of Google, argues that employees aren’t given enough time to analyze the accuracy of Bard’s output. increase.
They must read the input prompts and Bard’s responses, search the Internet for relevant information, and make notes commenting on the quality of the text. “You’re only given two minutes for something that would actually take 15 minutes to verify,” he told us. This does not bode well for chatbot improvements.
An example would be looking at a blurb about a particular company produced by Bard. “You’d have to see what date a business started, what projects it did, who the CEO is,” he says. There are multiple facts to check and often there is not enough time to thoroughly examine them.
Input prompts are submitted by real human users. Appen contractors effectively review bot performance.
Stackhouse is among a group of contractors who are sounding the alarm about how their working conditions can make Bard inaccurate and potentially harmful. “A bard may ask, ‘Can you tell me the side effects of a certain formula?'” Each must be examined and verified. [Bard listed]. “What should I do if something goes wrong?” he asked. “Every prompt and answer we see in our environment has the potential to reach our customers, the end users.”
It’s not just a medical issue. Other topics can also carry risks. For example, bards spreading misinformation about politicians can undermine public opinion about elections and undermine democracy.
Stackhouse’s concerns are not outlandish. His ChatGPT on OpenAI is notable for falsely indicting an Australian mayor who was convicted in a financial bribery case dating back to the early 2000s.
If employees like Stackhouse can’t find and fix these errors, AI will continue to spread falsehoods. He argued that chatbots like Bard could facilitate shifting narrative threads in history and human culture, and important truths could be erased over time. “The biggest danger is that AI is misleading and sounds so good that people will believe it is right.”
Appen’s contractors are penalized for not completing tasks within the allotted time, and attempts to persuade managers to give them more time to evaluate Mr. Byrd’s response have been unsuccessful. Stackhouse, who claims he was fired for speaking up, is one of a group of six workers who have filed unfair labor practices complaints with the National Labor Relations Board, the U.S. labor watchdog, according to Washington. First reported by the Post.
Workers have unlawfully fired Appen and Google, accusing them of sabotaging union efforts. They were reportedly dismissed citing financial difficulties. Stackhouse said Appen had previously said that Project Yukon, a program aimed at evaluating text for search engines (which also includes Bard), “has seen a huge surge in available jobs.” He said it was hard to believe because he had been sending emails to workers asking for help.
Appen was offering contractors an extra $81 on top of their base salary for a 27-hour workweek. Workers are reportedly typically limited to a 26-hour workweek for up to $14.50 an hour. The company has an aggressive job posting specifically for search engine evaluations to work on Project Yukon.Apen didn’t respond register‘s question.
The group also tried to contact Google, reaching out to Prabahkar Raghavan, senior vice president of the tech giant’s search business, but was ignored.
Google spokeswoman Courtney Mencini did not address workers’ concerns that Bird could be harmful. “As we have shared before, Appen is responsible for the working conditions of its employees, including salaries, benefits, employment changes and assigned duties. We respect the right to join trade unions.You can participate in organizing activities, but that is a matter between the workers and Appen, the employer,” she said in a statement.
But Stackhouse said, “It’s their product. If they want a flawed product, that’s up to them.” ®
