In early 2023, WorkStory, Inc. wanted to take its flagship product, VisualCV, to the next level while eliminating AI bias.
This Vancouver-based firm helps organizations manage multiple resumes. However, with the rise of generative AI, WorkStory wanted to upgrade his VisualCV with generative AI capabilities. Looking for an AI vendor, the resume builder found Armilla, a responsible AI platform vendor.
“This is the first time we have built these capabilities into a product,” said Jade Bourelle, CEO and co-founder of WorkStory. “We wanted to partner with a company that had experience as well as technology.”
Armilla and AutoAlign
Since 2019, Armilla AI has been helping companies test specific features of their AI model applications. With the rise of generative AI, the Toronto-based vendor introduced AutoAlign on June 22nd.
AutoAlign is a web-based AI platform that helps remove gender, race, and other biases from Large Language Models (LLMs) such as GPT-3 and Google Bard. This is achieved by fine-tuning open-source models such as Meta Llama to remove bias, or by acting as a safety guardrail for models that cannot be fine-tuned such as GPT-3.5 and GPT-4. By fencing non-open source products, companies can prevent models from responding to prompts in harmful ways.
During a TechTarget editorial demonstration, Armilla CEO and co-founder Dan Adamson showed how the open model can be tricked into providing instructions for creating dangerous substances such as napalm. . Instead, for companies that use AutoAlign as a guardrail, the platform prevents models from generating such information.
“It’s very powerful because you don’t need the full training set. You can create these training and evaluation sets and do your own tuning,” Adamson said. “You can look at these results and evaluate them, look at the training set and tune them, but in the end you will have a model that fits all of these tuning controls. “
AutoAlign allows companies to tune their LLMs for bias, run and compare different models against a set of controls to determine which best suits their goals and objectives. Become. For example, some customers started fine-tuning his one model, but when they ran the same fine-tuning tests against open-source models like Llama and RedPajama, they got better results, Adamson said. says.
“Making sure you’re using the right model is very powerful,” he said.
Armilla AI’s AutoAlign prevents hackers from jailbreaking LLMs and can trick base models into refusing to respond to prompts that give them information about manufacturing hazardous materials.
WorkStory and AutoAlign
The ability to fine-tune LLM for a particular purpose made AutoAlign attractive to resume writing companies like WorkStory. The company has access to a beta version of the product and has tested its generative AI capabilities into VisualCV.
“That’s added value, too.” [for us] “It’s about putting guardrails in place, one of which is being able to remove bias and limit what users shouldn’t do,” Bourrel said.
For example, WorkStory helps a variety of organizations update employee resumes, so system-generated resumes shouldn’t reflect poorly on employees.
“If you’re creating automated content and someone enters a description of a project they’ve worked on, and the AI is creating it, don’t make sure the project overview doesn’t provide information that you don’t provide. We need it, we want it,” Bourrel added.
overtune or undertune
This is the first time we have incorporated these features into a product. We wanted to partner with a company that had experience as well as technology.
Jade BourrelWork Story Co., Ltd. President and Representative Director
Brell said the platform hasn’t failed to deliver what WorkStory wants, but Adamson says customers have flipped or failed to adjust the model, with undesirable results. pointed out that there was
During the demonstration, he showed off models that were overtuned for their gender.
Adamson prompted the LLM to introduce the CEO. The original models all showed males, while the fine-tuned models all showed females. This is because it was tuned to be biased towards the opposite end of the spectrum instead of being gender-neutral.
“It’s part of the power of the system, but it means you have to be very careful,” Adamson says. “You also can’t apply things the right way, which means you can’t set the right alignment controls or you might miss something.”
But despite these issues, Bourrel said he expects the technology to continue to evolve.
WorkStory currently determines its cost model with Armilla according to resume company usage. A new and improved VisualCV with generated AI capabilities will be released in the fall.
Esther Ajao is a news writer covering artificial intelligence software and systems.
