Orca, an AI model with 13 billion parameters, can mimic and learn large language models like GPT-4.
Microsoft has partnered with OpenAI to steadily implement AI capabilities in our products and services, as well as build smaller, case-specific models. Microsoft Research has announced a new AI model called Orca that learns by imitating large language models. According to research papers, Orca is designed to overcome the limitations of smaller models by mimicking the inference process of larger underlying models like GPT-4.
Language models like Orca can be optimized for specific tasks and trained using large language models like GPT-4. Orca’s small size requires less computing resources to run and operate. Researchers can optimize their models according to their requirements and run them independently without relying on large data centers.
According to the research paper, Orca is an AI model with 13 billion parameters that can mimic and learn large language models like GPT-4 and is based on Vicuna. Rumored to have over a trillion parameters, he can learn explanations, step-by-step thought processes, and other complex commands with the help of GPT-4.
you’re exhausted
Limited number of free calls per month.
To read more,
Simply register or sign in
subscribe and read more
Please select a plan
all access
access to premium story
digital only
access to premium story
This premium article is free for now.
Subscribe to continue reading this story.
This content is for subscribers only.
Subscribe for unlimited access to premium articles exclusive to The Indian Express.
This content is for subscribers only.
Subscribe now for unlimited access to premium articles exclusively from The Indian Express.
Microsoft leverages large and diverse imitation data to drive progressive learning with Orca. Orca already outperforms Vicuna by 100% on complex zero-shot inference benchmarks like Big-Bench Hard (BBH). The new model is also claimed to be 42% faster than the traditional AI model on AGIEval.
In theory, Orca is said to be on par with ChatGPT in benchmarks such as BBH, despite being a smaller model. In addition, we have shown competitive performance on written exams such as the SAT, LSAT, GRE, and GMAT, although not as good as the GPT-4.
Microsoft researchers say Orca can learn using step-by-step human-created explanations and more sophisticated language models, potentially improving its skills and abilities.
© IE Online Media Services Pvt Ltd
Date first published: Jun 19, 2023 15:53 IST
