Microsoft yesterday launched a range of Copilot Plus PCs that include new AI features built directly into Windows 11. Behind the scenes, the company is currently running more than 40 AI models on Windows 11 thanks to the new Windows Copilot runtime. This also allows developers to: Please use these models in your app.
Today at Microsoft Build, the company is providing more details about how exactly this Windows Copilot runtime works. The runtime includes a library of APIs that developers can leverage into their own apps, with an AI framework and toolchain designed to allow developers to ship their own on-device models on Windows. It contains.
“The Windows Copilot Library consists of ready-to-use AI APIs such as Studio Effects, Live Captions Translations, OCR, Recall with User Activity, and Phi Silica, and will be available to developers in June,” Windows said. explains Pavan Davuluri, head of Surface.
Developers can now use the Windows Copilot Library to integrate studio effects, filters, portrait blur, and other features into their apps. Meta is adding Windows Studio effects to WhatsApp, so you can get features like background blur and eye contact during video calls. Live captions and new AI-powered translation features are also available to developers with little to no code.
Microsoft demonstrated its Recall AI feature yesterday. This allows Copilot Plus PC to document and save everything you do on your PC so you can recall your memories and search through your timeline. This is all powered by the new Windows Semantic Index, which stores this data locally, and Microsoft plans to enable developers to build something similar.
“We are providing this functionality so that developers with our Vector Embeddings API can build their own vector stores and RAGs within their applications and with application data,” Davuluri said.
Photo: Alison Johnson/The Verge
Developers can also improve the new Recall feature in Windows by adding context information to their apps that feeds into the database that powers this feature. “This integration allows users to pick up where they left off within an app, increasing app engagement and creating a more seamless flow for users between Windows and apps,” he said. Masu.
All of these improvements inside Windows for developers, along with new Arm-powered systems and NPUs coming soon from AMD and Intel, are very early building blocks for building more AI-powered apps. Microsoft is building a platform for developers to create AI apps for Windows, and now expects this to be a key part of his Windows development over the next decade. On stage at his Build today, Davuluri stood in front of a slide that read, “Windows is the most open platform for his AI,” demonstrating how important this moment is for Microsoft. .
