Microsoft wants to bring generative AI to the forefront of Windows and the PCs that run it.
In two keynotes at this week's annual Build developer conference, the company announced a new line of Windows machines it calls Copilot+ PCs and leverages AI such as Recall to help users find apps, files, and other content. announced the generation function. What I saw in the past. Copilot, Microsoft's brand of generative AI, will soon be more deeply integrated into the Windows 11 experience. And new Microsoft Surface devices are also coming.
Here are all the major announcements from Monday and Tuesday.
volume measurement app
Microsoft is bringing Windows Volumetric Apps (basically spatially aware, interactive VR apps) to the Meta Quest headset. Microsoft says that through its partnership with Meta, it will bring Windows 365 and local PC connectivity to the Quest headset, allowing developers to extend their apps into 3D space.
During Tuesday's keynote, Microsoft showed off a digitally exploded 3D view of the Xbox controller from the perspective of the Meta Quest 3 headset, a digital object that the wearer can manipulate with their hands. “We're deepening our partnership with Meta to make Windows the best-in-class experience on Quest devices,” Pavan Davuluri, Microsoft's CVP of Windows and Devices, said during the demo. .
Developers can sign up for a preview to access Microsoft's new volume API.
Copilot+ PC
Copilot+ PCs are flagship Windows hardware that represents Microsoft's AI-first vision. They all include a dedicated chip called an NPU to power AI experiences like Recall, and they ship with a minimum of 16GB RAM and paired with SSD storage.
The first Copilot+ PCs are powered by Qualcomm's Snapdragon X Elite and Plus chips, which Microsoft claims will deliver up to 15 hours of web browsing and 20 hours of video battery life. Chipmakers Intel and AMD are also partnering with various manufacturers, including Acer, Asus, Dell, HP, Lenovo, and Samsung, to develop processors for Copilot+ devices.
Pricing for the Copilot+ PC starts at $999, with select units available for pre-order starting today.
Surface Pro and Surface Laptop
Microsoft's newly announced Surface devices, Surface Laptop and Surface Pro, focus on performance and battery.
The latest Surface Laptops with 13.8-inch or 15-inch displays have been redesigned with “modern lines” and thinner screen bezels. The company says it lasts up to 22 hours on a single charge and is up to 86% faster than the Surface Laptop 5. It also supports Wi-Fi 7 and has a haptic feedback touchpad.
About the new Surface Pro, Microsoft says it's up to 90% faster than the previous generation Surface Pro (Surface Pro 9) and features a new OLED with an HDR display, Wi-Fi 7 (and optional 5G). states. Upgraded ultra-wide-angle front camera. Additionally, the removable keyboard is reinforced with additional carbon fiber and features haptic feedback.
recollection
Windows 11's upcoming recall feature will allow users to “remember” apps and content they accessed on their PC weeks or even months ago, for example when discussing clothes they were considering buying. Helps you find the Discord chats you're looking for. Users can use her Recall timeline to “scroll back” to see recent work or drill down into files such as PowerPoint presentations to see information that may be relevant to their search. You can.
Microsoft says Recall can create associations such as colors and images, allowing users to search for virtually anything on their PC in natural language (not unlike technology from startup Rewind). Developers can improve recall by adding contextual information to their apps. And Microsoft claims that all user data associated with Recall is stored privately on the device and, importantly, is not used to train AI models.
Here's more from Microsoft: “Your snapshots belong to you. They remain locally on your PC. From the system tray icon on your taskbar, you can delete individual snapshots, adjust and delete time ranges in settings, or pause at any point. You can also filter apps and websites to prevent them from being saved.”
Image editing and live translation
Windows has more AI than ever before, and some of it is exclusive to the new Copilot+ PCs.
A new feature, “Super Resolution,” can automatically enlarge and restore old photos, while Copilot can now analyze images to give users ideas for creative compositions, and a feature called Cocreator lets users generate images or even have an AI model follow what they're drawing to alter and restyle the image.
Also, Live Caption with Live Translation translates audio passing through your PC from YouTube or a local file into the language of your choice. Live Translation will initially support around 40 languages, including English, Spanish, Mandarin Chinese, and Russian.
A separate, related new feature in Microsoft Edge provides real-time video translation on sites like LinkedIn, YouTube, Coursera, Reuters, CNBC, and Bloomberg. Expected to be available in the near future, this feature supports translations from Spanish to English, English to German, Hindi, Italian, Russian, and Spanish, and audio content through both dubbing and subtitles. Translate live.
Team Copilot and Extensions
Team Copilot is the latest extension to Microsoft's growing suite of Copilot generative AI technologies. It integrates with Teams, the company's video conferencing app, so you can manage meeting agendas and create notes that all meeting participants can co-author. It also extends to Loop and Planner, Microsoft's collaboration and planning platforms, where you can create and assign tasks, track deadlines, and notify team members when their input is needed.
In somewhat related Copilot news, Microsoft has launched (in private preview) Copilot Extensions, which allows developers to extend GitHub's code generation tool, GitHub Copilot, with third-party apps and skills. Launch partners include DataStax, Docker, and LambdaTest. Extensions exist on his GitHub Marketplace, but developers can also create their own private extensions to integrate with internal systems and APIs.
Windows Copilot Runtime
Powering features like recall and super-resolution is the Windows Copilot runtime, a collection of about 40 generative AI models that make up what Microsoft calls a “new layer” of Windows. The Windows Copilot runtime works with Semantic Index, a vector-based system local to each individual Copilot+ PC, to enable generated AI-powered apps, including third-party apps, to run without necessarily requiring an internet connection.
“[The runtime] Studio effects, live caption translation, OCR, user activity recall, [more]will be made available to developers in June,” Davuluri said on Tuesday.
Microsoft says CapCut, the popular video editor from TikTok owner ByteDance, will use Windows Copilot Runtime and its new Windows Copilot Library, a set of APIs and AI development tools, to accelerate its AI capabilities. is. Meta also adds the aforementioned Studio Effects to WhatsApp, offering features like background blur and eye contact during video calls.
Upgraded bot builder
Azure AI Studio, a toolset within Microsoft's Azure OpenAI service, allows customers to combine AI models and build apps that “reason” about their data. Developers will soon be able to create apps using pay-as-you-go inference APIs (APIs). This allows developers to access and fine-tune generative AI models hosted on Azure infrastructure. Microsoft calls this the “as-a-service model,” and it's starting with the Nixtla and Core42 models.
In the adjacent Copilot Studio product suite, Microsoft releases the Copilot agent. The company describes it as an AI bot that “can independently tailor tasks to specific roles and functions.” (Copilot Studio is an AI-powered “copilot” for apps like Excel and Word.) (Provides tools for connecting Microsoft 365 to third-party data). Copilot agents can leverage memory and contextual knowledge to navigate different types of business workflows, learn from user feedback, and ask for help when they encounter situations they don't know how to handle. .
Snapdragon development kit
Qualcomm has a new development kit aimed at developers building apps for Copilot+ PCs powered by Arm chips
By the way, the $899.99 Snapdragon development kit for Windows (about the same width, height, and length as Apple's Mac Mini) includes Qualcomm's Snapdragon paired with 32GB of RAM, 512GB of storage, and tons of I/O. Equipped with X Elite chip. This development kit supports Wi-Fi 7 and Bluetooth 5.4 and can drive up to three of his 4K monitors simultaneously via various USB-C and HDMI ports.
Phi-3
Microsoft announced the addition of Phi, a family of generative AI models that can perform common visual analysis and inference tasks, such as answering questions about charts and images. This model can read both text and images and is efficient enough to run on mobile devices.
Phi-3-vision is available in preview, while the previously announced text-only models, Phi-3-mini, Phi-3-small and Phi-3-medium, are now generally available.
Partnership with Khan Academy
Microsoft will partner with Khan Academy and donate access to cloud computing infrastructure. This allows Khan Academy to offer U.S. educators free access to its AI-powered tools. The two companies will also work together to explore opportunities to improve AI apps for math instruction through generative AI, Microsoft announced Tuesday.
We're starting an AI newsletter! Sign up here Start receiving it in your inbox starting June 5th.
