Microsoft’s new and experimental Copilot features let AI analyze your files

Microsoft is expanding its ongoing efforts to make Windows 11 an artificial intelligence-focused operating system (OS) with several Copilot updates, including new Voice and Vision modes, as well as several new experimental features.

First, Microsoft wants to add the “Hey Copilot” word to Windows 11 devices. Users can enable the wake word in the Copilot app. Once enabled, users can say “Hey, co-pilot” to start a conversation using Copilot Voice. When finished, exit the game by pressing “X” to close Copilot, or simply say “goodbye”.

Microsoft goes on to say that it will make Copilot Vision available in all markets where Copilot is offered. Vision allows Copilot to view content on your screen. For example, users can share their desktop or applications so Copilot can analyze content, provide relevant information, and more.

This extends to applications such as Microsoft Office applications, allowing Copilot to analyze your documents or help you view your PowerPoint slides.

Another part of Vision is a feature called Highlights. With Highlights, users can ask Copilot for help completing tasks, and AI can show users where to click and how to complete a task.

Finally, Microsoft will soon introduce text-based interactions with Vision to Windows Insiders, giving people another way to interact with Copilot Vision.

Experiments with artificial intelligence, such as a new taskbar interface

Image credit: Microsoft

Speaking of Insiders, Microsoft is also bringing more experimental AI features to Windows Insiders and Copilot Labs. One of the new features Microsoft is testing is a new taskbar that puts “Ask the Co-Pilot” right on the taskbar.

There is also a preview version of Copilot Actions for Local Files, which can help users perform actions on files on their PC. Examples given by Microsoft include sorting photos and extracting information from PDF files.

Microsoft said it will start with a narrow set of use cases while optimizing the model's performance. As always, the company warns that the AI ​​may make mistakes.

Next are Copilot connectors, which allow users to connect external services to Copilot. This includes OneDrive and Outlook, as well as Google services such as Google Drive, Gmail, Google Calendar and Google Contacts. Once users opt in to these services, Copilot will be able to use personal information from these services in responses, allowing people to ask about calendar appointments, contact email addresses and more.

Image credit: Microsoft

Another integration is Copilot and Windows Settings, allowing users to ask Copilot to help them change PC settings, such as enabling Focus Mode or Dark Mode. Oddly enough, I thought this was already a feature – I previously asked Copilot to enable dark mode and it did! But when I checked it again while writing this story, it didn't work. I'm not sure when this feature was removed, but I think it's good to have it back.

Microsoft has announced a new generative artificial intelligence agent called Manus. It uses features of the Windows agent platform, such as the Model Context Protocol (MCP), to perform tasks. The company says Manus can perform complex tasks, such as building a website, using documents stored in photos on your computer. Manus is currently in closed preview.

Rounding out the experimental features, Windows Insiders with a Copilot Plus PC can use the new Zoom integration with Click to Do, which allows them to instantly schedule Zoom meetings by simply hovering over an email address.

Finally, Microsoft has released a beta version of Gaming Copilot (pictured above), available on the new ROG Xbox ally And Ally X gaming handheld computers. Gamers can access Gaming Copilot using keyboard shortcuts to ask for help and recommendations without even leaving the game they're in.

Some of the new features are already available, while many are currently only available to Insider testers or private previews. Follow Microsoft Windows Insider Blog to find out when new features will be available.