All the new features coming to Windows 11 and Copilot
Microsoft is bringing several exciting new AI-driven features to Windows 11 and its Copilot AI assistant. Some of these tools will be available to all Windows 11 desktops and laptops as part of the 24H2 Windows 11 update, while others will be exclusives specifically for Copilot+ laptops.
Look for new features like generative fill and AI erase in Paint, a new upscaling algorithm for the Photos app, and upgrades for Copilot so it can better understand what’s on the screen, and what you’re asking it to do -- including becoming a literal chatbot by listening to your voice and responding in kind.
Smarter Copilot in Windows
As useful as Copilot can be in Microsoft 365 and other applications, its everyday integration in Windows itself has been more limited. With the latest updates, though, it will have several new and impressive capabilities. Some of these will be available to everyone soon, while others will be rolled out more slowly to select Copilot Pro users as part of Microsoft’s Copilot Labs feature.
The first example of a more selective new feature is Think Deeper. This will use new reasoning models to make Copilot more capable of thinking through complicated requests, like solving harder math problems, or working out estimates for your next project based on supplied data. Think Deeper gives Copilot longer to respond, too, so it can take more time on its calculations to generate more useful insights.
Copilot Vision is an opt-in feature that will make it possible for the AI to understand what you’re looking at within the Edge browser, letting you ask questions about the content of the webpage or document you’ve opened in it. It can also help you navigate through the page, or suggest where to click or go next, but seems most useful for asking questions about images on a webpage.
Due to the potential sensitivity of what you see in your browser, too, Microsoft has made it clear that along with Copilot Vision requiring you to opt-in to use it, none of the audio, images, text, or your conversation with Copilot will be stored or used in training data. Once you close the feature, any data that was used for processing your requests is immediately discarded. It is also unusable on sensitive content pages and services and can only be enabled on a number of pre-approved sites for right now.
Microsoft is also making it easier to access Copilot in Edge. Simply type “@Copilot” into the address bar and hit Enter and you’ll start a Copilot conversation.
You’ll be able to find unique new Copilot functions in other Microsoft apps, too, like a super-resolution capability in Photos, to dynamically upscale your images, and a generative fill and erase tool in Paint.
Microsoft also suggested that the upcoming updates would lead to an improved Windows Search experience. It didn’t go into specific details, but improvements to Bing generative search would suggest Windows search will soon better understand what you’re looking for, and may be able to leverage Copilot to answer search queries more directly there.
Screenshot: Microsoft
A more personable Copilot
Alongside some of the advanced voice tools we’ve seen launching from ChatGPT, Microsoft is looking to make Copilot easier to converse with in a more personable manner. With the Copilot Voice update, you’ll be able to select from four conversational voices for Copilot to use, and it should understand you better, too, offering more personalized conversational asides as part of your discussion.
Microsoft gives the example that alongside with helping you to plan a trip, Copilot might offer you some advice on how to deal with the frustration of your seat being kicked by a child sat behind you.
Copilot Daily will help you start the day with a summary of what’s going on in yours, and the wider world, tailored to you and your preferences. It’ll let you know the local weather, any scheduled meetings or appointments that day, and useful news stories you might be interested in, all read to you by the chosen Copilot voice. It’s designed to be a cleaner, more succinct way to get up to speed on what you need to know, without all the clutter of more general websites and services.
Personalized Discover will give you more jumping-off points for conversing with Copilot too, making it easier to make requests for things you need in a way that will have it respond how you want. The prompts and suggestions will be customized based on your interactions with other Microsoft apps and services in the past, and will be further personalized over time.
When can I use the new features?
This very much depends on where you live, your Copilot subscription, and the hardware you’re running it on. Some of these features will be available to Windows Insiders only, some limited to Copilot+ PCs with a Copilot Pro subscription, and others available to the wider array of free Copilot users in the months to come.
For the best chance of getting the latest features as soon as you can, make sure to download the latest Windows 11 2024 update, known as 24H2, via Windows update. You should get access to Improved Windows Search, and updates to Photos and Paint first, with other features coming later.
If you’re an early Insider, though, you might have access already, including to the latest Recall preview.
