Ever wanted Windows to monitor what’s on your screen continuously and ask it to do stuff for you? Well, have I got news for you: Copilot Vision, as this feature is called, is now rolling out to users in the US.
Copilot Vision on Windows, now available in the U.S., is a new way to engage with your Windows 11 PC. When you enable it, it can see what you see on your device and talk to you about it in real time.
It acts as your second set of eyes, able to analyze content, help when you’re lost, provide insights and answer your questions as you go. Whether you’re browsing, working or deep in a project, Copilot Vision offers instant insights and answers.
↫ The Windows Experience Blog team
Do note that Copilot Vision will not keep its data on your device, instead sending it off to Microsoft. So, if you ever wanted to give Microsoft even more insight into exactly what you’re doing with your Windows installation, now’s your chance. Well, if you’re in the US, that is, and some its capabilities are only available on Snapdragon PCs, not on Intel or AMD machines.
In the same blog post, Microsoft also highlights a few actually useful features coming to Windows, like a colour picker in the screenshot tool, the ability to change the lighting in photos, or having the area selector in the screenshot tool snap to what it thinks is the important part you wish to actually take a screenshot of. However, Microsoft is also adding nonsense like sticker generators in Paint, text generators in Word, and stuff related to Teams that makes me even happier than I already am that I’m self-employed and work alone.
If you’re in the US, you can get these features now if you wish through Windows Update.
I have the “Blues”…
This, all the more makes me refuse to “upgrade” to WIndouche 11
spiderdroid,
While I do see AI’s potential, I am against forcing AI features on users who don’t want them. I think it would be better (ie more respectful) to have users explicitly request AI applications than to have them get shoved onto user’s computers via windows update. It becomes especially egregious when it comes to transmitting personal data to servers outside of the user’s possession – this is a huge red flag for me. Unfortunately the US doesn’t take consent and privacy that seriously, which is likely a factor in microsoft’s US rollout.
Would an AI “Clippy” help people make the switch ?
It does say in the blurb, and has been repeated plenty of times for anyone following the Copilot drama, that it is an opt-in feature. So at least for now, yes the bits are being shoveled in, but no one is forcing you to use them.
Moochman,
Insofar as the copilot options need to be clicked on to launch them and requires a subscription to use, then I can see why that can be called opt-in. When I try using it, it says I need to subscribe. Without a subscription, the copilot features basically serves as a permanent ad for copilot and I don’t see an intuitive way to remove this from applications.
Insofar as copilot is being automatically installed and loaded into applications, I can personally vouch that it is NOT opt in. It’s clearly being installed without asking users.
In terms of privacy, I would hope there’s zero copilot telemetry if there’s no user interaction with the feature, however I don’t know if that’s true. Do you have a source that clearly spells it out? When I wire trace a modern windows system there is so much encrypted network activity that I can’t make heads or tails of what data is being sent to microsoft. And I explicitly disabled all the data collection features I could!
Remember how fun, and amazing and new everything felt 25 years ago in computing? If you let it, AI can feel pretty close to that. But yeah, let’s not repeat ALL the mistakes. Especially the mistakes of web 2.0…
Anyone else get the feeling that technology hasn’t meaningfully progressed since 2016? That was when the last meaningful advancements in video technology happened (4K, HDR, and mass-market OLED TVs), when we got the Pascal architecture of GPUs, and roughly when Windows 10 was released and Windows became “a service” that only received relatively minor updates (Windows 11 being a rebrand of what was originally a relatively minor update for Windows 10).
The only advances I can think of are ray-tracing (which quite honestly doesn’t justify the huge performance hit it impacts, but I guess it’s a thing) and the Apple M-series of SoCs (which are basically scaled-up iPad chips and are more indicative of the stagnation of Intel than anything else, PC OEMs won’t use AMD CPUs due to backroom deals, so of course the M-series was a revelation compared to Intel CPUs built on an ancient process node).
So, other than hardware-accelerated ray-tracing, the only advances during the past 9 years have been iterative. A laptop from 8-years ago is still perfectly functional today and will even run Windows 11. And amazingly, so is a gaming rig if you disable ray-tracing in your games.
Hence why Microsoft has this urge to infuse PCs with LLMs: It’s their only play to convince users that their 2017 computer needs replacement, and it’s not working so far.
kurkosdr,
“Meaningfully” is subjective since not everyone appreciates the areas that have progressed, but I would say there’s been plenty of advancements for things like raytracing and AI assistants, content generation, etc.
Take music generation, which I didn’t really “need”, but count me impressed all the same, just wow. I have serious reservations about AI killing the livelihood of musicians (along with other affected disciplines). However just from a technical standpoint the advancements have been incredible. If I were in the market for custom music and didn’t have reservations about taking away human jobs (or if the person I paid hid the fact they were using AI to generate the music), then I think AI’s already more than good enough to do the job more competitively than humans can.
I agree.
If we’re focusing on desktop operating system technology, then I’d tend to agree that progress is “meh”. Although I do benefit from ray tracing for blender and cuda.
I am sure there will be a way to disable it, much like Windows Recall can be disabled.
But I do get the feeling that Microsoft bought OpenAI to stay relevant, and now they are trying to invent a way to stick it into Windows and other flagship products (for example Visual Studio) to justify the investment.
@kurkosdr
I am pretty sure that data harvesting off the screen of every Windows user comes with its own rewards.
It is like a Product Manager overheard someone say that Microsoft Recall was the most extreme privacy and security violating tool imaginable and took it as a challenge. Microsoft Vision is worse.
When I am looking at my banking information, private photos, medical records, trade secrets, confidential communications, or private encryption keys, the last thing that I want is “a second pair of eyes”.
An aspect of the growing encroachment of AI into software development is that it appears to be significantly shortening the longevity / lifecycle of any new Apps. Apps are appearing at a greater rate than ever before, evolving faster than before, and then disappearing just a quickly. It’s a huge problem for the end user if an when any significant portion of critical private data becomes dependant on access through a specific application.
It still takes far longer to do something in Office and Windows than it takes to say what you want to do, with zero ambiguity. If CoClippy can fix that then I am all in. Minor example. I had to add a safe sender email address to Outlook. I did not have an e-mail from the sender to right click. There is no obvious way on the desktop. It’s best to go to the web version and do it in settings there. How horrible is that!
I attended a live interview with Satya Nadella in Seattle soon after he was appointed. He said, more or less, that Microsoft would be focused on productivity vs protectionism going forward. We still have no Office on Linux and everything takes longer. It is getting ridiculous. And now is his chance to make good.
The golden age for Office was when the Mac versions were better than the PC versions, around 1993. That was the last time they got the right balance between functionality and simplicity. It was a pleasure to use. Excel destroyed Lotus. Word destroyed Word Perfect. PowerPoint destroyed Harvard Graphics. Outlook destroyed AOL (maybe an exaggeration). Now we have “Microsoft destroys Skype” and of course Microsoft destroyed Nokia. And there is Teams. The franken-app nobody wants to use but everyone has to. Please destroy Teams!!