For years now, people believe that their smartphones are listening to their conversations through their microphones, all the time, even when the microphone is clearly not activated. Targeted advertising lies at the root of this conviction; when you just had a conversation with a friend about buying a pink didgeridoo and a flanel ukelele, and you then get ads for pink didgeridoos and flanel ukeleles, it makes intuitive sense to assume your phone was listening to you. How else would Google, Amazon, Facebook, or whatever, know your deepest didgeridoo desires and untapped ukelele urges?
The truth is that targeted advertising using cross-site cookies and profile building is far more effective than people think, and on top of that, people often forget what they did on their phone or laptop ten minutes ago, let alone yesterday or last week. Smartphones are not secretly listening to you, and it’s not through covert microphone activation that it knows about your musical interests.
But then.
Media conglomerate Cox Media Group has been pitching tech companies on a new targeted advertising tool that uses audio recordings culled from smart home devices. The existence of this program was revealed late last year. Now, however, 404 Media has also gotten its hands on additional details about the program through a leaked pitch deck. The contents of the deck are creepy, to say the least.
Cox’s tool is creepily called “Active Listening” and the deck claims that it works by using smart devices, which can “capture real-time intent data by listening to our conversations.” After the data is captured, advertisers can “pair this voice-data with behavioral data to target in-market consumers,” the deck says. The vague use of artificial intelligence to collect data about consumers’ online behavior is also mentioned, with the deck noting that consumers “leave a data trail based on their conversations and online behavior” and that the AI-fueled tool can collect and analyze said “behavioral and voice data from 470+ sources.”
↫ Lucas Ropek at Gizmodo
Looking at the pitch deck in question, you can argue that it’s not even referring to smartphones, and that it is incredibly vague – probably on purpose – what “active listening” and “conversations” are really referring to. It might as well be simply referring to the various conversations on unencrypted messaging platforms, directly with companies, or stuff like that. “Smart devices” is also intentionally vague, and could be anything from one of those smart fridges to your smartphone.
But you could also argue that yes, this seems to be pretty much referring to “listening to our conversations” in the most literal sense, by somehow – we have no idea how – turning on our smartphone microphones, in secret, without iOS or Android, or Apple or Google, knowing about it? It seems far-fetched, but at the same time, a lot of corporate and government programs and efforts seemed far-fetched until some whisteblower spilled the beans.
The feeling that your phones are listening to you without your consent, in secret, will never go away. Even if some irrefutable evidence came up that it isn’t possible, it’s just too plausible to be cast aside.
Do you remember when you could opt out automatic updates for apps on Android? Do you notice how Whatsapp and Facebook update themselves even without your permission? I hope no one believes that configuration does really something…
I haven’t seen this behaviour on my Android device
You. You haven’t seen that behavior. I really don’t understand what’s your interest in this site when you discredit other opinions and also that from site’s editor so often.
For years, ever since I got my first Android phone back in 2012, I definitely felt that I was being “listened to” by my phone as I would time and again see an ad on Facebook or the mobile Chrome browser for something I had spoken of with someone but never searched for or otherwise input into any device I owned, often within an hour of mentioning the subject. I always chalked it up to confirmation bias; after all, smartphones were hardly powerful enough to do that kind of thing right?
Well interestingly, not long after I deleted my Facebook and Instagram accounts about seven years ago, I realized I hadn’t seen any of that creepy targeted ad stuff, at least not in relation to in person conversations. That was when I began to wonder if Facebook and related apps were actually turning on the mic and sending snippets back to the mothership, tied to my account. After all, upon installation Facebook required access to the microphone and camera, and before Android’s much improved sandboxing, you couldn’t turn access off without disabling the app altogether. As an experiment I asked my then wife to talk to me about something real but nonsensical to us, I believe it was killer whales, while her phone was nearby. That’s not a subject we ever discussed as far as I knew, and neither of us had searched for that animal that we knew of. We talked for several minutes about how cool killer whales were, and maybe we should make plans to go to Sea World to see them. Sure enough, a few hours later she was seeing ads on Facebook on her PC for Sea World and for killer whale and dolphin plushies. It was uncanny, and if not for the absolute need for Facebook on her phone for her book review website, she would have deleted it right away.
I still think a lot of occurrences like that today are confirmation bias or simply forgetting that you searched for a similar subject on another device, but that eerie feeling of being watched has never quite gone away. Now that Microsoft insists on shoving Recall down its users’ throats, combined with the increased capabilities of LLMs and similar AI-like tech, and revelations like this article, it’s only a matter of time before the Panopticon moves from science fiction to every day life for all of us.
Thom Holwerda,
Wayne Enterprises already did it.
https://www.youtube.com/watch?v=IRELLH86Edo
On a serious note though, privilege exploits that could do this aren’t unheard of. The wider the net cast, the more likely someone will detect it. High priority targets are more likely to be bugged than the general public and they’re unlikely to know when they are bugged.. There’s opportunity for agencies flush with cash to hire hackers to find exploits before they get discovered. And once the original exploit is discovered, it may take weeks/months for manufacturers to patch it. And even then it may be too late if the payload is already in control.
https://www.cvedetails.com/product/15556/
https://www.cvedetails.com/product/19997/
Simple Fishing attacks can be effective too if users are not alerted to anything suspicious.
Alfman,
Alfred resigning on the spot was probably the best fit for his character.
Even for “good reasons” such a technology should not exist.
If I’m not mistaken for voice control, the phone must constantly listen to capture the magic words like “Hey, Google”. Whatever the phone does afterwards with the non-magic words it captured is up to Google.
r0ller,
That may be what it’s programmed to do, but if we’re allowing for a scenario where somebody gains elevated access via a privilege escalation attack (or google use their backdoor, otherwise known as system updates), then all of that programming can be changed.
I think his point was that it *is* always listening if you have those tools installed. It would have to process the voice regardless of what was said.
Underphil,
Cannot know the current versions, but there was (and probably still is) a hardware component in Google devices that would listen to “Hey Google” and only activate the rest of the system when the trigger word was matched.
When the microphone is active you’d see LEDs to indicate that.
Of course this does not prevent:
1) Change in future hardware
2) Accidentally triggering by yourself or a kid
3) Hacking of the device
The solution is of course using the hardware switch that come with most devices. At least for Nest, they are physical switches that disconnect microphones:
https://www.theverge.com/circuitbreaker/2019/8/23/20828854/google-home-mini-mute-switch-button-privacy-microphones
sukru,
Is there documentation on this? I didn’t find evidence other than generic accelerators, which we know are there but these are programmable. Is this what you are talking about or are you implying there’s a single purpose ASIC chip that can only trigger on “Hey Google” and can’t be reprogrammed? The later would seem very unusual to me, I imagine “Hey Google” can be reprogrammed and bypassed.
It’s definitely an advantage to have physical confirmation, but it’s rare these days.
Alfman,
Digging a bit.
Yes apparently, the hotword can be changed. But the trigger mechanism is still a separate hardware.
In terms of Moto X, it is a TI C55x DSP:
https://www.anandtech.com/show/7235/moto-x-review/4
(Btw, it is very sad to see Anantech go)
One main reason to have this, in addition to privacy, battery life. You cannot have the main processor on all the time listening. The separate hardware essentially works like a wake up interrupt.
This is complicated.
The regular devices you buy from Google or Meta are very unlikely to listen directly to conversations. Not only it would be found out quickly, they are subject to accredited audits for privacy.
That being said…
As someone mentioned above, 3rd party software can hack the system for elevated privileged. It might not even need that if they have microphone access, or even accelerometer or other sensors which can be used as a make shift listening device.
What about others?
TVs are listening to you:
https://www.usatoday.com/story/tech/columnist/komando/2023/01/26/your-tv-may-spying-you-but-heres-how-stop-it/11113419002/
And this is terrible. Even the “high end” TVs now come with advertisement services. And if you use a streaming device, I think there is none left without some sort of marketing.
How would the TV, media streamer, or your cheap knockoff IoT device is able to influence Google shopping?
Well..
There are data brokerages that will sell user profiles. And IP based targeting profiles. There are also third parties that pull data from credit cards, phone service providers, supermarkets and other platforms and join them by either unique ID or estimated physical address.
https://hginsights.com/glossary/ip-based-targeting
That brings back to Google.
When an ad slot is available there is a real time auction for that slot that includes the profile (possibly generated from your TV listening to you and your bank selling credit card transactions along with the supermarket store loyalty card including all your purchases):
https://www.simpletiger.com/resources/glossary/real-time-bidding
And we have full circle, even though established reputable providers will *not* listen to you, or directly buy data from providers that listen to you, they will, occasionally, use indirect profiles for the same effect,..