on
Is Facebook Listening?
Disclaimer: this article isn’t in any way a defense of Facebook or its data collection methods, which I personally believe are immoral.
A few weeks ago, when I mentioned to a friend that I was planning on getting some new shoes, he joked that I should buy a pair of All Birds, the footwear of choice for stereotypical San Franciscans working in tech (like the “Midtown Uniform” for finance). This was a verbal, in-person conversation, but somehow, Facebook had heard about my “interest” in All Birds and advertised them to me across its various platforms for the next several days.
It’s a common phenomenon: expressing verbal interest in some product and being advertised that same product not long after, and the ubiquity of this phenomenon has given rise to and fueled the conspiracy theory that Facebook (and possibly other advertisers) is listening in on people’s conversations.
So, is Facebook listening in on your conversations?
No.
As you may recall, earlier in 2018, Mark Zuckerberg (Facebook’s cofounder and most public figure) had to testify in front of the Senate and the House about Facebook’s various alleged invasions of privacy and the Cambridge Analytica scandal, including if the Facebook app mines audio from users’ phones. When Senator Gary Peters questioned Zuckerberg about that allegation, if Facebook listens to users to collect information, Zuckerberg affirmed that the company did not.
But, if that testimony seems unreliable to you, consider that 1) mining audio from every user would be incredibly impractical and 2) Facebook doesn’t need to listen to your conversations because it already knows you so well through arguably more invasive surveillance methods.
The Inefficiency of Mining Audio
Facebook does not mine audio. Technical and casual investigations alike have confirmed as much.
And, to do so would be very impractical from a technical standpoint as well. Facebook would need to record every piece of audio that your microphone picks up while your phone’s on—the equivalent of a constant VoIP (Voice over Internet Protocol) call, which would run up about 130 MBs per day, per user. Across all of Facebook’s 150 million daily active American users, that’s a lot of data: an approximate 20 petabytes daily. To put that in perspective, Facebook’s entire data storage is estimated to be some 300 petabytes, from all its years of data collection. That’s about the same as 15 days’ worth of user recordings—barely two weeks (let alone several years). In fact, Facebook has a daily ingestion rate of about 600 terabytes, meaning it collects about 600 terabytes of user data per day. A day’s worth of user recordings (20 petabytes) is about 33 times that normal daily ingestion rate. And, that’s only on Facebook’s end. If every phone was to do this, the snooping would be almost immediately detectable, both because data uploads of that scale would take up very noticeable chunks of users’ data plans and because those same phones would suddenly slow down and become much less functional.
There’s always the Amazon Echo route as well: triggering the recordings when users say some keywords (e.g. “Alexa”), but this only works for the Amazon Echo because it was built to listen for these keywords, and there are only a few of these keywords to begin with. For Facebook to effectively do this, the app would need to listen to millions of possible keywords, and users’ phones would need to constantly be running the speech-to-text translation algorithms—the same translation algorithms that extremely powerful cloud servers sometimes struggle to run. Such an approach would be unreasonably taxing on phones and lead to untenable performance degradation.
The Amazon Echo line.
So, why did Facebook show me ads for those All Birds?
Multi-Dimensional Profiles
Facebook knows you (and me) so well that it doesn’t need to listen in. By essentially tracking everything users do online, and even tracking users offline in some cases, Facebook can algorithmically build “multi-dimensional” profiles of its users to deliver highly-targeted (or “relevant” in Facebook-speak) ads. These multi-dimensional profiles are essentially Facebook’s understanding of you as a person, created by analyzing your interests, behaviors, environments—in short, anything that may have had a hand in making you the person you are today (or at some moment in time). Here’s an excerpt of Facebook’s capabilities, compiled by the Electronic Frontier Foundation (an international non-profit digital rights group):
- tracks you through Like buttons across the web, whether or not you are logged in or even have a Facebook account
- maintains shadow profiles on people who don’t use Facebook
- logs Android users’ calls and texts
- absorbs unique phone identifiers through in-app advertising to associate your identity across the different devices you use
- tracks your location and serves ads based on where you are, where you live, and where you work
- tracks your in-store purchases to link the ads you see online with the purchases you make offline
- watches the things you start writing but don’t post to track your self-censorship
- linked purchases to Messenger accounts to allow sellers to send confirmation messages without affirmative user permission
- bought and advertised a VPN to track what users are doing on other apps and crush competition
- manipulated your Newsfeed to see if it can make you sad or happy
- files patents for emerging tracking technology, like tracking your location through the dust on your phone camera, for potential future use
But, those surveillance methods may seem a little abstract. What can Facebook do, realistically, with all that data? Let’s say some user Mark catches the flu. Mark goes to his local supermarket and buys tissues with his loyalty card. His purchase history can be tied to his loyalty card, which a third-party data collector consolidates with other users’ purchase histories and sells to a pharmaceutical company which can then use that data and Facebook tools to connect that offline data to online Facebook accounts and advertise to those Facebook accounts. So, Mark sees a targeted ad for flu medicine despite his illness not affecting his online habits at all. The Facebook app can also track location by considering what Wi-Fi networks are around you or even your direct permission to use your location. So, the pharmaceutical company can market its product to users in areas most affected by the flu, and if Mark is in one of those areas or even just a few blocks away from an affected areas, a marketer could make an educated guess and advertise to him since he’s more susceptible to illness than users in other areas. And of course, Facebook can track Mark’s online history through Pixel, a Facebook product that’s installed on thousands upon thousands of websites and gives Facebook and marketers access to his browsing habits and history. This method is likely the most well-known out of all the miscellaneous surveillance methods Facebook employs to build a multi-dimensional profile of Mark. And, by combining all this information it has about Mark, it can advertise pharmaceutical products to Mark with a high degree of accuracy.
In my example, Facebook could’ve advertised All Birds to me in any number of ways. It could’ve seen that I was looking for new shoes when I browsed the Adidas online store or found the physical location of the Michigan Avenue’s Adidas store on Google Maps. Knowing that I’m a Bay Area native and a computer science major—All Birds’ target audience—Facebook could make a very educated guess and show me ads for All Birds, despite me never expressing interest in them online.
Conclusion
To summarize, Facebook does not listen to your conversations and it doesn’t even need to. Through extremely sophisticated surveillance and data analysis, it can build a multi-dimensional profile of you and advertise products its algorithms (reliably) believe are relevant to you.