Facebook has long said that it doesnât use location data to make friend suggestions, but that doesnât mean it hasnât thought about using it.
In 2014, Facebook filed a patent application for a technique that employs smartphone data to figure out if two people might know each other. The author, an engineering manager at Facebook named Ben Chen, wrote that it was not merely possible to detect that two smartphones were in the same place at the same time, but that by comparing the accelerometer and gyroscope readings of each phone, the data could identify when people were facing each other or walking together. That way, Facebook could suggest you friend the person you were talking to at a bar last night, and not all the other people there that you chose not to talk to.
Facebook says it hasnât put this technique into practice.
âWeâre not currently using location [for People You May Know],â said a Facebook spokesperson. Facebook has previously told us that it only used location for friend recommendations one time during a brief test in 2015. But several of its patents show it thinking about using location, also recommending users friend each other, for example, if they âcheck into the social network from the same location at around the same time.â
In the course of our year-long investigation into how the social network makes its uncannily accurate friend recommendations to users, Facebook has told us many things it doesnât do, to ease fears about Facebookâs ability to spy on its users: It doesnât use proxies for location, such as wi-fi networks or IP addresses. It doesnât use profile views or face recognition or who you text with on WhatsApp. Most of Facebookâs uncanny guesswork is the result of a healthy percentage of users simply handing over their address books.
But that doesnât mean Facebook hasnât thought about employing usersâ metadata more strategically to make connections between them. Patents filed by Facebook that mention People You May Know show some ingenious methods that Facebook has devised for figuring out that seeming strangers on the network might know each other. One filed in 2015 describes a technique that would connect two people through the camera metadata associated with the photos they uploaded. It might assume two people knew each other if the images they uploaded looked like they were titled in the same series of photosâIMG_4605739.jpg and IMG_4605742, for exampleâor if lens scratches or dust were detectable in the same spots on the photos, revealing the photos were taken by the same camera.
It would result in all the people youâve sent photos to, who then uploaded them to Facebook, showing up in one anotherâs âPeople You May Know.â Itâd be a great way to meet the other people who hired your wedding photographer.
âWeâre also not analyzing images taken by the same camera to make recommendations in People You May Know,â said a Facebook spokesperson when asked about the patent. âWeâve often sought patents for technology we never implement, and patents should not be taken as an indication of future plans.â
The technological analysis in some of the patents is pretty astounding, but it could well be wishful thinking on Facebookâs part.
Vera Ranieri, an attorney at the Electronic Frontier Foundation who focuses on intellectual property, hasnât reviewed these specific patents but said generally that the U.S. Patent Office doesnât ensure that a technology actually works before granting a patent.
âA lot of patents are filed at the idea stage rather than the actuality stage,â said Ranieri by phone. âA tech company that files a patent has, hopefully, at least thought about how to do it. Youâd hope they could implement it if asked, but it doesnât mean they have done so before.â
Since being born into the world in 2004, Facebook has filed for thousands of technology patents in order to lock down its intellectual property and, like many in the field, stifle competition. In a search of those patents, we found a dozen, filed from 2010 to 2016, that were directly relevant to People You May Know, or PYMK as itâs called internally. They include techniques that Facebook could use one day to make friend suggestionsâor techniques it could sue someone else for using.
Taken in their breadth, they speak to the many sources of information Facebook could tap to learn more about us and our real-world social networks, thanks in large part to the sophisticated surveillance tools built by default into our smartphones, such as accelerometers, gyrometers, microphones, cameras, and endless, sprawling contact books.
The Facebook employees and contractors who authored the patents repeatedly explain why People You May Know is so crucial to the network: People with more friends use the network more and look at more ads. Without People You May Know, the $500 billion behemoth that is Facebook would be making less money. That may be why users arenât allowed to opt out of the feature, even when it carries risks for them.
âFor people with low friends countsâusually new to Facebookâweâve heard that the suggestions we provide help them feel more engaged, and weâre going to continue to try to make these suggestions as relevant as possible,â said the Facebook spokesperson by email. âConcerning how People You May Know works; we prioritize suggestions based on mutual friends because having friends in common is a good signal that you may want to be friends with someone on Facebook.â
Facebookâs earliest People You May Know patent was filed in 2010, two years after Facebook launched the feature. In it, employees from Facebook explain why friend suggestions are important:
âSocial networking systems value user connections because better connected users tend to use the social networking system more, thus increasing user engagement and providing a better user experience.â
In a patent filed two years later, employees on Facebookâs growth team explain why increased user engagement is so important. It leads to âa corresponding increase in, for example, advertising opportunities.â
In other words, People You May Know is crucial to Facebookâs bottom line. Thus, Facebookâs first PYMK patent was on the process of privileging friend recommendations for people who donât have very many friends. Another filing patents the act of aggressively displaying âPeople You May Knowâ to people who donât use Facebook very often.
Its second patent was for something Facebook doesnât currently let you do: sort its friend suggestions to you and rank them by hometown or number of mutual friends or their interests. (If youâre interested in actually being able to do that, try our PYMK Inspector, which will let you sort your friend suggestions by mutual friends.)
One of its patents is for figuring out who your family members are and suggesting them as friends. It says it could figure this out based on âexternal feeds, third-party databases, etc.â However, when Facebook suggested I friend a relative I didnât know I had, Facebook told me it doesnât use information from third parties or data brokers for People You May Know.
While Facebook says it often seeks patents for technology it never implements, one thing Facebook is doingâand that it has filed multiple patents on since 2012 because it works so wellâis building shadow profiles to connect users. Facebook collects all the contact information it can find for you from other usersâ address books and then associates it with your accountâthough not in a place you can see or delete. It then uses that information to connect you with other users who have those contact deets for you. In patent speak, this is âAssociating received contact information with user profiles stored by a social networking system.â
Hereâs how Facebook describes the process of figuring out everyone youâve ever met.
[U]ser profiles may include incomplete or outdated information, limiting the social networking systemâs ability to identify other social networking system users for connecting to an importing user. To more accurately identify users, the social networking system stores contact entries received from an importing user and associates a stored contact entry with a user profile including information matching information in the contact entry. Subsequently received contact entries are compared to user profiles and stored contact entries associated with the user profile to identify matching information. If information in a user profile or in a stored contact entry associated with the user profile matches a received contact entry, a user associated with the user profile is identified for establishing a connection. Associating received contact entries with user profiles supplements user profiles with received content information, allowing identification of more potential connections to users and increasing user interaction with the social networking system.
And, of course, more user interaction means more opportunities to look at ads.
As Facebook continues to grow, through app acquisitions and claiming new demographics and countries as users, it will do its best to connect those new users to its existing billion-plus members. We canât know when or if Facebook will ever actually scan digital photos for dust or tap into our phonesâ gyrometers to more fully map the relationships between all the people in the world, but we now know, thanks to the U.S. Patent Office, that Facebook at least thinks these things are possible.
With that kind of thinking happening internally at Facebook, itâs hard not to start thinking of it more as a spy service than a social network. If these techniques were put into practice, it would be an incredibly invasive level of tracking in service of suggesting you connect with people that you may not actually want Facebook to know that you know.
Contact the Special Projects Desk
This post was produced by the Special Projects Desk of Gizmodo Media. Reach our team by phone, text, Signal, or WhatsApp at (917) 999-6143, email us at tips@gizmodomedia.com, or contact us securely using SecureDrop.
