Meta Ray-Ban Hack enables facial recognition, which executives once considered

• Harvard students rigged their Meta Ray-Bans to do facial recognition, reports 404 Media.
• Meta’s smart glasses can’t do this by default, but in 2021, executives considered the idea.
• Andrew Bosworth, now Meta’s CTO, suggested it could help remember someone’s name at a party.

The idea of ​​smart glasses that can destroy people in real time – a long-standing futuristic concept – is getting closer to reality.

404 Media reported on two Harvard students who hacked their Meta Ray-Ban glasses to do just that. AnhPhu Nguyen and Caine Ardayfio created software that would scan a video stream from the smart glasses to find faces and then identify them using the Pimeyes facial recognition tool. Upon finding a facial match, the software would track other personal information such as addresses and jobs. In a demonstration video, the students showed examples of finding a stranger on the subway and identifying where that person worked.

It is important to note that the facial recognition was not performed on the glasses themselves. Meta Ray-bans can stream videos to your Instagram account or device; students entered that video into the software. All the rest—facial recognition and retrieving identifying details—was completed by a program on a laptop that ran separately from anything Meta had done.

Nguyen told Business Insider that the experiment wasn’t meant to show that Ray-Ban’s popular Metas are a problem.

“We don’t want this to be a criticism of their product at all, and we just had them on hand — this could have been done with a phone camera,” he said. “This is mainly a demonstration of what is possible today.”

Another goal of the experiment, he said, was to help spread awareness about how to protect yourself digitally against facial recognition software.

It’s not the first time the two students have played with dangerous homemade technology: they also made a flamethrower that blew Ardayfio off his feet.

Meta’s glasses cannot recognize faces by default

A Meta spokesperson told Business Insider: “To be clear, Ray-Ban Meta glasses do not have facial recognition technology. As far as we can see, these students are simply using publicly available facial recognition software on a computer that would work with the photos taken. on any camera, phone or recording device.”

The glasses are still more discreet than most recording and hands-free devices. (The Meta Ray-Bans have a small light that shines while recording. The light was easily covered with tape on the first model, but the newer version prevents people from turning it off.)

404 Media pointed out that other tech companies like Google experimented with the idea of ​​putting facial recognition in smart glasses and then scrapped the idea because it was too dangerous.

Business Insider’s Peter Kafka tested Meta’s new prototype Orion smart glasses and found they could do some really amazing things that showed huge promise for the future of AR/VR wearables. But one thing Orion does not do? Facial recognition.

It’s important to note, however, that Meta executives have thought about adding technology to its smart glasses.

Meta thought to add facial recognition

In early 2021, BuzzFeed News reported that Andrew Bosworth, now Meta’s CTO and head of Reality Labs, said at a company-wide meeting that Meta is weighing whether it has the legal ability to offer the software in its smart glasses.

“Facial recognition … might be the thorniest issue, where the benefits are so clear and the risks are so clear, and we don’t know where to balance those things,” Bosworth said in response to an employee question, according to BuzzFeed. He said one benefit could be helping you, or anyone with face blindness, remember someone’s name at a dinner party.

At the time, Bosworth followed up with a (now-deleted) tweet that said: “Facial recognition is a hugely controversial topic and for good reason, which we were talking about how we’re going to have to have a public discussion about the pros and cons.” Ray-Ban’s Meta Smart Glasses debuted a few months later in Fall 2021.

It’s been three years since Meta executives weighed the possibilities of facial recognition in smart glasses — and no such feature has yet been added. And not because it’s too technically challenging (hey, a few students can do it!).

It would appear that Meta decided that it was either too creepy for consumers or too thorny from a legal point of view. Illinois, Texas and the European Union have laws against certain uses of facial recognition technology.

Legal issues aside, consumer attitudes are changing. When the meta Ray-Bans came out, many people were intimidated by the idea of ​​camera glasses. But in recent years, glasses have become a successful product. Meta has sold more than 700,000 pairs since last October, according to market intelligence firm IDC. It’s possible that at some point in the future, facial recognition in glasses will seem like a welcome convenience rather than a privacy nightmare.