Facebook patents reveal how it plans to profit from the metaverse

According to an analysis of dozens of patents recently granted to Facebook’s parent company, pupil movements, body poses and frowns are among the flickers of human expression that Meta wants to harvest in building its metaverse.

Chief executive Mark Zuckerberg has pledged to spend $10 billion a year over the next decade on the nebulous and much-hyped concept of an immersive virtual world filled with avatars. Rivals such as Apple and Microsoft are also pursuing similar goals that Big Tech executives describe as part of the next evolution of the internet.

The Financial Times has reviewed hundreds of applications to the US Patent and Trademark Office, many of which were granted this month. They reveal that Meta has patented several technologies that use users’ biometric data in order to help power what the user sees and ensure their digital avatars are realistically animated.

But the patents also indicate how the Silicon Valley group intends to capitalize on its virtual world, with hyper-targeted advertising and sponsored content that mirrors its $85 billion-a-year advertising-based business model. .

This includes “virtual store” offerings where users can purchase digital goods or items that match real-world goods that have been sponsored by brands.

“For us, the Metaverse business model is all about commerce,” Nick Clegg, Meta’s head of global affairs, told the FT in a recent interview. “Clearly ads play a role in this.”

The patents don’t mean Meta will definitely build the technology, but they do offer the clearest indication yet of how the company aims to make its immersive world a reality.

Meta patent filing showing a “handheld magnetic sensor system”. Sketch gives the example of a soldier in sword and armor appearing in a virtual world © Meta patent

Some of the patents relate to eye and face tracking technology, usually collected in a headset via tiny cameras or sensors, which can be used to enhance a user’s virtual or augmented reality experience. For example, a person will see brighter graphics where their gaze falls, or make sure their avatar reflects what they do in real life.

A Meta patent, granted Jan. 4, shows a system for tracking a user’s facial expressions through a headset that will “then adapt media content” based on those responses.

There is a “wearable magnetic sensor system” to place around a torso for “body pose tracking”. The patent includes sketches of a user wearing the device but appearing in virtual reality as a soldier with a sword and armor.

Video: Nick Clegg’s first interview in the Metaverse

Another patent proposes an “avatar customization engine” capable of creating three-dimensional avatars from a user’s photos, using tools including a so-called skin replicator.

“Meta aims to be able to simulate you down to every pore of your skin, every strand of hair, every micro-movement,” said Noelle Martin, a legal reformer who has spent more than a year researching surveillance ambitions. Meta Human with the University of Western Australia. .

“The goal is to create 3D replicas of people, places and things, so hyper-realistic and tactile that they cannot be distinguished from what is real, and then to mediate any range of services. . . in truth, they are undertaking a global human cloning program.

Patent application meta image showing a

Image from Meta patent application showing an “avatar customization engine” capable of creating 3D avatars based on a user’s photos using tools such as a skin replicator © Meta patent application

The project has enabled the company, which has recently been dogged by other moderation and privacy scandals, to lure engineers from rivals such as Microsoft amid a fierce battle for talent between the biggest technology companies in the world.

Since changing its name from Facebook to Meta in late October as part of a rebranding, the company’s stock price has risen about 5% to $329.21.

Critics remain skeptical of the vision, suggesting the effort is a distraction from recent scrutiny after whistleblower Frances Haugen last year publicly accused the company of prioritizing hate over ‘benefit.

“What are they going to do with more data and how are they going to make sure it’s secure?” said Celia Hodent, former director of user experience at Epic Games who now works as a freelance consultant.

Some patents appear to be aimed at helping Meta in its ambitions to find new revenue streams amid concerns about declining interest from younger users in its main social media products such as Facebook.

Zuckerberg said the company plans to keep prices for its headsets low, but instead pull revenue into its metaverse from advertising and supporting sales of digital goods and services in its virtual world.

Patent explores how to present users with personalized augmented reality ads based on age, gender, interests and “how users interact with a social media platform”, including their likes and comments .

Another seeks to allow third parties to “sponsor the appearance of an item” in a virtual store that mirrors the layout of a retail store, through a bidding process similar to the company’s existing advertising auction process.

The patents indicate how Meta could deliver ads in its immersive world that are even more personalized than is possible in its existing web products.

Research shows that gaze direction and pupil activity can implicitly hold information about a user’s interests and emotional state, for example, if a user’s eyes linger on an image, this can indicate that he likes her.

“Obviously you could do something similar [to existing ad targeting systems] in the metaverse – where you’re not selling eye tracking data to advertisers, but to understand whether people are interacting with an ad or not, you need to be able to use the data to find out,” Clegg said.

Brittan Heller, Technology Lawyer at Foley Hoag, said: “My nightmare scenario is that targeted advertising based on our involuntary biological responses to stimuli will start showing up in the metaverse. . . most people don’t realize how valuable this could be. At present, there is no legal constraint on this subject.

Meta said, “While we do not comment on the specific coverage of our patents or our reasons for filing them, it is important to note that our patents do not necessarily cover the technology used in our products and services.

Additional reporting by Henry Mance in London

About Norman Griggs

Check Also

Use these quick tricks to ripen bananas

When you see a bunch of unripe bananas on the table, there are usually only …