Meta: a secret VR headset that could surpass Apple’s Vision Pro

It is one thing that all mixed reality headsets (and some virtual reality headsets) have in common. But this technique has not really perfected yet. This is the “passthrough”. A concept made possible thanks to a meticulous arrangement of cameras and lenses. It is thanks to this concept that headsets like the Meta Quest Pro, the Apple Vision Pro and even the Quest 2 can simulate your physical environment – to varying degrees – while they cover your real eyesight.

The “passthrough” allows the operation of augmented reality applications for interior decoration and interactive learning. Use cases that involve a certain degree of spatial awareness in the real world, and important computer processing for image simulation.

Naturally, it is not easy to get a visual experience similar to the one we have in real life. Especially if we take into account the distance between the location of the helmet cameras and that of our own “cameras”, that is to say our eyes. But Meta has a secret weapon hidden within the Display Systems Research team (DSR – Display Systems Research) of its Reality Labs laboratories which, according to it, has managed to make a great breakthrough in the field of “passthrough”.

Light Field Passthrough, inspired by insects

So here is Flamera, a mixed reality headset still in pre-production that uses the “Light Field Passthrough” to accurately reproduce your environment. Come again? By adding a grid between the lens arrays and the camera sensors. This is enough to obtain an optimal capture of the light, regardless of the movement of the eyes, and a natural recording of the distance and the movement of the objects that surround you.

Flamera pre-production unit. Meta

This opening module gives what can be described as the appearance of an insect’s eye, with holes pointing in all directions and from all angles. The helmet takes the raw sensor data captured by all these holes and reconstructs them through a reconstruction according to depth, according to Meta. You can see below a comparison between the “passthrough” mode and reality.

“Flamera’s optical design works best when the helmet is thin, which allows us to place the passthrough cameras as close to the user’s eyes as possible,” explains Grace Kuo, researcher in Meta’s DSR team.

Last month, the Flamera headset was demonstrated at Siggraph 2023, an annual conference on interactive techniques. “This was without a doubt one of the best quality augmented reality “passthroughs” I have ever seen,” said Anshel Sag, senior analyst at Moors Insights & Strategy.

Sag also pointed out the reduced field of view of the beta headset, which can be a problem for VR/AR applications that require looking from side to side.

The fact remains that Flamera could be the beginning of something important. And the list of new mixed reality headsets that will be released in the coming months, culminating in the launch of Apple’s Vision Pro at the beginning of next year, makes the technology developed by Meta’s DSR team something to keep an eye on.

Source: “ZDNet.com “