It has been a long time since I write (been very busy), so much has changed, so much has stayed the same. Case in point XR, ie, the Metaverse.

Why are we here in the 1st place?

In short: PR. Let’s elaborate. For many of us in the tech world, the advent of AR/VR (XR) is a certain fact. The only unknown variable is when. Slowly but steady, the technology is closer today, nevertheless it’s still not ready. Regardless, this reachless proximity cycle is what triggered Mark Zuckerberg to acquire -in 2014- the promising VR company: Oculus. This acquisition in itself triggered the serious dawning of the VR software industry.

You see … for many years Facebook (in turn Zuckerberg) was a cool guy. Their conference (called Connect) were highly anticipated events for the tech world, on which Facebook displayed and influence where the technology was headed. Zuckerberg became a star. As I wrote some time ago, the reason of Facebook success wasn’t the social app, it was the possibility to allow software developers to exploit the social functionality of the site. And exploit they did!

As we all know, Facebook was being used for good and bad things (like any other human endeavour). However, Zuckerberg quickly went from giving softball interviews on the media, to giving testimonies on the US Congress. Facebook is still being grilled hard. Needless to say, this is not a comfortable position to be in.

And as the company with the motto Move fast and break things, Facebook re-baptized itself as Meta, putting Facebook under the Meta umbrella and turning on the afterburner on that far ahead dream/idea of the VR world. This movement implies that now Zuckerberg can point someone to be the Facebook CEO and still deal with the current grill, so he can focus on the company’s future without being vilified.

Facebook has -at least- 8000 people worldwide developing XR. So this is Meta’s future. But it’s not the only company betting on that future, Apple also is. If Facebook kept the current reputation, the brith people need to make the Metaverse a reality, will not be interested on working for such a company, making the dream farther away.

So now that we cleared the way for the real reason of this technological push, let’s quickly evaluate the challenges.

Tech, Tech, tech, money, money, money

Technological changes are enabled by economics. The cheaper the economics of the technology, the wider and faster of its adoption. Here, we’ll have to split XR into VR and AR, as both have technological challenges, just different.

Side note: We have computing power, which is the calculus capacity of a computer. And we have power consumption, which is how much electricity it needs to run. The former one is handled by the CPU/GPU, the latter one is handled by the battery.

VR

On the VR side (the Meta’s -the company- approach) the current challenge is the quality of the graphics. Needles to say, the better the graphics the better the experience. Remember, VR objective is to isolate you from the current world and immerse you on the Metaverse. The idea then, is to provide a VR headset that has such a high quality of display, that indeed the user becomes absorbed on the experience, motivating them to spend more time on the Metaverse. Keyword: Headset. The VR device must be carried on the head of the user. Therefore, it can’t be heavy, nor it can’t become too hot. Both of these situations will make it uncomfortable, alienating their use. But to provide high quality graphics, computing power is needed, which generates heat and requires a big battery to power it. As such, until the computing power becomes small and powerful enough that heat and power is not an issue, the Metaverse can’t be practical.

In the path to achieve this goal of immersion, Sony’s next PSVR 2 it’s using one clever trick: Foveated rendering. As previously written, displaying a full interactive -real time rendering- VR experience requires significant computing power. However, the human eye focus on certain sections of the experience, not the whole one. Well, what foveated rendering does is to track the eye movement of the user to locate where the eye is focused. With this information, the eye tracker can tell the CPU/GPU which section of the whole VR environment has more attention. Which this, only that section can have a higher image quality while at the same time the rest of the VR environment is blurred (less quality, less computing power used). Consequently, the user perceives a higher quality graphics on the whole experience, while the CPU-GPU-Battery combination provides a good power-display/consumption ratio. Brilliant!

AR

Apple, Google & Microsoft approach. AR objective is to enhance your surrounding, not to absorb you from it. In the AR world graphic quality (thus computing power) is not that important. What’s important is the field of view (FOV). Current AR devices display the FOV as in the left in the below image. The ideal scenario would be as in the right. To achieve this, the projector on the glass has to be small, low cost and low consumption in power. Not an easy requirement. Such a thing could be achieved with today’s porjection technology, but it’s still expensive to offer.

Which one is closer to hit the sweet spot? I would guess AR (I’m biased though 🙂 ). Nevertheless both are here to stay. I truly wish that when they becomes a practical manner, we still appreciate and enjoy the beauties the real world offers. There’s a time and space for everything.

Image source: Chetraruc