Movements in XR (or Cross Reality)
X Reality (XR or Cross Reality): what it is and how it works
It seems so long ago since 3D, VR and AR have appeared in our everyday lives - on our smartphones, on our computers and consoles, in our cinemas - but in reality these technologies have developed in a handful of years, not without a few hitches along the way.
The beginning
It all began back in 1996 when Nintendo introduced the Virtual Boy, the world's first ever virtual-reality console. At the time, however, the technology offered to us by the gaming industry was extremely immature and - to be honest - not very successful. At the same time, the movie industry was already giving us a taste of what the world could have been like in 20 years with iconic movies such as “The Lawnmower Man” inspired by a Stephen King novel or “Johnny Mnemonic” based on William Gibson's steampunk genius work. It took some time for videogames to catch up on that level of coolness.The turning point
The real turning point was 2012 with the appearance on the market of Oculus's Rift virtual reality viewer (later acquired by Facebook) which gave new stimulus to immersive experiences, both in virtual reality and augmented reality. The immersive experiences we can enjoy today descend directly from that first technology. Since then the 3D tech has been slowly - but steadily - infiltrating our domestic life and even our work life. Many of us have bought their first 3D devices for gaming really early on or first encountered VR sets at some trade fair and then decided to buy one to use it at home or at work. With caution, we have opened ourselves to a world of work and play experiences in virtual worlds in which we are totally immersed. However, the transition to the immersive world was not as smooth as one would hope.The first issue encountered by the general public with immersive technologies was “motion sickness”, a disorder that we experience when we perceive an inconsistency between actual and expected motion. The effect is due to the fact that our real life movements are still too fast compared to the speed with which the VR headsets are able to reproduce the scenes in computer graphics.
The result is that our body perceives a certain speed that does not correspond to that perceived by the eyes - which is in fact slightly slower. Another limit that we immediately found in the use of VR headsets is the 3DoF - an acronym that stands for “3 degrees of freedom”, which is a limitation that we encounter in the simulated movement. Using the first VR headsets we could in fact only rotate the head by making only 3 movements: pitch, yaw, and roll - as shown in figure 1.
Fig. 1
This limitation was caused by the sensors the first headsets were equipped with and by the algorithms. The risk was to encounter the "drift" or drift effect: moving in space, the headset could not exactly keep track of our trajectories, producing a drift effect as in figure 2 which could eventually lead to inaccuracy of the spatial tracking.
Fig. 2
This limitation is acceptable if we are faced with 360 ° images, but it becomes frustrating if we want to move and explore entire virtual worlds. In order to walk in virtual spaces with the first VR headsets it was therefore necessary to rely on special joysticks or more futuristic solutions such as omnidirectional treadmills (such as the Virtuix Omni).
A few years and two tracking techniques later, we were finally freed from obsolete devices and we could experience the 6DoF (6 degrees of freedom) with the ability to move in space in width and height.
For the math geeks, here are some nerdy details on how this tech innovation came to be. It all starts from quaternions. These are a mathematical entity introduced by William Rowan Hamilton in 1843 as extensions of complex numbers. Quaternions provide convenient mathematical notation for representing orientations and rotations of objects in three dimensions (Figure 3). Compared to Euler angles, they have simpler functions to compose and avoid the problem of the cardan block. Compared to the rotation matrices, they are more stable numerically and perhaps more efficient.
Fig.3
Outside in & Inside out tracking
We are now ready to delve into the techniques for positioning a body in virtual reality. Let's start with "Outside in tracking": this is a technique that involves the use of sensors to be positioned in the room to keep track of our position - as shown in figure 4.
Fig. 4
The second tracking technique used in VR tech is called “Inside out tracking”: by mounting the cameras (even low resolution ones) on the headset and using image recognition algorithms, it is possible to obtain the same effect as the outside tracking (figure 5). A headset that already uses this technique is the Oculus Quest, whose cameras will soon also allow us to recognize fingers, freeing us from the use of controllers and gloves.
The same algorithms that reproduce inside out tracking allow us to create mixed reality experiences on our smartphones.
Fig. 5
I disegni in questo articolo sono stati realizzati da Sara Torre, illustratrice. Per info e contatti, clickare qui.