How does virtual reality work?

From a strictly technical point of view, virtual reality is a simple concept, combining hardware (helmets, screens, sensors...), and synthetic environments generated by computer. This technology allows to immerse users in simulated environments.

Virtual reality helmets are machines designed to allow users to experience real sensorimotor and cognitive activities in a synthetic 3D environment, which is purely virtual. Increasingly light and easy to wear, they display an image calculated according to the user's movements, thus promoting a very good immersion. Other equipment makes the experience even more realistic, such as haptic systems, which provide the sensation of touch and allow the user to feel objects as if they were really there. "If I'm in a virtual reality experience and I reach for an object, my hand will fall through, which may break the illusion. Haptic systems will give me the impression of really touching the object, of manipulating it, and that changes everything," says Martin Hachet, head of the Potioc project team, which designs, develops and evaluates new approaches that exploit multimodal interaction in order to promote a stimulating user experience.

But to be effective, that is to say to succeed in immersing the user in the universe in front of which it is, virtual reality is not only a question of good material. "What is important is to know what is behind all this technology. It's not enough to have a good headset and a good computer, you also have to know how to script the experience, how to best evaluate the user's interaction with his environment... All this has to be worked on," explains Martin Hachet.

Living beings base their perception of the world on the rules developed by their own experience. It is on this way of perceiving things and interacting with the world, shared by humans, that virtual reality designers base themselves to create environments that seem as credible as the world around us. The technology is based on several important concepts and techniques.

• Stereoscopic vision

The first component of a successful virtual reality experience is stereoscopic vision. Virtual reality headsets are based on our physiology, presenting images that are slightly shifted from one eye to the other in order to reproduce binocular vision (in reality, our eyes capture two different images that our brain combines to create a single three-dimensional image).

• Motion tracking

Movement is also one of the most important criteria for a convincing virtual reality experience. The environment must be able to adapt to the position of the user's body as it moves forward, turns, or moves backward.

For this, virtual reality today uses motion sensors, which detect the user's actions and adjust the view on the screen in real time accordingly. The helmets, for their part, integrate a head movement tracking system capable of recovering information on the user's orientation in space so that the decor adapts according to the user's position.

This concept is based on an important fact: the computer's ability to react in almost real time to the user's actions. This reaction time, which must be at most a few milliseconds, is of the utmost importance to avoid "cybermalesis", a feeling of nausea, fatigue, disorientation or headaches, caused, among other things, by a too high processing time between the user's movement and the computer's reaction.

• The field of view

A third important concern for virtual reality developers is to imitate the field of vision of the users, or at least to get as close as possible to it. The average human being can see the environment around him/her over an area of about 200 to 220 degrees on the horizontal axis. Where the vision of the left and right eyes overlap, there is a 114-degree binocular stereoscopic field of view, within which we can actually see in 3D.

To date, no affordable consumer headset can yet offer the perfect imitation of the full natural field of view. Some come close, like the StarVR (210 degrees), but the most common headsets on the market currently offer a field of view of about 110 degrees.

• The concept of presence and the notion of interaction

Last but not least is the concept of "presence", which defines the user's impression of being physically in the virtual environment, rather than in the real environment.

Several techniques are used today to promote this feeling of presence, and allow the user to fully immerse himself in the virtual reality experience. The quality of the feeling of presence can be assessed through subjective questionnaires, or through objective measurements based on physiological data: faster heart rate, sweaty palms and faster breathing, for example, in certain situations.

"When someone plays a car driving video game, the experience is considered successful when he leans back in his couch in the curves, because he anticipated them," says Franck Multon, head of the MimeTIC project team, in an article dedicated to virtual reality in the field of sports.

"The notion of virtual embodiment, which allows to appropriate one's virtual body, one's avatar, is also not to be neglected", adds Anatole Lécuyer. It is therefore of increasing interest to technology specialists, who are working to facilitate the user's embodiment before the virtual reality experience. The Avatar Challenge, founded in 2018, is working in this direction, aiming to design better embodied, more interactive and more social avatars.

Six Fingers est une plateforme expérimentale de réalité virtuelle sur l’incarnation virtuelle (virtual embodiement) et l’avatar, développée par les équipes HYBRID et MIMETIC. © Inria / Photo J.C. Moschetti

Finally, the notion of "interaction" is essential in a virtual reality experience, and is largely what differentiates it from the viewing of a 360-degree video, during which the spectators are passive.

As advanced as it already seems to us, virtual reality will nevertheless continue to improve widely and rapidly over the next few years, to make the user experience ever more immersive and interactive. Devices, still expensive, will become more democratic, and technologies, such as screen resolution, will become more refined to further enhance the user experience. "In the future, designers will be able to take more into account the physiological and psychological aspects of virtual reality. We will have to understand and characterize them, especially in fragile people, to have real recommendations on their use. There are also many ethical, ecological and data protection issues to be dealt with in this field, and there is still work to be done before the technology and everything that goes with it reaches maturity," concludes Anatole Lécuyer.