Creating populated immersive experiences

“If you’re immersed in a virtual street, you expect to see people walking about, going shopping or talking to each other. If you are immersed in a train station environment, then it will be passengers waiting to board trains and friends come to see them off that you'll expect, as well as people who have just come to buy cigarettes from the tobacconist’s. Our team studies the way in which these specific environments can be populated using virtual alter egos which behave like people in real life”, explains researcher Julien Pettré.

To develop a better understanding of human activity, whether in a train station or on a street, the researchers employ immersion in a simulation of these environments. There are two objectives here. The first relates to technology, the aim being to identify ways and means of creating autonomous virtual humans and having them walk around in a simulation in the presence of a person wearing a VR headset. The second concerns making scientific use of the observations from this immersive simulation. “We have questions which this simulator will provide answers to.”

The researchers all study different subjects. Julien Pettré, for instance, is focused on crowd behaviour. “You wouldn’t think this had anything to do with virtual reality. For me, it’s just a tool, a method that I use in an attempt to understand how people move around in crowds, how people interact with their neighbours, the distances people try to keep, etc."

There are two more advantages to using virtual reality to carry out experiments: cost and speed. “For a standard experiment on group behaviour, you need 60 participants, four months of preparation, and then a year spent putting the data together. It’s half a PhD's worth of work.” Then there’s the cost of a motion capture system, which can be the best part of a million euros.

“This would slow us down considerably. With virtual reality, the aim is to accelerate the cycle.” For an experiment at an underground station, all that’s needed is one person wearing a VR headset walking around among their virtual fellow passengers. These characters will just be replicas of the person, recorded during their previous visits to the station. “One day is enough to acquire this data, which is already processed, meaning it can be analysed right away.” Are these experiments faithful to reality? “Bias can crop up, but we know how to measure it in order to be able to disregard it.”

Data sets ten times more extensive This method can also provide additional information. The aim of one of the experiments is to follow the gaze of a pedestrian in the street. In a real situation, you could capture the movement of their eyes and film the scene viewed by the individual. On the video, a marker would then indicate what they are looking at. But measuring the movements of a person's body is significantly harder. In the virtual world, this problem doesn’t exist. “We have a complete record of their movements and the movements of the other pedestrians. This gives us a data set that is ten times more extensive than would be the case for real life. This makes analysis much easier and much more detailed.”

Linked to this method, the future team is exploring three areas of research.

• The first of these relates to animating virtual humans. “Characters perform a movement, which lets us work out what they're doing. You have to find a way of linking individual movements to activities.” But these characters aren't restricted to walking. “One question which might be of interest to us is finding out how someone would react if the person facing them seemed aggressive. Would we stare back at them? Or would we keep our distance? From a technological standpoint, the aim is to animate virtual humans with the capacity to express themselves and convey their state of mind. That’s not easy...”
• The second area of focus relates to the collective aspect. The mix of human activities has to be consistent with the environment they find themselves in. “In our virtual street, for example, we have people running into each other, people talking. The characters have to be choreographed in such a way as to ensure the scene matches reality. This is crowd simulation. The understanding of collective behaviour which we develop through our experiments then feeds into the techniques we use to carry them out.”
• The third and final area of focus relates to the immersive experience itself, as well as how it is evaluated and used for scientific purposes. This tool could be of particular use in the social sciences.

The researchers are set to demonstrate three aspects of their work at Laval Virtual. “My colleague Marc Christie studies the concept of the virtual studio used in cinema, devising new ways of interacting for designing animations. This research is being conducted as part of the EU project INVICTUS.

Then, in the context of crowd immersion, we will be presenting new technology which we call field of interaction, developed as part of the EU project PRESENT. This provides a much quicker way of creating new behaviours for our virtual humans. Previously, for each interaction, we had to come up with a model outlining how each character behaved in relation to others. This took a lot of development time. This new method can be used to generate behaviours from a basic sketch. We can, for instance, arrange a game of hide-and-seek between two characters, where one is looking to hide behind an obstacle.”

The third demonstration will relate to animating characters plus techniques developed as part of the EU project CLIPE. On the screen: a scene in a bar. “We record all of the behaviours you might find in a bar: the waiter, customers leaning against the bar, others sitting at tables, etc. We use virtual reality to create convincing and appropriate animations, which is really quick and really easy. No animation skills are required. We then immerse someone in this scene.” This person might get an extra surprise when they notice characters turning towards them. “We simulate visual attention, tracking the person’s gaze depending on what’s going on around them. This gives the impression of characters being aware of the presence of the newcomer.” This further reinforces the feeling of immersion. “These three demonstrations will give visitors an opportunity to see what we’re capable of doing in terms of content: animating virtual humans, both individually and collectively, and providing a new immersive experience with scientific applications. It’s important to remember that we are not just content creators. We use virtual reality to carry out our research.”