Picking up good vibrations from virtual reality

Improving social interactions in the virtual sphere

« As you know, when you speak, you emit vibrations”, says Anatole Lécuyer, head of the project team Hybrid at the University of Rennes Inria Centre. “These vibrations are already used to improve speech recognition for people with hearing difficulties. We thought these vibrations could be of use in the context of the EU programme GuestXR, the aim being to improve social interactions in the virtual sphere. This is something that had never been done before »

Virtual reality headsets with haptic feedback

© Inria / Justine Saint-Aubert

The team at Inria Rennes achieved a “first”: they developed a virtual reality experiment designed to study the impact of vibrations perceived by an individual when holding a conversation with avatars.

“We gave a group of people virtual reality headsets that would put them into the same immersive space as avatars”, explains Justine Saint-Aubert, a postdoctoral researcher with the project team Hybrid. “Each of them held a vibrator which was able to convert the vocal flow of the avatars into vibrations in the palm of their hand.” Specialising in the use of haptic perception for virtual reality, the young researcher based this on a small vibrator produced by the start-up Actronika, the frequency and amplitude of which will vary depending on the sound signal.

Those sending vibrations to participants 15% more convincing  

The first exercise performed using this device involved listening to two identical chatbots, with the same synthetic voice and saying the same insignificant things, ensuring uniform conditions for the experiment. The speech flow of the first avatar didn’t send any vibrotactile feedback to the participants’ hands, while the second sent vibrations that varied depending on the sound of the voice.

Twenty testers then had to fill in questionnaires aimed at evaluating their trust in the avatars as well as their conviction, leadership and social presence (the feeling that they are actually there in the room with you).

Boosting self-confidence through vibrations

And that’s not all. 20 other participants were asked to take part in a variant on the first exercise. This involved reading a neutral text to two chatbots in turn, receiving vibrotactile feedback when speaking with the first character but not when speaking to the second. Once again, the participants were asked to fill in a questionnaire that was almost identical to the one from the previous exercise.

“We were surprised once again as the results were even clearer. When they felt vibrations while reading the text, participants felt they were 28% more convincing, as though the vibrations were stimulating their self-confidence.” These highly promising results, which were covered in a paper, pave the way for further research and new uses, which the researchers will be presenting for the first time at the IEEE VR conference in Shanghai in late March 2023.

Encouraging people to be kinder to each other in the metaverse

When Justine Saint-Aubert came up with the idea for this “speech-to-touch” device, the team’s aim was to encourage people who are normally reluctant to speak to contribute at meetings (whether virtual or physical, business or personal), boosting their confidence and enhancing their ability to persuade people. They would do this by synchronising vibrations with people’s contributions. “We are keen to ensure our research isn't used inappropriately, whether it’s to further an already authoritarian line or to stigmatise certain points of view”, explains Anatole Lécuyer.

With this in mind, technology will need to be developed with the capacity to activate or deactivate the vibrating devices on request. The aim is to further expand on the team's results, by testing how sensitive individuals are to these vibrations or by measuring the impact of different tones. “The impact of vibration on persuasion that we have demonstrated opens up a huge field of research to explore. This is just the start, but the possibilities are endless”, says Anatole Lécuyer.