I have set myself two objectives under this project, which is called VHIA (Vision and Hearing in Action). Firstly, to offer a mathematical representation of audio-visual objects, in this instance the face of a person who is speaking. There is much to be done because specialists in images and sound are not used to working together. The data concerned, it is true, are of very different types. For example, the propagation of an acoustic wave and the reflection of light rays on objects obey different laws of physics. Creating one model to unify vision and hearing is therefore a real challenge.
Secondly, we are going to work such that the perception and analysis of audio and visual sensory signals cause a reaction in a machine. To achieve this, we are going to make use of the humanoid robot NAO , from Aldebaran Robotics, modifying the original sensors on it. It should then be capable of spontaneously interacting with a group of people, moving towards them if it sees that it has attracted their attention, and communicating verbally and with gestures. The ultimate aim of this project is to lay the foundations for a robot with well developed social behaviour, able to understand a situation and communicate with the people around it.
My background is that of a specialist in artificial vision, a fascinating area at the very edge of artificial and experimental science. Defining a process that can be used to extract useful information from an image is a stimulating challenge. I gradually wanted to gain exposure to perception more broadly, as I saw that there is great potential in researching a merger between audio and visual. And then, five years ago, I was able to meet the founders of Aldebaran Robotics, with whom I worked as part of the European Humavips social robot project, which ended in March 2013. The VHIA project is essentially a continuation of that initiative – there is much still to be done in the area of interaction between people and robots. We are at the dawning of a new era, because I am convinced that these robots will feature much more in our society, in assistance roles, and indeed to encourage people to relearn how to communicate with each other.
I was very pleasantly surprised because it was the third time that I had submitted an application for this grant. Most researchers generally give up after two rejections, but I was convinced, rightly, that I still had room to move forward. Bouncing back after disappointment is part of our job, because the competition is such that we cannot always win. It is also recognition for the work that I have accomplished in recent years.
I am glad that a project like this one, which is partly like something out of a dream, was able to receive such a prestigious distinction. It will enable me to plan long term and focus on the scientific aspects, without having to hunt for funding. It is a unique opportunity for me and my team.
