Talos, the latest flagship of humanoid robotics

A robot avatar controlled by teleoperation

The team uses Talos to develop the technology for “immersive teleoperation”: the robot is not left to its own devices, but is designed to be used as an avatar sent into dangerous situations, controlled and operated remotely by a human. Talos mimics the movements of an operator equipped with a virtual reality helmet and a motion capture suit, with man and machine moving in unison. There are currently many machines capable of being remotely operated, including robotic arms. But the immersive teleoperation we see with Talos is on another level, using the full body of the robot and featuring a humanoid robot body. “Talos doesn’t just imitate its operator - it adapts its movements to its own dynamic, its strength and its weight, imitating the human without falling or making a misstep.” The robot is housed at the Inria Nancy centre by Jean-Baptiste Mouret, a director of research who specialises in machine learning applied to robotics, working in tandem with Serena Ivaldi, an Inria researcher who supplies her skills in humanoid robotics and human-robot interaction. Talos is well-supported.

The LARSEN team at the controls of robotics

Both of these scientists came to the Inria Nancy centre in 2014 to take part in setting up the LARSEN project team. With around thirty or so members, this team carries out research at the crossroads between machine learning and robotics, imagining robots capable of tackling unknown situations, learning from their errors and interacting with other robots or humans. Serena Ivaldi and Jean-Baptiste Mouret are responsible for overseeing the team's research into humanoid robotics. With Talos, the two researchers are able to draw on solid experience acquired over a number of years of research with Inria and their previous institutions. Examples include the ERC project Resibots, which involved a six-legged robot capable of recovering from damage (after losing one of its parts), and Andy, an H2020 project focused on predicting and anticipating human movements in order to improve interaction with robots. “The challenge for humanoid robots is to synchronise their motors in order for their whole bodies to be able to move. The software we are working on is a whole-body controller through which the body can be thought of as a whole, using the left arm for balance when moving the right arm, for example”, explains Jean-Baptiste Mouret. When you break down a movement as complex as walking, you get an idea of the challenges facing humanoid robotics. Talos is designed to study how to overcome a number of obstacles in this field of research.

Talos: a robot that looks like a human beings

At the research centre, Talos is programmed to follow the movements of its operator. To prevent it from falling, which could damage its costly machinery, it currently operates suspended from a harness. The aim of these experiments is to answer three research questions. First of all, it must be capable of adapting to unpredictability: should the Wi-Fi signal cut out temporarily or be delayed, Talos must be able to predict the rest of a movement in order to continue it independently, “guessing” what the operator was intending to do or making up for missing commands. The second obstacle involves using the same type of prediction to adapt the robot’s movements, taking the specific dynamics of its metal body into account. Lastly, the robot must also learn how to continue working after sustaining damage to one of its components, such as a leg. In short, it must be able to limp as well as walk. Meeting these objectives requires solid models of equations in order to translate the robot's dynamic into mathematics and machine learning algorithms so that it learns from its experience: “Talos is able to process and solve internal problems on its own. Between 500 and 1,000 times a second, it will find the solution to problems with multiple objectives (following the correct trajectory) and multiple constraints (kinetics, angular limits, etc.). If it has to lift its foot, Talos will calculate the optimum position for all of the joints of its 32 motors in order to identify the optimum movement. It doesn’t have to be explained this by the operator.” Jean-Baptiste Mouret and Serena Ivaldi's work involves studying these algorithms through which the robot is able to calculate movements, adapt them and anticipate future commands.

Opensource software

The aim of this research undertaken at the Inria Nancy - Grand Est research centre is the furthering of knowledge. Further south, in Toulouse, the Gepetto project team at the LAAS (CNRS), which is headed up by Olivier Stasse, has also been working with Talos. Jean-Baptiste Mouret and Serena Ivaldi are currently using part of the code they developed to calculate Talos’ movements and save themselves precious time. Because the software developed in Nancy is opensource, the researchers in Toulouse will soon be able to utilise this research. In keeping with Inria policy, this is managed by the Software Heritage project, the aim of which is to make the source code for all available software accessible to all in order to build a vital archive for IT research. Once shared, this research will benefit the entire humanoid robotics community. It’s not a particularly big one: only five or so teams are currently exploring this subject in France. Within it, Inria and LARSEN have set themselves apart through their capacity to bridge the gap between machine learning and robotics: “Few teams put as much learning into their robots”, explains Jean-Baptiste Mouret. “With Talos, Inria is one of the few teams in France to be exploring wholel-body teleoperation in humanoid robotics and putting human movement front and centre in a project involving a robot avatar.”