“In-person” human-computer interaction at the Collège de France

Getting the most out of computers

Rethinking perspectives to empower users. This is the challenge facing research in human-computer interaction (HCI), according to Wendy Mackay. The researcher is head of the Ex-situ project team at the Inria Saclay Centre and has devoted her career to the subject. She will also be dedicating a chair to it at the Collège de France from 24 February to 24 May this year.

“Although our work is respected and recognised worldwide, we still need to explain what this field of research is to the general public and future students”, she says. “The aim of this chair is to present human-computer interaction and show that it’s not ergonomics, that we don’t just design pretty interfaces but are fundamentally changing the way humans use computers, we create technology”.

Fundamental research that is easily applied

As proof, in January this year recent PhD graduate from Wendy Mackay’s team Viktor Gustafsson will create a start-up dedicated to a new perspective on computer games through the Inria Startup Studio. “On the one hand, designers lack content for their games”, says Wendy Mackay. “On the other hand, the gamers themselves are very creative, so we had the idea of transforming the user experience to allow them to contribute to the narrative, interact with the game and extend this interaction to other players!”

Another example is “swipe typing”, which allows users to type on their smartphone without taking their finger off the screen. Algorithms are used to predict the words the user wants to write without them having to tap each letter. “Those who design these algorithms aim to remove what they call “noise” and focus on the word alone”, explains Wendy Mackay. “But our research aims to incorporate this noise as user expressiveness. We factor it in to allow users to change things like the font style or colour of their text simply through their gestures”.

HCI and AI: multiple challenges ahead

This research illustrates the new challenges raised for HCI by the development of artificial intelligence (AI). “Most of the time, AI researchers focus on the performance of their algorithms”, the scientist continues. “But we turn the question around to consider things from the human perspective: how can AI be used to augment human capabilities?”

Wendy Mackay gives the example of an algorithm developed to guide sailors to port. It was initially rejected by professionals who felt it reduced their decision-making ability. A second version was proposed where the sailor indicated to the tool which route they wished to take and then received suggestions from the software. It was accepted because, this time, the sailor stayed in control! “It is important to work with actual users to assess the long-term impact on how they do things”, says Wendy Mackay.

From psychology to digital technology

The researcher has applied the same approach throughout her career, which began in an unusual way. After starting out in experimental psychology, she had to use a minicomputer during her PhD to design automated experiments on monkeys. “I wanted to know how the device worked and especially how and in what way it could help me”, she recalls. So she switched from psychology to digital technology, obtaining her PhD on innovation management at MIT (Massachusetts Institute of Technology). “I had understood how humans learn to use a computer”, she says. “I wanted to continue exploring the creation of interactive systems that allow the average user to benefit from computer science”.

In 1984, she directed a research group in an American company called Digital Equipment Corporation, a pioneer in the computer industry which developed the first software allowing a computer to explain to the user by itself how to use it in a video. “When I discovered the video hardware that had been created by another researcher, I thought it had huge potential! It took a lot of time and energy to convince the management though”. In the end, the innovation was adopted and was the start of a long series of projects.

Wendy Mackay then worked at Xerox PARC (now Palo Alto Research Center) on graphic interfaces to allow computers to be used by non-specialists of digital technology as well as interactive pens and paper. These devices are not only designed to detect what has been written and convert it into digital text, but also offer multiple possibilities for interaction such as recording and playing back a sound or calculating; all without having to use buttons or menus but simply using the user's natural movements! It was one of the applications of this technology (to adapt augmented paper for use by air traffic controllers) that brought Wendy Mackay to France.

Eight classes to learn more

The researcher stayed in France and joined Inria in 2000. “There was a strong focus on mathematics in France at the time and this approach was in Inria’s roots. But that's not what I do”, she says. “I was very lucky: the Institute invested in the subject, funded PhDs and research, and now we are a world leader in HCI!” 

This explains why the Collège de France decided to hold a new chair on “Interacting with computers". On 24 February, Wendy Mackay will give an inaugural lecture on “Rethinking our interaction with the digital world”. Eight hour-long classes, each followed by a talk by a specialist on the subject covered, will then be held until May. A closing two-day symposium will take place at the end.

“Computers are very powerful tools that can be used for good or bad”, says the researcher. “The aim of HCI research is to find out how to do things better for humans. It’s a fascinating subject, thanks in particular to the contact it offers with a wide range of users and the constant advance of technology”. Wendy Mackay hopes to share her enthusiasm with the general public at the Collège de France in February… See you there!

More details about Wendy Mackay's research

• Créer des partenariats Humain-Machine, Wendy Mackay, Inria, 20/04/2017.

• Les interfaces homme-machine, Wendy Mackay, Le blob, l’extra-média (Universcience), 7/02/2018

• Human-Computer Partnerships, Wendy Mackay - GOTO 2018, 27/03/2019 (in English)