Dialogue on canvas with a machine: art as a metaphor for the influence of digital technology

The multitude of articles on the miraculous feats achieved through AI and the hyperbole surrounding its capacity to become truly ‘smart’ in the future - and, who knows, to one day replace us - distract us from the fact that we are already in contact with machines that influence us on a daily basis. Smartphones and tablets, to name just two, have already transformed the way in which we communicate and flirt, as well as our relationship with knowledge: with information available at our fingertips, we no longer need to remember it. 

Demonstrating the influence of human-computer interaction

The influence machines have on humans is what is of interest to Vivien Cabannes (Inria), Thomas Kerdreux (Inria) and Louis Thiry (ENS). Vivien and Thomas have been friends for a long time and are both PhD students in applied mathematics within the Sierra team, which focuses on machine learning, while Louis is in the IT department at the ENS Paris. Running alongside their PhD work, and self-funded, their project Dialogue on canvas with a machine is a metaphor for the influence digital technology has on our lives.

The people responsible for actually creating their ‘algorithm-based paintings’ are Tina&Charly (see inset), a duo formed by Tina Campana and Charly Ferrandes. For a number of years, the two artists have worked to develop ‘schematic dialogues’ based on a vocabulary, a set of symbols agreed upon for expressing emotions and concepts, in addition to their own freedom of interpretation and inspiration.

Their way of working has been standardised: after having decided on a theme, represented on a blank canvas by a black symbol, Tina begins by drawing a red line, which Charly then responds to using green. They continue to exchange in this way until they both agree that the dialogue is over and the picture is finished.

Three-way dialogue: the artist, the mathematician, the computer

When our PhD students met Tina&Charly, the artists were looking to collaborate with mathematicians in order to add a degree of rigour to their newly created language, and so the young researchers decided to offer them a third partner: a computer. Fed initially by simple drawings, before graduating to Tina&Charly’s symbol-based vocabulary, the machine became capable of joining in the discussion by making its own contributions to the picture.

Vivien, Thomas and Louis designed a computer program that would take a photo of the picture after each contribution made by Tina and Charly and analyse the lines drawn by the artists, before using a rear projection system to add its own brushstroke, which the researchers could then choose either to go over in blue or ignore. What the program does has an impact on the painters, restricting them or giving them inspiration. The same process is then repeated. The choice of a projector as opposed to a robotic arm for drawing the lines stresses that the influence of machines on our daily lives should remain suggestive.

AI and painting: a breakthrough practice?

Interactive projects between AI and painters are rare. The best-known is surely the work of the artist Sougwen Chung, who works with robots and robotic arms in order to explore similarities between interpersonal communication and human-machine communication. The work of Vivien, Thomas, Louis and Tina&Charly is in keeping with these pioneering propositions, through both its interactivity and the questions that it raises. Operating as a collective, they jointly presented their work at a creativity and design workshop at Neurips in Vancouver in December 2019, the world’s biggest machine learning conference.

By presenting their work, which combines the sensitivity of a visual artist with that of a scientist, the group had the opportunity to showcase their perception of applications in both science and art. Their aim was to present the vision of the technicians themselves, but also of citizens reflecting on a society where the way in which we interact with machines has become essential in understanding how to control them.

And this is just the beginning. Although Thomas’s PhD viva is scheduled for the summer of 2020, the young researchers are set to continue exploring various other interactive avenues in painting, including with Tina&Charly, the goal being to fine-tune their thinking on modern society. On the horizon, the use of an EEG (electroencephalography) headset in a system where a machine projects an interpretation of its reading of the painter’s brain activity onto the canvas.