Is predicting thoughts an achievable challenge?

Towards a new way of assessing brain activity

“Brain activity is currently observed via functional MRI , which enables us to identify an increase in oxygen consumption in different areas of the brain when a person is performing a particular task, such as reading or moving their fingers,” explains Demian Wassermann, Senior Researcher in the Mind project team at Inria (a joint undertaking involving the Inria Saclay centre and the CEA). “But these experiments are subject to biases linked to their design or to the expectations of the experimenter, who runs the risk of focusing only on what they are actually seeking to observe. Our aim was therefore to establish an objective measurement of brain activity, which would enable us to find out whether the fMRI-based studies carried out to date were relevant and whether any cognitive functions were still to be revealed.” 

To this end, an international research project has been launched. In France, it brings together the Neurofunctional Imaging Group (GIN – CNRS/CEA/University of Bordeaux) and the Mind project team, which specialises in models and inferences for neuroimaging data. 

The first human cognition “morphospace”

How did the researchers proceed? First of all, they designed artificial intelligence algorithms capable of extracting 506 maps linking cerebral activity observed via fMRI with cognitive functions from an imaging database (Neurosynth), which were then compiled to create a “morphospace” (a 3D graphical representation). This one, which looks surprisingly like a neuron, has several branches, each representing a major cognitive function: memory, language, movement, and so on, and which contain the various related domains. For example, the “emotions” branch includes fear, joy, sadness, etc. As for the morphospace centre, it groups together the more transversal and still poorly defined functions, such as consciousness. 

The team then proceeds with the validation of this organisational structure by projecting more than 800 maps from another imaging database into the morphospace (Neuroquery). Outcome: the researchers have found that the activities located by these maps closely correspond to the zones defined by the morphospace. “This verification has shown that not only was our morphospace well designed, but that the fMRI studies were equally well designed, since their images fell within the correct domains of the morphospace,” notes Valentina Pacella, a postdoctoral researcher at the GIN at the time of the study and now a researcher at the Cognitive Neuroscience Centre at the University School of Advanced Studies in Pavia (Italy).

Show me your brain and I'll tell you what you're thinking

This initial conclusion validates the first stage of the project... and much more, because the successful development of this new way of understanding and visualising cognition offers a major opportunity: being able to guess, from looking at an image of the brain, what type of cerebral activity is occurring. To prove this, the researchers have opted for an original experiment: trying to read the minds of people watching scenes from films.

“The activities that participants in fMRI studies usually perform are fairly artificial and designed to meet the experimenter's expectations,” explains Demian Wassermann. One example involves selecting the image on a screen that corresponds to a word you hear. Some laboratories have found that watching a film or listening to the radio provides a more realistic, more ‘natural’ understanding of human brain activity. What's more, the data generated is very extensive because it is linked both to the fact of watching a film, but also to what happens in the film itself." Scientists from Mind and GIN therefore draw on such studies, project the maps obtained from them into their morphospace, and are able to predict what the participants are thinking every second of each scene. 

For example, among people who observed the scene in the film “Home Alone” where the child goes down into the basement and is terrified by the boiler, the morphospace decoded an increasing activation of the cerebral areas linked to attention, rhythm and memory as the child's fear grew, followed by an activation of the areas linked to imitation when he escapes from the basement. "It's as if the participants were personally involved in the scene and were identifying with the main character,” remarks Valentina Pacella with amusement. 

Similar findings were observed among people who watched a scene from “Star Wars” Episode V, in which Luke Skywalker is attacked by a snow monster: before the attack, the morphospace detected an activation of the consciousness and decoding zones; and during the attack, it detected an activation of zones linked to more active states, such as directional attention and the processing of visual information. In short, “we have been able to assign semantic characteristics to cerebral activity”, enthuses Demian Wassermann. 

Filling the “gaps” in cognition

Will we be able to use this new tool to read minds simply by copying fMRI images into the morphospace? Not just yet, say the researchers. “Human cognition is so complex and neural networks are so intricate that it is difficult to predict cognitive functions precisely,” states Valentina Pacella. But the creation of the morphospace is a giant step into the future.” 

The team intends to continue along these lines: Demian Wassermann and his colleagues will continue to develop the morphospace by integrating even more data and designing more precise AI algorithms, which will lead to more accurate thought prediction in the future.

In the meantime, their study is already paving the way for two applications. The first will enable researchers to validate an approach: “Scientists conducting experiments with fMRI will be able to use our morphospace to check that their results are in line with our representation, and therefore that their protocol is well designed," explains Valentina Pacella.

The second stems from the additional objective originally pursued by the researchers: identifying new cognitive functions. “We think that the remaining gaps in the morphospace represent functions that have not yet been described or studied with sufficient precision," continues the neuropsychologist. Researchers will be able to hypothesise about them and project them into our representation in order to supplement it.” 

Demian Wassermann compares the morphospace to Mendeleev's table of elements: “When he developed it, some elements were missing. This doesn’t mean that his system wasn't perfect, on the contrary: it indicated that there were still elements to be discovered.” In the same way, the morphospace paves the way to a better understanding of human cognition... Now it is up to neuroscientists to take it on board.

Find out more

• A language for improving the evaluation of brain activity, Inria, 7/9/2023.
• Cette IA révolutionnaire lit dans les pensées et les transforme en texte, Science et Vie, 10/1/2024.
• Prédire toutes les activités cognitives à partir d'images du cerveau en fonctionnement, c'est possible, CEA, 30/6/2022.
• Our brain anticipates the meaning of words while listening to speech, ENS, 30/3/2023.