Artificial intelligence in radiology: what benefits can it provide?

"In November 2016, Geoffrey Hinton, one of the pundits of artificial intelligence, declared that ‘‘in five years’ time, deep learning will be so powerful that we will no longer need to train radiologists’’”, remembers Thibaut Jacques, a radiologist at Lille University Hospital. “It was controversial, but it showed that, in the field of computing, there was huge potential for AI applications in health… at a time when nobody was really interested in the field. So specialists in both health and digital technology had to work together to assess whether deep learning really could deliver feasible and pertinent solutions for radiology.” 

Quite logically, Thibaut Jacques and his colleagues turned to the Scool team at Inria, led by Philippe Preux, since both structures are supervised by the University of Lille. When the two men met in 2018, they shared the same conviction: AI was far from being able to replace radiologists. But they were also curious to know how far such tools could go. “In our scientific community, we talk of high-performance algorithms, but on images that are all the same size, or with elements to identify that any human could spot”, explains Philippe Preux. “In radiology, it’s completely different: the images are not all the same size and non-expert human eyes can hardly make out anything. It’s a completely different context. It was an interesting challenge to see how public research, with limited means, would be able to adapt.”

The first key step was to obtain funding. In the framework of a joint State-Region “data” contract plan led by Inria, Philippe Preux submitted a project called RAID: Radiology AI Demonstrator. A sum of €100,000 was granted at the end of 2018, allowing them to hire an engineer for 18 months and buy a computer that would be up to the task.

The next stage was to choose the type of pathology that the AI would have to identify. The team opted for elbow radiology in paediatric traumatology. “This is particularly complex and leads to many diagnostic errors,” says Thibaut Jacques. “This makes it an interesting sub-field, for which the Lille University Hospital has data but which remains a niche area, outside of what other teams in the world are doing.”

Data scientist Franck Valentini was hired for the project and began work in autumn 2019. To save time, he used pre-trained artificial neural networks available online, meaning they were already able to recognise certain geometric shapes. He refined their training with paediatric elbow radiographs labelled by Thibaut Jacques and his intern, Clémence Rozwag. “These images presented several difficulties for the neural networks,” says Philippe Preux. “On the one hand, they were not all the same size, and on the other hand, the zone of interest was not necessarily centred and sometimes appeared at different angles. Lastly, radiologists base their diagnosis on a series of scans. On some of them, the pathology is clearly visible but on others not at all… and yet they are all labelled as corresponding to the same pathology!”

However, two algorithms developed by the team successfully overcame these obstacles with an accuracy rate of over 90%.

“It is a very satisfactory result given the resources available to the project,” says Philippe Preux. “However, it confirms what we thought and what the majority of researchers in the field agree on today: machines will support humans, but they won’t replace them.

This raises another question that the project team is also trying to answer: how will these tools interact with humans in a given environment? In other words, how will humans take account of the information produced by a machine that can make mistakes?

“To find out, we sent a hundred scans to be read by eight fellow radiologists. We compared the diagnosis they made before they had the result of the analysis produced by our algorithms and after,” explains Thibaut Jacques. “It would appear that one of the models slightly improves the accuracy of their diagnosis... but the other reduces it!” An added complexity is that it is currently impossible for the researchers to know what these differences are due to, especially as the radiologists did not all react in the same way, with some being misled less by the algorithm than others. “This goes to show that the performance of an algorithm will not be at all the same in the laboratory as in real conditions,” continues the radiologist. “So we must be careful with these tools. There is currently a lack of robust studies on the results of AI in health in practice.

With this project, experts in health and digital technology may have succeeded in developing a fairly powerful tool, but it still raises the crucial question of human-AI interaction. Thibaut Jacques plans to recruit larger cohorts of radiologists and repeat the exercise using algorithms that already have the CE mark (meaning they are available on the market) to assess their impact on diagnosis. It is the only way, in his view, to distinguish between solutions that are really useful, ineffective, and downright dangerous.

“We can develop as many increasingly powerful algorithms as we like, the question of interaction will always be there, in whatever domain AI is applied,” says Philippe Preux. “It is a fascinating issue that I would love to explore, but which requires us to consider things other than from a purely technical perspective.” An open door to future collaborations? The partnership between the radiologists and Scool is set to continue, at any rate. “This first project has built ties between the teams, led to a Ph.D. and a scientific article is currently being written,” adds Thibaut Jacques. “Why not continue and try to improve our models even further or see if they can be applied to other areas of radiology? There is no shortage of questions and ideas!”

• Scool team
• Game of Go (only in French), interstices.info
• Pixyl : When AI deciphers medical imaging