Cerebral aneurysms: augmented reality guiding surgeons

How can surgeons be helped to operate in the best possible conditions? Erwan Kerrien is a researcher in the Magrit team, and has been working on this topic since the beginning of his career. He has fostered an ongoing relationship with the surgical team at Nancy Regional University Hospital, one of the best in the world in the area of aneurysms. Over the years, he has attended several operations to more clearly identify the difficulties involved.

Accessing the brain via the femoral artery

"That's when you realise how delicate these procedures are", explains the researcher. surgeon is not operating on an organ that they can feel and see from different angles. They slide a catheter into the femoral artery in the groin and then move it up to the brain via the vascular system. When they reach the aneurysm, which measures no more than 5 mm in diameter, the surgeon releases tiny platinum coils into it which strengthen the distended walls and block blood circulation until a clot forms to seal it off. All of this is done by manipulating the other end of the catheter, which is roughly a metre and a half long".

To guide their movements, the surgeon has nothing but 2D black-and-white images of the vascular system displayed on monitors. flatten perspective and force the user to picture the scene in 3D themselves, with the ensuing risk of getting it wrong. It's a lot for the mind to be juggling". The Magrit team's solution? "Augment" the 2D images – in other words, enrich them with 3D images that provide additional information.

3D view of the aneurysm on surgical monitors

The starting point for Erwan was the advent of a new radiological technique called rotational angiography. Using image reconstruction algorithms, rapidly acquired views from different angles reveal the cerebral vasculature in 3D and more specifically the characteristics of the aneurysm, such as its exact location in the artery, size, shape and, especially, whether multiple lobes are present.

But for this 3D image to be truly useful, it must be possible to superimpose it onto the 2D images on the monitors. "Developing this was highly complex. To give the surgeon the impression that they were looking at a single image required very precise 3D image registration from each angle, adding shadows to render relief, making sure not to blot out important details of the real scene, and so on". In 2007, this technology was transferred to GE Healthcare, a world leader in medical imaging which has since marketed it in many countries.

Artery, aneurysm and surgical device in the same image

Erwan then threw himself into another challenge, that of "reconstructing the device". He wanted to add a second 3D image to the monitors, this time that of the catheter. Hence the surgeon would see the artery, aneurysm and surgical device simultaneously, and so re-connect what their eyes were seeing with what their hands were doing.

But how to go about creating a real-time 3D image on a computer that would represent a flexible metre-and-a-half-long tube moving through the arteries, twisting round and even forming loops? "Initially, the simulation speeds were far too slow", acknowledges Erwan.

But then the team realised that the catheter's behaviour was largely predictable, since it moves through the vascular system (which can be mapped in 3D), bends and curves but never breaks and never forms a right angle. "We turned to our colleagues from the Mimesis team, which is jointly affiliated with the University of Strasbourg and which specialises in real-time mechanical simulation in blood vessels. They helped us to develop reliable real-time algorithms".

 "Achieve 100% reliability to ensure medical professionals adopt this technology"

The Magrit team has since been working through the phases of validation. The team has conducted conclusive tests using a life-size see-through model of a blood vessel. "The images of the catheter reconstructed using two synchronised cameras must coincide with the simulation calculated by the computer”.

The most difficult scenarios remain to be tackled, like when the tip of the catheter is pointing at the surgeon: how can they tell whether it's moving forward or looping back on itself? Erwan's most recent research is into information-fusion algorithms to ensure constant, accurate superimposition of the simulated catheter onto the catheter seen in the intra-operative image. must achieve 100% reliability for healthcare companies and medical professionals to adopt this technology. Reconstructing the instrument will save the surgeon time and make their computer-based training easier".