Watch: How holograms, virtual reality, and AI are transforming the operating room

Surgeons may one day be able to rehearse life-saving operations before even touching a patient.

A mother donating part of her liver to her sick baby could, for example. undergo a safer, more predictable procedure because doctors have already explored the organ in three dimensions, turning it, slicing it and studying every detail in advance.

A Norwegian start-up believes this is no longer the stuff of fiction but is fast becoming part of modern medicine.

Holocare is developing virtual reality technology that lets surgeons view a patient’s organ as a 3D hologram, built from the same MRI or CT scans used in every hospital.

Instead of looking at flat grey images, doctors can step inside a virtual reconstruction and plan their approach together.

“So you imagine five surgeons trying to plan a complicated case, they all have different opinions, different ways of thinking about it, younger ones don't dare to say too much to the professor,” Alison Sundset, HoloCare’s chief executive, told Euronews Next.

“So then you have this very unique opportunity to collaborate and look at the images and expand the images, go inside the images and actually plan better surgery,” she added.

Seeing inside the body before surgery

The system uses artificial intelligence (AI) to convert 2D medical images into detailed 3D models.

Surgeons can then explore them through virtual reality (VR) goggles, enlarging parts of the organ, rotating it or cutting into the model with virtual tools.

In a demonstration to Euronews Next, two product designers swung their arms and pinched their fingers in the air.

To observers, it looked like a mime performance, but inside their headsets, they were examining the real liver of a patient with colorectal metastasis.

They were practising a resection plan, testing how to remove a tumour while calculating how much healthy liver would remain.

They are simulating a resection plan to simulate how the tumour can be cut out. This process allows doctors to calculate the volume of the liver and the remnant of the resection.

“What we have learned from the clinicians is that the real value is the spatial feeling and the understanding of the anatomical structures that you get from looking at it from different angles, being able to expand it, and being able to cut into it with different tools,” Dag Otto Lund, Holocare’s design lead, told Euronews Next.

The technology has recently been authorised for use in liver cases, with more organs expected to be authorised soon.

Hospitals in Norway, Germany, France, Spain, and the United Kingdom are already using the platform, according to the company.

A second chance for patients

This clarity matters because, traditionally, surgeons have had to render flat scans into mental 3D models in their heads.

It is a skill that takes years to master and often leads to different interpretations among team members.

“Younger surgeons find it very difficult[to translate 2D images into mental 3D models] and everyone ends up with their own interpretation,” said Sundset.

“With this tool, they are completely aligned,” she added.

Sundset said the technology has already changed real patient outcomes. One surgeon had declared a case inoperable and sent the patient home, but later reviewed the scans in 3D while teaching junior colleagues.

“He realised that actually he could operate,” she said. “He brought the case back in, and the man is alive and well with his children”.

Holocare says its browser-based AI system can process images into a 3D model in about ten minutes.

It is designed so surgeons can load images directly into the software, reducing reliance on radiologists and saving time.

However, the company underscores that its AI software is not a diagnostic tool and that poor-quality imagery input will lead to poor output.

“The AI only gives suggestions. The clinician or the surgeon, him or herself, needs to go through all the labelling [of lesions] and verify that it's correct,” said Lund.

The company is now working on bringing real-time holographic guidance into the operating room through a surgical robot.

“Our ultimate goal is that a surgeon would have on a lens and be looking at the patient so the hologram would be overlaid and fixed onto the body of a patient... and you would operate in 3D through the hologram,” said Sundset.

For more on this story, watch the video in the media player above.