Using VR to Guide Physicians Through The Patient’s Body
Ludovic Avot and Yannick Le Berre are big video game fans with Avot’s favorite being “Fallout 4, which guides players through post-apocalyptic Boston”. The game is bleak but immersive, and playing it gave the GE Healthcare designer an idea. What if doctors could use video game technology to step inside the human body — like the heroes of the sci-fi movie “The Fantastic Voyage” — to inspect organs and tissues and search for disease? “We were inspired by the photo-realistic rendering techniques of the high-quality games,” Avot says.
Le Berre, an engineer in medical imaging, wanted to create a way for doctors to walk through the human body using virtual reality. After a few simple but promising prototypes, he decided, with Ludovic, to move things further.
Working at GE Healthcare’s Global Center of Excellence in Medical Imaging Software in Buc, France, the pair spent a hack week — time where GE designers can work on a project outside of their current work — with VR design tools and other gaming software. They used detailed 3D information from CT and MRI body scans to build a virtual experience complete with color, texture, light and other features.
Doctors can use it to observe the glistening pleura of the lungs or ashen pink matter of the brain. They can also “enter” a specific part of the body and closely examine it for things like polyps, tumors and lesions.
Le Berre explained that they used a tracking system capable of following movements so accurately, it doesn’t make users dizzy, like some other VR environments. The prototype, which can be used with off-the-shelf VR headsets like the Oculus Rift®, could be a vital turning point in offering doctors a fresh interpretation of a medical image or to better prepare them for surgery.
“Imagine walking into a virtual room that is your patient’s brain,” says Francois Lenfant, general manager for Global UX Brand & Design Language at GE Healthcare. “You can enlarge the area of interest to the point where you can go inside the brain cells. When you’re immersed in an image, you can work intuitively and find new perspectives.”
Top image and above: “Imagine walking into a virtual room that is your patient’s brain,” says Francois Lenfant, general manager for Global UX Brand & Design Language at GE Healthcare. Images credit: GE Healthcare